Nelson LaiAuthor
- Building
Sentinel build log
A record of how we built Sentinel on HarnessRouter during the hackathon: what we did and when, the
real numbers from our agent runs, what broke and how we fixed it, and candid feedback on
HarnessRouter itself. Everything here comes from the actual build: the session history, git log,
and the HarnessRouter API responses for our runs.- Product: Sentinel, an agentic security analyst. Paste code, drop in files, or paste a GitHub
repo. A sandboxed agent audits it and returns a graded, triaged findings report with fixes, and
you can ask the same agent for the hardened code. - Agent backend: one HarnessRouter harness,
Security Analyst
(chrn_01a74a33efe94123a15d3d8958abe16d), baseqwen(Qwen Code CLI), modelqwen3.8-max,
Workspace Default Workspace. - Stack: Next.js 15 (App Router, TypeScript) for Vercel, TiDB Serverless for task records,
ElevenLabs for the demo voiceover only. - How we built it: the team directed an AI coding agent through HarnessRouter's own onboarding
path:agents.mdplus the integration Skill, vendored at.agents/skills/harnessrouter/. - Size: 40 source files (about 4,500 lines of TypeScript and CSS, excluding bundled sample
data), 28 unit tests, 9 commits.
Timeline (Nov 18, 2026, UTC)
Time What happened 02:10 Kickoff with the HarnessRouter hackathon prompt. Read agents.mdand the Skill (SKILL.md plus the API, runtime, files and security references).02:17 Our coding assistant's own safety layer paused the session on a "cyber" flag and we had to resend the request. It happened again at 03:27. A small, real example of the over-refusal problem Sentinel is built to solve. 02:25 Stored the key in a gitignored, owner-only .env.harnessrouter. Discovery (GET /v1/me) answered 200 and reportedcapabilities.memories: false, so we send nomemoryfield.02:30 Created the harness with the Skill's harness_config.py ensure: name, base, model and standing instructions (output contract plus security guardrails and arole_mismatchrule). Saved the reviewed copy asharnessrouter.json.02:30-03:20 Built the app: Next.js API routes call POST /v1/responseswithbackground: trueand pollGET /v1/responses/{id}. Added signed, sandboxed report previews, owner checks on every route, and a signed cookie id sent asuser.03:05 Bug: cookie and preview signatures failed between routes. Each route bundle had generated its own fallback secret. Fixed with one stable secrets module. 03:18 First real data on timing: thorough reviews on qwen3.8-maxtook about 4-20 minutes. One run finished in 1,069 s for $0.44; another hit our initial 20-minute cap and endedincomplete. We raised budgets to 3,600 s / 200 steps / $4.03:24 Verified end to end through the app's own routes: a completed review (231 s, 10 findings, 2 Critical), preview 200 with the sandbox CSP, tampered token 404, another user 404, and no upstream ids in any response. 03:30-04:14 Judges README and the first demo video (slides plus UI stills). Merged to main.04:16-04:22 Added TiDB Serverless as the durable store (HTTP serverless driver, so no connection pool to exhaust from Vercel functions). 18:02-18:07 Verified TiDB live: created the harnessrouterdatabase and round-tripped a real task record through the app.18:50
How Sentinel uses HarnessRouter
HarnessRouter feature What Sentinel does with it Harness resource ( POST/GET/PUT /v1/harnesses,ETag+If-Match)The whole agent: base, model, instructions, guardrails, budgets. Updated twice after creation, with no app deploy. POST /v1/responseswithbackground: trueEvery scan and follow-up. Serverless-friendly: the function returns in milliseconds. GET /v1/responses/{id}Polling, and recovery: every poll can settle a finished task even if the original request died. Idempotency-KeySent with every new task and stored with the record, so retries never start a second run. POST /v1/files+input_fileA pasted GitHub repo becomes one tarball upload (up to 25 MB) that appears in the agent's working directory. GET /v1/sessions/{id}/turnsThe live activity feed (tool names mapped to friendly labels, never arguments) and accurate "Completed in" times from completed_at.GET /v1/sessions/{id}/files,?changed=true, file contentReading findings.jsonandmanifest.json, the sandboxed HTML preview, downloads, and separating follow-up outputs from the original report.previous_response_id+metadata.session_idFollow-ups ("show the fixed code") run in the same sandbox with the repo and the report still on disk. max_step,timeout_seconds,max_cost_usd,incomplete_details.reasonPer-task ceilings, plus a "stopped early, partial results" state in the UI. POST /v1/responses/{id}/cancelCancelling runs we started only to record the demo, so they don't burn their full budget. userThe signed-in person's id on every task, set by the server, never by the browser.
Real runs
Durations are from turn timestamps. Spend is what
GET /v1/sessions/{id}reports, which is the
session total, so a scan plus its follow-up share one number.Run Input Duration Result Tokens (in / out / cache read) Session spend Express API review (v1) pasted file 231 s and 1,069 s 10 findings, 2 Critical n/a $0.44 (1,069 s run) Container review Dockerfile + compose 13.7 min 17 findings 76k / 36k / 0.66M $0.54 Invoice API audit pasted app.py14.3 min 15 findings 65k / 36k / 1.2M $2.13 for both turns ↳ follow-up: hardened code same session 23.7 min 545-line drop-in, 41-test suite, 87 offline assertions passed 135k / 60k / 3.4M (included above) we45/Vulnerable-Flask-Appaudit #1GitHub repo 16.4 min 20 findings, 5 Critical 89k / 38k / 1.9M $4.03 for both turns ↳ follow-up: hardened code same session failed at 38.3 min provider_error208k / 92k / 8.7M (included above) we45/Vulnerable-Flask-Appaudit #2GitHub repo 25.5 min 24 findings, 6 Critical, PoCs reproduced in the sandbox 141k / 55k / 3.6M $4.05 for both turns ↳ follow-up: hardened code same session failed at 33.8 min provider_error207k / 79k / 6.6M (included above) Audit #3 GitHub repo cancelled at ~7 min used for the live-feed shot n/a n/a What this taught us about the product:
- Per-task costs. A single-file review costs roughly $0.50-0.75; a whole-repo audit with
sandboxed proof-of-concept testing is roughly $1-2. Writing a full hardened version plus tests is
the most expensive step. The per-turn figures are estimated from token shares, because spend is
only reported per session. - Long follow-ups are fragile. Writing a complete hardened version plus tests ran 24-38 minutes,
and two of three such runs hit provider errors. In production we'd scope them smaller (one file
or one finding at a time) and give them a tighter time budget. - Duration. A task takes 4-26 minutes, so background mode is the only sane design on serverless,
and the UI must make waiting feel productive (live activity, "you can leave this page"). - Real work in the sandbox. The agent ran the target code there: it rendered
{{7*7}}as 49 to
prove template injection, forged a JWT with the leaked key, and ran its own regression tests.
That's the edge we were after.
Problems we hit, and fixes
Problem Fix Cookie and preview signatures failed between routes (each Next route bundle generated its own random fallback secret). One shared secrets module, with stable COOKIE_SECRETandPREVIEW_SIGNING_SECRETenv vars in production.A review hit our 20-minute time cap and ended incomplete.Budgets raised to 3,600 s / 200 steps / $4, configurable by env. The UI shows partial results instead of an error. Task records live in memory by default, which breaks on serverless (polls land on other instances). TiDB Serverless via its HTTP driver, with Vercel KV as a fallback. The sandbox network blocks GitHub's archive host. A git clone --depth 1fallback (validated args, no shell). On Vercel the archive path is used."Completed in" times were wrong when a task was settled late. Use the turn's completed_atfrom the session turns.A follow-up failed with provider_error. Our UI showed the agent's half-finished last sentence as the error, and blocked further follow-ups because the latest turn was not continuable.Failed runs now sh - Product: Sentinel, an agentic security analyst. Paste code, drop in files, or paste a GitHub