- Building
Campus Event Assistant Build Log
Campus Event Assistant is a web app that help students find campus events offering free food or drinks at Stanford or at Berkeley. Nobody reads hundreds of listings a week to find the few that mention lunch, so agents do that reading and the page shows only the result.
The Spotlight at the top shows the most interesting coming up. Visitors sign in once with a Stanford or Berkeley email to see the full list. Live site · Demo video
Build week
November 17 · Where the food is announced
We started by finding out where free food is announced on campus, and verified each place we found. Then we wrote a design doc. It set the rule the rest of the week followed: when the page says pizza, there is pizza.
November 18, overnight · A first version, and a first failure
A first demo had a single agent browse event pages for several minutes and return a digest. The real build has the agents work ahead of time, and the page shows what they last found, so a visitor never waits on an agent.
The first live run started at 12:42 am and gathered 144 Stanford events. The first agent task then failed in 18 seconds, refused by the model provider. Nothing was listed.
November 18, morning · Real results, and one idea that did not work
We changed the model and ran again. This time all 144 events were read: 21 stated the food outright, and 20 only hinted at it.
The first results changed the product. An early version showed hinted events with a muted "Likely" chip. We removed it. A visitor walks across campus for the food, so "reception to follow" is never shown. We also added a food or drink filter, which the design had first ruled out.
We tried having an agent search the web for free boba events. The search engine turned the automated searches away with a verification page, and we removed the feature within the hour. The app does not work around such checks.
November 18, afternoon · The Spotlight and a second campus
The Spotlight banner took fourteen commits to settle. It began as three slides, then became one event: the most interesting food or drink coming up, with the event title as the headline and a real photograph filling the banner.
Then we added Berkeley. A switch in the header picks the campus, and the page never mixes the two. The first Berkeley run read 179 events. Across both campuses the page then listed 56 events out of 358 read: 35 at Stanford and 21 at Berkeley.
November 18, evening · Calendar, sign-in and sharing
Each event row gained an add to calendar button, and the site gained a home screen icon, a pizza pennant. We built a school email sign-in. A signed-out visitor sees the Spotlight and how many events each day has. A Stanford or Berkeley address is emailed a six-digit code, and typing it signs the device in. There are no passwords.
We registered
whatsoncampus.app, added a Share link with a preview card for chat apps, and merged the pull request at 12:35 am.November 19 · Live on its own domain
We recorded the demo video and submitted the project. The site is live at whatsoncampus.app, refreshes its listings every day, and has the email sign-in switched on.
Where HarnessRouter fits
Agents on HarnessRouter do the reading a person would otherwise do: they go through event listings and decide whether food or drink is really provided to people who attend. The site does everything else, including the listing, the time rules and the sign-in.
Three things keep the page trustworthy:
- Only an event whose listing states the food outright is shown. A hint is not enough.
- Everything an agent returns is checked by the site before it can appear on the page.
- The page shows saved results, so it stays up and fast when an agent task fails or runs long.
What we learned about shipping
A spending limit was the first safeguard to fire. On the first live run we had set the limit very low, and it paused most of the work before any money was spent. We kept the safeguard and tuned the limits.
Accuracy came from saying no. The instructions we give the agents spend more words on what does not count than on what does: a lunchtime concert, a talk about food.
The same event often appears in more than one place under slightly different titles. Showing it once, with the best information from each place, needed its own rules and tests.
November 19 snapshot: 54 commits on main · 1 merged pull request · 182 passing tests · 2 campuses · 56 events listed in the first full run
- Blocked
1. A refused model looked like a refused key
What happened: our first live task failed 18 seconds after it started. The error said our key was refused, with a 403 from the model provider. The same key worked a few hours later on another model, so we believe the model was what the provider refused.
To reproduce
- Create a harness on a model the provider refuses for your account.
- Start a background task on it.
- Read the finished task. It failed, and the message points at the key, not at the model.
What we did: we changed the model and ran again. Nothing told us about the problem when we created the harness, so we found it on the first real run, after midnight. (See Want 1.)
2. An agent that was refused found another way
What happened: we gave an agent a web search to do. The search engine answered with a verification page. The agent did not stop. It moved to other search engines and wrote its own scripts, and the task ran for twelve minutes before it was cancelled. That one task was more than half of our agent spend for the week.
To reproduce: give an agent with web access a search task, and have the search engine refuse it.
What we did: we rewrote the instructions to say stop at once when refused, then removed the feature the same hour. Apart from the task's time and cost limits, instructions were the only brake we had. (See Want 2.)
- Loved
- A client we did not have to write. We copied HarnessRouter's published Node adapter into the project unchanged. Background tasks, polling, file downloads and safe retries all worked on the first day.
- A task never runs twice. Every task starts with an idempotency key. When our server restarts in the middle of a task, it picks the same task up again and never sends it a second time.
- Limits on every task. Each task carries a cost cap and a time limit, so a confused agent cannot run up a bill overnight.
- The real cost of each task. HarnessRouter reports what each task cost. We show it per run on our own status page, and stop the day's work when the month's limit is near.
- Harnesses as code. We create and update harnesses through the API from files in our repository. An update carries a version tag, so it cannot silently overwrite a change someone made in the console.
- Changing the model is one field. After the first failure we changed the model with a one-line edit and one command.
- Want
1. A model check when a harness is created, and an error that names what was refused
Problem: a harness can be created on a model that will fail on every task, and the failure then blames the key (Blocked 1).
Who it affects: any team setting up a harness for the first time, or changing its model.
How it would help: the mistake shows up at setup, in daylight, and not on the first scheduled run. An error code that separates "this key is refused" from "this model is not available to you" tells the developer which of the two to fix.
Loading