<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Albedo Lab]]></title><description><![CDATA[Notes from building and measuring: distributed systems, AI agents, databases]]></description><link>https://albedolab.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Thu, 12 Nov 2026 09:29:29 GMT</lastBuildDate><atom:link href="https://albedolab.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[48 closed cards, no passing design: a multi-agent build with Hermes Agent, part 1]]></title><description><![CDATA[AI agents on a Kanban board built one game level in 38 hours. 56 cards closed, 22 hours idle, and the coordinator cost eight times the workers.]]></description><link>https://albedolab.hashnode.dev/building-a-system-design-game-with-hermes-part-1</link><guid isPermaLink="true">https://albedolab.hashnode.dev/building-a-system-design-game-with-hermes-part-1</guid><category><![CDATA[ai agents]]></category><category><![CDATA[llm]]></category><category><![CDATA[Software Engineering]]></category><category><![CDATA[AI]]></category><category><![CDATA[kanban]]></category><dc:creator><![CDATA[Nikolay Fokin]]></dc:creator><pubDate>Thu, 12 Nov 2026 02:38:34 GMT</pubDate><content:encoded><![CDATA[<p>I wanted to find out whether a team of AI agents could build a feature mostly on their own. The feature was one new level for a system design browser game, described in a detailed document. I used <a href="https://github.com/NousResearch/hermes-agent">Hermes Agent</a>, an open-source agent from Nous Research that can run a team of agents from a Kanban board. The run took 38 hours, and four things went differently from what I expected:</p>
<ol>
<li>Cards closed while the level did not work. With 48 cards closed, no design could pass it.</li>
<li>Most of the time nobody was working. For 22 of the 38 hours, no agent was running.</li>
<li>One card held up the rest. It took 17 runs, with twelve other cards waiting behind it.</li>
<li>Fifteen minutes of my own play found defects that 56 reviewed cards had missed.</li>
</ol>
<p>In the end the level was accepted with two scenarios out of the six that were planned. At API list prices the agents on the board used about $40 of tokens. The coordinating agent that kept the board moving used about $350. That is the price of the tokens, not what I paid: most of the models ran on flat subscriptions.</p>
<p>The game is live: <a href="https://albedoweb.github.io/system-design-game/">albedoweb.github.io/system-design-game</a>. The <a href="https://github.com/albedoweb/system-design-game">source is public</a>.</p>
<img src="https://cdn.hashnode.com/uploads/gql/6aefe354f8418419009a8845/445fa8b5-f423-4610-8d4b-4cf94ee1a0d3.png" alt="The Global Cart level: a design on the canvas after a passing run" style="display:block;margin:0 auto" />

<h2>The game and the task</h2>
<p>System Design Game is a browser game where you draw an architecture and then run load through it. In the Global Cart level, you build a shopping cart for customers in San Francisco, Frankfurt and Tokyo. You assemble the system from a set of components, and it has to pass several tests. The simulation runs two scenarios against your design: normal load and a rush.</p>
<p>Behind this is a deterministic discrete-event simulation with queues for CPU, disk and network links. It has to keep its books straight: a write that was never delivered must have no effect, and a lost acknowledgement must not undo a write that did happen.</p>
<p>The game already existed. The task was to add this one level.</p>
<h2>Why Hermes</h2>
<p>I wanted to experiment with a multi-agent harness. What I liked about Hermes is that the work is built around task cards on a Kanban board. A card has an assignee, dependencies and a reviewer, like a ticket in a real development team, and that way of working is familiar to me.</p>
<h2>The setup</h2>
<p>Every card is assigned to a profile: a role with its own model and instructions. I started with five roles.</p>
<table>
<thead>
<tr>
<th>Role</th>
<th>Model</th>
<th>What it does</th>
</tr>
</thead>
<tbody><tr>
<td>Lead</td>
<td>GPT-6 Astra</td>
<td>Writes the engineering contract and splits the work into cards</td>
</tr>
<tr>
<td>Game master</td>
<td>GPT-6.1 Sol</td>
<td>Owns the player's side: the concept, the balance and the final verdict on the level</td>
</tr>
<tr>
<td>Programmer</td>
<td>DeepSeek V4 Flash</td>
<td>Writes the code, in two profiles: one for the simulation, one for the interface</td>
</tr>
<tr>
<td>Tester</td>
<td>DeepSeek V4 Flash</td>
<td>Writes independent checks and takes measurements</td>
</tr>
<tr>
<td>Reviewer</td>
<td>GPT-6 Luna</td>
<td>Reviews every card before it can close</td>
</tr>
</tbody></table>
<p>During the run I added programmers on stronger models: one on Sol, then several on Claude Sonnet 5.5.</p>
<p>One more agent stood outside the board: a coordinating agent, a long Codex session on Astra. It prepared the cards and their git worktrees, restarted work that had stopped, and was the one to be alerted when a card got blocked.</p>
<p>Three units of work appear below:</p>
<ul>
<li>A run is one session of one agent on one card. It ends when the agent hands the card on, or when a limit stops it.</li>
<li>A step is one model call with its tool call inside a run.</li>
<li>A round is one author run followed by a review.</li>
</ul>
<p>The limits at the start were 15 minutes and 64 steps per run, three rounds per card and at most three runs at once. I raised several of them during the run.</p>
<p>The lead split the level description into 24 cards with 44 dependency links.</p>
<h2>Cards closed, level not working</h2>
<img src="https://cdn.hashnode.com/uploads/gql/6aefe354f8418419009a8845/5d3a91e2-86f0-4edd-98f7-f703823291ec.png" alt="Cards on the board and cards closed over 38 hours" style="display:block;margin:0 auto" />

<p>The board grew from 24 cards to 68, and 56 of them were closed, each after an independent review. The other 44 cards were added by the coordinating agent along the way: repairs and diagnostics after each failed integration or acceptance, and at the end the cards for a reduced level.</p>
<p>For the first 27 hours, the count of closed cards kept rising while the level did not work. A design here is one architecture built in the game. The game master first ran the whole level at hour 20, with 24 of the 30 cards closed: seven test runs, no passing design. At hour 27, with 48 cards closed, three more complete designs were run against all six scenarios, and again none passed. Checkout latency at the 95th percentile was 953 to 974 ms against a limit of 750 ms.</p>
<p>At that point I cut the level from six scenarios to two. The four I dropped covered customers who move between regions, two devices editing one cart, a network partition and a race for the last items in stock. Ten hours later, 7.3 of them spent waiting, the two-scenario level passed acceptance.</p>
<p>Reviews did their job: the reviewer sent work back 30 times, on 20 different cards. But each review checked one card against its own description, and nothing ran the whole player path until hour 20.</p>
<h2>Most of the time, nobody was working</h2>
<p>The whole run took 38 hours by the clock, and for 22 of them no agent was running. At least one agent was working for 16 hours, and two or three at once for under 3 of those.</p>
<p>Most of the idle time was three long waits, 14.6 hours together. Each was caused by a limit that did not fit the work, or by an alert that never reached the coordinating agent.</p>
<p>The first was 3.2 hours. A run reached its 64-step limit just as it handed a finished card to review, and the limit handler in Hermes overwrote the handoff. The alert about the blocked card went to the wrong conversation of the coordinating agent, so nothing woke it.</p>
<p>The second was 4.1 hours. Five repair cards were done, and the next step was an integration that only the coordinating agent could do. Again the alert did not reach it.</p>
<p>The third was 7.3 hours. A card that had already passed its checks had used up its reviews, and only I could allow one more.</p>
<img src="https://cdn.hashnode.com/uploads/gql/6aefe354f8418419009a8845/73d621f8-130e-41fb-ba7b-d37f0e7bcc07.png" alt="How the 276 agent launches on the board ended" style="display:block;margin:0 auto" />

<p>The board recorded 276 launches of an agent. In 43 of them the agent never started, 26 times because the card had used up its rounds or the shared budget. Seven more are bookkeeping entries. Of the 226 runs that did start, 49 ended without handing the work on, which is about one in five. Of those, 17 hit the step or time limit, 17 stopped to ask for a decision, 9 crashed and 6 failed in other ways.</p>
<h2>One card held up the rest</h2>
<p>The parts of the simulation were built in parallel, and the card that joined them turned out to be a redesign of how the simulation moves time forward. The DeepSeek programmer made six author runs on it without getting it through review, and it was then split into five smaller repair cards. In the end it took 17 runs, and at one point twelve other cards were waiting behind it.</p>
<p>About half of the cards took the short path: 32 of the 60 that were started closed in two or three runs.</p>
<p>Most of these problems came from planning, mine and the coordinating agent's, and I do not hold them against the tool. Hermes had two defects of its own, which I patched locally. The overwritten handoff was one of them, and part 2 goes into both.</p>
<h2>The result and the cost</h2>
<p>The level with two scenarios was accepted: a reference design passes, and designs that should fail do fail. The level as first specified was not built. Twelve cards stayed open, all of them work for the original six scenarios.</p>
<p>Only DeepSeek was billed by the token, about $4.50. The other models ran on subscriptions, so I estimated what their tokens would have cost at API list prices.</p>
<table>
<thead>
<tr>
<th>Role</th>
<th>Model</th>
<th>Tokens in / out</th>
<th>Cost*</th>
</tr>
</thead>
<tbody><tr>
<td>Programmers and tester</td>
<td>DeepSeek V4 Flash</td>
<td>281 M / 2.6 M</td>
<td>$4.50, billed</td>
</tr>
<tr>
<td>Extra programmers</td>
<td>Claude Sonnet 5.5</td>
<td>63 M / 0.6 M</td>
<td>about $20</td>
</tr>
<tr>
<td>Reviewer</td>
<td>GPT-6 Luna</td>
<td>90 M / 0.4 M</td>
<td>about $2</td>
</tr>
<tr>
<td>Game master and one programmer</td>
<td>GPT-6.1 Sol</td>
<td>39 M / 0.2 M</td>
<td>about $9</td>
</tr>
<tr>
<td>Lead</td>
<td>GPT-6 Astra</td>
<td>3 M / 0.03 M</td>
<td>about $7</td>
</tr>
<tr>
<td>Coordinating agent</td>
<td>GPT-6 Astra</td>
<td>290 M / 0.6 M</td>
<td>about $350</td>
</tr>
</tbody></table>
<p>* The cost of the tokens at API list prices, not the cost of a subscription. Only DeepSeek was really billed this way. The other models ran on flat subscriptions, so this is not what I paid.</p>
<p>The agents on the board come to about $40. The coordinating agent comes to about $350, more than eight times as much. It ran on the most expensive model, and in one long session it re-read a large context on every step: 99% of its input tokens were cache reads. I did not count my own time.</p>
<p>For the same money, one strong agent with a cheap reviewer could probably have done this work without the overhead of the board. I have not measured that yet.</p>
<h2>Fifteen minutes of play</h2>
<p>The version you can play took one more evening in a simpler setup, on top of the work the board had done: one coordinating agent, Claude Code, handing small tasks to cheap sub-agents while I played the level between rounds. Fifteen minutes of play found things that 56 reviewed cards had not:</p>
<ul>
<li>The design checklist always reported purchased circuits as missing.</li>
<li>A connection dropped on a component, a few pixels away from its small port, was silently lost.</li>
<li>A failed run opened a report 64,000 pixels tall, on the scenario that had passed.</li>
<li>A rule survived from the scenarios I had cut: every application had to read storage in all three regions. No review noticed, because every review tested the reference design, which obeys the rule.</li>
</ul>
<p>The first three are fixed in the version you can play. The rule is still in the level, and the game now explains it.</p>
<h2>What I learned</h2>
<ul>
<li>Split the project into smaller and more concrete tasks. The card that joined the simulation parts was a super-task: the DeepSeek programmer could not get it through in six runs, and twelve cards waited behind it.</li>
<li>Check the dependencies between tasks before the start. I need a dependency viewer, or a checker that finds a card with twelve others behind it before the first run.</li>
<li>Set the limits with care. Limits are needed, but mine did not fit the work: a run that hit its 64-step limit lost a finished handoff, and a card that had passed its checks waited seven hours for one more review. Badly set limits and a broken alert path to the coordinating agent caused the idle time.</li>
<li>Cards that are each tested and reviewed do not add up to a working game. It is obvious, and still worth saying: 48 cards were closed while no design could pass the level.</li>
<li>Full autonomy did not work out, and acceptance has to start early. Fifteen minutes of my own play found what the board had missed.</li>
</ul>
<p>At the start the task looked very simple to me, and I did not expect this many problems. The preparation before the first run deserves far more work than I gave it.</p>
<h2>Next</h2>
<p>Two more parts will follow:</p>
<ul>
<li>Part 2, the cost of coordination: the board read as a distributed system, with its limits, lost handoffs and alerts that went to the wrong place.</li>
<li>Part 3, Hermes next to a simpler harness: what I would use each for.</li>
</ul>
]]></content:encoded></item><item><title><![CDATA[From Hacker News threads to PG Finds]]></title><description><![CDATA[How I turned Hacker News discussions into a source-linked PostgreSQL reading list.]]></description><link>https://albedolab.hashnode.dev/building-pg-finds</link><guid isPermaLink="true">https://albedolab.hashnode.dev/building-pg-finds</guid><category><![CDATA[PostgreSQL]]></category><category><![CDATA[Databases]]></category><category><![CDATA[data-engineering]]></category><dc:creator><![CDATA[Nikolay Fokin]]></dc:creator><pubDate>Sun, 08 Nov 2026 04:53:39 GMT</pubDate><content:encoded><![CDATA[<p>After reading <a href="https://karpathy.bearblog.dev/hacker-tube/">Andrej Karpathy’s original post about HackerTube</a>, I wanted to try the same idea for PostgreSQL: turn useful recommendations buried in Hacker News discussions into something easy to browse.</p>
<p>The result is <a href="https://albedoweb.github.io/pg-finds/">PG Finds</a> — <strong>263 articles, tools, guides, and other resources</strong>, each linked to its original source and the HN conversations where it appeared.</p>
<p><img src="https://cdn.hashnode.com/uploads/gql/6aefe354f8418419009a8845/732db8ca-194e-4b15-805d-3eed6b6983c8.png" alt="PG Finds: topic filters, search, and resource cards with links to original sources and Hacker News discussions." /></p>
<h2>What you can do</h2>
<ul>
<li><strong>Find something to read:</strong> search the catalogue and filter by topic, format, or HN mention year.</li>
<li><strong>See why it’s included:</strong> each resource has a short description and its own page.</li>
<li><strong>Go straight to the source:</strong> open the original article, documentation, or project.</li>
<li><strong>Follow the conversation:</strong> jump back to the Hacker News discussions for context.</li>
<li><strong>Save things for later:</strong> keep a reading list in your browser, without creating an account.</li>
</ul>
<p>The catalogue is a static site hosted on GitHub Pages. Its <a href="https://github.com/albedoweb/pg-finds">site source is public</a>.</p>
<h2>The agent team</h2>
<p>I used several agents with different jobs:</p>
<ul>
<li><strong>Astra — coordination.</strong> The more expensive lead agent planned the work, delegated tasks, reviewed results, and brought the pieces together.</li>
<li><strong>Luna and Terra — implementation and testing.</strong> Cheaper agents worked on data processing, site development, review, and browser testing of the catalogue.</li>
<li><strong>DeepSeek Flash — analysis and selection.</strong> It screened candidates in batches and reviewed extracted source text to help classify and annotate useful material.</li>
</ul>
<p>My role was to set the direction, try the results, and ask for changes. Using smaller agents for focused tasks kept the lead agent available for decisions and coordination.</p>
<h2>How it worked</h2>
<p><strong>Collect → Select → Build → Test</strong></p>
<ol>
<li><strong>Collect.</strong> Start with the open <a href="https://huggingface.co/datasets/open-index/hacker-news">Hacker News archive</a>, find PostgreSQL-related links, and preserve their discussion context. I began with 2024, then expanded to 2023, 2025, and the available part of 2026, supplemented by recent HN search results.</li>
<li><strong>Select.</strong> Remove duplicates, prioritize recent material, check links, and review promising pages. Agents produced short descriptions and flagged uncertain dates or project status.</li>
<li><strong>Build.</strong> Turn the selected resources into searchable cards with source links, HN references, and a reading list.</li>
<li><strong>Test.</strong> Use separate agents to check the catalogue in a browser, report issues, and record an acceptance-test scenario for future changes.</li>
</ol>
<p>The input covered about <strong>15.1 million HN records</strong>; the catalogue grew from 27 pilot entries to 263 resources. This was automated filtering and review, not millions of articles read by me. Coverage is selective, especially for 2026.</p>
<p>Recorded DeepSeek calls cost about <strong>$0.49</strong>. That is a partial processing cost: Astra, Luna, Terra, and some interrupted calls are excluded.</p>
<p>I’ll explore the collection’s statistics in a separate post. For now, <a href="https://albedoweb.github.io/pg-finds/">browse PG Finds</a> and see what catches your interest.</p>
<p>Thanks to the original authors and Hacker News contributors. The source archive uses the <a href="https://opendatacommons.org/licenses/by/1-0/">ODC Attribution License</a>. PG Finds is independent and is not affiliated with Hacker News.</p>
]]></content:encoded></item></channel></rss>