You open a channel for an audience to ask you anything, and within minutes you have a wall. The questions overlap, repeat, and contradict each other in wording while asking the same thing underneath. You cannot read them live, and nobody wants to scroll them.
What you actually have is a handful of questions wearing hundreds of costumes. The job is not to display the flood or to answer it item by item. The job is to collapse it — down to the few questions that, once answered, cover everything anyone asked.
Queuena is the product ML LABS built around exactly that collapse: a six-letter short code and a QR that turn any surface — a slide, a poster, a table tent, a screen behind a stage — into a question box. It lives in the internal portfolio at Escape Velocity Labs, a sister practice under the same owner. The submission side is trivial. The summarization is the product, and its economics are the reason the product is buildable at all.
Merge Intent, Not Wording
The first decision is what "the same question" means. Two questions phrased nothing alike are frequently the same question, and two questions sharing most of their words are frequently different ones. Collapsing on surface text produces false merges and false splits at the same time.
Queuena merges on intent — on what the asker wants — and it merges hard. The umbrella summary targets roughly five questions and refuses to exceed ten, no matter how many arrive underneath. That cap is not a performance limit. It is the product's spine: a system with room for forty umbrellas will produce forty, and forty umbrellas is the flood with better formatting.
Merge aggressively. Four well-merged umbrellas beat eight narrow ones, because the presenter can hold four in their head and act on them.
The bias toward aggressive collapse is a product judgment, not a technical default. A summary that faithfully preserves every nuance has failed at the only job it had. Stuffing the whole accumulated set into one long prompt does not rescue it either — long contexts bury information in the middle (Liu et al., 2023), so the fidelity you thought you were buying is spent on tokens the model reads least carefully.
A wrong merge and a wrong split fail differently, and the asymmetry decides how the policy should lean. A split that should have merged shows up on screen as two near-identical umbrellas, and the presenter fixes it in a second by answering both at once. A merge that should have split is invisible: the question that got absorbed simply never appears, and the person who asked it watches their question go unanswered while the summary looks complete.
So the model's job is deliberately narrow. It proposes the umbrella structure; the cap, the ordering, and the merge policy are code, and none of them are negotiable by the model. Keeping a model's scope tight and its authority explicit (Anthropic, 2024) is what makes the output predictable enough to put in front of a live audience without a human in the loop for every refresh.
Regeneration Cost Decides Viability
The summary is worthless if it is stale, which means it has to regenerate as questions arrive. Queuena regenerates every five new questions. That is arrival-triggered rather than clock-triggered, and it is the right trigger: a live talk that draws a burst of questions gets a fresh summary immediately, while a poster in a lobby collecting a trickle over a month does not pay to recompute a summary nothing has changed.
Now look at what that cadence costs. Regenerating every fifth question means the summarizer runs constantly during the moments that matter most — and a naive implementation re-sends the entire system prompt and the whole accumulated context on every single call. The arithmetic is not subtle. Continuous re-summarization becomes a feature you ration, and a rationed summary is a stale summary.
The fix is to cache the part of the work that does not change. Prompt caching that reuses a fixed prefix across calls (Anthropic, 2024) is applied to Queuena's system prompt, which makes regenerations roughly ten times cheaper after the first call. That single ratio is what moves continuous regeneration from a cost problem to a non-issue, and it is why the cadence could be set by the audience rather than by the budget. A cheap operation can run continuously; an expensive one gets rationed until it is stale. Which of the two summarization becomes depends entirely on whether the prefix was cached — an architecture decision made long before the first question arrives, and one that is far cheaper to make then than to retrofit under load.
When The Questions Are The Asset
There is a boundary where the summary stops being the point. Every question an audience asks is a demand signal — an unfiltered record of what people do not understand, want, or fear. At low volume that record is anecdote. At scale it is market research the audience is volunteering, and the moment it crosses that line, throwing away the raw questions after summarizing them destroys the most valuable output the system produces.
The collapse is lossy by design, and lossy is correct for the moment on stage. It is wrong for the quarter. A system built to serve only the live moment will discard the corpus that answers the strategic question — what this audience does not understand about the thing you are selling them — and the discard is invisible, because the summary still looks great. Keep the raw questions, keep the merge decisions, and treat the accumulated corpus as a retrievable asset in its own right: retrieval before generation (Lewis et al., 2020) is as useful over your own question history as it is over any other corpus.
The other half of that boundary is operational. A summarizer running continuously is a model in production, with all the obligations that implies — ML systems accrue hidden debt wherever monitoring and feedback are missing (NeurIPS, 2015), and a merge policy that starts collapsing distinct questions together will do so fluently, confidently, and without an alarm.
First Steps
- Set the hard cap before you write a prompt — pick the number of umbrella questions a human can actually act on, and force the system to prioritize into it rather than expand past it.
- Cache the stable prefix of every regeneration before you scale, so continuous re-summarization is affordable rather than rationed into staleness.
- Trigger regeneration on question arrival rather than a clock, so a live burst refreshes immediately and an idle surface costs nothing.
Summarize For Decision, Not Fidelity
Summarization is a decision aid, not a compression exercise. Its job is to let one human act on a crowd in seconds, which means collapsing hard, capping tight, and caching so that the collapse is cheap enough to run continuously. Fidelity to every distinction and usefulness under a hard cap pull in opposite directions, and the product has to pick a side out loud.
The merge policy and the cache strategy are the same decision, and neither is a prompt tweak. Both are architecture, both are set before the first question arrives, and both are the kind of thing that is cheap to get right at the start and expensive to retrofit — which is the general shape of every path from an AI pilot to production, and the same discipline that makes open-web signal extraction an ingestion problem rather than a model problem. When the model call is the cheap part, what your AI spend actually buys is decided upstream of it.
Back at the question box: the difference between a wall of text and a decision a presenter can act on is a merge policy, a hard cap, and a cached prefix. That is one contained workflow, and it is exactly what a production workflow build is scoped to deliver — acceptance targets written into the SOW before work starts, the system run against those targets before you accept it, and its first 30 days of operation run by the person who built it. The flood is not the problem. The flood is the raw material, and the collapse is the product.
References
- Anthropic. Prompt Caching. Anthropic Documentation, 2024.
- Liu, N. F., et al. Lost in the Middle: How Language Models Use Long Contexts. arXiv, 2023.
- Lewis, P., et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv, 2020.
- Sculley, D., et al. Hidden Technical Debt in Machine Learning Systems. NeurIPS, 2015.
- Anthropic. Building Effective Agents. Anthropic Engineering, 2024.
