From clusters to Issues

In a customer demo, I was walking through a screen I had designed and built: circles of agent failures, grouped by type and sized by frequency. The feedback was simple. It looked like a dashboard. What were they supposed to do with it?

They were right. I had made the problem visible. I hadn’t made the next step clear.

The problem behind the traces

I joined BentoLabs in March 2026 as the first hire. The founders were about to leave for YC, and there was a product idea to turn into something people could use. The CTO owned the backend. I owned the customer-facing product: the design system, onboarding, trace viewer, and what came next.

BentoLabs was building production infrastructure for AI agents: monitor what runs, improve what fails. On research and pilot calls, I kept seeing the same workflow. Teams sampled a few traces, pasted the bad ones into a model, guessed at the pattern, and tried a change.

They had the raw material. They needed a way to decide where to spend their attention.

Three questions the clusters couldn’t answer

The clusters view grouped similar failures together. But across demos, customers kept pulling the conversation toward three questions:

• Evidence. Why did you classify this as a failure?
• Severity. Does it need attention, or can we ignore it?
• Scale. How many people does it affect?

A PM needed to take an understandable problem to an engineering team, explain why it mattered, and agree on what to fix. Our screen still left that work to them.

A familiar unit of work

Developers and PMs already had a shared model for this: an issue. I proposed turning each failure pattern into a concrete, prioritised item, with a plain-language summary, a category, severity, and the evidence behind it.

The category mattered: a data leak and a hallucination call for different responses. The summary mattered: it needed to read like something an engineer could bring to standup. And the ranking mattered: the first item should give the team a place to start.

Every claim needs a way back

One rule shaped the whole product: every pattern must link back to the exact evidence that produced it. If an insight lived in Bento but its trace lived elsewhere, checking our work meant leaving the product.

That rule exposed another problem. A trace is whatever a customer chooses to log. One team might record a single exchange; another might put a long conversation inside one trace. We needed a consistent unit of analysis.

We introduced trajectories: one user message and everything the agent does until the next one. Issues were built on trajectories, and each trajectory linked back to the raw trace. It added a concept to explain, but gave the evidence a consistent shape.

Designing it in the working product

I took the proposal to both founders, and they backed it. Kaushik built the backend: summary generation, affected-user mapping, and ranking. I worked through the interface in Figma, then moved into code while there were still decisions to make.

A working screen with seeded data helped us judge things a mockup couldn’t. Did a summary make sense at a glance? Could you inspect the evidence without losing your place? Did the details still work when an issue had thousands of trajectories?

For evidence, I tried both tables on the page and a side panel. The panel kept the issue in view and avoided adding another competing trajectories table to the main page.

The summaries needed the most iteration. Early versions were cluster labels stretched into sentences. We kept refining them until they described a specific problem someone could discuss and act on. About a week after the whiteboard, we had a working demo.

The feature became the pitch

The conversation changed. Customers could see the failure, inspect why we had flagged it, and bring it to the rest of their team. One pilot’s first pass surfaced eight issue categories across 1,500 supplied traces, including unresolved conversations, data leaks, tool-parameter errors, and retry storms.

The next request was revealing: tell us what action to take for each issue. The conversation had moved from understanding the analysis to deciding what to do with it. That became the next direction for the product.

It also changed our demos. We had spent long stretches explaining what we did with people’s traces. Now we could open the Issues screen and start with a concrete problem.

Fixed meant watching

The fix introduced another design question: when is an agent issue actually done? With a reproducible software bug, you can often test a change against the exact condition that broke. An agent can pass that example and still repeat the same failure on a different request or with different context. Fixing one trace doesn’t settle the pattern.

That changed the lifecycle we designed. A new issue could move into drafting a fix, then a proposal for someone to review. Marking it fixed moved it straight into Watching. Future traces became the test: if the pattern appeared again, the issue could return for investigation.

“Done” meant a change had been made and its effect was being watched. The interface needed to show that continuing responsibility, rather than promise that one fix had closed the problem forever.

What that didn’t prove

A useful product did not settle the business. Some teams still chose to build internally. At its peak, BentoLabs had six live pilots, several paid; the founders paused the product in September as they reconsidered the kind of business they wanted to build.

I left with a lesson beyond interface design: ask about build versus buy early. A team agreeing that a problem is real is different from a team deciding to buy your solution.

What I’d carry into the next product

Start with the decision someone needs to make. Give them language they can share with their team. Make every claim inspectable.

The question I ask now, before building a screen, is simple:

What would you do after seeing this?

Issues worked because there was finally an answer.