The art of reading traces

I’ve never been good at reading a wall of text. Give me a diagram, or something I can move through, and I start to understand it. Give me the same thing as a long transcript and I skim, miss something, and go back to the beginning.

That gets awkward when you spend your days working on agents. So much of what matters is sitting inside their traces: the ask, the tool calls, the corrections, the moment a conversation changes direction. All recorded. Still surprisingly hard to see.

At BentoLabs, I worked on ways to turn those records into problems a team could investigate. I kept coming back to a smaller question: what would it take to make one session worth reading all the way through?

A familiar kind of watching

Think about watching a session recording. A cursor pauses over a label. Someone opens a menu, closes it, then opens it again. They click the same thing three times. Nothing has crashed, but you already have questions about the design.

A conversation with an agent has moments like that. Someone repeats a constraint. They narrow an instruction the agent took too broadly. They ask whether anything is still happening. Those moments tell you where to look, even when the final answer seems reasonable.

They don’t tell you exactly how the person felt. A short reply might mean frustration, or it might just mean they were busy. I want to notice the moment without pretending to read their mind.

The interesting part is often in the middle

One of the synthetic sessions in my trace reader starts with an intermittent checkout failure. The agent runs a test. It fails. The agent runs it again. It fails again. Then the user steps in:

Please check whether the retry uses the refreshed token, rather than just re-running the tests.

That is the turn I care about. The user has supplied a direction the agent hadn’t taken. A count of failed calls tells me where something broke. Reading the exchange shows me how the work got back on course, and who had to steer it there.

Later, the same test command runs again and passes. Three identical calls, serving different purposes. If I compress the session into “three repeated calls,” I lose the difference between trying again and checking a change.

The final message matters too. In this example, the agent says the focused tests passed and the full integration suite wasn’t run. That boundary is part of the experience. It tells the person what they can rely on and what still needs attention.

Make the evidence easy to reach

My first sketch was a web player. Time became space. A long pause would look long. A curve would let you move through the conversation, with notes beside the turns that seemed important. I wanted the feeling of watching a recording rather than opening a log file.

I still like that direction. But a convincing picture can make a weak interpretation feel certain. A mood curve looks authoritative even when the messages give you very little to go on. Before adding more interpretation, I wanted a shorter path back to what was actually recorded.

The tool I’m building now, trace-analyser, starts there. It reads local coding-agent sessions, surfaces a few observations, and lets you follow each one into the event timeline and its source. The opening request sits beside the last assistant message. You can read the distance between them for yourself.

Its language is deliberately careful. A call without a matched result might be unfinished, missing from the capture, or in a format the reader doesn’t understand. That uncertainty belongs in the interface. Otherwise the reader creates another story you have to debug.

This is product work

An agent’s status messages, its questions, the way it explains a failure, and the point at which it hands control back are all things a person experiences. Some of those decisions live in a prompt or a tool definition. They still deserve design attention.

I’d like more designers and product people to spend time here. Start with a session. Read what the person asked for. Follow the agent’s actions. Notice what the person had to repeat, explain, or repair. Then ask what would have made that exchange easier.

One session gives you something concrete to question. Finding the same thing across many sessions can help a team decide what to change. You need both scales, but the larger pattern is much more useful when you can still open an example and understand it.

The next time an agent gives you a good final answer, read how it got there. There may be a better product hiding in the middle.