Systems | Development | Analytics | API | Testing

Last week we introduced AI Transport v0.2.0 and made one idea the centre of the design: the session is the channel. Every input, output, and lifecycle event for an AI conversation is just a message published to an Ably channel, which is what makes a session durable, multi-party, and resumable. In v0.3.0, we added first-class support for presence and LiveObjects to AI sessions, allowing you and your agent to see who's online and update shared state in real time.

We audited our skills library a few months ago and found twelve dashboards hiding in it. Not dashboards. Skills that built dashboards. Someone needed a view of some data, asked Claude to put it together, got a long HTML page out of it, and then wrapped the whole thing in a skill so others could run it again. Twelve times over, by different people, for different questions.

You've built an AI chat app on the Vercel AI SDK. It works in development. The model responds, the stream comes through, and the UI updates cleanly. Then you ship to production, and the transport layer starts showing its edges. Most of these failures are quiet: things that work in demos and break in ways that are hard to pin down until you know where to look. They share a common cause: DefaultChatTransport is built for HTTP, and HTTP has structural properties that some production requirements exceed.

I ran a webinar on this recently and had more to say than the time allowed, so this is the written version: the argument I was making, some context on the demo, and the questions that came up from people watching. The recording is below if you'd rather watch than read. The thesis: AI products are being let down by the user experience, not the model.

Version v0.2.0 of @ably/ai-transport reorganises the SDK to better support a wide range of interaction patterns. Everything in an AI session – input, output, agent lifecycle, control signals – is captured durably, allowing you to easily build the sophisticated interaction patterns that support modern AI user experiences. When we first built @ably/ai-transport, we modelled an AI conversation the way most people first picture it: as a request and a response.

Your AI agent is mid-task, waiting on the result of a search tool call it made 30 seconds ago. The user is watching a spinner. Then a network blip drops the connection. The application reconnects in under a second, fast enough that most monitoring wouldn't flag it. But the tool call result that came back during the gap is gone, and so are the 200 tokens the agent generated before the silence began. The reconnect succeeded - but the session didn't.

When Temporal launched, a lot of people had the same reaction: "We have queues and retries. We don't need this." (Temporal's own blog addressed this directly.) That reaction made sense. Queues solve queue problems and they do it well. What Temporal gave you was something different: a named execution context that survives a server restart and picks up from its last checkpoint. Not a better queue. A different abstraction entirely. If you built with it, you couldn't imagine going back.

At Ably we've been able to make a transformational change in how we build SDKs as a result of what's now possible with LLMs. In this post, I explain how.

You're mid-conversation with an AI support agent. You've explained the problem, the agent is halfway through a response, and the connection drops. When you reconnect, the response is gone. You type the same question again. The agent asks the same clarifying questions again. Three minutes of context, gone. Not because the model forgot it, but because the delivery layer stored nothing.