# When Conversation Needs an Interface > Building Gaia's personal assistant channel shaped my view of hybrid AI interfaces: conversation for intent and explanation, visual controls for precise work, and shared state that makes moving between them coherent. - Author: Kostas Karolemeas - Published: 2026-08-31 - Topics: AI interfaces, product design, agentic AI, Gaia - Canonical URL: [https://www.voxelperfect.com/writing/when-conversation-needs-an-interface](https://www.voxelperfect.com/writing/when-conversation-needs-an-interface) When I built the personal assistant channel in Gaia, I used `ui-layouts` to bring visual interaction into the conversation. The assistant could work through language, while forms, records, and canvases gave people something they could inspect and change directly. That combination expresses my view of where AI interfaces should go. I want to explain what I am trying to achieve without first learning an application's navigation. But when the work involves comparing options, correcting a field, or choosing exactly what should happen, I often want controls in front of me. Conversation and traditional UI solve different parts of the same task. The product needs to make moving between them coherent. Felix Haas's [_Invisible Interfaces_](https://designplusai.com/p/invisible-interfaces), published on August 28, is a useful starting point. He describes agents becoming the layer through which people reach services, making the interfaces underneath less visible. He also makes room for transparency and carefully designed approval moments. His argument is about where interaction moves, rather than a literal disappearance of every screen. I agree with that direction. Where I would take it further is the role of the interface that remains. Approval is one reason to show a person something. Exploring, comparing, editing, and taking over part of the work are reasons too. **Conversation lets people express intent. A visual interface lets them inspect, compare, and change the result. A hybrid product needs both to operate on the same work.** ## Where language helps, and where I want controls Consider an escalation review, the kind of workflow described in Gaia's [runtime canvas guide](https://thegaia.ai/user-guide/conversations/scenarios/runtime-ui-canvas-artifact-workflow). The following is an illustrative walkthrough of that interaction pattern, not a report of a customer deployment or a measured usability study. I ask the assistant to open a review workspace for an escalation and explain the recommended next action. Language is useful here: I can describe the task, ask for missing context, and challenge the recommendation without finding the right sequence of menus. Then I need to inspect the record. Perhaps the proposed owner is wrong, the follow-up date needs changing, or the recommendation depends on an assumption I do not share. A form can keep those fields visible together. I can change the date directly and add a reviewer note without writing a paragraph that describes two small edits. I might then ask, “Does that change your recommendation?” That belongs in the conversation again. The assistant needs to read the current values, including my edits, before answering. If it reasons from the earlier version, the experience has broken even if both panels look polished. Finally, I choose an action. The interface should distinguish an edited proposal from a submitted decision and show whether execution succeeded. A confident sentence from the assistant is not enough evidence that a record changed. For this workflow, I would use conversation to express the goal and discuss the reasoning, a visual surface to inspect and edit the proposal, and an explicit action to commit it. The person should be able to move among those activities without reconstructing the context each time. That is a design preference, not a claim that every task needs a screen. A straightforward lookup may need only a sentence. A familiar, authorized routine may need no intervention at all. Adding a canvas to either can be unnecessary work. ## How I approached this in Gaia Gaia's [UI Layouts](https://thegaia.ai/user-guide/conversations/ui-layouts) provide structured visual definitions for forms, dashboards, and other views. They can be reused across surfaces, including the split canvas alongside a conversation. The [Gaia 3.0 release write-up](https://thegaia.ai/blog/3-0-ui-layouts-canvas-and-portal-experience) documents inline UI responses, layout routing, and persistent canvas artifacts. The distinction that matters most to me is between **the layout definition** and **the work happening inside it**. A reusable review form describes how information and controls are arranged. A particular review has its own values, progress, and history. Changing the reviewer's note should not require rewriting the form definition. Reopening the conversation should not create a fresh review that forgets the work already done. Gaia's runtime canvas represents this explicitly: a `ui-layout-canvas` artifact holds a layout reference or snapshot, with session values in `state.values`. The assistant can find and reopen an existing artifact, read it, and update it. The [documented runtime workflow](https://thegaia.ai/user-guide/conversations/scenarios/runtime-ui-canvas-artifact-workflow) keeps those updates separate from editing a saved layout. There are also different lifetimes for different interactions. An [inline UI response](https://thegaia.ai/user-guide/conversations/scenarios/inline-ui-layout-responses) can serve a short exchange inside the reply. An entity canvas can expose a business record. A runtime canvas can hold a working session across turns. A saved layout provides a reusable definition. These should be deliberate choices about what the task needs and what must persist. The personal assistant channel can also be configured with a default canvas that opens beside the conversation and is reused on later opens. I like the implication of that choice: users do not always have to ask the assistant for an interface before they can begin. For a recurring task, a familiar workspace can already be there. ## Shared state is the hard part Putting a chat panel beside a form is straightforward compared with making their behavior consistent. If a user edits a value, the next agent action must use that value or explicitly resolve a conflict. If an operation fails, the screen must not silently present the proposed result as completed. If a conversation reopens, the user needs to know which artifact and version they are looking at. ![Conversation and interface connect to the same working state: ask and explain through language, inspect and edit through controls.](/writing/when-conversation-needs-an-interface-linkedin-shared-state.webp) _The design target: two ways to work with the same current state. This is a conceptual diagram, not a product screenshot._ Those requirements turn interface design into application architecture. The model's account of what happened, the browser's displayed values, and the persisted result can diverge unless the product defines how they are reconciled. One concrete Gaia mechanism is the optional mapping of successful synchronous workflow outputs back into canvas state. The [runtime guide](https://thegaia.ai/user-guide/conversations/scenarios/runtime-ui-canvas-artifact-workflow) specifies that mapped outputs and the new artifact revision are saved together; missing outputs or persistence failures should not leave a partially updated canvas. That is a specific consistency mechanism, not a guarantee that every possible concurrent edit is resolved. The same guide exposes a useful trade-off. A canvas can reference the latest saved layout, or carry a snapshot of the layout for that conversation. The reference can follow later layout changes; the snapshot preserves the captured definition. The documented implementation does not add drift warnings or rebase prompts for references. Choosing between the two means deciding how much change an ongoing task should absorb. I would treat that decision as part of the experience. An interface that unexpectedly rearranges itself halfway through a review can undermine the familiarity that made visual controls useful in the first place. ## Hybrid interaction has a history This combination predates today's language models. In his [1999 paper on mixed-initiative interfaces](https://www.microsoft.com/en-us/research/publication/principles-mixed-initiative-user-interfaces/), Eric Horvitz examined how automated services and direct manipulation could work together. The example system concerned scheduling and meeting management. That history is helpful because it moves the discussion beyond declaring either agents or graphical interfaces the winner. There is a long-standing design problem here: how to distribute work between people and automation while allowing each to contribute effectively. The [2019 Guidelines for Human-AI Interaction](https://www.microsoft.com/en-us/research/publication/guidelines-for-human-ai-interaction/) offer a more recent foundation. Among their recommendations are making correction efficient, explaining system behavior, and adapting cautiously. The authors' [practical explanation](https://www.microsoft.com/en-us/research/articles/guidelines-for-human-ai-interaction-eighteen-best-practices-for-human-centered-ai-design/) also discusses helping users finish a task themselves when the system is uncertain. These sources support principles for human control and correction. They do not prove that Gaia's implementation is better than a conversational or conventional alternative. That would require evaluation of actual tasks with actual users. ## What I would test in a hybrid product My test would start at the point where a person stops asking and starts changing something. Can they correct a value directly? Does the assistant use the correction in its next response? Can they ask why a recommendation changed? Can they leave, return, and continue from the saved state? If an action fails, can they tell what was saved and what remains only a proposal? I would also test whether the visual surface deserves to exist. For a simple task, does it help, or merely add another step? For a repeated task, do stable controls make the work easier to recognize and resume? Keyboard access, clear labels, and an understandable focus transition matter when a canvas opens; producing valid layout data is only part of that job. These are the outcomes I would measure: task completion, correction effort, recovery from failures, and whether people correctly understand the resulting state. Counting fewer clicks or more conversational turns would not tell me enough. Haas's essay asks us to imagine software that no longer requires visiting every application. I find that compelling. My experience building Gaia points me toward a related design goal: people should be able to delegate work and still have a clear, precise way to participate in it. Sometimes that means a sentence. Sometimes it means a form that stays open while we talk. The interface should remain available for as long as it helps the person do the work. ## Sources - Felix Haas, [Invisible Interfaces](https://designplusai.com/p/invisible-interfaces), August 28, 2026: the contemporary argument about agents mediating access to applications, transparency, and approval. - Eric Horvitz, [Principles of Mixed-Initiative User Interfaces](https://www.microsoft.com/en-us/research/publication/principles-mixed-initiative-user-interfaces/), CHI 1999: combining automated assistance with direct manipulation. - Saleema Amershi and colleagues, [Guidelines for Human-AI Interaction](https://www.microsoft.com/en-us/research/publication/guidelines-for-human-ai-interaction/), CHI 2019, and the authors' [practical explanation](https://www.microsoft.com/en-us/research/articles/guidelines-for-human-ai-interaction-eighteen-best-practices-for-human-centered-ai-design/): guidance on correction, explanation, and predictable adaptation. - Gaia's [UI Layouts guide](https://thegaia.ai/user-guide/conversations/ui-layouts), [inline response scenario](https://thegaia.ai/user-guide/conversations/scenarios/inline-ui-layout-responses), and [runtime canvas workflow](https://thegaia.ai/user-guide/conversations/scenarios/runtime-ui-canvas-artifact-workflow): the documented interaction modes, state model, workflow updates, and layout-reference trade-offs discussed here. Implementation details checked against the Gaia repository on August 31, 2026. - Gaia team, [Gaia 3.0 — UI Layouts, Canvas, and Portal Experience](https://thegaia.ai/blog/3-0-ui-layouts-canvas-and-portal-experience), June 10, 2026: the public release account of inline layouts and durable canvas artifacts.