VoiceXD made building chatbots easy. Debugging them was still painful.
Context
VoiceXD is a collaborative platform that allows teams to build, test, and publish conversational AI agents (chatbots) without writing code.
VoiceXD demo (Created using Rive)
When a bot failed in testing, designers had to scroll through long, unstructured chat logs or parse complex JSON payloads to understand what went wrong. There was no tool to identify why a conversation broke down, no guidance on how to fix it, and no way to visualize the logic behind the bot's responses.
To position VoiceXD as an enterprise-grade platform, we needed to move beyond creation tools and turn raw conversation data into actionable insights, using the (then-new) capabilities of Large Language Models (LLMs).
My core objectives were to:
- Reduce cognitive load: Transform raw JSON logs into a human-readable format.
- Enable AI-powered debugging: Use AI to pinpoint where conversations broke down and provide actionable insights.
- Ensure scalability: Design a system that could handle thousands of message turns without breaking the UI or the backend.
My role: Product Designer
Responsibilities: User Research, Product Strategy, Interaction & Visual Design, Prototyping, User Testing
Team: CEO, CTO, and 2 engineers
Timeline: 6 weeks
Discovery and Insights
I conducted a competitive audit of platforms in the conversation design space, talked to our early users, and partnered with our internal Subject Matter Experts in Conversation Design to audit their existing debugging workflows.
The clearest finding: users traverse two distinct mental modes when reviewing transcripts.
The "Reader" mode: they read the chat linearly like a script to understand the user's sentiment, flow, and frustration.
The "Engineer" mode: they need the exact variable state, API status (200/404), and slot-filling logic at every turn.
One transcript, two readers: the human following the story, and the engineer tracing the logic.
Users needed both views, and they needed to switch between them instantly within the conversation.
The competitive audit added a second insight: other platforms covered the basics of transcript management, like browsing, categorizing, and managing statuses. Advanced analysis and debugging capabilities were an open space.
The Obvious Answer: Analyze Everything
The most ambitious part of this project was the AI analysis feature, which critiques conversations (e.g., "Why did the user get frustrated here?"), identifies root causes, and provides actionable insights.
My initial idea placed this feature in a sidebar, letting users run analysis on the entire transcript at once.
Where It Broke
Testing and engineering reviews surfaced two problems with the analyze-everything approach:
-
Users didn't want it. They usually cared about a specific failure point ("Why did the bot fail at step 4?"), and whole-transcript analysis buried that answer in noise.
-
The context window couldn't hold it. We were using the OpenAI GPT-3.5-turbo-16k model with a 16k token context window. Long conversation histories would exceed that limit and the feature would simply fail.
Scoped selection kept every request inside the token limit and matched how designers actually debug.
I pivoted the interaction model from "Analyze All" to a Start Point → End Point selection.
Users select a specific range of messages, and the analysis runs on exactly that slice.
Why this direction worked:
- Technical feasibility: The data sent to the LLM always stayed within token limits.
- Focused debugging: Users go straight to the section they care about, which makes root causes easier to isolate.
- Reduced cognitive load: Analyzing a smaller subset keeps the output digestible.
Reaching this required multiple iterations and close collaboration with engineering. Each round surfaced new constraints and refined the design until it was both technically viable and pleasant to use.
Iteration and Feedback - Example 1
Iteration and Feedback - Example 2
Shipped Designs
Debug Mode
One toggle between story and logic
The direct answer to the Reader/Engineer insight. Toggle off, and the interface reads like a standard messenger: clean and human. Toggle on, and every message bubble expands to reveal the backend details: input type, extracted variables (e.g., destination: paris), and intent classifiers.
Backend Processes
Failures visible where they happen
Visual indicators show the success and failure states of API calls directly within the chat stream, so a broken integration is spotted in context instead of hunted through logs.
Transcript Management
A home for everything around the conversation
The right panel holds extracted variables, tags for categorization, and additional metadata, keeping transcript management one glance away from the conversation itself.
Key Outcomes
When this feature launched in 2023, none of the competitor platforms we had audited offered AI-powered debugging. It shipped to 500+ users, anchored VoiceXD's Product Hunt launch (reaching a #32 day rank), and completed the platform's core loop: teams could now build, test, debug, and fix assistants without leaving VoiceXD.
~30%
increase in weekly platform activity in the months after launch
~2x
more assistant updates published per team, as debugging insights fed directly into fixes
The Debug Toggle's progressive disclosure abstracted complex backend data (API status, variable slots) into a readable UI. Users could confidently debug and optimize their assistants without needing engineering help to read logs.
The feedback from users validated the direction:
Oh, I love this, I would say it's definitely a 5/5. It's a feature of other platforms (competitors) that I've wished had more to it
Conversation designer, agency working across multiple chatbot platforms
“That debug toggle is brilliant!”
User feedback on the Show Only Required Details toggle
Retrospective
Constraints help shape design. The 16k token limit felt like a blocker, but it actually forced a better UX pattern (Scoped Selection) that made the tool more precise and useful for the end user.
How to better navigate ambiguity. Form well-defined, informed hypotheses early, then validate them through in-depth conversations with users, engineers, and stakeholders. Every conversation converts an unknown into a decision.