Daily News

ElevenLabs has introduced a voice agent designed to efficiently handle user inquiries related to its documentation, achieving a resolution rate of over 80%, according to ElevenLabs. The voice agent processes approximately 200 calls daily, demonstrating significant success in addressing user queries.

Performance and Evaluation

The voice agent, powered by a large language model (LLM), has been evaluated for its ability to solve or redirect inquiries effectively. Human validation of 150 conversations revealed an 81% agreement rate between the LLM and human evaluators on successfully resolved inquiries. The agent also demonstrated an 83% agreement on maintaining adherence to the knowledge base.

Furthermore, 89% of relevant support questions were either answered or correctly redirected by the documentation agent, showcasing its capability in managing straightforward queries.

Strengths and Limitations

Strengths

The LLM-powered agent excels in resolving specific questions that align well with the available documentation. It effectively guides users to relevant pages and provides initial guidance on complex queries, proving beneficial for questions such as API endpoints, language support, and integration queries.

To optimize its performance, ElevenLabs recommends targeting users with clear questions and utilizing redirects for more complex inquiries, enhancing the efficiency of the support process.

Limitations

Despite its strengths, the agent encounters challenges with vague or account-related inquiries that require deeper investigation. The voice medium is less suited for sharing code or handling complex technical issues, prompting ElevenLabs to suggest redirecting users to documentation or support channels for such queries.

Development and Configuration

The voice agent is configured with a system prompt that guides its responses, ensuring it remains focused on ElevenLabs products. A comprehensive knowledge base, including a summarized version of all documentation, supports the LLM in providing accurate answers.

Three primary tools are integrated into the agent’s functionality: redirecting to external URLs, email support, and documentation, offering versatile pathways for user inquiries. The agent’s evaluation tooling assesses conversations against predefined criteria, ensuring ongoing improvement and reliability.

Continuous Improvement

ElevenLabs recognizes the limitations of LLMs in solving all types of queries, particularly in a rapidly evolving startup environment. However, the company emphasizes the benefits of automation, allowing its team to focus on complex challenges as the community expands the potential of AI audio technology.

The agent, powered by ElevenLabs Conversational AI, serves as an effective tool for navigating product and support questions, continuously refined through automated and manual monitoring, reflecting the company’s commitment to enhancing user support experiences.

Image source: Shutterstock