LLM-based Conversational AI Knowledge Assistant for MyBuddy Humanoid Robot
Posted by Yanjun Li
Author：
Hanxiao Chen
Research Area:
I & LLM Robotics · Human-Robot Interaction · Speech Recognition
Summary:
An LLM-powered conversational AI assistant that enables MyBuddy 280 to understand natural speech, answer knowledge-based questions, and engage in continuous multi-turn conversations.
Traditional robot dialogue systems rely on predefined responses and limited knowledge databases, making it difficult to handle open-ended questions and maintain conversation context. This project explores LLM-based interaction to make humanoid robots more intelligent and adaptive.
Key Challenges:
●Understanding natural speech and complex questions.
●Maintaining context across multiple conversations.
●Integrating real-time knowledge retrieval.
●Generating natural speech and emotional responses.
Solution:
The researcher developed a cascading speech-to-speech AI system integrating Silero VAD, Whisper-Fast, Llama 3.1 8B, and Kokoro-82M TTS. Using the Agno agent framework, MyBuddy 280 can access Google Search, Wikipedia, and arXiv for knowledge retrieval, enabling intelligent voice conversations, contextual reasoning, and emotion-based interactions.
