ChatdollKit

uezo/ChatdollKit is ranked 11,043 among these 8 peers; google-gemini/deprecated-generative-ai-js is ranked 11,037.

ChatdollKit 3D virtual assistant SDK that enables you to make your 3D model into a voice-enabled chatbot. She’s multilingual, so you can ask her something like "Let's talk in Japanese" when you want to switch languages. 🍎 iOS App: OshaberiAI A Virtual Agent App made with ChatdollKit: a perfect fusion of character creation by AI prompt engineering, customizable 3D VRM models, and your favorite voices by VOICEVOX. ✨ Features **Generative AI Native**: Supports multiple LLMs like ChatGPT, Anthropic Claude, Google

Gemini Pro, Dify, and others, with function calling (ChatGPT/Gemini) and multimodal capabilities. **3D model expression**: Synchronizes speech and motion, controls facial expressions and animations autonomously, supports blinking and lip-sync. **Dialog control**: Integrates Speech-to-Text and Text-to-Speech (OpenAI, Azure, Google, VOICEVOX / AivisSpeech, Aivis Cloud API, Style-Bert-VITS2 etc. ), manages dialog state (context), extracts intents and routes topics, supports wakeword detection. **Multi platforms**: Compatible with Windows, Mac, Linux, iOS, Android, and other Unity-supported platforms, including

VR, AR, and WebGL. 16 **🎙️ WebSocket Streaming STT**: WebSocket-based streaming speech recognition offloads VAD to the server and completes recognition during turn-end detection, reducing overall response latency by several hundred milliseconds. **🗣️ Barge-in Support**: Users can now interrupt AI speech mid-sentence with their voice, making conversations feel more natural and responsive. **💃 ModelController Refactoring**: Extracted speech handling into and face expressions into , improving maintainability and extensibility. 🕰️ Previous Updates (click to expand) 0. 15 **🌏 WebGL Enhancements**: Add Silero VAD

support, camera switching (front/rear) with correct aspect ratio handling, file upload for images, optimized microphone data transfer, and fixes for lip-sync when muted. **🥁 Stronger Noise Resistance**: Combine multiple voice activity detection methods (e. , Silero VAD + built-in energy-based VAD) to better capture user speech even in noisy environments like event venues. 14 **🎙️ Echo Cancelling Support**: Add native microphone support for Android, iOS, and macOSX that support AEC, noise cancelling and other features for voice conversation. **🗣️ Conversation Improvement

**: Prevent conversation breakdown caused by turn-end misrecognition and improve conversation experience with features like automatic volume control when users interrupt during AI speech **💠 Platform Expansion**: Support for Aivis Cloud API TTS, AIAvatarKit TTS/STT, and GPT-5 parameter 0. 13 **🥳 Silero VAD Support**: ML-based voice-activity detection vastly improves turn-end accuracy in noisy settings, enabling smooth conversations outdoors or at events. **🪄 TTS Pre-processing**: Optional text pre-processing lets you fine-tune pronunciation (e. , convert “OpenAI” to

katakana) before synthesis. **🤝 Grok & Gemini Compatibility**: Removes OpenAI-specific params from the OpenAI-style endpoint, so Grok, Gemini, and other API-compatible models work out of the box. **🌐 WebGL Improvements**: Upgraded mic capture to modern for lower latency and reliability; stabilized mute/unmute handling; improved error handling to immediately surface HTTP errors and prevent hangs; fixed API-key authorization in WebGL builds. 10 **🌎 Dynamic Multi-Language**: The system can now autonomously switch languages for both speaking and listening during conversations

. **🔖 Long-Term Memory**: Past conversation history can now be stored and searched. Components are provided for ChatMemory, but you can