ExpressBuddy Used AI
**ExpressBuddy** is a first-of-its-kind mobile app that uses a cartoon-style AI avatar to help children with autism, speech delays, or social anxiety improve nonverbal communication and social-emotional skills. The app creates a safe, engaging space where students can practice eye contact, emotion recognition, conversation turn-taking, and verbal expression. Powered by speech-to-text models and an emotion-aware LLM, the avatar responds naturally with animated facial expressions and social cues.
Designed for use in elementary and middle school classrooms, **ExpressBuddy** supports special education, ESL, and social-emotional learning (SEL) goals. I actually don't own the pandabot asset and it doesn't belong to me.
Followers
No followers yet
Ship Your Project
Get ready!
Once you ship this you can't edit the description of the project, but you'll be able to add more devlogs and re-ship it as you add new features!
Timeline
Created a new feature called Silence Detection where Pico can automatically detect when the child is stuck answering questions and give suggestions to them. I created a popup menu where you adjust the timeout before Pico nudges you to respond, allowing it to be personalized to each child.
It helps children with autism who sometimes find it difficult to answer questions posed by Pico.
So I am building a new feature that allows children to play an emotion game. Where they are faced with different types of scenarios and they pick the most appropriate emotion to display. Helping train their emotional intelligence.
I brought back the favorite Pico character kids love into this screen, where it reads the questions to kids with EdgeTTS.
ADDED MEMORY FEATURE. PICO CAN NOW REMEMBER EVERYTHING TELL IT. IT USES GOOGLE GEMINI API TO CALL TOOLS AND SAVE KEY,VALUES TO LOCAL STORAGE
Completed a major architectural update for Panda Piko. I successfully integrated the avatar into Gemini Live and created a backend FastAPI Kokoro TTS pipeline that generates high-quality audio and lip-sync animations for Piko. Gemini Live models enable Piko to understand kids in real-time, making progress toward our goal of helping children with autism, speech delays, or social anxiety improve their nonverbal communication and social-emotional skills.
I was continuing to improve the LipSync feature for my avatar. I also implemented word-level subtitles using kokoro tts timestamped.
Today, I created an interactive AI avatar built with a custom stylized 3D panda and real-time text-to-speech lipsync. Powered by Kokoro TTS and the Rive animation runtime, users can input text and watch the panda speak it back with expressive, synced mouth motion and character animation.
Today, my goal was to clone the repo https://github.com/aigc3d/LAM_Audio2Expression and figure out how to run it in my local environment. I figured it out and it required me to use WSL and conda. I also created an Expression Engine apart from the one from the audio because I wanted custom control over expressions.