Galaxic dev

I already had a prototype in one waterpit with bad prototype code, during this 2 hours I revamped the system to use the new MQTT infrastructure for better communication and improved power savings. We also use MQTT to allow for duplex communication so we can add a system that will send commands from the APP to the sensor using MQTT (for example to put it in config mode without opening the case)

System components:
We use an firebeetle ESP32 v4 because it has excellent deep sleep which is important for battery life.
We connect an JSN SR04M sensor to it to measure the water level, we use this because its an simple, easy and cheap sensor while being waterproof.
we use a simple 3.7V lipo battery from amazon to power it, along with an charger board that supports solar input and usb in case of an emergency.
The solar panel is a simple 5V 1W solar panel we got from repurposing a solar powered water fountain pump (it was better quality and cheaper this way🤣)
We use this one since the charger board supports 5V 200mA input (P=UI | P=5v200*10-3 -> 1W) what is the exact solar panel specs we got.
I thought the battery life during the winter would be very bad, but I actually had the prototype running from january till now to test battery life before improving and really building it now. Let's just say I was wrong and the battery almost never got too empty to power the sensor, if even.
Next hours I will try to implement an OTA system, a config website that temporarily runs on the ESP32 to configure it, improve error detection and remove lingering bugs, improve app design, ...

-Added math game
-added XP/level system
-added extra commands for the math game and the xp/level system
-added /daily where you can play one round of each game for some extra XP.

-rewrote the lobby system to make it modular for other games
-added GeoGuess game where you get the outline of a country and need to guess the name
-added leaderboard for the GeoGuess game
-fixed bug where you were able to select all options and get the point
-changed the /triviabattle command to accept params, instead of having 2 seperate commands

Since the lobby system for the GeoGuess game was 99% the same, I rewrote the script to be modular and work with any game.
For the GeoGuess game I downloaded a JSON map from https://mapshaper.org/ and got help from AI to write a script to turn this JSON into a outline image. Aside from creating a image we also added every country to our database with 4 countries to choose from (the one in the image and 3 countries next to it)
Then I just displayed an embed with the image and a dropdown with the 4 options.
I wanted the user to be able to change their answer, but with the old answer system I had it remembered all you answers for this round. Because of this you could just select every one of them and get a point. To fix this I changed from a Set() to a Map() and made it so it only saved the last answer from every user.

-added a leaderboard for the triviagame
-fixed the round timer going down using discord relative timestamp
-fixed the answer reveal to be a nice embed after the round ended

At first each round was a new embed, but this really filled the chat, so I changed it to edit the old embed to the new round embed every time.
Originally I told the user if he was correct or wrong when he answered in a message only he could see, but this was very annoying to always click away (especially for phone users). Because of this I change the round to last 20s, then after the round we show the answer for 5s and start new round.
I tried to make the timer count down, but this reset the dropdown menu every time, which is very annoying for the user and potentially game breaking. To fix this I just used the native discord timestamp to display the time left.

-Added 1400 trivia questions ranging from easy to hard
-Made the lobby and challange actually start a trivia game

I found a big library with a lot of trivia questions (https://opentdb.com/), they provided a free API to fetch questions. I wrote a small seeding script that will fetch multiple choice questions with random difficulties from 5 different categories. I opted for a mongodb database because they are fast and fairly easy to work with (and the most important, FREE 😅) Here I stored all the questions and the information with them that I needed.
Then in discord.js when a trivia game is started we just fetch a random question from the pool every round and shuffle the answers to randomize it a bit.
Still need to add a system that it remembers the questions it had and doesn't ask them again in the same game, but the chances for that are very minimal.
I had a lot of issues with the interactions needing to be editted, replied too, followed up. It used to be a lot easier to just send a message in a channel.

For this project I choose node.js with discord.js.
This hour I created a basic discord bot with a /command handler.
I added a command /triviagame with 2 options (start/challange).When you choose start it starts a lobby where unlimited players can join in 60s to play the game with you. When you choose challange you are asked to provide another user, this user will then get a request to play a game of trivia with you.
Currently only the lobby mechanism works, the game doesn't actually start.

• Added automatic gain control to SmartListener
• Changed the AI interface to use ollama's package instead of using API calls
• Added a tool system to the AI so it can fetch real time weather information
• Choose a new AI model (Qwen2.5:7b) because this model supports tools (model might change later)
• some small other fixes

explanation:
Since I saw that my mic had was a little bit too quiet for the transcriber to consistently pick up, I added an automatic gain system which will calculate the current -dB and add a gain until its a preconfigured decibel level (-30 or -40dB)

I tried integrating my own tool system using JSON in the responses, but this was giving issues with the TTS (due to bad stream integration) and the model was often hallucinating actions that didn't exist or not using them at all.
After reading some blog posts (ollama's docs and medium post) I saw that ollama had a package to interact with self-hosted models. This package included a native tools function for model that support it. Here I passed along all the functions that the model can use (currently only getWeather) using the openAI's JSON system (you can also just pass along the functions, but I choose this to give more context)
Half the models I downloaded that supported tools didn't actually use this, don't know yet if this is a issue on my side. Qwen2.5 did use it and had decent assistant answers, so that is the model I am using as of right now. Still testing a few more models.

There were a lot of audio glitches that needed to be debugged in the activation system, in short the conclusion is that I got a bad mic 🤣
To make sure you got the right channel for the voice recognition, I also added a small tkinker GUI where you can debug your microphone.

I have also experimented a lot with different AI's and AI engines to find the best one for our use case. First I used transformers with a big 7B model but this took 2minutes 37seconds to generate a small response. After reading some reddit and blog posts I learned that you shouldn't use transformers and rather use ollama.
I switched over to ollama llama3 model and it gave a response within a second (with streaming on), which is very good for our use case.
I'm currently searching for the best model to run on ollama and integrate this in our flow.

Integrated an AI model to test how good its conversational skills are. Still needs finetuning and access to real world current data. Also integrated a system that will listen to your voice and turn your speech into text so the AI can understand it. This system is integrated with privacy filters so your commands will only be passed on if you show intent to ask the AI anything.
(didn't have anything to showcase, so added a short video of asking the AI a question)