Nedimark

Hi My last Devlog here! I just published the Launch video on Youtube go check it out: https://www.youtube.com/watch?v=GRMrwrrrwkE Thank youuuu!!!!

Today I fixed a rly annoying bug, Yes I have said that before and Yes its the same one but this time its so fixed that I made a second release! So the Solution was to just handle the seed step which is basically the thing you start out with as a chip and just integrate it into the existing system! Looking forward to shipping soon!

Hi! What a wild Ride it has been. I even made an AD for this thing check it out below! ..... The hours came from the fact that the for the second time I found bugs in the app while making the Demo video(you see it in the mockup). Plus also I just want to say I have a terrible relationship with Davinici resolve.... LIKE OH MY GOD why are so many simple things so darn it hard???? At the same time why is there for any function i expect to watch a 20 min video for, like three shorts on YT already??? But in the end it was fun!

OMG I had such an annoying bug today it took me sooooo long to fix it. It's crazy (crazy? I was crazy once...) but it all happend while making the demo video when there was just the pop-up didnt want to go away. Long story short the streamlit ui refresged (even though it shouldnt),right when I called the OpenAI API! And that messed up with the states and everything....

Yipeee I finally got the Installer to work! plus I have already used the platform to create a set of my own LLMs go check them out on: https://huggingface.co/CanisAI . I will be shipping this project soon!

I spent the most time in Figma experimenting with the design and for the most part I used the summer of Making owns color, whichc I think added a rly nice touch. And yes the design is inspired by the cards on HCB....

OMGGG IT LOOKS SO FIREEEEEE. I am just Putting on the finishing touches like step deletion, Batch cancel, better batch tracking and more. LOOK AT THAT DARK MODE real gourmet UI right there

Soooooo.... Where to start uhhh a lot of things in parallel. First I strated wokring on tryin out how data cleaning works, thwe connections keeped getting misplaced so now there is this smart builder helper to ceate new steps. Also I understood that the data cleaning setup required like 4 + for each lable another node at least! so for my case 9 NODES!!! thats a lot to have on the screen at the same time! So I decided to build so called chips they are built from at least one llm tool step and multiple code tools. This result in only one node having to be on screen for the whole data classification/cleaning step! (Plus the parsing goes from 2 to 1 nodes). Also I have finalized and cleaned my first dataset YAY. I will be reporting on the results soon but probably i will first clean up the UI at a few places and get it into shape to be shipped!

So I just burned 15$ of my OpenAI credits... While not noticing a bug in my code. So yeah I am still on a hunt for bugs and giving up my credits for the greater good! Currently off my list is correcting the directory managment to not make it machine dependent, the seed file generation now works (like actually (fr this time(I swear))) and added more preview functionality to be able to view generated data and not only work with blackbox data...

Uh so yeah uhm its taking way loooonger to implement a UI than i thought! (Oh what a common humanity between developers to always underestimate their goals ...) So here is a small preview of what I have been doing! And yes its a streamlit based UI cuz uh porting all the python code to a web langauge would this would be a huge hassle even with AI involved! And if youre wondering if AI helped to create the UI: yes it did, because in this case i wrote the code thats needed to run the engine of the synth Data generation and now LLMs are bad at deveolping new systems but are good at making things that exist so instead of a app with ui for Dataset generation it only has to make a UI for a fully developed Python App!

FINALLY it works! And I have also done all the tools i wanted implemented. So for llm side tools you have a away to parse to the alapaca style isntruct or the Chat-ML Template for conversations the same if you want to derive them from context! plus a clean function that classufies and cleans your data! Now the more complex ones are the code tools: merge: merge two datasets of same length together great for adding messages in conversations! bind: if you have per row multiple dialogues for example they will now get each their own entry segregate: the completion part for clean here you get the cleaned dataset seperated according to the classification; expand want to a pply a single line of information to multiple rows of data USE EXPAND ! With select you can specifically select one of the segregated datasets and get acces to it only; count well counts the entries and with percantage you can for example see how much data fell through the classification! Pheeeeeew that was A LOT. Now this train is transforming into a jet cause its flying straight to UI-city!!!1!

I think I have a sickness ... and that is to rewrite the whole codebase ... it's now the third time... But now you have more flexivility to use tools on your data. For one you have llm tools like clean, parse and derive where the data is uploaded via batches and prewritten templates and code tools which are python functions that allow for example to connect to data_files finalize your dataset and more... but for this flexibility and ability to easily add more tools was really hard like look at the time. So today is just a touchpad drawn image illustrating my pain knowing I still have to debug......😿

First time using the AMPCode agent to rewrite code into a differen language (me no speak no web dev). The whole App now is live on Github Pages! Also as a future idea it might be cool to a gallery with the coolest ascii art creations!

Hi there! I just decided that why not use the code i made for iframe the ysws and make it into a lil nifty tool for everyone!

As developers we all believe in the sunk cost fallacy... It's in our nature to believe in our self written code, it may be ugly, but it works!1!2! So today i have decided not to be one in a krillion🐠 and rewrote all my code using a node based system. Now not only should workflow creation be simplified but even more complex ones are possible. (I hope it was worth the pain 😿😿😿)

Finally after such a looooong night 😴😴😴 I have managed to implement JSON Parsing as a processing step after generating the seed (raw) data. Now the next step would be finalizing the JSON formated Data into a pure JSON only Dataset. that would logically then be the ... (looks up the life stages of a tree) ... sapling stage?

Before moving along my To-Do List I have decided to make the UX more ✨nice✨.
Now next stop!!! 🚂 (this time for real(I hope)) JSON Parsing!

After figthing the gods of pyhton for a whole day... I finally have gotten my first generated bit of synthetic data! While its still not pretty formated in JSON, it is a start! Therefore next I'll be heading to the sacred Valleys of JSON Parsing. ⛰️🏔️🚞

Before Ending the Day I also created a template for creating your own synthetic seeds. Feedback is welcome and wanted 😸😸😼

Hello! Today was a loooooot of brainstorming on what would be the best way to combine a lot of different prompts for an LLM to generate a great amount of Synthetic conversations. The Result this JSON Template filled out with an prompt, that took quite a while to get right ... I really might make an custom GPT or similiar to just generate these in a conversation... If u have any ideas dont hesitate to comment !😸😸😸

Today I decided to pick this project up again. And decided to switch to the OpenAI API and did a lil experimenting (cuz who doesnt love some good experiments), because i got a grant from the toolsmith YSWS ... With that in Mind i created a plan how to build the Application (see Attachment)

Sometimes you just have days when the code aint coding and you may have written thousands of lines, but in the end u push none.. 😿😿😿