Grall

shipping this, am tired. updated the delay logic a bit, should be all good

completed the openai api wrapper and wrote a UI for the demo, chunking was a pain

now looking at the demo myself people not knowing the context of what grall is will probably think this is slop lol

oh well, please read the devlogs before voting :3

this is absolutely silly, worked on a sequential chunked training system but i got clapped at the merging part so hard. it was fast and produced correct models but didnt really run. wasted 6 hours and oh it didnt get logged cool

the unix socket api is done

started working on chunked training because that will massively boost the training speeds i imagine. im working on training chunks sequentially atm but every step there is always support for making it run in parallel.

i almost completed the sequential chunked training i just need to check for half-serialized files and then k-way merge all trained model chunks then ill proceed with the parallel training

and then i gotta work on the openai api and then a demo ui

working on the unix socket api rn

im not doing the training optimizations rn because i plan to change a few things about how it fundamentally works

so far so good, worked on an stdio api but it turned out shitty because of the throughput i needed.

this also means i can use the same grall process across multiple dependent processes and each connection uses a different thread, therefore performance!!!

after some small optimizations we hit our first 5MB/sec. training is very slow so ill optimize that, ive figured that array insertions are causing the slowdown so ill move to a different data structure for training. im glad i completely separated the training and runner chains from the beginning.

currently microoptimizing the hell out of a string compare function for a very specific case but will generalize soon. hits 1MB/sec otherwise but i want it to go faster. shoutout to a fine maiden named pbl for a c implementation that uses linked list and is blazingly fast, although its word-word not byte-byte BUT I WANT IT TO GO FASTER RAHHHH

https://github.com/bitpbl/ferrite

completed the base markov runtime, working on features like ending styles and yaml serializations (already completed the file serialization)

still need to work on chunking the input text data across multiple models and k-way merge them

so far, 222mb textual data after training, uses 15MB on disk and ~~150 MB ram during runtime (must fix!!) and generates text at ~~600KB/sec

markov chains are silly