HyperQuark

Today I implemented yet more blocks: the operatormathop block (which does things like sqrt, abs, ceil, floor, (a)sin/cos/tan, ln, log, exp and pow), operatorrandom, and direction-related blocks. This means that my 'colourful smiley face simulator' test project (http://scratch.mit.edu/projects/951679565) now compiles (although it doesn't quite work properly yet). I suspect it's something to do with nested loops but i'm not totally sure.

Today I implemented the operator_mod block, which wasn't too difficult in of itself. However, at the same time, I tried to simplify some of the type analysis so that blocks themselves don't need to consider the cases where they might receive boxed inputs (because that never happens in practice because blocked inputs are sent off to monomorphised implementations). This led to some interesting code to find the actual output type for when there are boxed inputs, including the attached almost-triple-question-mark which thankfully didn't make it into the final commit.

Today I implemented the pen_setPenColorParamTo block (which allows setting the hue/saturation/brightness/transparency of the pen colour individually). Was slightly challenging as the parameter menu is an input rather than a field (so accepts arbitrary values) :(
Image: the same as before but even more colours

Ok so I'm not quite sure how I've managed to rack up so many hours recently but anyway here's the devlog.
Since I started the rewrite late last year, I've been using a reusable pattern/trait/type that I call the 'registry pattern', which is really just an insertion-ordered key-value map with some associated functions. There are some cases where the keys and values that are passed in are constant (known at compile time) and used in various places across the codebase. It would be convenient if these could be stored in importable constants, but the issue then arises that I've got to make sure that the correct key is used with the correct value. To resolve this issue I create a new registry type called a NamedRegistry, inspired by reading through the codebase of typst, which allows me to define a zero-sized struct with associated constants from which an arbitrary (but consistent) name is derived and the value reused. Now rather than writing something like func.registries().tables().register("exampleTable", /* a bunch of info */) many times, I can write func.registries(),tables().register::<ExampleTable>() which is much nicer. And the code is quite abstract which makes me feel clever ;)
Using this new registry pattern I added a new registry for static utility functions, which were required for hsv<->rgb colour conversions, so I've was then able to implement colours as a primitive value (RGB and ARGB colours require separate primitive values because they behave slightly differently) and the pen_setPenColourTo block.

Attached image: the same as before, but colourful.

Ok so since my last devlog I've added a bunch of blocks but I was coding on my phone using vim so my time wasn't recorded ._.
Here's a complete (maybe) list of blocks that I've added since then:
- set size to
- switch costume to
- show
- hide
- pen down
- pen up
- clear (pen)
- forever
- go to (x) (y)
- set pen size to

Still more left to do and then i want to think about either lists or broadcasts.

(image: scratch cat drawing with pen)

I discovered that the preview deployment on github pages doesn't work even though my local build does - it turns out that vite treats a certain dependency differently depending on if it's building for dev or prod. To simplify things, I just set it so that it works in prod, and I'm using prod mode locally as well. Also updated my nightly rust toolchain locally so it matches CI and all my checks pass, for the first time in quite a while. yay!
this is probably a good point for me to push all the changes in my branch to the main repo and deploy as a new version.

With all my recent changes, my unit tests had got out of sync with the main codebase. I fixed these up to match recent api changes, and also replaced a lot of code that was duplicated elsewhere in the codebase and doing quite a bad job of it. Now all tests pass! (Although I definitely need more unit tests elsewhere in the codebase that don't just test that the generated wasm compiles!)

I've now started implementing some more basic blocks. I decided to start with the switch costume to [] block as it's probably the simplest block that requires some rendering. Rather than hardcoding offsets for the in-memory positions of various sprite properties, I made a macro that takes the names of properties and their sizes, and keeps track of their offsets automagically. Noice. Implementing the block itself was very easy but I had an absolute nightmare trying to get the sprite to actually display properly - this is probably where most of the supposed coding time came from, but in reality it was a one-line fix - I had mistyped target.direction as target.rotation. This is one of the few times I wish I was using typescript rather than javascript (although that would be horrible to work with scratch's quite-complicated json schema).
I also came across an instance of a Weak reference being dropped before usage, so I created a small reference cycle as a workaround - although this will cause a memory leak, it shouldn't be a particularly significant one, and I plan to reduce my usage of reference-counted pointers in the future anyway, which should get rid of the root problem. Hopefully.

Attached image: scratch cat displaying it's second costume successfully. Yippee!

I discovered yet another bug in the variable type analysis system (after I'd claimed that I'd fixed them all) - yippee! Turns out that doing funky shennanigans with variables means that we sometimes get missing type casts that we need to reinsert. Fixing this was ostensibly very simple (just needed to run the insertcasts procedure after VTA), but I also decided to address a long-standing issue where I was using hacky workarounds to allow procedures to return multiple values, without allowing multi-valued returns in the type system. This wasn't a particularly large change but was spread across many files so took quite a while to do. This was important to fix because `insertcasts` was getting a bit confused because I never made it consider procedure calls as a special case, since prior to now it never encountered multi-valued-returning procedures (as these are only added during VTA). While I was there I also spotted a possible bug involving transforming variable access to procedure argument access, which failed to consider the presence of regular procedure arguments.
With these changes, I could now run the benchmark project that I was using prior to the mega rewrite earlier this year, so today's image is some meaningless out-of-context benchmark numbers. You're welcome!

Fixed a little bug to do with procedure parameters - boxed inputs [1] were not being automatically boxed, but now they are! Also linted. Sorry for the short devlog but there's really nothing more to say!

[1] a boxed value is one that doesn't have its type known at runtime, so it is stored along with a tag of its type

It's been a while since my last devlog; I've been working on other projects. For hyperquark, I've been fixing more small and subtle bugs in the VTA (variable type analysis) system. These are really annoying to debug but my favourite one (i.e. the most frustrating) was where I was getting errors about the type stack being larger than the number of variables it was being assigned to. In the execution logs, it appeared that one step (a collection of operations) was being visited twice by the VTA module, but with a different set of opcodes. This was really very confusing, until I realised that steps can be accessed through more than one route - for instance through the different branches of an if/else block - and so visited more than one time. However, visiting a step can also modify that step (e.g. to write to outer-scoped variables), so on the second visit, any additional instructions were duplicated and so we ended up with a type stack double the size than was expected. It was a simple fix once I identified that, but it took a very long time to get there!

I don't really have any fun images of things working to give you, so have a pic of my latest error instead.

Today I fixed a bunch of little bugs around all sorts of things... so many I can't even remember what I've done. I think there are still some instances of variable graph node entries being in the wrong order (fun to debug 🙄) but hopefully those won't be too bad to fix - and once those are done I can move on to some slightly more interesting items on my todo list; then I can push the last few weeks of changes to the main branch and release v0.0.5. The image is a funky iterator adapter that I wrote today which I quite like. Apparently I've spent 2h26m since my last devlog, so this time must include some other time from the last few days that I haven't devlogged for - no idea what I was doing in that time tbh but was probably just more nitty gritty bug hunting.
Have a great Thursday :D

Just devlogging so I don't lose a little bit of time from yesterday - started refactoring how I handle string constants, using the js string imports extension rather than manually inserting them into a table. To do this I'm needing to add an additional lazy instruction for global indices, because we can't know what index a global will have until we've inserted all the necessary global imports, which can't happen until all steps are compiled.

Finally managed to iron out all the various bugs surrounding procedures in VTA. Turns out procedures need a lot of special casing because not only can they return multiple values, they also need any boxed inputs to remain unboxed - usually, boxed inputs are unboxed and passed into monomorphised blocks, which currently doesn't work with procedures - I would like to do procedure monomorphisation at some point but I need to figure out if it will be worth the trade off for code size. Not many lines changed for a long period of time but a lot of that was spent trying things out and then deleting them again.

Just adding a devlog so I don't lose my very important 8 minutes and 56 seconds of working today... I spent that time trying to debug an interesting issue in the VTA where a node is emitted which has more variable writes than there are elements on the stack. I know this must occur somewhere around the entry/exit to procedures, but I'm not sure where yet. Although I haven't yet fixed that bug, I did manage (i think) to preemptively fix another big which will occur on the exit to procedures. Maybe.

VTA (variable type analysis) now considers the flow of variables in and out of procedures; or rather, it will do once I've ironed out the bits that don't work. This required changing procedures to have return values (not yet implemented in WASM gen), along with considering procedures in the VTA graph generation - this included changing first-time global variable reads in procedures to argument access. Hopefully this'll allow for some nice optimisations once it's working. The commit is quite large but this includes changes before I started recording time for this hackclub event!