Draedon

Applied some very basic styling to the buttons and inputs. Honestly, this reminded me of why I don't like Web Dev very much. I don't plan to make it look much nicer because I'm just after functionality

Added the options to change the image dimensions, along with an additional option to lock the aspect ratio and a button to copy the text to clipboard. Again, super easy to implement. Next I just need to make it look pretty, because I don't have any proper styling for the buttons and inputs.

Technically works now! It was super easy to convert the brightness values into text. I just found a palette of characters and put them all into a string, side by side. Then, to figure out which one to use, I just indexed into it with an index of: floor(brightness * (palette.length - 1)). This essentially maps the brightness values from [0, 1] to [0, pallete length - 1], allowing it to index the big palette string. Next up is setting a custom width because currently, the generated images are MASSIVE

Grayscale conversion works! This was super simple to get working: average the red, green, and blue components and multiply by the alpha. I might try different colour spaces in the future, such as Oklab

Implemented a simple drag and drop functionality for files! It's been ages since I've done this so I had to do a bit of googling for this one. Next up is probably loading the image and getting the brightness of each pixel in preparation for turning it into ASCII

10 hours just to try and implement a few math equations and it's still broken. All I need to do is implement the RKF45 algorithm for numerical integration but it's just not working and I have no idea why. Hopefully I can fix it within another 10 hours.

Updated web port with YouTube demo videos and audio! The biggest challenge was probably to get the animation running at exactly 30fps and to have it sync with the audio. It turns out that requestAnimationFrame and setInterval aren't great for precise looping, so I had to do a little hack with a recursive setTimeout with a self-adjusting delay. But once that worked, getting the audio to sync was really easy. Still no real styling since I don't see a point in it. After all, the project is getting Bad Apple to render on the calculator, not the web.

Not a real devlog but a reminder that there are two demos! One is a video of the actual emulator (https://youtu.be/qDuJ1DxRWTY) and the other is the web port (https://draedon123.github.io/HP-Bad-Apple/), both found in the README of the GitHub repo.

Added a web port of the program! This was super simple to do since all the hard work has already been done. In fact, I wrote the port in one go and it worked first try, with no bugs, which is definitely a first. The port just uses the 2D Canvas API to render the frames so it should work on most, if not all devices. I didn't spend any time making it look pretty though, so it's just a canvas on a white screen.

After one (big) change, the file size of the compiled .hpprgm file has dropped by a whopping 62.7%! This brings it down from 98.8mb to 36.9mb. This was done by changing the internal representation of a frame from a list of numbers to a single binary string. So instead of having two numbers for one internal chunk of a frame being, say, 0,12743, which takes up 7 bytes, it now only requires two UTF-8 characters. The first one has a code point of 0+161=161 and the second 12743+161=12904. This takes up a total of 5 bytes. This by itself is only a 27% decrease, but I'm guessing the .hpprgm compiler can represent a binary string much more compactly than a list of numbers. Finally the program can run on my physical calculator... although it's only reaching 3-6fps depending on the frame, and has very visible screen tearing since the screen can't render the rows/columns fast enough.

Fixed the bugs. There were three main ones which all contributed to very slightly incorrect rendering. The first one was the easy one to track and fix. When my scripts read the individual frames from a folder, it didn't sort the files first. So everything would work well until you get to frame 99. Then, it renders frame 100, then 1000, and finally 101, because that's how the default alphabetical sorting goes. Next was I had the incorrect dimensions hardcoded. The screen is actually 320x240, but I misinterpreted a diagram and had 319x239. Finally, the last bug was an off-by-one error. Simply put, if I have to draw a horizontal line starting at (x, y) that's 5 units long, I need to draw it from (x, y) to (x + 4, y), not (x + 5, y). It renders smoothly on the emulator, but the built file is 98.8mb, which my physical calculator can't actually handle.

Optimisation is really hard to do. The first change was swapping LINEP to RECTP to draw lines. I'm guessing that RECT_P has less overhead and that's why it's faster. Then, to reduce the number of draw calls, I added a parameter (per frame) to determine which way to draw the rows. Some frames require much less draw calls if the direction is changed, and this reduced the build size by around 38%. It's still bugged though. Introducing this change made it so the bottom row of pixels doesn't work as expected, but I decided to just make a devlog before fixing it. All these optimisations brought up the fps on the physical calculator from 3fps to something I can't eyeball; it's almost smooth and just runs slower than it's supposed to.

Delta encoding was also very easy to get working. All I needed to do is store what pixels and render the changed pixels only, which massively improves performance at the cost of around a 2x increase in build size. I'm sure I can optimise the file size somehow though. I think delta encoding is a great optimisation for this project in particular because the Bad Apple video is just a simple animation, so not that many pixels change between frames as compared to other videos.

WARNING: VIDEO CONTAINS INTENSE FLICKERING. So video rendering does kind of work, because the functionality was actually implemented in the last devlog. But there are issues. Firstly, the flickering. The calculator just can't handle redrawing the entire screen every frame. I might have to fix this with a delta-based approach, where instead of storing the data for each individual frame, only the difference with the previous frame is stored and drawn. Next, the speed. It should be rendering at 30fps, but clearly that's not the case. And the last issue I found is that although it kind of runs smoothly on the emulator, it runs much slower on my actual calculator. Optimisations here I come.

Surprisingly wasn't hard to render a single frame! All it took was parsing the frames and drawing single-width lines across the screen, wrapping around to the next row when needed. Next step is rendering multiple frames and probably optimising

Wrote the algorithm to encode the video as an array usable by the HPPPL (HP Prime Programming Language, which the calculator uses). It first uses ffmpeg to get the individual frames of the video, and then uses JIMP (JavaScript port of GIMP) to resize the images. Then, all the pixels are converted to greyscale (a single number representing brightness). Pixels above a certain threshold (set to 200 right now) are assigned a value of 1. Otherwise, they get a 0. These values are then appended to a string in order, giving something like 0000000111000010. To compress this, Run Length Encoding is used, which basically writes out the character and how many times it's written in a row. In the above example, it would be 0,7,1,3,0,4,1,1,0,1. But since there's only two possible characters (0 and 1), I can omit the character and just write down how many times the character appears. Of course, the string has to be prepended by the starting character though, giving 0,7,3,4,1,1, which is much more compressed. This would be far better if I could write it out in raw bytes, but HPPPL can't parse binary.

Can't believe it took almost 4 hours to ray trace a simple black hole. Granted, this is different to traditional ray tracing, which mathematically calculates if it intersects with shapes. This ray tracer is closer to ray marching than it is ray tracing so far, since it marches the ray forward a certain distance and checks to see if it's inside the black hole. If it is, then the pixel is black.

Initial commit! I couldn't be bothered writing out a bunch of code I've written before (for cameras, ray tracing, skyboxes, etc) so I copied code from a few previous projects. However, a lot of the classes require other classes (shared between projects) with different interfaces, which made combining all the code a pain. Maybe I should've just written it all out by hand. But that aside, we have a skybox! It's currently just the Milky Way, and there aren't any black holes yet. But everything is super buggy still (moving the camera breaks it instantly) and the code is a mess since it's all copied.

Added the configuration panel, finally. This just allows basic tweaking of settings, such as the maximum number of bounces and rerenders, and the gamma of the render. This was really easy to implement as it was just a bit of copy-paste on the HTML side and a few functions in JS. Also added a tiny bit of error handling in case the GPUDevice is lost, which will pretty much only happen if the GPU is overtaxed.

Implemented dielectric materials! Basically, just transparent materials. Overall, it wasn't that hard to implement. It was just Snell's Law, Fresnel's Law of Reflectance (with Schlick's approximation), and Total Internal Reflections. It also required a slight modification of the material system. Previously, since there were only diffuse and metal materials, I simply had a boolean flag in the Material struct to determine if it was a metal or not. But now, that has changed to be a u32 to accommodate more materials. This doesn't actual use more memory though, since previously, the bool was actually stored as a u32 due to wgsl limitations. Also, this project should be close to being shipped. Maybe a configuration panel is next.

Implemented some additional controls and a rotatable camera! Sadly, the camera isn't interactive yet, but that'll come eventually. The new controls include the Tab key, which changes scenes (currently there are only two scenes: the mirror room and the Cornell Box), and Ctrl + S, which downloads the current render, without the info box in the top left. Next up is probably transparent and translucent materials, which I think falls under the category of Dielectrics.

Metals implemented. How are metals different from a typical diffuse material, I hear you ask? Well, instead of scattering light in a nearly random fashion (I implemented a cosine-weighted BSDF for diffuse materials), metals are very smooth and scatter light almost perfectly. This makes them look very smooth since incoming light rays with the same direction will bounce off in the same direction too. Except, there is also a roughness factor, which offsets the bounces a little bit, making the surface of the metal look a little fuzzy, which can be used as a stylistic choice.

Honestly, metals were very easy to implement as it only involved checking to see if the material was metallic or not and changing the BSDF accordingly. I also found a bug with my Sphere object struct, which wasn't taking into memory alignment. I didn't find this bug earlier because I only ever had one sphere.

Spent ages trying to fix NEE and got nowhere. So this is filler devlog, without much progress. Some code cleanups were made, and I experimented with dynamic scenes instead of static scenes. However, since this is a progressive ray tracer (scenes rendered over the course of many frames), there will be much more noise when the scene changes every frame. Since we can't use the data from previous frames, we are effectively rendering with one spp (sample per pixel), which is incredibly noisy. I might just give up on fixing NEE for the time being and go implement something interesting, like metals and/or dielectrics.

Implemented the Next Event Estimation (NEE) algorithm. If you look at the previous devlog, you'll see how it's super dark and relatively noisy, as it's pretty rare for a random ray to hit the light. What NEE does is in addition to shooting out a ray on every intersection (simulation bouncing), it also shoots out another ray at a random light source. This brightens the scene a little and reduces noise as it's proactively seeking out the lights instead of waiting for a ray to randomly hit one. Or at least, it's meant to reduce noise, but it doesn't really do that here. I'm almost certain that my implementation is a little broken.

Lighting, finally. Lighting should've been a quick implementation, but it required me to completely change the material system. Honestly, I should've done that ages ago, since previously, all objects simply had a colour attached to it. But now, they have a material index, which points to a material in a materials array, passed to the GPU. Also fixed many bugs along the way, such as normals facing the wrong way for planes, sometimes. Finally, I changed the scene to the famous Cornell Box.

Made the rendering loop progressive. I already gave a brief explanation of what that is, but basically, before, 30 rays were being shot out per pixel, per frame. But since that's really expensive, it now shoots only 1 ray, but accumulates the results across frames to effectively increase the frame rate. Now, instead of the <4 fps, I'm getting 80-100fps, although it does take longer for the image to converge. There's also the issue of floating-point precision issues, but I'm mostly avoiding that by limiting the number of rerenders.

Implemented rendering of quads. Even though the entire point of this project was for me to learn the maths behind path tracing, I don't think I want to learn the entire derivation behind ray-quad intersections. Fortunately, quads are just a general 2d primitive and it's really easy to change it to other 2d shapes, most notable of which being the triangle. But on the flip side, with 30 samples per pixel at a screen resolution of 1920x1080, I'm getting below 4fps. Next up is probably progressive path tracing (rendering one sample per frame and averaging the result with the last frame). If that isn't enough, then I'll have to optimise properly.

Added some basic performance metrics. These include the time per frame spent on the GPU ONLY, and the fps as calculated by 1/frameTime. These aren't super useful now but will come in handy later, when I start optimising. As you can see from the screenshot, with 30 samples per pixel, my crappy laptop is struggling to reach even 40fps on 1920x1080. But to be fair, over 62 million rays are being shot out per frame.

Shading is now ray traced! Currently, it just uses a basic Lambertian BRDF to determine how the rays bounce. Also took the the time to add in anti-aliasing (currently at 30 samples per pixel) and gamma correction (gamma = 1.5). Gamma right now is hardcoded because I can't be bothered making and binding another buffer.

Implemented some ray traced spheres. Don't get me wrong - rays are being shot into the scene and there is maths calculating the nearest collision, but the lighting isn't ray traced at all, but rather, just computed with a basic diffuse model, where the sphere's colour is multiplied by the dot product of the the surface normal and the light direction. Even though the shading technically isn't ray tracing, it was still useful to implement as proof that my surface normals are correct.

Technically have some actual ray tracing now! Rays are being shot out of the camera into the (empty) scene and returning the sky colour, which is a gradient depending on the vertical direction of the ray. Already ran into so many bugs though, including my worst nightmare: memory alignment issues in the shader. Next up is adding some spheres into the scene

Here we go. The start of my second path tracer. Not much to say about this first devlog. All that's happened is a boilerplate really. A framebuffer rendered to by a compute shader, which is displayed to the screen. People familiar with computer graphics will recognise the classic UV gradient that's being rendered.

Added horizontal displacement vectors! How the simulation worked before is that it calculates the height of every vertex in the mesh and moves them up and down accordingly, but you can see that it looks almost static. However, if we move all the vertices horizontally a little bit too, it looks far better. Normals still broken though.

Not much of an update this time. Just changed the skybox texture and updated the README to include all my references for the project. Normals are still broken because I haven't gotten around to fixing those yet. Unfortunately, I'm going to be busy for around two weeks so not much work (if any) will happen in that time.

Massive improvement boost and basic code clean up. This pushed my performance from around 30fps to >60fps. Honestly, I'm not sure how it was this much of an improvement. All I really did in terms for performance was implement workgroup sizes across all my compute shaders

Hacked in some half-broken normals according to various papers. At first glance, it seems to work fine, until you look into the vertex shader and see that the slope vector is multiplied by some arbitrary value of 20. Basically, the shape of the slopes in my texture holding them looks fine, but they're ridiculously small. Probably because in the final phase of the ifft, it normalises every value, dividing them by the square of the texture size. And my texture size? 512x512. So every value is smaller by a factor of 512*512=262144. Yeah. No idea how to fix that properly.

I can't believe it took me that long to find the bug. It wasn't in the wave spectra or the IFFT algorithm, which I spent hours inspecting, but the random number generator. And I should've fixed this ages ago, considering that my old height map wasn't complete nonsense, which would've happened if either of those were broken. Tessendorf's paper literally says that different random number distributions will result in different stylised waves, which hinted at my broken random number generator. And the bug was in my method of preventing division by 0.

The final step of the gaussian random number distribution was to return v/u, and my check for u was to set it to 0.0001 if its absolute value was less than that, which definitely skewed my result. The fix? Just discard it and try again if abs(u) is less than some threshold.

Now I can finally start coding the interesting things, like accurate normals, wave caustics, sea foam, subsurface scattering, and all the other pretty things I haven't gotten to yet

IFFT still not working, but I took a slight detour to put in some finite difference normals, which suck. I'll probably implement exact normals with more IFFTs and then fix the waves properly. Not sure if it's the IFFT algorithm which is broken (I swear it's correct but idk) or my wave spectra

Spent ages on the ifft algorithm, and it half works. Basically, it's giving some values (which kind of make sense but don't). Better than having it completely broken at least

This isn't really a devlog, but I thought that I should mention it. This took a lot longer than 3 hours to make, and majority of it was made before the Summer of Making. I just used this time to clean up the code and make it look good.

Spent far too long implementing the ifft from Jerry Tessendorf's Simulating Ocean Water paper. And the funny thing is, it's still broken, I'm still using an idft instead of ifft, and I don't know what's wrong

Honestly, most of the maths is flying over my head. Simulating Ocean Water by Jerry Tessendorf isn't making much sense to me. It defines the Wave Spectrum as a function which takes in a 2d vector, but all the spectra I can find online (even the Phillips spectrum in the paper) just takes in a frequency. This is going to take me a lot of reading to figure out

Wasted hours trying to get an ifft algorithm...
All that hard work and all I have right now is a half-baked random number generator
Here we go...

Just a really quick settings tweak and loading screen. Made the ocean mesh 10x bigger without increasing the number of vertices, which didn't seem to affect the quality. Hopefully the other small tweaks made it looks a little bit more like an ocean. Fine-tuning the settings was a pain so I might make a little configuration panel sometime.

Skybox and Fresnel reflections implemented! Water now looks a lot better, but still looks like plastic/acrylic. Not quite sure how to fix that though. I wish I could have a skybox with a slightly brighter sky though

Did the code clean-up. Honestly surprised that it didn't take too long, and there were surprisingly few changes. Not complaining. But did implement a browser popup which would happen in any errors were thrown

Added a skybox to replace the boring black background! As much as I'd love to make it look even nicer with stuff like reflections of the skybox on the water, the code is such a mess from trying to hack together the skybox that I'll definitely have to spend an hour or so just cleaning it up.

Finally added specular highlights to the water! Instead of making it look better and more realistic, it just made it look even more like plastic. Maybe I'll turn down the shininess coefficient in the shader and apply Fresnel Reflections later.

Just some typical code clean-up and optimisation. Nothing much. But for some reason, the waves changed a little bit. Not catastrophic; just different. I wonder when I'll deal with the tiling issues. Not now, because there are more important things to do, like Specular Reflections and making the environment look nice.

Waves actually look okay now. Fixed the incorrect lighting direction and implemented a fBm (Fractional Brownian Motion) thing to the waves, which decreases how much influence the later waves have. This adds finer details to the waves and increasing the number of waves won't make the ocean look like Miller's Planet.

Lighting! Took long enough to get here. It was hard enough looking at flat colours on my screen and trying to figure out if it was working. Also, the purple is finally replaced with a more ocean-like colour, blue.

Just some general code clean-up. Most notable of which was actually splitting the shader into multiple .wgsl files and concatenating them at runtime. Honestly not sure why I didn't do this earlier, when the functionality was already built-in to the Shader class.

Got some basic wave moment! Currently it's using the Sum of Sines approximation proposed by GPU Gems 1, with 3 randomly generated waves. Without lighting, it's a bit hard to see the waves, but that will be fixed soon™

Subdivided the initial square. That's it. It will be used later though :)

I lied. Camera movement is NOT a silent update! Got some basic movement and camera rotation in place, which should make debugging incorrect meshes later on easier. And it's just cool being able to explore the world.

Added a basic perspective camera! No input handling yet, but that might be a silent update. Also changed the square colour to purple. Finally don't have to look at that ugly yellow!

Basic square rendering! Well, it's not actually a square because its size is dependent on the aspect ratio, but it's a start. Pretty much just a WebGPU boilerplate with a little bit of abstraction. Honestly, I'm not sure why I made the square yellow. Yellow sucks.

Fully styled now, with not-default buttons and sliders. Responsive design added to. This might be the last devlog before it's ready for release!!!

Basic styling 1. Added some very basic colours and styling to the project