Universe Simulator

I don't entirely know what the secret sauce was, but I rebuild the entire render pipeline from scratch starting from just drawing individual points, and I've made it back here: I can draw cubes using instanced rendering! Huzza!

Next, I'll make them look a bit more pretty and scale/color based on mass. Then, I'll try to integrate the physics simulation with it.

See that little black dot in the middle of the screen? That means my vertex shader isn't properly seeing the entities' positions. Jeez, Louise! More news at 6:00.

Rendering In Progress!

Bit of a teaser here-- I'm in the trenches squashing bugs; the latest issue is that, while I can get the render pipeline to accept all the data buffers properly without error, I can't get the damn thing to actually put pixels on the screen how I want them. :(

More progress on the main sim loop!

Lower effort devlog. I've finished the first draft of the main sim loop and written tests for entity management and buffer resizing, as well as keyboard and mouse inputs for camera control. Tests are passing nicely so far.

I also drafted the renderer using a basic render pipeline for BGFX that I painfully extracted information about from Claude. Thanks EnglishGarfield for suggesting Claude, it's actually pretty good and much better than ChatGPT for code.

I'm still procrastinating testing the rendering because I'm not excited to debug it :/

Drafting the Main Simulation Loop

This will take a lot of effort to complete, but I'm working on filling out the main simulation loop. This means I need to implement:
1. Camera and keyboard controls with SDL. Already drafted this, mostly complete as far as I can tell.
2. Logic for safely creating new entities and adding them to data buffers. This means:
1. Utilities for finding space in GPU buffers to slot new entities
2. Ways to track entities that exist on GPU side from the CPU side without knowing their positions etc.
3. Performing buffer resizing whenever more space is needed, again using another specialized compute shader.
3. Graphical rendering! I'll have to figure out some way to draw objects when only the GPU can know where they are or that they exist at all.
4. GUIs for interacting with the app. I will probably use Dear ImGui for this; it's supposed to be pretty easy to use with BGFX.

I have written a little bit of this code, but I still haven't gotten around to unit tests yet. I think this will mean a lot of manual labor coding followed by extensive sessions in the bug mines. Fun times, but once this is through I should have a minimum functional v1.0.0 to show off. Hopefully, my devlogs won't just be IDE photos much longer ;)

More debugging, and I fixed the gravity calculations.

Yes, another IDE photo. This time, I got the shader for gravity calculations fixed. I switched to use a 2D multithreading structure, making a sort of times table. For example:
. 1 2 3 4 5
1|
2|
3|
4|
5|
We run once for each cell here. The shader calculates force applied between the two entities it's run on, and adds the acceleration deltas to the entity on the X side. Pretty cool! I also had to use the atomicAdd() function to avoid race conditions adding to the same entities at once. Either way, the photo below features my test case working :)

More BGFX Weirdness!!!

Okay, so I got the first test with adding, reading, and killing entities working just fine. Now, though, when I try to compute accelerations for gravity, I'm having issues. The BGFX frame call is hanging, but it goes through when I step through it with a debugger??? I'm still working through it, but I'll keep you posted. For now, I need to sleep.

Adding and reading single entities WORKS!!! I know it only says ~ 1 hour of time, but I forgot to get WakaTime set up on my Linux dev box for a while, so there's probably a lot of time here that didn't get recorded.

I tried so much stuff I honestly don't know what did the trick. The last thing I did was setting the texture image format to RGBA32F instead of RGBA8, which allows you to actually store a full set of floats, and reading data doesn't give back gibberish anymore. Not sure why that didn't occur to me before, but here we are. Now, you can add an entity to the data buffers and read it from CPU. Small steps. Oh yeah, and I got everything working on Linux... now to bring it to parity on macOS.

Big devlog coming soon, I think.

There is some WEIRD stuff happening in this poor project; I'm trying to test adding data to GPU buffers and reading them back to CPU with textures, but the data that's being spat up is pure nonsense. I'm now creating an Xcode project based off this CMake project so I can run the tests in Xcode and use the GPU frame capture debugger there to hopefully gain a bit of insight into what's going wrong.

Wish me luck!

Another IDE photo! I worked on basic scaffolding for the project and wrote drafts of the compute shaders for Velocity-Verlet integration and Newtonian gravitation.

It turns out data buffers for compute shaders are very unintuitive and a bit annoying to use. Basically, you can create buffers in CPU, and when doing so you flag them as readable and/or writable. You can never read data from a buffer on the CPU side, and you can only write data to a buffer if it's NOT marked as writable by the GPU. The only way, therefore, to write data to a GPU-writable buffer is to use a specialized compute shader that writes data on the GPU side.

The only way to read data back from the GPU side in your CPU code is to create a writable texture, write to it in your GPU shader, copy it to a non-writable texture with the readback flag in GPU using bgfx::blit(), then copy the data from the texture to an already created space malloc'd in memory. All of this takes multiple frames to complete. Very annoying.

On the more algorithmic side of things, I've organized the procedure for calculating gravitation and motion. Basically, I will have six data buffers: old and new for each of position, velocity, and acceleration. According to the Velocity-Verlet algorithm, each frame, I will:
1. Calculate positions for each entity, using data from velocity_old and acceleration_old and depositing results in positions_new
2. Calculate gravitation for each entity, inputs positions_old and outputs positions_new (positions also stores mass in the w component)
3. Calculate velocities for each entity, inputs velocities_old and accelerations_old and outputs velocities_new
4. Swap old and new buffers.

I won't go into too much detail but I'm sure you can work out how that would work. Anyway, this was long. Hopefully it all comes together nicely!

Sorry for all the IDE photos! I learned how to get GPU compute working and use data buffers to send data to the GPU; designed a quick, simple compute example. Now, I will get working on designing the architecture for the universe simulator proper... no less than 12 hours into the project :/

Shader compilation is FIXED!! Oh man, was this rough. I swear.

It turns out, the bgfx.cmake library I was using to make bgfx work nicely with CMake has a strange collection of issues that meant I couldn't get the shader compiler to include the directories I wanted. BGFX wants shaders to #include <bgfx_shader.sh> or #include <bgfx_compute.sh> in order to work, but these mysterious .sh files are hidden in the source directory somewhere.

I couldn't get that included with the utility functions provided by bgfx.cmake, so I wrote my own utility functions for interfacing with shaderc. I also had to deal with some weird errors that I took way to long to realize were happening because of trying to compile shader types that aren't compatible with my toolchain. Regardless, I finally managed to compile a Metal-compatible compute shader on macOS, and I should (in theory) be able to do the same for Vulkan and OpenGL on other platforms.

Woah, BIG refactor. I had to start over from the very very beginning. I tried and tried, but it became clear that metal-cpp would not work. It was too weak of a library to do much more than GPU compute, so I would have had to interface with Swift or Objective-C, but at that point, I might as well just write the whole thing in one of those languages for how much of a pain it would be to bridge.

I decided that, if I'm going to refactor what I'm doing, I might as well just build it from the ground up to be cross-platform. The point of locking into Apple was that Metal is supposed to be a walk in the park... but, at this point, it isn't. So, I decided to build it around bgfx (using bgfx.cmake to make it play nicer with CMake), with SDL3 as a backend. It was a huge pain to get started, but it all seems to basically be working now. Finally, I can get started working on GPU compute and the simulation architecture. Hopefully.

GPU Compute seems to be working! I got metal-cpp all set up (finally). This beautiful article on Apple Developer, translated into C++, is quite helpful for getting the basics down. Now, the more difficult task: how do I scaffold an actual freaking universe simulator??

jeez guys.... scaffolding a project that uses CMake, Metal, and metal-cpp is a surprisingly complex endeavor. I think I'm finally getting there, though.