orpheusmarket

I added parallel downloads to the images part of my backup script. As you can see in the video, the time decreases by so much from like 1 minute to 5 seconds (if you ignore the broken link at the end lol). I used concurrent.futures and put the logic into a function that the concurrent.futures executor exectutes. I also changed the timeouts so that I have a better chance of actually getting the images, since the cdn is kinda broken. (you can see in the video at the end that it takes until to the last request)

I added logging to the backup script using python's logging module. I didn't add it to my app program (the flask app) because I already get logging from gunicorn and journalctl. To do this, I create a logging.basicConfig() and do some basic configuration, including setting the log level to info (since this is in prod), setting to format to time - log level - message and creating 2 handlers. The first handler is a simple StreamHandler, which just shows logs in the terminal, and the second one is a file handler, which saves all of the logs from this run to a file. Then it gets a logger with the config and uses that for the rest of the script (eg logger.info("test") instead of logging.info("test"). I also added retry logic, since HC's cdn is not that reliable. It basically tries to make a request and if it fails it waits 2 seconds and tries again. If it fails after 3 attempts, then it just skips it and the website will use the cdn link instead of the local on

I added a function that actually checks if the backup date (the images and api) are the same before making a new one because I'm hosting this on Nest which has very limited storage :( Basically it works by calling getlatestimages from a script I made like 5 devlogs ago which returns the real path of the location of the latest backup. It then compares the api and file data by using json.dump with sorting and if they are the same, it sets the api_match variable to true.

The image check is mostly the same, but I first load the file from getlatestimages(), and then iterate over it and remove any keys named date, because obviously it will always be different and trigger a new backup. Then I build the expected images (which is basically what the backup should be), in a dictionary, compare it with what the api returns and sets img_match to true if they are both the same

uhh then my code just checks if both apimatch and imgmatch are true and if it is then it exits and if it isn't it just runs the rest of the backup code

I made a way to collapse the options because they can get really messy and in the way on small screens. In theory, it should have been simple, it's a button that changes the css of the div holding the dropdowns. I also wanted to make it save whether or not it was visible, so if you reloaded the page, it would stay the same. However, Flask is really slow at updating the session (the cookie), so if you reloaded the page and pressed the button within around 3 seconds, it wouldn't update (since i was using a post route that only updates it after it returns and finishes). Fortunately, I found out you could use session.modified = True to make it change and that's why it took me so long lol :D

I added the time machine feature! Basically, my main function uses a helper script to get all of the backups in a directory (hosted in /static) and show them in a menu. You can select a date and it will save it in your session (or cookie) just like the shop and dates. Unfortunately, the older backups are not compatible as i used a different structure (hence why the 20th is empty in the video) but i'm slowly building up a catalog of backups. Also I learnt a really important lesson lol, i need to MAKE MY CODE MODULAR, because it took me an hour when it should have been 10 minutes to implement this 😭

I started to lay the foundations for the time travel / archive feature. I changed the backup script to not copy images anymore and instead save them directly to static (since there is no reason not to). I also created a new function in the helper program that gets the latest backups, which retrieves all of the existing backup apis in static, which I will use to show a dropdown to select a date

I made some small quality of life improvements including fixing the website title, adding a favicon and fixing some bugs (a result of using quotes inside of quotes). Finally, I added some meta and opengraph tags to the rendered html, so platforms like slack can preview orpheusmarket better, with a description, title and image! Check the image for a better idea of what I'm talking about :D I also made it so that the backup script deletes the oldest backups to keep the count within 30 (only on nest because i have only 15gb of storage, I will keep all of the old backups on my raspberry pi, for the future time travel feature).

wow! This took me a while, but basically all of the images are locally hosted instead of being on the hc cdn (in case it breaks again 😭)! I had to change all of the paths in the backups script (because it was originally made for my rasperry pi, which runs linux) using os so it works on any platform (Posix or NT). I also modified it to save a new JSON file named images.json which contains all of the data for the main flask application to find images and their location. Since I'll probably set a cron job to run the backup script every 12 hours, I added a fallback, so if the image doesn't exist, it just uses the cdn image. tysm for actually reading!

Mahad changed the API a bit to account for the stickerlode so I had to fix my script (basically change if item["blackMarket"] == True to if item["shopType"] == "blackMarket). I also added another dropdown which allows you to browse the regular SOM shop in the same website (it's also much faster than the SOM website because most of the data is cached). I had to make some changes to the css, which involved a media query (@media (min-width: 768px) {} to make the dropdowns stacked because smaller screens can't accommodate the space needed for them being side by side. Next, I will probably either add support for stickerlode or have a time machine function where you can see stock for a particular date (courtesy of my backups lol). Thanks for reading :D

I implemented a backup system because last week, when I was testing this, both @mahad's api and the Hack Club cdn were down :( I originally had it running on my raspberry pi and it does something similar to the actual app. It makes a request to @mahad's api, iterates over everything and saves the images to a directory with a timestamped name. The helper script that actually gets the black market items checks if the api returns a abnormal HTTP code (anything other than 200) and if it does, it runs another function to get the path of the latest backup and then loads that!

I created a readme and banner using inkscape!

I styled the button with css to make it bigger and put it into a card, like the items. I also added a picture of orpheus on the top bar, and some css (overflow-x: hidden; and max-width: 100vw;) to prevent horizontal scrolling, a really annoying issue that I have come across many times when scrolling through websites on a phone, which you can see near the end of the video. Also, the dropdown didn't appear in the video (it does appear in the actual website) for some reason, so ignore that

I added lots of CSS to the cards and replicated the theme of heidimarket. Basically, I just have a variable where I add CSS and inject that into a <style> tag. I also added a region dropdown that allows you to get the correct cost for your region. When you change your selection on the dropdown, it adds ?region=regionhere and reloads the page. If the region is a valid region (US, EU, IN, CA, AU or XX) and then changes the items displayed to you and your cookie (so it stays persistent)

I created the basic webpage using Flask. It uses my script from the previous devlog (but modified to print anything) to get the items which are then iterated over and made into a card as seen on line 20-25. I just created a really simple boilerplate for the html and added {{card_html|safe}} in the body to render all of the cards. I had to use the safe option, else it would have been escaped by flask and wouldn't work. Right now, I have the images, buy links, title and description. However, I still need to add proper CSS because right now it looks like a webpage from the 2000s lol

I created the first basic script to get the items from @mahad's api, split them into blackmarket and the normal shop and display them with the shell count. I used the Colorama module for the colored output and the requests library to get the api. There is a spaces variable that is set to 5 and subtracted by the length of the shell cost. This just makes everything aligned and is by far the most overengineered part so far. The script also gets the regions from the api and puts it all in a set, so you can get accurate costs for your region