A shop monitor for the Summer of Making shop!
Zoe
Check their projects out: Decode, readitt, Neeter Browser, Wayline, Tail & Ears, Dimensity, Midnight Cabin (v2!!!), RuthArm, about://me
zrl
Check their projects out: arker, terminal-wakatime
Toshit
Check their projects out: epoxy-tls, SoM Grand Survey: Mount Kablooey Summit Visitor Center, m3-dreamland, dreamland.js, scramjet, SoM Harbor, powerd, Summer Of Making
Once you ship this you can't edit the description of the project, but you'll be able to add more devlogs and re-ship it as you add new features!
I added Stickerlode scraping! The bot will now detect new stickers every day. Also cleaned up some code to make it even easier to add scrapers in the future.
I migrated the monitor to a new DOM parsing library called Linkedom. Here's my postmortem:
- SOM Monitor uses a library called Happy DOM to parse the pages
- The latest version of Happy DOM does not work on the latest version of Bun (see capricorn86/happy-dom#1762)
- So commit 43e1aeb downgraded it to 17.1.8 from 18.0.1, which doesn't have this issue
- However, 17.1.8 has a memory leak
This change has been great. At initial launch, the LinkeDOM version uses ~200MB less RAM than the happy-dom version. And unlike the happy-dom library, LinkeDOM doesn't suffer from memory leak bugs - the old version used to use up to 20GB (!!!) of RAM after just a few hours.
So many cool updates!
- Now scrapes the Black Market
- Made the bot way more stable and has a lot less bugs
- Switched over to @channel pings
- The bot is now really modular, so updating scrapers is a lot easier
- New API! https://summer.skyfall.dev/api/shop
Fixed bugs, TS config. The shop code updated overnight and I fixed the bot so it can read prices properly.
Made notification summaries in the Activity tab more helpful (thanks @Jeremy!)
(They used to just say @shop-watchers for every single update :pf:)
Cleaned up code, started working on improving channel ping notification messages in your Activity tab.
Observability is important. Slack said that my API requests were incorrect, and @GGORG mentioned how it would only send a maximum of one update, even when there were more than one!
To fix this, I added Sentry (https://sentry.io) to my project and now log all Slack API requests and items. The end result is that in case the SOM devs change up the shop code (the bot uses HTML scraping), I'll be notified and can fix the bot.