Please sign in to access this page

Illustrative Math Archiver

Illustrative Math Archiver

7 devlogs
10h 4m
•  Ship certified
Created by will

A bash (Linux console) script that automatically archives math courses from Illustrative Mathematics' accessim.org

Timeline

Ship 2

1 payout of shell 66.0 shells

will

3 months ago

will Covers 4 devlogs and 2h 24m
will
will
1h 31m 3 months ago

Just finished fixing SVGs on Chrome! Now I'm going to a full test run and then I'll probably ship. Miraculously, mv "./accessim.org/" "./accessim.org/.svg" and the search pattern 's//.svg/g' (for sed) are both non destructive, which means I can skip checking whether or not the array is empty (much more difficult than it sounds :sob:) and silence the errors instead.

Update attachment

I did some housekeeping for the code (cleaning up stuff, maintaining best practice, etc.). I used a program called shellcheck to analyze my code. Before, I had ~22 warnings! But now I have zero.

Update attachment

I just fixed the issue that caused an error on Chromium-based browsers! What I had to do was not only block JavaScript, but all <script> tags that fetched JavaScript (which I didn't catch before because of my request blocker). Next up is fixing SVGs!

Made a demo video!

Ship 1

1 payout of shell 127.0 shells

will

4 months ago

will Covers 3 devlogs and 7h 39m

I figured since there's no demo (command only works on Linux and maybe macOS) I'd show a little about how it works. First, you go to the github repo and copy the command that runs the script. After running the command, you are greeted by a cow (see devlog notes) and prompted to select a course group. Once you do, it fetches all the courses in that course group. Then, simply select the course you want to archive and the rest is automated. It downloads the entire course to your computer (a course is typically one grade level's worth of work). Once it's done, it will tell you where the files are.

[ACTUAL DEVLOG STARTS HERE] I added two quick things in this devlog: it now has a little cow that says the name of it, and it tells you where the downloaded lessons are on your computer. I also made a basic cover for the project (attached).

Update attachment
will
will
4h 16m 4 months ago

The first release is finished! I fixed both of the issues that occured in the previous devlog. Here's how I solved each:

  1. The CLI. I fixed this by adding a function that takes in the option selected and the valid options. If the option selected isn't part of the valid options, it exits the program.
  2. The images. This problem was significantly harder to fix, but I just had to break it into steps. The first step was to identify what assets needed to be downloaded. I found out all the images came from a specific URL, so I used a grep command to get a list of assets that needed to be downloaded. Then, I looped through each URL and curl'd them into the asset/ folder in the accessim.org directory. The last step uses a sed command to replace all the links that led to online resources with links that led to the assets folder.

While I was at it I made it rename the files to remove the URL parameter.

I also polished up the readme and added a single command you can use to run it. Attached is a pic of what happens when you run the command.

I will probably do one more devlog adding extra features and then ship it. 2 doesn't feel like enough and even 3 feels like not enough but then again this is a small project.

Update attachment
will
will
2h 55m 4 months ago

The first prototype is done! It has a basic CLI, supports (I think) all courses, and has mostly working webpages!

Here are the two main issues I need to fix next:
1. The CLI does not enforce the options it gives you. If you type in random gibberish, it will still carry out the rest of the code as if it's a normal answer. This may cause unwanted behavior.

  1. The webpages -- while appearing to work as if they were online -- are actually missing a few elements. If you open the HTML pages in Firefox and use the Disable cache feature and block 'http://' and 'https://' features (to simulate a fully offline and unused experience), you'll see the images on the site will be replaced with their accessibility labels. This is demonstrated in the attached picture. This is because (I assume) the images are not properly downloaded for offline viewing. The reason it shows it before you do this is the offline site makes a web request to get the image from the online server. Even if you turn your internet off, it may still show the images as your browser caches them-- meaning it downloads them somewhere on your computer (intended to make pages load faster).

In the next update, I'd like to both fix these two issues and update the CLI. Later, I will make it download the pages faster and add better error handling as well as Windows support.

Update attachment