Manuel Hofmarcher

Implement pixel chunking to make website faster.

Started to improve the canvas db (before i had a db entry for every pixel - now im putting 50x50 pixel into one tuple and saving the color data in a data column. For some reason I experience this problem currently. Still not sure how this happens.

The website is now hosted on my publicly available server. Took some time cause I had to change some thing in the compose files and in the db.

added thiccnes slider and colour picker (they dont do anything yet, but the values of those items are correctly transmitted accross components). Also changed the sizing of some components to make the site more responsive on 4k displays.

Created custom angular material theme but for some reason this weird glow doesnt go away.

Took a way easier approach by just adding an event listener to the window once is resized. The listener calls a function that calculates the right scale by checking the window height. Making the canvas tag responsive is hard because it has to has this fixed logical size of 1000x500, my current approach is set a multiplier for the curser calculation.

Tried some time making the canvas zoomable. Then decided to throw that idea away and go with a new approach. The canvas should have fixed breakpoints and adjusts its size using them. Firstly this approach sounds way easier. Only issue is that my canvas still has a logical size of 1000 x 500 pixel. I somehow have to figure that one still out. Idk what picture i should insert so i just show what currently happens when u zoom out.

Added websocket functionality. Now the frontend sends messages live to the backend, while the backend sends out updates to all users.

convert all ebooks feature added. Had to spend some debugging because the script always expected the first subbook collection which leaded to a NoSuchElementException in selenium.

Canvas element added. API is now initially called to retrieve current Canvas state. Site also waits for API call to finish, before that a text is shown. Next step is to implement the websocket feature to also send the inputs to the db. I am also not currently able to draw lines, the drawn pixels are apart from each other.

worked some time on the ui

tinkered with the design of the site a bit. Also switched burger icon

looked for some fonts, icons and tried to import the downloaded font into tailwind. couldnt figure it out.

fucking hate browser caching. Tried solving an issue with nginx that was already fixed probably half an hour ago. switched to angular btw.

finished websocket support. Created svelte project and nginx config. Got it to forward me to the api trough nginx. Learned much about Svelte (first time use)

kind of started the websocket support. But I learned what WSGI and ASGI means. For the websocket support, I implemented a consumer on the server side and I installed the channels app for Django. I also finally found out what Redis does in this whole system and set a password for it.

Learned how to create Django Apps, how Models work and what they are. I also created some test apps and models, but I deleted them in the end. Then I created a model for the canvas table and the first API endpoint that returns the current canvas state.

Completely tested the script on Windows and Linux, fixed some loading times and added ASCII Art :)

Finally got all my book types working with the script (some ebooks redirect u from digi4school to some other site, and they all ofc use different layouts).
Still found some bugs ill be working on the next days. Also added some docstrings. Sleeping times are still not fully dynamic (despite me having said that in a previous devlog, but at least now they are partially dynamic using webdriverwait)
Fixed the next page popup issue.
Most of my time I spent on building the mechanism for the BiBox books, cause I tried many ways of obtaining the crop coordinates (BiBox uses a canvas for the book <- makes it way harder).

Discoveries:
Books hosted by scook use iframes
One of my math books cannot be found using XPATH for some reason
DigiBox books use a canvas (makes the process of obtaining the crop coordinates harder)

What I spent my time on:
Made the script compatible with EBooks hosted by DigiBox and Scook. Therefore I split up the savebookaspdf function into the book-specific part (selecting first page, obtaining crop coordinates, closing pop-ups and adjusting the zoom) and the general part "savebooksaspdf_main" which now only saves the images and crops them using the already gained values.

I also just noticed that I never mentioned when I switched to Chrome. The reason for the change was that chrome supports the print page feature. I no longer use this feature, nevertheless I stayed with chrome.

Fixed a bug with a Math book (for some reason selenium couldnt click the name of the book, so i changed it from searching for the bookname to searching for the nth entry heading). the script also closes other popups that may occur.

normalized sleep times
moved exports to their own directory
started the all option in the book selection (not finished)
created subbok selection (some ebooks contain subdirectories for ressources, audios and the real ebook)
fixed the mechanism of how the window is zoomed and the pics are cropped
changed the way of how the savebookas_pdf function notices the end of the book

I found out that some books have a different width to height ratio. My previous approach involved using fixed window sizes and a fixed zoom that I got with trial and error. Now I have spent a lot of time trying to build functions and testing different approaches. I think I will continue using a fixed zoom, but crop the image dynamically according to the div on the site. I also fixed some issues that arose with some other books like specific pop-ups or weird page naming schemes. Rest were some minor changes.

Tried some other methods of capturing the pages (for a better pdf quality). In the end I decided to stay with my previous approach. Sleeping times are now dynamic (the script checks every 100ms if the page finished loading). The user can now specifiy the amount of loading time between the pages (so ppl with slower internet can set it higher). And some other minor changes.

It took me a bit to find the best way to capture the pages. My approach is to take a png of the whole page (before the capture, the script zooms accordingly), crop the image and after all the images have been collected, the pdf is created. The script still lacks some features like: deletion of already used images, dynamic sleep times (currently the script always waits the same amount of seconds that I think the page needs, but the script also could check every like 100ms if the page already loaded). I also added timestamps to the print commands.

Spent a lot of time trying to create a good capture of the pages. Done for today.

Now all the books are presented and u can select one of them. After u have selected one, the chosen book will be opened and the first page will be shown.

Started this project a few months ago, didnt come very far. Now I firstly updated the current script to match the new website (website got huge overhaul). Then I wanted to add the feature of listing the owned books (so the script can ask the user which of the owned books should be converted). In the process of writing this part, I found out about shadow roots and shadow DOMs. That made it a little more complex. Got not as much done as I planned, but its still smth.

In the attachment are the current features shown.