This project has been deleted.

Media Toolbox

Media Toolbox Used AI

8 devlogs
29h 10m
Created by spyguy

Website + API powered by ffmpeg, openai-whisper, and pillow that can convert media (images, video, audio) across many formats and transcribe audio into subtitle files or text. All secure, all data deleted immediately after use, and completely free!

Timeline

Ship 1

1 payout of shell 584.0 shells

spyguy

about 1 year ago

spyguy • Covers 8 devlogs and 29h 10m

added some quick cards on the home page to go to the conversion and transcription pages

Update attachment
Earned sticker

I was unhappy with the UI, so I decided to redesign it - and make it more mobile friendly in the process. Website is on the 1.2.0 version, with all features from the backend available in the frontend (including transcription, which wasn't available in the first release). I'm super happy with the final design of the website as well! This took way too much time tho lmao

Live @ https://media-toolbox.shashp.dev/

Update attachment

First iteration of my UI! With making the website, I had some challenges - namely, I found that my API was lacking features; quick fixes later, I have a semi-complete product. Still no transcription on the frontend, but shouldn't be too hard to implement (hopefully)

Update attachment

Finally was able to make everything into a docker image, and all tests pass in a container running linux!

Update attachment

After doing some research, I was able to include a faster (albeit more limited) implementation of openai-whisper called faster-whisper, based on CTranslate2 and able to use quantization. It also uses less RAM (!!!) and less CPU time. This is under a new endpoint because I have not done enough testing with it, so it is considered a BETA feature for now. I also included tests for this and made the endpoint largely the same except for the exclusion of a language field.

Update attachment

After fiddling around with language support and ensuring that the API does not use a ridiculous amount of resources, I was finally able to improve the transcription feature and write tests for it. It now supports translation (albeit only with the 'tiny' model) and is covered by tests.

Update attachment

After a long fight with ffmpeg, I finally found compatible codecs and ffmpeg args for all conversions that this platform plans to support.

I also wrote 172 tests (most are parametrized) for the API, file handling (upload, download, deletion) and file conversion (images, audio, video). In addition, I moved image conversion over to PIL instead of ffmpeg, improving performance.

TODO: write tests for transcription, implement cleanup of old files

Update attachment

After making some edits to the openai-whisper codebase and submitting a PR, I finally have some core features complete in the API:

  • File upload/download (stored with random uuid)
  • File conversion (powered by ffmpeg, so incredibly versatile)
  • File transcription (powered by openai-whisper, with a custom progress callback function to track progress through the transcription because it takes a while!)
Update attachment