formsMD (fMD)

These 3h have been recorded over the last 3 days, so tbh I don't really know everything I've done anymore (I know I should devlog more frequently). Probably as I know myself jumped around a lot and did some small things here and there. I definitely did start on the matrix questions, as visible in the screenshot. And I think I did the mentioned options compile rework, as that doesn't seem to be covered in a previous devlog. For this rework, I've basicaly gone from a system that was robust at first, but fell apart when new things were just taped on to a new system built with all these improvements in mind already. Specifically, previously I was first replacing anything not fitting with special escape sequences (random uuid4s), but now I just go over it character by character and have variables for e.g. value enquoted, etc.

I first tried making an indicator for minlength and maxlength attributes of an input, but I didn't really know how to do that, so I switched over to styling the select/dropdown elements to look better on Chromium-based browsers. They were already looking well on Firefox, but not as much on mobile browsers and other desktop browsers (Chromium based center aligned the options in a really weird way).

It took me way longer than it should to implement the aforementioned feature of passing attributes.
While trying to implement this, I noticed, that my new seemingly more robust option compiling system has had some unrobustnesses, for example not accepting equal signs, even when enquoted or not stripping attributes correctly.

Got text input fields up and running (again, bc compiler rework...)!

You can define a datalist, like I did here or just make a noral text input. Now to work on getting attributes conveyed to that input, so things like placeholders, etc. can be set up.

I honestly don't know what the 3h are for, they kinda just appeared (I did not hack), but I did some things after the last devlog:
I've been jumping around a lot from different pieces that need work, one of them being figuring out, why the elements of an optgroup don't render correctly (indented in FF), after a lot of trial and error, I finally found, that <hr> tags are allowed everywhere else in a select tag, but not there. And yeah, I did many of such smaller problems and the time stacked up.

I edited the none option from the last Devlog to have a customizeable text using an option as well as fixed a bug with the optgroups not wrapping correctly (essentially HR elements aren't allowed in an optgroup)

Added the beforementioned None option along some other bugfixes and minor improvements (I've gone through ./todo.md, but forgot what I did exactly). We also hit 60hrs already!

(img: I've kinda gone with the food example of my classmate, so here is another food question, this time with a none option and inside a select)

Added the next question type, dropdowns. It took me some time to figure out the best way for required/optional questions to work, but I've found a good way by auto-selecting an option that just says please select (that option is only there if nothing is preselected by the way). Another thing to do now, is add a none option for optional questions that don't have the please select element.

Today, one of my classmates wanted to see my code and what it does, so I coded a quick food quiz.

JS code is something for me that requires much time for less progress than Python. Not because I know Python better, but because JS is more often reviewed and edited by me. So this whole devlog is just covering one small JS feature, which was there previously, but recoded by me, as the previous implementation was bad.

It is the weird way to handle other input fields for radio buttons and checkboxes, as they are weird. I know have a solution I mostly like, except not automatically changing focus to the text field when clicking the label / radio button / checkbox. The solution is close to what Google Forms does, so if typing in the text field, the radio button gets selected.

50th devlog at nearly 60hrs!
I have continued some work on JS to make the form feel native and accessible. Todays improvement: Make the other input disappear and reappear depending on if the checkbox/radio button is checked or not. First I did it with localStorage, but then changed to a way smarter system. The input is there all the time, however with a CSS style of color: #00000000 (fully transparent) it appears invisible. Now it's time to think about and add the necessary JS so these inputs appear connected (e.g. selecting radio/checkbox when typing in text field)

I wasn't all too wrong, the checkbox was indeed just some copy/paste magic and it's finished already. One quick JS bug fix and it's fully finished and working!

I just now stumbled across my first devlog with the time estimation of around 10hrs, and I've nearly laughed out aloud. I've calculated the times of each devlog and now, with only radio buttons working again, I'm at 16hrs already! Took more time then I'd have liked, but the result is definitely way better.

I thought it was a super smart idea to combine radio inputs and checkbox inputs, then spend a lot of time doing this just to realize their fundamentally different in some cases (mainly name/value meanings and required meaning), so it's better to make them separately while copy/pasting some code (or even making new functions)
Then I moved some code around:
Previously my ./compiler.py was way to crowded and ./filter.py did nearly nothing, so ./filter.py now got the code for specific blocks and ./compiler.py the code for specific processing of blocks.

So 30mins pretty useless as most got reverted, so I had somewhat of a struggle to find a good image. So these are my current Python files (./compiler_old.py is the old version, bwfore I started rebuilding)

small fix: My ./main.py script I wrote at the beginning of fMD used a longer form to get the options, now it's way more compact.

Old Form:

try:
option = options[key]
except KeyError:
option = default

New Form:

options = options.get(key) if key in options else default

After adding a background image again (Wikimedia Commons Picture), I noticed the CSS needs additional changes, that everything is well visible. Additionally I've added the full valid/invalid colour scheme to the submit button as well as the fieldsets. While adding it to the submit button I noticed, how close I am to getting a Pages System working. I just need to use JS to only allow continuation to the next page when the current page is valid. I always thought it would involve custom validity checking.

Now first appearance, not just one but infinite other options for radio and/or checkbox questions! I did not see this on any other form builder service yet!

this was the main thing done it these 20mins, along with manditory bug fixes for this feature.

JS improvements and bug fixes...

One new feature I added, is the abbillity to automatically highlight the text input when clicking on the radio button or label. This is something I've wanted for some time now and just now came around and have done it.

My last devlog didn't post yesterday, so two devlogs in one:

1. the last devlog I tried posting didn't post I think, so here it is again:

I've made it look slightly more clean again discovering the use of fieldsets and legends for a cleaner group look (the fieldsets will get a custom styling soon™)
And I revisited my plan from the reading session to just use CSS properties :valid and :invalid to style the input accordingly.
And some quality of life improvements, such as finally selecting the radio when clicking the text field of the other option.

somewhere in between these devlogsI also moved from setting radio groups as optional to adding an option no answer, as optional radio groups are a joke (once one radio button is selected it can't be unselected).

1. done some CSS work including a new asterisk to show required questions to users and a custom styling for fieldsets to appear connected.

Some new projects just appeared out of the wild. Mostly different projects from online research (Stack(overflow/exchange), etc.), so here is another hour!

Finished some last details with the multiple choice question. Although this question took a long time now, I think the rest of the compiler.py rework will be way faster as it's at least somewhat copy-paste stuff.

Important things I've learned:
- value if condition else value (e.g. True if answer == yes else False)
- crazy usefulness of f-strings (known them before, but not used as much)

»My alma mater was books, a good library…. I could spend the rest of my life reading, just satisfying my curiosity.« (Malcolm X, Muslim minister, human rights activist)

Well I continued reading myself into just <input>s of HTML. At the end of this project I'll be a master to create <input>s in HTML, but wouldn't know how to do cursive text (really don't know rn). The image shows the articles I want to read to code cool new stuff.

(Again notice for the approvers: I have Wakatime for Firefox installed, however set to only include sites I only visit when coding on this project. If I should, for some reason visit them outside this project, I'll make sure to open them in a Private Window, where extensions are disabled)

As I've installed Wakatime for Firefox, now reading documentations for formsMD also counts. I've read the MDN docs (developer.mozilla.org) article about the input element, as it's so versatile and a core componenent for every form.

What I've learned:
- custom form validation (for pages system and css styling) doesn't need to be custom, there are already built in JS and CSS options!
- way more options to customize rules to values in a form than I thought, so these will be implemented as well.

Our Informatics teacher at school allowed me (and one other person who knows to code already) to do my own projects in class, so I could continue working on fMD! He even wants me to do a short presentation about this project in an upcoming lesson, to show the class what can be done with code.

I didn't do too much, as I was half-listening to class and half-coding, but I fixed some smaller bugs I have encountered, such as the options not compiling correctly (I've forgot to return the compiled options, duh) or replaced some statements, so the IDs are generated at the right time and not after markdown compiling hit in and made <p> tags around it.

After long work the first few question types compile again. I'm questioning how useful this rework really was, but just want to get over it, so I can start on the backend. I hope to get everything finished soon.

Continued in the rework of compiler.py with coding the function that compiles all answer lines into an answer HTML for the finished HTML. Additionally some smaller fixes of slight issues PyCharm showed, such as overshadowing and similar.

Wrote a function that generates a unique ID based on just some text. Even if the text is in there multiple times, it gives a unique ID to each. I'm kinda tired, so this took longer than expected.

Kind of long duration for a devlog, I forgot to post more...

I continued to improve the compiler.py rework, adding some new functions to make my life easier and code less repeated. Specific new things:
Compile Lines: Lines are now compiled for all questions in a function!
And another function I worked on, but then discarded, becuase it was kind of useless.

I have that weird thing, where I just can't leave my code even when it's running and I've heard never change a running system way too often. So, I decided to just rewrite my biggest python file, /core_convert/compiler.py, and have just reworked the options compile.

But every time I rework code, I make it better and more robust, so it's actually a good thing? This new options compile definetely is more robust already and other things should rely on functions way more.

Rough Time estimation:
I took 1hr for one function, so at least 6 more to go means 10hrs probably just for this.

Oh, and I decided to code my own backend, just waiting for hackclub.app to approve me. First something similar, than increase complexity.

Looked at different other backend options but found NONE! So, please if you have any... I need something free with no limitations and a good file support. I might need to code my own.

Btw, the image represents the nothingness I found...

I've done some smaller things here and there, as well as worked on some forms using this script which revealed some points where I'll need to improve my script.

Additionally after tons of work, I fixed a very minor bug where the inputs for multi-line labels would display at the bottom of the label instead of the top. That took me way too much time...

Anyways I though I'd explain the system on how I convert everything:

1. I start with the raw input
   
2. Then each block of one question is extracted.
   
3. Each block gets analyzed mostly line by line and all important componetnts are extracted into variables
   
4. for checkbox, radio, etc. with unlimited inputs, another function generates the full answer HTML already
   
5. using f-strings, the HTML for that question is builded and returned
   
6. giving everything into an Markdown converter
   
7. adding the document structure of the html
   
8. finished HTML document!
   
9. cloning over CSS and JS assets.
   

If you couldn't watch the vid of the last dev-log, here is the vid in mp4: (same vid)

Ok, the problem from yesterday aside, here is a first function video I didm showing the process of creating a form. I know it's kinda lenghty, I might cut this in the future.
Oh, and welcome to follower no 1! What do you think?

I heavily tested my tool by making a complex form for a competition sign-up. While doing this I got some new things on my /todo.md list and discovered that file uploads already work really well, just some minor improvements, than it's done.

I only did code changes, so no good image.

I fixed a bug where checkboxes and radio buttons wouldn't auto-save properly as well as after I fixed that not auto-load properly.

As my /todo.md list was getting very short, that needed a fix, so I did a market research with the biggest competitors (Google & Microsoft Forms, Limesurvey), to find some new inspiration for different questions.
So here is a sneak-peak at what features are coming to fMD:

Added the big feature of matrix questions, to get quantity data from users. Often used in forms, this is essential for customer satisfaction surveys. A documentation entry is still due for this feature, but I'll need some sleep now.
Thanks for ×16 multiplier btw!

Fun Fact: I drank more than 3l of water during just this hour!

I have started by trying to make my website more mobile-friendly, but I'm horrible at doing a good website for mobile, so I just wrote it on my /todo.md list. However while going into CSS I changed some measurements from absolute to relative.

Then I did some work on form accessibillity:
- Labels for all questions
- Document Language

And yeah, I was tired, so that took longer than normal...

(I don't know about the image, I just added the little placeholder for some more accessibility)

• Reworked docs for the new syntax
• Added auto-expand to textareas (take more height if more text is entered
• Changed from hiding text inputs for other option if corresponding checkbox/radio select isn't checked to disabling these inputs

That means only logic is marked as important on my todo list now.

Finally! After 1h 6min of cool coding time, I have done one of my features marked with IMPORTANT on my todo list:

Save between sessions, automatically! Using browser storage, entered data into any form is now saved and instantly re-loaded if you open the page again.

I needed to fix a small bug. Thankfully I noticed it before shipping:

My /core_convert/filter.py was coded really weirdly (by me), so not every case was covered. I first didn't know what happened, but after I found the error it was a quite simple fix using some regex instead of the Python native str in str.

Additionally I created a new thumbnail to hopefully get more votes on this project!

I did some things from /todo.txt, to make the service a better product in general:
- complete CSS rework
- other text inputs only visible when other option is selected (may change to active in the future)
- filter.py full rework, as it was very bad code quality before.

This is also the release of v1.0, as I'm kinda afraid it wouldn't go through until end of August otherwise. So everything (should) work(s) now!

I have worked around 2 weeks on this with a very bad internet connection (max. 8kB/s), so Summer of Making didn't load at all. Here is a devlog of these two weeks and 5½ hours of work:

This project is getting close to a production-ready state, however my to-do list of things I could somewhen do is incredibly loooong... I have, however managed to already publish one form with fMD, and it worked great!

Features added:
- DROPDOWN - select your element out of a dropdown
- DATALISTS - get context-based suggestions for text inputs - BROKEN (MD processing breaks everything there)
- GLOBAL OPTIONS - set preferred submit method, title, etc.
- WORKING SUBMIT - using different services (formsubmit, mailto: link, etc.) the form can FINALLY be submitted! using formsubmit, it is a clean user experience, as no mail program, etc. needs to be installed.
- probably more I forgot about

Continued on the documentation for a new feature I will reveal once implemented! As a small hint: It's included in the Syntax of the screenshot.

improved a lot on the documentation (GitHub wiki), made a doc page for every question type, currently working on global options.
Additionally made a getting started guide for new users.

Added docs for different question types, so users can easily review the syntax.

added 'other' field for checkbox and radio questions

Started work on the wiki, so anyone can understand and use this tool

Minor bug fixes, e.g. newline chars in textareas not being transmitted to the mailto: URL
small improvement: changed a more complicated while statement to a regex '-+', to turn multiple hyphens in a generated qid into 1.

New Feature!
textareas, now you can promt your users to submit longer feedback, by including a multi-line textarea, just like the one you write your DevLogs in.

Bug Fixes:
I've fixed many minor bugs, the biggest of them being MD in divs not being rendered. Using the Markdown extension mdinhtml, I was able to still have it rendered though.

I've finished the stylesheet to make it look good, I've used multiple variables, so I can update them via JS and global options in the document in future DevLogs.

Here is the link for the live prview: https://0yqc.github.io/formsMD/output/page1.html

I started some work on the CSS to not only make it functional but also visually appealing, this is not finished yet, I'll make it look more beautiful tomorrow.

What's planned already?
- Options for the result, e.g. some CSS colors, info about where to send emails, etc.
- logic support, e.g. dynamically hide or show questions/answers
and more...

(In case you wonder, the submit action on the demo page doesn't work yet as the demo tool doesn't support JS)
(The currently generated page is just some random text and options I wrote to test everything out.)

first thing I did: added the support for all text input fields, as I started this yesterday it wasn't as difficult.

second thing: added a simple submit function with some JS cloning, I'll need to change it, since currently it always sends all form submissions to my personal email address. the plan is, to include these options at the top of the file:
- document name for the HTML title and email subject
- receiving email address for receiving all form submissions

I started to make the compiler for open-ended input questions (, etc.), however for that I needed to implement an options system where questions can implement options. And that took longer than expected...

So here I am after 1h37m and haven't finished text input yet, but am nearly done with the options, which will be a time-saver also for other question types.

If you wonder, why the code in the screenshot looks so short for that long coding time, it's because I tried a lot of other options before coming up with this complicated regex.

Added support for multiple-choice questions!

Now you can ask your survey or form participants a question where they need to decide themself.

It was basically just copying over the code from checkboxes and changing the generated HTML a bit to now include radio buttons.

Next up will be text input fields (+ email, etc.) and textareas.

Reworked Compiler!

Now I included more reusable code and made it cleaner in general by utilizing f-strings. Additionally the output now looks way cleaner and is even better styleable using CSS, since I used more IDs and classes.

with the new compiler I'll probably be able to do other question types in the near future...

finished up the checkbox conversion and fixing some bugs

added the markdown library to convert markdown to HTML (sudo apt install python-markdown-doc), so now I get the first outputs

as you can see on the screenshot, this project needs a lot of work to be competitive with the major forms platforms, but I'll maybe get there eventually

finished basic compiling for checkbox questions, the screenshot shows all the code I did for that.
is it the most efficient solutions? i don't think so
but does it work? yes, so I'm gonna keep it (at least for now)

next up will be multiple choice questions and then text input fields. In the end I would like logic and advanced options, let's what this project will evolve to...

Quick DevLog: Started to convert checkbox and multiple-choice questions to HTML elements

core conversion system: Big progress!

✅ open and process files for compiling
✅ filter out formsMD blocks for further compiling
⏳ compile fMD blocks into HTML elements

If you want to see this project process, consider following this project. If you have questions, just drop them in the comments or send them to [email protected]
If you like this project, you may want to visit the GitHub repository.

Don't have a new image, but I started to do some work on the Core Conversion system, which is written in Python.
I'm not as fast, so in this hour I've mainly set the files up and done some script, that they can open input files.

My Plan:
- Filter out lines that are from formsMD and not regular MD
- let them compile using a custom compiler I will write
- insert the compiled HTML
- a MD library will handle the rest

Started to work on how the markup syntax looks like. It may change in the future though :)
Basically this should in the end be a python tool that converts this Markdown to a HTML/JS site. If you have any ideas on how to save the results using only a front-end site (e.g open-source online-services), please send them to [email protected]. Thanks!