From ac7534b921319ae38226d4f86d89eaa99f704e19 Mon Sep 17 00:00:00 2001
From: Alex <alex.corcoles@veecle.io>
Date: Fri, 21 Feb 2025 17:34:37 +0100
Subject: [PATCH] Add notes about document formats

Refactor some other content to point to it
---
 IDEAS.org                |  7 +----
 INTERESTING_PROJECTS.org | 13 ++++----
 misc/document-formats.md | 65 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 71 insertions(+), 14 deletions(-)
 create mode 100644 misc/document-formats.md

diff --git a/IDEAS.org b/IDEAS.org
index 375c95b..bbf30a7 100644
--- a/IDEAS.org
+++ b/IDEAS.org
@@ -178,12 +178,7 @@ https://github.com/alexpdp7/ubpkg/ (also discusses some alternatives)
 
 * Extensible lightweight markup language
 
-- A language similar to AsciiDoc...
-- But with a well-defined AST
-- Complex nested lists
-- Styling of code blocks (highlighting, user input, replaceable...)
-- See https://github.com/jgm/djot / https://github.com/opendevise/asciidoc-parsing-lab/
-- Helper for "prose linters".
+See [[misc/document-formats.md]]
 
 * Touch controller/meeting controller/desktop shortcuts + calendar
 
diff --git a/INTERESTING_PROJECTS.org b/INTERESTING_PROJECTS.org
index bff9595..855f404 100644
--- a/INTERESTING_PROJECTS.org
+++ b/INTERESTING_PROJECTS.org
@@ -259,15 +259,12 @@ Also keeping things in a Org mode means I can classify and add notes.
    - https://github.com/Dakkaron/Fairberry - add Blackberry keyboard to other phones
 * Technical writing
 ** Document formats
-   - https://github.com/nota-lang/nota - Proper parser, Markdown, LaTeX...
-   - https://github.com/markdoc/markdoc
-   - https://github.com/christianvoigt/argdown - Argumentation
-   - https://github.com/jgm/djot - JGM designs a simpler Markdown, with an AST
-   - https://github.com/typst/typst - technically sound modern LaTeX replacement
-   - https://github.com/nvim-neorg - org-mode alternative, TreeSitter-grammar-first
-   - https://github.com/podlite/podlite/ - powerful lightweight markup language with styling in code blocks
+
+See [[misc/document-formats.md]].
+
 *** Markdown
-    - https://github.com/lezer-parser/markdown
+   - https://github.com/lezer-parser/markdown
+   - https://github.com/markdoc/markdoc
 **** Rust Parsers
     - https://github.com/kivikakk/comrak
     - https://github.com/raphlinus/pulldown-cmark
diff --git a/misc/document-formats.md b/misc/document-formats.md
new file mode 100644
index 0000000..132e337
--- /dev/null
+++ b/misc/document-formats.md
@@ -0,0 +1,65 @@
+# Document formats
+
+Most of the time, when writing a document, I want a document format with the following properties:
+
+* Fast to write using a plain text editor
+* Easy to parse into an AST
+
+An AST is a programming-friendly representation of a document.
+ASTs reduce the effort required to write tools such as a program that validates links in a document.
+Ideally, ASTs contain information to track a document element to the position it occupies in the original document.
+With this information, if you write a tool such as a spell checker, then you can highlight misspelled works precisely in the original document.
+
+On top of that, some features that I don't always need:
+
+* Math support
+* Sophisticated code blocks.
+  For example, being able to highlight arbitrary parts of code blocks (not syntax highlighting).
+* Diagram support
+
+## Existing formats
+
+### Markdown
+
+* Easy to write using a plain text editor
+* Has good AST parsers with position information
+* Has math support
+* Does not support sophisticated code blocks
+* There are many extensions with support for math, diagrams, and many others
+* Is very popular and supported everywhere
+* However, there is a wide variety of variants and quirks
+* Especifically, because Markdown was not designed with parsing in mind, so tools based on different parsers can have differences in behavior
+
+### Djot
+
+It is very similar to Markdown, except:
+
+* It is designed for parsing, so independent parsing implementations are very compatible with each other
+* It is not so popular, so there are less extension and tool support
+
+### AsciiDoc
+
+Compared to Markdown:
+
+* It's more complex to write, but mostly because it's different and more powerful
+* There are attempts to write better parsers, but good parsers with position information are not available yet
+* Supports sophisticated code blocks
+* It has a smaller ecosystem than Markdown, but many good quality tools such as Antora
+
+### Typst
+
+Checks all my boxes, except:
+
+* It is designed for parsing and it has an AST, but it is not easy to access
+* Currently Typst is very oriented towards generating paged documents (e.g. PDF)
+* It includes a full programming language, which is mostly good (very extensible), but this might increase complexity undesirably
+
+Typst is very new and is not yet very popular.
+
+### TODO: other formats
+
+- https://github.com/nota-lang/nota
+- https://github.com/christianvoigt/argdown
+- https://github.com/nvim-neorg
+- https://github.com/podlite/podlite/
+- https://orgmode.org/
-- 
2.47.3