Re: Org-Mode as DSL

[Tom Gillespie <tgbugs@gmail.com>]

Hi Mauro,
    Welcome! This is definitely not a crazy idea (or at least if it is
then you are in good company).
Org already has wide support as a markup language, and tools like
https://github.com/tecosaur/org-pandoc-import exist that leverage
this,
but the syntax is not standardized and as a result the elisp
implementation is the defacto standard.
Depending on what editors your colleagues use and the extent to which
features of Org they need to
use there might be an org-mode plugin that meets their needs (editing
Org as markup yes, using Org babel, no).
See the In Other Editors section of https://orgmode.org/install.html#org70be9a1.

Beyond its use as a markup language, there are efforts toward making
the more powerful parts of Org portable so that runtimes
beyond Emacs can correctly interpret and enable the dynamic nature of Org.

See for example this thread on formalizing org syntax
https://lists.gnu.org/archive/html/emacs-orgmode/2020-10/msg00344.html.
Parsers for org already exist in a variety of languages, but they vary
in their correctness and their support for org features.
In part this is because some of those features (such as footnotes)
look like they are easy to implement, but support syntactic variants
that parsers do not implement correctly or at all (e.g. the org-ruby
parser). Formalizing what it means to be a compliant parser could
help in those cases. There is also a need to specify some additional
underlying semantics about how to correctly construct trees
from the parsed elements and how to associate keywords to elements
etc. I am writing up an executable grammar for Org using Racket
and #lang brag that I will share in the mentioned thread.

Beyond surface syntax and basic semantics I see two key areas that
need attention in order for other full org implementations to be
viable.

The first obstacle is the fact that the semantics of org babel source
blocks are inconsistent and that it is hard to know what features a
particular ob-lang supports.
See https://lists.gnu.org/archive/html/emacs-orgmode/2020-10/msg00244.html
for a discussion of some of the challenges here.
I have been drafting an email on babel regularization for quite a
while, and it will show up on this list in the near future.

The second obstacle that I see is the fact that there are many places
where Org is deeply integrated with Emacs Lisp.
For example it is possible to include raw Emacs Lisp expressions in
certain areas of an Org file that will be evaluated
at a certain time (the time is not the same in all cases).
Supporting this functionality is something that other implementations
must be able to do
if we do not want Org files to be split into those that use elisp and
those that do not (somewhat like the current sorry state of markdown).
I have a couple of thoughts about how this could be addressed, but in
the absence of a portable implementation of elisp there would
be a divide between Org files that use elisp closures outside of src
blocks and those that do not.
In the event that other languages could someday be used in the top
level of an org file, it would be easier for the
reference elisp implementation of Org mode to run such files by
leveraging the ob-lang implementation for those languages.

A third area (of less immediate concern to me) that might need
standardization is the export backends.
Should different implementations of Org produce the same latex output?
Probably. How much trouble
would it be to specify? Probably quite a bit. So it might be easier to
just run Emacs behind the scenes
in those cases and not worry about trying to standardize.

Best!

Tom