Thoughts about Prejournal architecture

[michielbdejong]

Even though Prejournal is still very young, we use it as an R&D project so it has cycled through a few designs already.
Prejournal started as a collection of commands that can:

• 1: convert source documents to a REA-like representation of the world
• 2: generate bookkeeping journals from that representation

We soon added:

• 3: generate export documents in other formats
• 4: convert smaller bits of data (essentially one-liner source documents) to a REA-like representation of the world
• 5: persist the REA-like representation in a database.

With these additions, Prejournal can be run as a server instead of as a script.
The way we use Prejournal for our own bookkeeping at Ponder Source, we still basically run it as a script from source documents.
This sometimes clashes with the expectations of others who use https://time.pondersource.com as a server with an API.
For instance, each time we our source documents change, we regenerate the database contents from scratch, and this means items that were added from other sources may be lost, or items may be renumbered.

A third way to use Prejournal would be as an event-sourced hosted application, so that all edits get logged, and that log then becomes our source document store.

It's probably best if we refactor Prejournal to that architecture, but I'm still trying to think the consequences of that through properly, and working out what the best timing for that would be.

[michielbdejong]

One product that would be useful is a gateway that doesn't store anything. We could use it at Ponder Source to connect our books. We would still need something else to actually persist our books, but that can be just a CLOGS repo or a super simple web app around a database schema.

[michielbdejong]

We could also use multiple tracking and bookkeeping tools at the same time, all linked through the gateway.
Not sure whether 'Prejournal' would still be the best name to describe it. Maybe the software should also be called 'ConnectYourBooks'.

[michielbdejong]

And the gateway wouldn't do CLOGS because it's not storing anything.

[michielbdejong]

The network as a whole would take on the role of generic persistence with CLOGS and event sourcing

[michielbdejong]

We decided to split out https://github.com/pondersource/CYB from Prejournal to become the gateway,
and Prejournal will be a CLOGS-based bookkeeping system.

I'm thinking of using TigerBeetle for this, see also:

• This is getting slow #125
• balance invariants in TigerBeetle