Track transaction progress

[hmottestad]

I've had some requests for tracking the progress of SHACL Validation. And I easily see a similar need for tracking progress while loading data or while querying data.

With the new(ish) transaction settings support I was thinking that we could allow users to pass in their own identifier when starting a transaction that we could use to emit events from various sails during transaction processing.

Something like con.begin(TransactionID.of(19473826492L)); to pass an identifier (as a long) to the transaction. Then we can listen for events by registering a handler TransactionEvents.register(TransactionID.of(19473826492L), event -> System.out.println(event));. We would naturally require that the transaction ID was registered before the transaction starts.

When loading data the NativeStore would emit events like new WriteProgressEvent(100000) to indicate that 100'000 statements have been read. I feel it makes more sense if the events are stateless, eg. emitting an event for every 100'000 statements rather than emitting one with the accumulated progress. But I'm not certain about this. Might want to look at how other event based systems work.

Another option would be to add some method to the base connection interface to allow a user to attach an event handler directly to the connection. This would be a more invasive change though. And we would need to make sure that the event handler is passed down the sail stack, something that is already implemented with the transaction settings code.

Some important aspects to consider are:

• what are others doing at the moment (eg. GraphDB)
• how to allow for slow adoption and implementation so that it doesn't matter if a certain sail in your stack doesn't support emitting events
• performance overhead
• error handling and lost transactions