Workload tracking improvement - emit event both at the beginning and end of each workflow operation

[ashvina]

LST-Bench workload comprises of various operations, including phase, task, and statement. Currently, for tracking purpose, the workload executor emits telemetry events at the end of each operation. The event payload indicates operation start and end times in addition to status of the operation execution. However, this approach poses a problem in error situations and also makes it harder to track progress till an operation completes. It becomes unclear whether the operation had even started or not. To address this issue, I propose enhancing our workload execution tracking by emitting an additional event at the beginning of each operation, in addition to the existing event at the end. Note, missing end event would indicate operation failure.

This approach partially borrows from popular event frameworks like XEvent in Sql Server. For e.g., SQL Server emits an xevent when SQL statement execution starts, and can emit an event when the statement completes or fails.

Another advantage of this change is its potential for extension to other components of the benchmark, going beyond just the workload tracking.

As such, each event will carry the following fields:

1. experiment id: experiment run identifier)
2. timestamp
3. event name (e.g. operation started, operation completed, operation failed, etc.)
4. event id (e.g. operation instance identifier. Start and end events of an operation carry the same id)
5. operation name (e.g. operation identifier from the workload config file)
6. event payload [optional]

This change will require changes to analysis scripts that depend on both start and end times.

[ashvina]

@jcamachor what do you think about the proposal?

[jcamachor]

Thanks for putting together this proposal, @ashvina ! I think it makes sense and heads in the right direction. As the project evolves, if more flexibility is needed, maybe we can create more complex extensions, that if I'm not mistaken is something that the XEvent framework also considers?
When it comes to the analysis scripts, I hope we can get testing for the metrics module shortly (#22), including some integration testing between core and metrics modules as well. That way we can catch any issues more easily moving forward.

CCing @anjagruenheid for awareness.