Add additional time types to Discovery

[jasonbreimer]

Data in this category is typically "found" or "discovered" after the actual event time. (Event time = time when thing actually happened). While an event time may be present we should add additional time value types to capture "discovered" times. This would allow maximum flexibility for producers and consumers. Additional optional time values would allow data producers, when possible, to add additional time context to discovered data. Consumers could then filter on time values, according to their use case, and execute more performant rules/queries.

Both existing classes (non-dev) have the following time types either directly or via base:
start_time - The start time of a time period, or the time of the least recent event included in the aggregate event.
time - The normalized event occurrence time.
timezone_offset - The number of minutes that the reported event time is ahead or behind UTC, in the range -1,080 to +1,080.

The following should be added as optional to both classes. This allows unique time values to be captured at creation, when it was first seen, when it was last seen, and when it was last modified. Not all producers will have all of these types, the should be optional.

Created Time created_time Optional Timestamp The time when the finding was created.
First Seen first_seen_time Optional Timestamp The time when the finding was first observed.
Last Seen last_seen_time Optional Timestamp The time when the finding was most recently observed.
Modified Time modified_time Optional Timestamp The time when the finding was last modified.

[floydtree]

Are the base descriptions of these 4 fields still applicable to the Discovery classes?

[jasonbreimer]

Yes, I would keep the existing descriptions for these 4 types. Its very similar to what I use currently in my own data.

I'm also easy going if there's a more accurate description.

[pagbabian-splunk]

As I added them to the Device object in the PR, I changed the descriptions to be about the device. If we discovery Users in the future, we'll do the same. I noticed that per JP's comment, most of the extra time attributes are in objects.

[jp-harvey]

@jasonbreimer these are very useful fields and I think they should be added.

Recently we were discussing the structure and design of OCSF in a different context, and talked about how the Objects were time-independent "entities" that had properties that defined them, and the Classes were a record in time of an event associated with one or more Objects.

In that context, could it make sense to add them to the base class of object instead of to a single class? An attribute first_seen_time on an object is semantically "the first time this object was seen" where the first_seen_time attribute on an event is semantically "the first time this event was seen". (We would be having a description to clarify if we did it the second way of course, just illustrating)

Thoughts?

[pagbabian-splunk]

I will create a PR for this. I'll add them to the Device object (see discussion below).

One thing: I wouldn't use the term 'finding' specifically even though it is a finding of sorts, so as to not confuse with the Findings category, which is more of an analytic category. These timestamps are very much about the discovery process and likely make sense in the base discovery.json superclass.

To JP's point, sometimes the object itself can be timestamped if it were to persist after the event was captured. These two Discovery classes are both about Device and having the timestamps in the event classes would necessarily affect the device as if the timestamps were 'in' the Device object.

If you look at the uses of first_seen_time or expiration_time etc. many of these are in objects as well as events, so we should be free to add to the object (in this case Device).

There is a caveat though: at the time of the event (i.e. in the class) presumably that would be the last_seen_time and the object, let's say it's stashed away somewhere after discovery, would maintain its first_seen_time but if it had a last_seen_time, that value would need to be updated on every discovery run.

Hence, capturing the envelope on every discovery run, necessary for the last_seen_time, may just as well capture both. Also, if we put these two (or at least first_seen_time) in object.json or entity.json, it often would be redundant with the event time, since most events don't come as a repeated event due to a discovery pass.

start_time and end_time are meant for batch transfers.

Therefore, to be consistent with how we've done other objects, I suggest adding the timestamps to the Device object, rather than every object, as many of those other objects have more specific timestamps (e.g. last_run_time for a Job object). I could see in the future we might add to other entity objects though.

(I could see in the future these 4 timestamps could be an object that applies to many entities, but likely not all entities. At the moment the PR has them separate, per Jason, but in the Device, per JP.

[pagbabian-splunk]

These timestamps have been added to the Device object in RC3. Closing the issue.