diff --git a/index.html b/index.html index ad67ad5c..d5f33fac 100644 --- a/index.html +++ b/index.html @@ -92,6 +92,24 @@

Preview for branch clarify-connection-st diff with main +

Preview for branch security-stuff

+ + + + + + + + + + + + + + + + +
HTTP/3 qlog event definitionsplain textsame as main
Main logging schema for qlogplain textdiff with main
QUIC event definitions for qlogplain textsame as main

Preview for branch clarify-close-trigger

diff --git a/security-stuff/draft-ietf-quic-qlog-h3-events.html b/security-stuff/draft-ietf-quic-qlog-h3-events.html new file mode 100644 index 00000000..6e88baa6 --- /dev/null +++ b/security-stuff/draft-ietf-quic-qlog-h3-events.html @@ -0,0 +1,2691 @@ + + + + + + +HTTP/3 qlog event definitions + + + + + + + + + + + + + + + +
+ + + + + + + + + + +
Internet-DraftHTTP/3 qlog event definitionsJune 2024
Marx, et al.Expires 29 December 2024[Page]
+
+
+
+
Workgroup:
+
QUIC
+
Internet-Draft:
+
draft-ietf-quic-qlog-h3-events-latest
+
Published:
+
+ +
+
Intended Status:
+
Standards Track
+
Expires:
+
+
Authors:
+
+
+
R. Marx, Ed. +
+
Akamai
+
+
+
L. Niccolini, Ed. +
+
Meta
+
+
+
M. Seemann, Ed. +
+
+
+
L. Pardue, Ed. +
+
Cloudflare
+
+
+
+
+

HTTP/3 qlog event definitions

+
+

Abstract

+

This document describes concrete qlog event definitions and their metadata for +HTTP/3-related events. These events can then be embedded in the higher +level schema defined in [QLOG-MAIN].

+
+
+

+Note to Readers +

+ +

Feedback and discussion are welcome at +https://github.com/quicwg/qlog. Readers are +advised to refer to the "editor's draft" at that URL for an up-to-date version +of this document.

+

Concrete examples of integrations of this schema in +various programming languages can be found at +https://github.com/quiclog/qlog/.

+
+
+
+

+Status of This Memo +

+

+ This Internet-Draft is submitted in full conformance with the + provisions of BCP 78 and BCP 79.

+

+ Internet-Drafts are working documents of the Internet Engineering Task + Force (IETF). Note that other groups may also distribute working + documents as Internet-Drafts. The list of current Internet-Drafts is + at https://datatracker.ietf.org/drafts/current/.

+

+ Internet-Drafts are draft documents valid for a maximum of six months + and may be updated, replaced, or obsoleted by other documents at any + time. It is inappropriate to use Internet-Drafts as reference + material or to cite them other than as "work in progress."

+

+ This Internet-Draft will expire on 29 December 2024.

+
+
+ +
+
+ ▲

+Table of Contents +

+ +
+
+
+
+

+1. Introduction +

+

This document describes the values of the qlog name ("category" + "event") and +"data" fields and their semantics for the HTTP/3 protocol [HTTP/3] and some +of extensions (see [EXTENDED-CONNECT], [H3_PRIORITIZATION] +and [H3-DATAGRAM]).

+
+
+

+1.1. Notational Conventions +

+

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", +"MAY", and "OPTIONAL" in this document are to be interpreted as +described in BCP 14 [RFC2119] [RFC8174] when, and only when, they +appear in all capitals, as shown here.

+

The event and data structure definitions in ths document are expressed +in the Concise Data Definition Language [CDDL] and its +extensions described in [QLOG-MAIN].

+

The following fields from [QLOG-MAIN] are imported and used: name, category, +type, data, group_id, protocol_type, importance, RawInfo, and time-related +fields.

+

As is the case for [QLOG-MAIN], the qlog schema definitions in this document +are intentionally agnostic to serialization formats. The choice of format is an +implementation decision.

+
+
+
+
+
+
+

+2. Overview +

+

This document describes how HTTP/3 can be expressed in qlog using the schema +defined in [QLOG-MAIN]. HTTP/3 events are defined with a category, a name (the +concatenation of "category" and "event"), an "importance", an optional +"trigger", and "data" fields.

+

Some data fields use complex datastructures. These are represented as enums or +re-usable definitions, which are grouped together on the bottom of this document +for clarity.

+

When any event from this document is included in a qlog trace, the +"protocol_type" qlog array field MUST contain an entry with the value "HTTP3":

+
+
+
+
+$ProtocolType /= "HTTP3"
+
+
+
Figure 1: +ProtocolType extension for HTTP/3 +
+
+
+
+

+2.1. Usage with QUIC +

+

The events described in this document can be used with or without logging the +related QUIC events defined in [QLOG-QUIC]. If used with QUIC events, the QUIC +document takes precedence in terms of recommended filenames and trace separation +setups.

+

If used without QUIC events, it is recommended that the implementation assign a +globally unique identifier to each HTTP/3 connection. This ID can then be used as +the value of the qlog "group_id" field, as well as the qlog filename or file +identifier, potentially suffixed by the vantagepoint type (For example, +abcd1234_server.qlog would contain the server-side trace of the connection with +GUID abcd1234).

+
+
+
+
+
+
+

+3. HTTP/3 Event Overview +

+

This document defines events in two categories, written as lowercase to follow +convention: h3 (Section 4).

+

As described in Section 3.4.2 of [QLOG-MAIN], the qlog "name" field is the +concatenation of category and type.

+

Table 1 summarizes the name value of each event type that is defined +in this specification.

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+Table 1: +HTTP/3 Events +
Name valueImportanceDefinition
h3:parameters_setBase + Section 4.1 +
h3:parameters_restoredBase + Section 4.2 +
h3:stream_type_setBase + Section 4.3 +
h3:priority_updatedBase + Section 4.4 +
h3:frame_createdCore + Section 4.5 +
h3:frame_parsedCore + Section 4.6 +
h3:datagram_createdBase + Section 4.7 +
h3:datagram_parsedBase + Section 4.8 +
h3:push_resolvedExtra + Section 4.9 +
+
+
+
+
+
+

+4. HTTP/3 Events +

+

HTTP/3 events extend the $ProtocolEventData extension point defined in +[QLOG-MAIN]. Additionally, they allow for direct extensibility by their use of +per-event extension points via the $$ CDDL "group socket" syntax, as also +described in [QLOG-MAIN].

+
+
+
+
+H3EventData = H3ParametersSet /
+              H3ParametersRestored /
+              H3StreamTypeSet /
+              H3PriorityUpdated /
+              H3FrameCreated /
+              H3FrameParsed /
+              H3DatagramCreated /
+              H3DatagramParsed /
+              H3PushResolved
+
+$ProtocolEventData /= H3EventData
+
+
+
Figure 2: +H3EventData definition and ProtocolEventData extension +
+
+

HTTP events are logged when a certain condition happens at the application +layer, and there isn't always a one to one mapping between HTTP and QUIC events. +The exchange of data between the HTTP and QUIC layer is logged via the +"stream_data_moved" and "datagram_data_moved" events in [QLOG-QUIC].

+

HTTP/3 frames are transmitted on QUIC streams, which allows them to span +multiple QUIC packets. Some implementations might send a single large frame, +rather than a sequence of smaller frames, in order to amortize frame header +overhead. HTTP/3 frame headers are represented by the frame_created +(Section 4.5) and frame_parsed (Section 4.6) events. Subsequent +frame payload data transfer is indicated by stream_data_moved events. +Furthermore, stream_data_moved events can appear before frame_parsed events +because implementations need to read data from a stream in order to parse the +frame header.

+
+
+

+4.1. parameters_set +

+

The parameters_set event contains HTTP/3 and QPACK-level settings, mostly +those received from the HTTP/3 SETTINGS frame. It has Base importance level; see +Section 9.2 of [QLOG-MAIN].

+

All these parameters are typically set once and never change. However, they +might be set at different times during the connection, therefore a qlog can have +multiple instances of parameters_set with different fields set.

+

The "owner" field reflects how Settings are exchanged on a connection. Sent +settings have the value "local" and received settings have the value +"received".

+
+
+
+
+H3ParametersSet = {
+    ? owner: Owner
+
+    ; RFC9114
+    ? max_field_section_size: uint64
+
+    ; RFC9204
+    ? max_table_capacity: uint64
+    ? blocked_streams_count: uint64
+
+    ; RFC9220 (SETTINGS_ENABLE_CONNECT_PROTOCOL)
+    ? extended_connect: uint16
+
+    ; RFC9297 (SETTINGS_H3_DATAGRAM)
+    ? h3_datagram: uint16
+
+    ; qlog-specific
+    ; indicates whether this implementation waits for a SETTINGS
+    ; frame before processing requests
+    ? waits_for_settings: bool
+
+    * $$h3-parametersset-extension
+}
+
+
+
Figure 3: +H3ParametersSet definition +
+
+

The parameters_set event can contain any number of unspecified fields. This +allows for representation of reserved settings (aka GREASE) or ad-hoc support +for extension settings that do not have a related qlog schema definition.

+
+
+
+
+

+4.2. parameters_restored +

+

When using QUIC 0-RTT, HTTP/3 clients are expected to remember and reuse the +server's SETTINGs from the previous connection. The parameters_restored event +is used to indicate which HTTP/3 settings were restored and to which values when +utilizing 0-RTT. It has Base importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+H3ParametersRestored = {
+    ; RFC9114
+    ? max_field_section_size: uint64
+
+    ; RFC9204
+    ? max_table_capacity: uint64
+    ? blocked_streams_count: uint64
+
+    ; RFC9220 (SETTINGS_ENABLE_CONNECT_PROTOCOL)
+    ? extended_connect: uint16
+
+    ; RFC9297 (SETTINGS_H3_DATAGRAM)
+    ? h3_datagram: uint16
+
+    * $$h3-parametersrestored-extension
+}
+
+
+
Figure 4: +H3ParametersRestored definition +
+
+
+
+
+
+

+4.3. stream_type_set +

+

The stream_type_set event conveys when a HTTP/3 stream type becomes known; see +Sections 6.1 and 6.2 of [HTTP/3]. It has Base importance level; see Section 9.2 of [QLOG-MAIN].

+

Client bidirectional streams always have a stream_type value of "request". +Server bidirectional streams have no defined use, although extensions could +change that.

+

Unidirectional streams in either direction begin with with a variable-length +integer type. Where the type is not known, the stream_type value of "unknown" +type can be used and the value captured in the stream_type_bytes field; a +numerical value without variable-length integer encoding.

+

The generic $H3StreamType is defined here as a CDDL "type socket" extension +point. It can be extended to support additional HTTP/3 stream types.

+
+
+
+
+H3StreamTypeSet = {
+    ? owner: Owner
+    stream_id: uint64
+    stream_type: $H3StreamType
+
+    ; only when stream_type === "unknown"
+    ? stream_type_bytes: uint64
+
+    ; only when stream_type === "push"
+    ? associated_push_id: uint64
+
+    * $$h3-streamtypeset-extension
+}
+
+$H3StreamType /=  "request" /
+                  "control" /
+                  "push" /
+                  "reserved" /
+                  "unknown" /
+                  "qpack_encode" /
+                  "qpack_decode"
+
+
+
Figure 5: +H3StreamTypeSet definition +
+
+
+
+
+
+

+4.4. priority_updated +

+

Emitted when the priority of a request stream or push stream is initialized or +updated through mechanisms defined in [RFC9218]. For example, the priority +can be updated through signals received from client and/or server (e.g., in +HTTP/3 HEADERS or PRIORITY_UPDATE frames) or it can be changed or overridden due +to local policies. The event has Base importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+H3PriorityUpdated = {
+    ; if the prioritized element is a request stream
+    ? stream_id: uint64
+
+    ; if the prioritized element is a push stream
+    ? push_id: uint64
+
+    ? old: H3Priority
+    new: H3Priority
+
+    * $$h3-priorityupdated-extension
+}
+
+
+
Figure 6: +H3PriorityUpdated definition +
+
+
+
+
+
+

+4.5. frame_created +

+

The frame_created event is emitted when the HTTP/3 framing actually happens. +It has Core importance level; see Section 9.2 of [QLOG-MAIN].

+

This event does not necessarily coincide with HTTP/3 data getting passed to the +QUIC layer. For that, see the stream_data_moved event in [QLOG-QUIC].

+
+
+
+
+H3FrameCreated = {
+    stream_id: uint64
+    ? length: uint64
+    frame: $H3Frame
+    ? raw: RawInfo
+
+    * $$h3-framecreated-extension
+}
+
+
+
Figure 7: +H3FrameCreated definition +
+
+
+
+
+
+

+4.6. frame_parsed +

+

The frame_parsed event is emitted when the HTTP/3 frame is parsed. It has Core +importance level; see Section 9.2 of [QLOG-MAIN].

+

This event is not necessarily the same as when the HTTP/3 data is actually +received on the QUIC layer. For that, see the stream_data_moved event in +[QLOG-QUIC].

+
+
+
+
+H3FrameParsed = {
+    stream_id: uint64
+    ? length: uint64
+    frame: $H3Frame
+    ? raw: RawInfo
+
+    * $$h3-frameparsed-extension
+}
+
+
+
Figure 8: +H3FrameParsed definition +
+
+
+
+
+
+

+4.7. datagram_created +

+

The datagram_created event is emitted when an HTTP/3 Datagram is created (see +[RFC9297]). It has Base importance level; see Section 9.2 of [QLOG-MAIN].

+

This event does not necessarily coincide with the HTTP/3 Datagram getting passed +to the QUIC layer. For that, see the datagram_data_moved event in +[QLOG-QUIC].

+
+
+
+
+H3DatagramCreated = {
+    quarter_stream_id: uint64
+    ? datagram: $H3Datagram
+    ? raw: RawInfo
+
+    * $$h3-datagramcreated-extension
+}
+
+
+
Figure 9: +H3DatagramCreated definition +
+
+
+
+
+
+

+4.8. datagram_parsed +

+

The datagram_parsed event is emitted when the HTTP/3 Datagram is parsed (see +[RFC9297]). It has Base importance level; see Section 9.2 of [QLOG-MAIN].

+

This event is not necessarily the same as when the HTTP/3 Datagram is actually +received on the QUIC layer. For that, see the datagram_data_moved event in +[QLOG-QUIC].

+
+
+
+
+H3DatagramParsed = {
+    quarter_stream_id: uint64
+    ? datagram: $H3Datagram
+    ? raw: RawInfo
+
+    * $$h3-datagramparsed-extension
+}
+
+
+
Figure 10: +H3DatagramParsed definition +
+
+
+
+
+
+

+4.9. push_resolved +

+

The push_resolved event is emitted when a pushed resource (Section 4.6 of [HTTP/3]) is successfully claimed (used) or, conversely, abandoned (rejected) +by the application on top of HTTP/3 (e.g., the web browser). This event provides +additional context that can is aid debugging issues related to server push. It +has Extra importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+H3PushResolved = {
+    ? push_id: uint64
+
+    ; in case this is logged from a place that does not have access
+    ; to the push_id
+    ? stream_id: uint64
+    decision: H3PushDecision
+
+    * $$h3-pushresolved-extension
+}
+
+H3PushDecision = "claimed" /
+                 "abandoned"
+
+
+
Figure 11: +H3PushResolved definition +
+
+
+
+
+
+
+
+

+5. HTTP/3 Data Field Definitions +

+

The following data field definitions can be used in HTTP/3 events.

+
+
+

+5.1. Owner +

+
+
+
+
+Owner = "local" /
+        "remote"
+
+
+
Figure 12: +Owner definition +
+
+
+
+
+
+

+5.2. H3Frame +

+

The generic $H3Frame is defined here as a CDDL "type socket" extension point. +It can be extended to support additional HTTP/3 frame types.

+
+
+
+
+; The H3Frame is any key-value map (e.g., JSON object)
+$H3Frame /= {
+    * text => any
+}
+
+
+
Figure 13: +H3Frame type socket definition +
+
+

The HTTP/3 frame types defined in this document are as follows:

+
+
+
+
+H3BaseFrames = H3DataFrame /
+               H3HeadersFrame /
+               H3CancelPushFrame /
+               H3SettingsFrame /
+               H3PushPromiseFrame /
+               H3GoawayFrame /
+               H3MaxPushIDFrame /
+               H3ReservedFrame /
+               H3UnknownFrame
+
+$H3Frame /= H3BaseFrames
+
+
+
Figure 14: +H3BaseFrames definition +
+
+
+
+
+
+

+5.3. H3Datagram +

+

The generic $H3Datagram is defined here as a CDDL "type socket" extension +point. It can be extended to support additional HTTP/3 datagram types. This +document intentionally does not define any specific qlog schemas for specific +HTTP/3 Datagram types.

+
+
+
+
+; The H3Datagram is any key-value map (e.g., JSON object)
+$H3Datagram /= {
+    * text => any
+}
+
+
+
Figure 15: +H3Datagram type socket definition +
+
+
+
+

+5.3.1. H3DataFrame +

+
+
+
+
+H3DataFrame = {
+    frame_type: "data"
+    ? raw: RawInfo
+}
+
+
+
Figure 16: +H3DataFrame definition +
+
+
+
+
+
+

+5.3.2. H3HeadersFrame +

+

This represents an uncompressed, plaintext HTTP Headers frame (e.g., no QPACK +compression is applied).

+

For example:

+
+
+
+
+headers: [
+  {
+    "name": ":path",
+    "value": "/"
+  },
+  {
+    "name": ":method",
+    "value": "GET"
+  },
+  {
+    "name": ":authority",
+    "value": "127.0.0.1:4433"
+  },
+  {
+    "name": ":scheme",
+    "value": "https"
+  }
+]
+
+
+
Figure 17: +H3HeadersFrame example +
+
+
+
+
+
+H3HeadersFrame = {
+    frame_type: "headers"
+    headers: [* H3HTTPField]
+}
+
+
+
Figure 18: +H3HeadersFrame definition +
+
+
+
+
+
+H3HTTPField = {
+    name: text
+    ? value: text
+}
+
+
+
Figure 19: +H3HTTPField definition +
+
+
+
+
+
+

+5.3.3. H3CancelPushFrame +

+
+
+
+
+H3CancelPushFrame = {
+    frame_type: "cancel_push"
+    push_id: uint64
+}
+
+
+
Figure 20: +H3CancelPushFrame definition +
+
+
+
+
+
+

+5.3.4. H3SettingsFrame +

+

The settings field can contain zero or more entries. Each setting has a name +field, which corresponds to Setting Name as defined (or as would be defined if +registered) in the "HTTP/3 Settings" registry maintained at +https://www.iana.org/assignments/http3-parameters.

+

An endpoint that receives unknown settings is not able to log a specific name. +Instead, the name value of "unknown" can be used and the value captured in the +name_bytes field; a numerical value without variable-length integer encoding.

+
+
+
+
+H3SettingsFrame = {
+    frame_type: "settings"
+    settings: [* H3Setting]
+}
+
+H3Setting = {
+    ? name: $H3SettingsName
+    ; only when name === "unknown"
+    ? name_bytes: uint64
+
+    value: uint64
+}
+
+$H3SettingsName /= "settings_qpack_max_table_capacity" /
+                   "settings_max_field_section_size" /
+                   "settings_qpack_blocked_streams" /
+                   "settings_enable_connect_protocol" /
+                   "settings_h3_datagram" /
+                   "reserved" /
+                   "unknown"
+
+
+
Figure 21: +H3SettingsFrame definition +
+
+
+
+
+
+

+5.3.5. H3PushPromiseFrame +

+
+
+
+
+H3PushPromiseFrame = {
+    frame_type: "push_promise"
+    push_id: uint64
+    headers: [* H3HTTPField]
+}
+
+
+
Figure 22: +H3PushPromiseFrame definition +
+
+
+
+
+
+

+5.3.6. H3GoAwayFrame +

+
+
+
+
+H3GoawayFrame = {
+    frame_type: "goaway"
+
+    ; Either stream_id or push_id.
+    ; This is implicit from the sender of the frame
+    id: uint64
+}
+
+
+
Figure 23: +H3GoawayFrame definition +
+
+
+
+
+
+

+5.3.7. H3MaxPushIDFrame +

+
+
+
+
+H3MaxPushIDFrame = {
+    frame_type: "max_push_id"
+    push_id: uint64
+}
+
+
+
Figure 24: +H3MaxPushIDFrame definition +
+
+
+
+
+
+

+5.3.8. H3PriorityUpdateFrame +

+

The PRIORITY_UPDATE frame is defined in [RFC9218].

+
+
+
+
+H3PriorityUpdateFrame = {
+    frame_type: "priority_update"
+
+    ; if the prioritized element is a request stream
+    ? stream_id: uint64
+
+    ; if the prioritized element is a push stream
+    ? push_id: uint64
+
+    priority_field_value: H3Priority
+}
+
+; The priority value in ASCII text, encoded using Structured Fields
+; Example: u=5, i
+H3Priority = text
+
+
+
Figure 25: +h3priorityupdateframe definition +
+
+
+
+
+
+

+5.3.9. H3ReservedFrame +

+
+
+
+
+H3ReservedFrame = {
+    frame_type: "reserved"
+    ? length: uint64
+}
+
+
+
Figure 26: +H3ReservedFrame definition +
+
+
+
+
+
+

+5.3.10. H3UnknownFrame +

+

The frame_type_bytes field is the numerical value without variable-length +integer encoding.

+
+
+
+
+H3UnknownFrame = {
+    frame_type: "unknown"
+    frame_type_bytes: uint64
+    ? raw: RawInfo
+}
+
+
+
Figure 27: +UnknownFrame definition +
+
+
+
+
+
+

+5.3.11. H3ApplicationError +

+
+
+
+
+H3ApplicationError = "http_no_error" /
+                       "http_general_protocol_error" /
+                       "http_internal_error" /
+                       "http_stream_creation_error" /
+                       "http_closed_critical_stream" /
+                       "http_frame_unexpected" /
+                       "http_frame_error" /
+                       "http_excessive_load" /
+                       "http_id_error" /
+                       "http_settings_error" /
+                       "http_missing_settings" /
+                       "http_request_rejected" /
+                       "http_request_cancelled" /
+                       "http_request_incomplete" /
+                       "http_early_response" /
+                       "http_connect_error" /
+                       "http_version_fallback"
+
+
+
Figure 28: +H3ApplicationError definition +
+
+

The H3ApplicationError extends the general $ApplicationError +definition in the qlog QUIC document, see [QLOG-QUIC].

+
+
+; ensure HTTP errors are properly validated in QUIC events as well
+; e.g., QUIC's ConnectionClose Frame
+$ApplicationError /= H3ApplicationError
+
+
+
+
+
+
+
+
+
+
+

+6. Security and Privacy Considerations +

+

The security and privacy considerations discussed in [QLOG-MAIN] apply to this +document as well.

+
+
+
+
+

+7. IANA Considerations +

+

There are no IANA considerations.

+
+
+
+
+

+8. Normative References +

+
+
[CDDL]
+
+Birkholz, H., Vigano, C., and C. Bormann, "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, , <https://www.rfc-editor.org/rfc/rfc8610>.
+
+
[EXTENDED-CONNECT]
+
+Hamilton, R., "Bootstrapping WebSockets with HTTP/3", RFC 9220, DOI 10.17487/RFC9220, , <https://www.rfc-editor.org/rfc/rfc9220>.
+
+
[H3-DATAGRAM]
+
+Schinazi, D. and L. Pardue, "HTTP Datagrams and the Capsule Protocol", RFC 9297, DOI 10.17487/RFC9297, , <https://www.rfc-editor.org/rfc/rfc9297>.
+
+
[H3_PRIORITIZATION]
+
+Oku, K. and L. Pardue, "Extensible Prioritization Scheme for HTTP", RFC 9218, DOI 10.17487/RFC9218, , <https://www.rfc-editor.org/rfc/rfc9218>.
+
+
[HTTP/3]
+
+Bishop, M., Ed., "HTTP/3", RFC 9114, DOI 10.17487/RFC9114, , <https://www.rfc-editor.org/rfc/rfc9114>.
+
+
[QLOG-MAIN]
+
+Marx, R., Niccolini, L., Seemann, M., and L. Pardue, "Main logging schema for qlog", Work in Progress, Internet-Draft, draft-ietf-quic-qlog-main-schema-08, , <https://datatracker.ietf.org/doc/html/draft-ietf-quic-qlog-main-schema-08>.
+
+
[QLOG-QUIC]
+
+Marx, R., Niccolini, L., Seemann, M., and L. Pardue, "QUIC event definitions for qlog", Work in Progress, Internet-Draft, draft-ietf-quic-qlog-quic-events-07, , <https://datatracker.ietf.org/doc/html/draft-ietf-quic-qlog-quic-events-07>.
+
+
[RFC2119]
+
+Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/rfc/rfc2119>.
+
+
[RFC8174]
+
+Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/rfc/rfc8174>.
+
+
[RFC9218]
+
+Oku, K. and L. Pardue, "Extensible Prioritization Scheme for HTTP", RFC 9218, DOI 10.17487/RFC9218, , <https://www.rfc-editor.org/rfc/rfc9218>.
+
+
[RFC9297]
+
+Schinazi, D. and L. Pardue, "HTTP Datagrams and the Capsule Protocol", RFC 9297, DOI 10.17487/RFC9297, , <https://www.rfc-editor.org/rfc/rfc9297>.
+
+
+
+
+
+
+

+Acknowledgements +

+

Much of the initial work by Robin Marx was done at the Hasselt and KU Leuven +Universities.

+

Thanks to Jana Iyengar, Brian Trammell, Dmitri Tikhonov, Stephen Petrides, Jari +Arkko, Marcus Ihlar, Victor Vasiliev, Mirja Kühlewind, Jeremy Lainé, Kazu +Yamamoto, Christian Huitema and Hugo Landau for their feedback and suggestions.

+
+
+
+
+

+Change Log +

+

This section is to be removed before publishing as an RFC.

+
+
+

+Since draft-ietf-quic-qlog-h3-events-06: +

+
    +
  • +

    ProtocolEventBody is now called ProtocolEventData (#352)

    +
    • +
  • +

    Editorial changes (#402)

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-h3-events-05: +

+
    +
  • +

    Removed all qpack event definitions (#335)

    +
    • +
  • +

    Various editorial changes

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-h3-events-04: +

+
    +
  • +

    Renamed 'http' category to 'h3' (#300)

    +
    • +
  • +

    H3HTTPField.value is now optional (#296)

    +
    • +
  • +

    Added definitions for RFC9297 (HTTP/3 Datagram extension) (#310)

    +
    • +
  • +

    Added definitions for RFC9218 (HTTP Extensible Prioritizations extension) (#312)

    +
    • +
  • +

    Added definitions for RFC9220 (Extended Connect extension) (#325)

    +
    • +
  • +

    Editorial and formatting changes (#298, #258, #299, #304, #327)

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-h3-events-03: +

+
    +
  • +

    Ensured consistent use of RawInfo to indicate raw wire bytes (#243)

    +
    • +
  • +

    Changed HTTPStreamTypeSet:raw_stream_type to stream_type_value (#54)

    +
    • +
  • +

    Changed HTTPUnknownFrame:raw_frame_type to frame_type_value (#54)

    +
    • +
  • +

    Renamed max_header_list_size to max_field_section_size (#282)

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-h3-events-02: +

+
    +
  • +

    Renamed HTTPStreamType data to request (#222)

    +
    • +
  • +

    Added HTTPStreamType value unknown (#227)

    +
    • +
  • +

    Added HTTPUnknownFrame (#224)

    +
    • +
  • +

    Replaced old and new fields with stream_type in HTTPStreamTypeSet (#240)

    +
    • +
  • +

    Changed HTTPFrame to a CDDL plug type (#257)

    +
    • +
  • +

    Moved data definitions out of the appendix into separate sections

    +
    • +
  • +

    Added overview Table of Contents

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-h3-events-01: +

+
    +
  • +

    No changes - new draft to prevent expiration

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-h3-events-00: +

+
    +
  • +

    Change the data definition language from TypeScript to CDDL (#143)

    +
    • +
+
+
+
+
+

+Since draft-marx-qlog-event-definitions-quic-h3-02: +

+
    +
  • +

    These changes were done in preparation of the adoption of the drafts by the QUIC +working group (#137)

    +
    • +
  • +

    Split QUIC and HTTP/3 events into two separate documents

    +
    • +
  • +

    Moved RawInfo, Importance, Generic events and Simulation events to the main +schema document.

    +
    • +
+
+
+
+
+

+Since draft-marx-qlog-event-definitions-quic-h3-01: +

+

Major changes:

+
    +
  • +

    Moved data_moved from http to transport. Also made the "from" and "to" fields +flexible strings instead of an enum (#111,#65)

    +
    • +
  • +

    Moved packet_type fields to PacketHeader. Moved packet_size field out of +PacketHeader to RawInfo:length (#40)

    +
    • +
  • +

    Made events that need to log packet_type and packet_number use a header field +instead of logging these fields individually

    +
    • +
  • +

    Added support for logging retry, stateless reset and initial tokens (#94,#86,#117)

    +
    • +
  • +

    Moved separate general event categories into a single category "generic" (#47)

    +
    • +
  • +

    Added "transport:connection_closed" event (#43,#85,#78,#49)

    +
    • +
  • +

    Added version_information and alpn_information events (#85,#75,#28)

    +
    • +
  • +

    Added parameters_restored events to help clarify 0-RTT behaviour (#88)

    +
    • +
+

Smaller changes:

+
    +
  • +

    Merged loss_timer events into one loss_timer_updated event

    +
    • +
  • +

    Field data types are now strongly defined (#10,#39,#36,#115)

    +
    • +
  • +

    Renamed qpack instruction_received and instruction_sent to instruction_created +and instruction_parsed (#114)

    +
    • +
  • +

    Updated qpack:dynamic_table_updated.update_type. It now has the value "inserted" +instead of "added" (#113)

    +
    • +
  • +

    Updated qpack:dynamic_table_updated. It now has an "owner" field to +differentiate encoder vs decoder state (#112)

    +
    • +
  • +

    Removed push_allowed from http:parameters_set (#110)

    +
    • +
  • +

    Removed explicit trigger field indications from events, since this was moved to +be a generic property of the "data" field (#80)

    +
    • +
  • +

    Updated transport:connection_id_updated to be more in line with other similar +events. Also dropped importance from Core to Base (#45)

    +
    • +
  • +

    Added length property to PaddingFrame (#34)

    +
    • +
  • +

    Added packet_number field to transport:frames_processed (#74)

    +
    • +
  • +

    Added a way to generically log packet header flags (first 8 bits) to +PacketHeader

    +
    • +
  • +

    Added additional guidance on which events to log in which situations (#53)

    +
    • +
  • +

    Added "simulation:scenario" event to help indicate simulation details

    +
    • +
  • +

    Added "packets_acked" event (#107)

    +
    • +
  • +

    Added "datagram_ids" to the datagram_X and packet_X events to allow tracking of +coalesced QUIC packets (#91)

    +
    • +
  • +

    Extended connection_state_updated with more fine-grained states (#49)

    +
    • +
+
+
+
+
+

+Since draft-marx-qlog-event-definitions-quic-h3-00: +

+
    +
  • +

    Event and category names are now all lowercase

    +
    • +
  • +

    Added many new events and their definitions

    +
    • +
  • +

    "type" fields have been made more specific (especially important for PacketType +fields, which are now called packet_type instead of type)

    +
    • +
  • +

    Events are given an importance indicator (issue #22)

    +
    • +
  • +

    Event names are more consistent and use past tense (issue #21)

    +
    • +
  • +

    Triggers have been redefined as properties of the "data" field and updated for most events (issue #23)

    +
    • +
+
+
+
+
+
+
+

+Authors' Addresses +

+
+
Robin Marx (editor)
+
Akamai
+ +
+
+
Luca Niccolini (editor)
+
Meta
+ +
+
+
Marten Seemann (editor)
+ +
+
+
Lucas Pardue (editor)
+
Cloudflare
+ +
+
+
+ + + diff --git a/security-stuff/draft-ietf-quic-qlog-h3-events.txt b/security-stuff/draft-ietf-quic-qlog-h3-events.txt new file mode 100644 index 00000000..9a634409 --- /dev/null +++ b/security-stuff/draft-ietf-quic-qlog-h3-events.txt @@ -0,0 +1,1013 @@ + + + + +QUIC R. Marx, Ed. +Internet-Draft Akamai +Intended status: Standards Track L. Niccolini, Ed. +Expires: 29 December 2024 Meta + M. Seemann, Ed. + + L. Pardue, Ed. + Cloudflare + 27 June 2024 + + + HTTP/3 qlog event definitions + draft-ietf-quic-qlog-h3-events-latest + +Abstract + + This document describes concrete qlog event definitions and their + metadata for HTTP/3-related events. These events can then be + embedded in the higher level schema defined in [QLOG-MAIN]. + +Note to Readers + + Note to RFC editor: Please remove this section before publication. + + Feedback and discussion are welcome at https://github.com/quicwg/qlog + (https://github.com/quicwg/qlog). Readers are advised to refer to + the "editor's draft" at that URL for an up-to-date version of this + document. + + Concrete examples of integrations of this schema in various + programming languages can be found at https://github.com/quiclog/ + qlog/ (https://github.com/quiclog/qlog/). + +Status of This Memo + + This Internet-Draft is submitted in full conformance with the + provisions of BCP 78 and BCP 79. + + Internet-Drafts are working documents of the Internet Engineering + Task Force (IETF). Note that other groups may also distribute + working documents as Internet-Drafts. The list of current Internet- + Drafts is at https://datatracker.ietf.org/drafts/current/. + + Internet-Drafts are draft documents valid for a maximum of six months + and may be updated, replaced, or obsoleted by other documents at any + time. It is inappropriate to use Internet-Drafts as reference + material or to cite them other than as "work in progress." + + This Internet-Draft will expire on 29 December 2024. + +Copyright Notice + + Copyright (c) 2024 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents (https://trustee.ietf.org/ + license-info) in effect on the date of publication of this document. + Please review these documents carefully, as they describe your rights + and restrictions with respect to this document. Code Components + extracted from this document must include Revised BSD License text as + described in Section 4.e of the Trust Legal Provisions and are + provided without warranty as described in the Revised BSD License. + +Table of Contents + + 1. Introduction + 1.1. Notational Conventions + 2. Overview + 2.1. Usage with QUIC + 3. HTTP/3 Event Overview + 4. HTTP/3 Events + 4.1. parameters_set + 4.2. parameters_restored + 4.3. stream_type_set + 4.4. priority_updated + 4.5. frame_created + 4.6. frame_parsed + 4.7. datagram_created + 4.8. datagram_parsed + 4.9. push_resolved + 5. HTTP/3 Data Field Definitions + 5.1. Owner + 5.2. H3Frame + 5.3. H3Datagram + 5.3.1. H3DataFrame + 5.3.2. H3HeadersFrame + 5.3.3. H3CancelPushFrame + 5.3.4. H3SettingsFrame + 5.3.5. H3PushPromiseFrame + 5.3.6. H3GoAwayFrame + 5.3.7. H3MaxPushIDFrame + 5.3.8. H3PriorityUpdateFrame + 5.3.9. H3ReservedFrame + 5.3.10. H3UnknownFrame + 5.3.11. H3ApplicationError + 6. Security and Privacy Considerations + 7. IANA Considerations + 8. Normative References + Acknowledgements + Change Log + Since draft-ietf-quic-qlog-h3-events-06: + Since draft-ietf-quic-qlog-h3-events-05: + Since draft-ietf-quic-qlog-h3-events-04: + Since draft-ietf-quic-qlog-h3-events-03: + Since draft-ietf-quic-qlog-h3-events-02: + Since draft-ietf-quic-qlog-h3-events-01: + Since draft-ietf-quic-qlog-h3-events-00: + Since draft-marx-qlog-event-definitions-quic-h3-02: + Since draft-marx-qlog-event-definitions-quic-h3-01: + Since draft-marx-qlog-event-definitions-quic-h3-00: + Authors' Addresses + +1. Introduction + + This document describes the values of the qlog name ("category" + + "event") and "data" fields and their semantics for the HTTP/3 + protocol [HTTP/3] and some of extensions (see [EXTENDED-CONNECT], + [H3_PRIORITIZATION] and [H3-DATAGRAM]). + +1.1. Notational Conventions + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and + "OPTIONAL" in this document are to be interpreted as described in + BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all + capitals, as shown here. + + The event and data structure definitions in ths document are + expressed in the Concise Data Definition Language [CDDL] and its + extensions described in [QLOG-MAIN]. + + The following fields from [QLOG-MAIN] are imported and used: name, + category, type, data, group_id, protocol_type, importance, RawInfo, + and time-related fields. + + As is the case for [QLOG-MAIN], the qlog schema definitions in this + document are intentionally agnostic to serialization formats. The + choice of format is an implementation decision. + +2. Overview + + This document describes how HTTP/3 can be expressed in qlog using the + schema defined in [QLOG-MAIN]. HTTP/3 events are defined with a + category, a name (the concatenation of "category" and "event"), an + "importance", an optional "trigger", and "data" fields. + + Some data fields use complex datastructures. These are represented + as enums or re-usable definitions, which are grouped together on the + bottom of this document for clarity. + + When any event from this document is included in a qlog trace, the + "protocol_type" qlog array field MUST contain an entry with the value + "HTTP3": + + $ProtocolType /= "HTTP3" + + Figure 1: ProtocolType extension for HTTP/3 + +2.1. Usage with QUIC + + The events described in this document can be used with or without + logging the related QUIC events defined in [QLOG-QUIC]. If used with + QUIC events, the QUIC document takes precedence in terms of + recommended filenames and trace separation setups. + + If used without QUIC events, it is recommended that the + implementation assign a globally unique identifier to each HTTP/3 + connection. This ID can then be used as the value of the qlog + "group_id" field, as well as the qlog filename or file identifier, + potentially suffixed by the vantagepoint type (For example, + abcd1234_server.qlog would contain the server-side trace of the + connection with GUID abcd1234). + +3. HTTP/3 Event Overview + + This document defines events in two categories, written as lowercase + to follow convention: h3 (Section 4). + + As described in Section 3.4.2 of [QLOG-MAIN], the qlog "name" field + is the concatenation of category and type. + + Table 1 summarizes the name value of each event type that is defined + in this specification. + + +========================+============+=============+ + | Name value | Importance | Definition | + +========================+============+=============+ + | h3:parameters_set | Base | Section 4.1 | + +------------------------+------------+-------------+ + | h3:parameters_restored | Base | Section 4.2 | + +------------------------+------------+-------------+ + | h3:stream_type_set | Base | Section 4.3 | + +------------------------+------------+-------------+ + | h3:priority_updated | Base | Section 4.4 | + +------------------------+------------+-------------+ + | h3:frame_created | Core | Section 4.5 | + +------------------------+------------+-------------+ + | h3:frame_parsed | Core | Section 4.6 | + +------------------------+------------+-------------+ + | h3:datagram_created | Base | Section 4.7 | + +------------------------+------------+-------------+ + | h3:datagram_parsed | Base | Section 4.8 | + +------------------------+------------+-------------+ + | h3:push_resolved | Extra | Section 4.9 | + +------------------------+------------+-------------+ + + Table 1: HTTP/3 Events + +4. HTTP/3 Events + + HTTP/3 events extend the $ProtocolEventData extension point defined + in [QLOG-MAIN]. Additionally, they allow for direct extensibility by + their use of per-event extension points via the $$ CDDL "group + socket" syntax, as also described in [QLOG-MAIN]. + + H3EventData = H3ParametersSet / + H3ParametersRestored / + H3StreamTypeSet / + H3PriorityUpdated / + H3FrameCreated / + H3FrameParsed / + H3DatagramCreated / + H3DatagramParsed / + H3PushResolved + + $ProtocolEventData /= H3EventData + + Figure 2: H3EventData definition and ProtocolEventData extension + + HTTP events are logged when a certain condition happens at the + application layer, and there isn't always a one to one mapping + between HTTP and QUIC events. The exchange of data between the HTTP + and QUIC layer is logged via the "stream_data_moved" and + "datagram_data_moved" events in [QLOG-QUIC]. + + HTTP/3 frames are transmitted on QUIC streams, which allows them to + span multiple QUIC packets. Some implementations might send a single + large frame, rather than a sequence of smaller frames, in order to + amortize frame header overhead. HTTP/3 frame headers are represented + by the frame_created (Section 4.5) and frame_parsed (Section 4.6) + events. Subsequent frame payload data transfer is indicated by + stream_data_moved events. Furthermore, stream_data_moved events can + appear before frame_parsed events because implementations need to + read data from a stream in order to parse the frame header. + +4.1. parameters_set + + The parameters_set event contains HTTP/3 and QPACK-level settings, + mostly those received from the HTTP/3 SETTINGS frame. It has Base + importance level; see Section 9.2 of [QLOG-MAIN]. + + All these parameters are typically set once and never change. + However, they might be set at different times during the connection, + therefore a qlog can have multiple instances of parameters_set with + different fields set. + + The "owner" field reflects how Settings are exchanged on a + connection. Sent settings have the value "local" and received + settings have the value "received". + + H3ParametersSet = { + ? owner: Owner + + ; RFC9114 + ? max_field_section_size: uint64 + + ; RFC9204 + ? max_table_capacity: uint64 + ? blocked_streams_count: uint64 + + ; RFC9220 (SETTINGS_ENABLE_CONNECT_PROTOCOL) + ? extended_connect: uint16 + + ; RFC9297 (SETTINGS_H3_DATAGRAM) + ? h3_datagram: uint16 + + ; qlog-specific + ; indicates whether this implementation waits for a SETTINGS + ; frame before processing requests + ? waits_for_settings: bool + + * $$h3-parametersset-extension + } + + Figure 3: H3ParametersSet definition + + The parameters_set event can contain any number of unspecified + fields. This allows for representation of reserved settings (aka + GREASE) or ad-hoc support for extension settings that do not have a + related qlog schema definition. + +4.2. parameters_restored + + When using QUIC 0-RTT, HTTP/3 clients are expected to remember and + reuse the server's SETTINGs from the previous connection. The + parameters_restored event is used to indicate which HTTP/3 settings + were restored and to which values when utilizing 0-RTT. It has Base + importance level; see Section 9.2 of [QLOG-MAIN]. + + H3ParametersRestored = { + ; RFC9114 + ? max_field_section_size: uint64 + + ; RFC9204 + ? max_table_capacity: uint64 + ? blocked_streams_count: uint64 + + ; RFC9220 (SETTINGS_ENABLE_CONNECT_PROTOCOL) + ? extended_connect: uint16 + + ; RFC9297 (SETTINGS_H3_DATAGRAM) + ? h3_datagram: uint16 + + * $$h3-parametersrestored-extension + } + + Figure 4: H3ParametersRestored definition + +4.3. stream_type_set + + The stream_type_set event conveys when a HTTP/3 stream type becomes + known; see Sections 6.1 and 6.2 of [HTTP/3]. It has Base importance + level; see Section 9.2 of [QLOG-MAIN]. + + Client bidirectional streams always have a stream_type value of + "request". Server bidirectional streams have no defined use, + although extensions could change that. + + Unidirectional streams in either direction begin with with a + variable-length integer type. Where the type is not known, the + stream_type value of "unknown" type can be used and the value + captured in the stream_type_bytes field; a numerical value without + variable-length integer encoding. + + The generic $H3StreamType is defined here as a CDDL "type socket" + extension point. It can be extended to support additional HTTP/3 + stream types. + + H3StreamTypeSet = { + ? owner: Owner + stream_id: uint64 + stream_type: $H3StreamType + + ; only when stream_type === "unknown" + ? stream_type_bytes: uint64 + + ; only when stream_type === "push" + ? associated_push_id: uint64 + + * $$h3-streamtypeset-extension + } + + $H3StreamType /= "request" / + "control" / + "push" / + "reserved" / + "unknown" / + "qpack_encode" / + "qpack_decode" + + Figure 5: H3StreamTypeSet definition + +4.4. priority_updated + + Emitted when the priority of a request stream or push stream is + initialized or updated through mechanisms defined in [RFC9218]. For + example, the priority can be updated through signals received from + client and/or server (e.g., in HTTP/3 HEADERS or PRIORITY_UPDATE + frames) or it can be changed or overridden due to local policies. + The event has Base importance level; see Section 9.2 of [QLOG-MAIN]. + + H3PriorityUpdated = { + ; if the prioritized element is a request stream + ? stream_id: uint64 + + ; if the prioritized element is a push stream + ? push_id: uint64 + + ? old: H3Priority + new: H3Priority + + * $$h3-priorityupdated-extension + } + + Figure 6: H3PriorityUpdated definition + +4.5. frame_created + + The frame_created event is emitted when the HTTP/3 framing actually + happens. It has Core importance level; see Section 9.2 of + [QLOG-MAIN]. + + This event does not necessarily coincide with HTTP/3 data getting + passed to the QUIC layer. For that, see the stream_data_moved event + in [QLOG-QUIC]. + + H3FrameCreated = { + stream_id: uint64 + ? length: uint64 + frame: $H3Frame + ? raw: RawInfo + + * $$h3-framecreated-extension + } + + Figure 7: H3FrameCreated definition + +4.6. frame_parsed + + The frame_parsed event is emitted when the HTTP/3 frame is parsed. + It has Core importance level; see Section 9.2 of [QLOG-MAIN]. + + This event is not necessarily the same as when the HTTP/3 data is + actually received on the QUIC layer. For that, see the + stream_data_moved event in [QLOG-QUIC]. + + H3FrameParsed = { + stream_id: uint64 + ? length: uint64 + frame: $H3Frame + ? raw: RawInfo + + * $$h3-frameparsed-extension + } + + Figure 8: H3FrameParsed definition + +4.7. datagram_created + + The datagram_created event is emitted when an HTTP/3 Datagram is + created (see [RFC9297]). It has Base importance level; see + Section 9.2 of [QLOG-MAIN]. + + This event does not necessarily coincide with the HTTP/3 Datagram + getting passed to the QUIC layer. For that, see the + datagram_data_moved event in [QLOG-QUIC]. + + H3DatagramCreated = { + quarter_stream_id: uint64 + ? datagram: $H3Datagram + ? raw: RawInfo + + * $$h3-datagramcreated-extension + } + + Figure 9: H3DatagramCreated definition + +4.8. datagram_parsed + + The datagram_parsed event is emitted when the HTTP/3 Datagram is + parsed (see [RFC9297]). It has Base importance level; see + Section 9.2 of [QLOG-MAIN]. + + This event is not necessarily the same as when the HTTP/3 Datagram is + actually received on the QUIC layer. For that, see the + datagram_data_moved event in [QLOG-QUIC]. + + H3DatagramParsed = { + quarter_stream_id: uint64 + ? datagram: $H3Datagram + ? raw: RawInfo + + * $$h3-datagramparsed-extension + } + + Figure 10: H3DatagramParsed definition + +4.9. push_resolved + + The push_resolved event is emitted when a pushed resource + (Section 4.6 of [HTTP/3]) is successfully claimed (used) or, + conversely, abandoned (rejected) by the application on top of HTTP/3 + (e.g., the web browser). This event provides additional context that + can is aid debugging issues related to server push. It has Extra + importance level; see Section 9.2 of [QLOG-MAIN]. + + H3PushResolved = { + ? push_id: uint64 + + ; in case this is logged from a place that does not have access + ; to the push_id + ? stream_id: uint64 + decision: H3PushDecision + + * $$h3-pushresolved-extension + } + + H3PushDecision = "claimed" / + "abandoned" + + Figure 11: H3PushResolved definition + +5. HTTP/3 Data Field Definitions + + The following data field definitions can be used in HTTP/3 events. + +5.1. Owner + + Owner = "local" / + "remote" + + Figure 12: Owner definition + +5.2. H3Frame + + The generic $H3Frame is defined here as a CDDL "type socket" + extension point. It can be extended to support additional HTTP/3 + frame types. + + ; The H3Frame is any key-value map (e.g., JSON object) + $H3Frame /= { + * text => any + } + + Figure 13: H3Frame type socket definition + + The HTTP/3 frame types defined in this document are as follows: + + H3BaseFrames = H3DataFrame / + H3HeadersFrame / + H3CancelPushFrame / + H3SettingsFrame / + H3PushPromiseFrame / + H3GoawayFrame / + H3MaxPushIDFrame / + H3ReservedFrame / + H3UnknownFrame + + $H3Frame /= H3BaseFrames + + Figure 14: H3BaseFrames definition + +5.3. H3Datagram + + The generic $H3Datagram is defined here as a CDDL "type socket" + extension point. It can be extended to support additional HTTP/3 + datagram types. This document intentionally does not define any + specific qlog schemas for specific HTTP/3 Datagram types. + + ; The H3Datagram is any key-value map (e.g., JSON object) + $H3Datagram /= { + * text => any + } + + Figure 15: H3Datagram type socket definition + +5.3.1. H3DataFrame + + H3DataFrame = { + frame_type: "data" + ? raw: RawInfo + } + + Figure 16: H3DataFrame definition + +5.3.2. H3HeadersFrame + + This represents an _uncompressed_, plaintext HTTP Headers frame + (e.g., no QPACK compression is applied). + + For example: + + headers: [ + { + "name": ":path", + "value": "/" + }, + { + "name": ":method", + "value": "GET" + }, + { + "name": ":authority", + "value": "127.0.0.1:4433" + }, + { + "name": ":scheme", + "value": "https" + } + ] + + Figure 17: H3HeadersFrame example + + H3HeadersFrame = { + frame_type: "headers" + headers: [* H3HTTPField] + } + + Figure 18: H3HeadersFrame definition + + H3HTTPField = { + name: text + ? value: text + } + + Figure 19: H3HTTPField definition + +5.3.3. H3CancelPushFrame + + H3CancelPushFrame = { + frame_type: "cancel_push" + push_id: uint64 + } + + Figure 20: H3CancelPushFrame definition + +5.3.4. H3SettingsFrame + + The settings field can contain zero or more entries. Each setting + has a name field, which corresponds to Setting Name as defined (or as + would be defined if registered) in the "HTTP/3 Settings" registry + maintained at https://www.iana.org/assignments/http3-parameters + (https://www.iana.org/assignments/http3-parameters). + + An endpoint that receives unknown settings is not able to log a + specific name. Instead, the name value of "unknown" can be used and + the value captured in the name_bytes field; a numerical value without + variable-length integer encoding. + + H3SettingsFrame = { + frame_type: "settings" + settings: [* H3Setting] + } + + H3Setting = { + ? name: $H3SettingsName + ; only when name === "unknown" + ? name_bytes: uint64 + + value: uint64 + } + + $H3SettingsName /= "settings_qpack_max_table_capacity" / + "settings_max_field_section_size" / + "settings_qpack_blocked_streams" / + "settings_enable_connect_protocol" / + "settings_h3_datagram" / + "reserved" / + "unknown" + + Figure 21: H3SettingsFrame definition + +5.3.5. H3PushPromiseFrame + + H3PushPromiseFrame = { + frame_type: "push_promise" + push_id: uint64 + headers: [* H3HTTPField] + } + + Figure 22: H3PushPromiseFrame definition + +5.3.6. H3GoAwayFrame + + H3GoawayFrame = { + frame_type: "goaway" + + ; Either stream_id or push_id. + ; This is implicit from the sender of the frame + id: uint64 + } + + Figure 23: H3GoawayFrame definition + +5.3.7. H3MaxPushIDFrame + + H3MaxPushIDFrame = { + frame_type: "max_push_id" + push_id: uint64 + } + + Figure 24: H3MaxPushIDFrame definition + +5.3.8. H3PriorityUpdateFrame + + The PRIORITY_UPDATE frame is defined in [RFC9218]. + + H3PriorityUpdateFrame = { + frame_type: "priority_update" + + ; if the prioritized element is a request stream + ? stream_id: uint64 + + ; if the prioritized element is a push stream + ? push_id: uint64 + + priority_field_value: H3Priority + } + + ; The priority value in ASCII text, encoded using Structured Fields + ; Example: u=5, i + H3Priority = text + + Figure 25: h3priorityupdateframe definition + +5.3.9. H3ReservedFrame + + H3ReservedFrame = { + frame_type: "reserved" + ? length: uint64 + } + + Figure 26: H3ReservedFrame definition + +5.3.10. H3UnknownFrame + + The frame_type_bytes field is the numerical value without variable- + length integer encoding. + + H3UnknownFrame = { + frame_type: "unknown" + frame_type_bytes: uint64 + ? raw: RawInfo + } + + Figure 27: UnknownFrame definition + +5.3.11. H3ApplicationError + + H3ApplicationError = "http_no_error" / + "http_general_protocol_error" / + "http_internal_error" / + "http_stream_creation_error" / + "http_closed_critical_stream" / + "http_frame_unexpected" / + "http_frame_error" / + "http_excessive_load" / + "http_id_error" / + "http_settings_error" / + "http_missing_settings" / + "http_request_rejected" / + "http_request_cancelled" / + "http_request_incomplete" / + "http_early_response" / + "http_connect_error" / + "http_version_fallback" + + Figure 28: H3ApplicationError definition + + The H3ApplicationError extends the general $ApplicationError + definition in the qlog QUIC document, see [QLOG-QUIC]. + + ; ensure HTTP errors are properly validated in QUIC events as well + ; e.g., QUIC's ConnectionClose Frame + $ApplicationError /= H3ApplicationError + +6. Security and Privacy Considerations + + The security and privacy considerations discussed in [QLOG-MAIN] + apply to this document as well. + +7. IANA Considerations + + There are no IANA considerations. + +8. Normative References + + [CDDL] Birkholz, H., Vigano, C., and C. Bormann, "Concise Data + Definition Language (CDDL): A Notational Convention to + Express Concise Binary Object Representation (CBOR) and + JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, + June 2019, . + + [EXTENDED-CONNECT] + Hamilton, R., "Bootstrapping WebSockets with HTTP/3", + RFC 9220, DOI 10.17487/RFC9220, June 2022, + . + + [H3-DATAGRAM] + Schinazi, D. and L. Pardue, "HTTP Datagrams and the + Capsule Protocol", RFC 9297, DOI 10.17487/RFC9297, August + 2022, . + + [H3_PRIORITIZATION] + Oku, K. and L. Pardue, "Extensible Prioritization Scheme + for HTTP", RFC 9218, DOI 10.17487/RFC9218, June 2022, + . + + [HTTP/3] Bishop, M., Ed., "HTTP/3", RFC 9114, DOI 10.17487/RFC9114, + June 2022, . + + [QLOG-MAIN] + Marx, R., Niccolini, L., Seemann, M., and L. Pardue, "Main + logging schema for qlog", Work in Progress, Internet- + Draft, draft-ietf-quic-qlog-main-schema-08, 4 March 2024, + . + + [QLOG-QUIC] + Marx, R., Niccolini, L., Seemann, M., and L. Pardue, "QUIC + event definitions for qlog", Work in Progress, Internet- + Draft, draft-ietf-quic-qlog-quic-events-07, 4 March 2024, + . + + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, + DOI 10.17487/RFC2119, March 1997, + . + + [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC + 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, + May 2017, . + + [RFC9218] Oku, K. and L. Pardue, "Extensible Prioritization Scheme + for HTTP", RFC 9218, DOI 10.17487/RFC9218, June 2022, + . + + [RFC9297] Schinazi, D. and L. Pardue, "HTTP Datagrams and the + Capsule Protocol", RFC 9297, DOI 10.17487/RFC9297, August + 2022, . + +Acknowledgements + + Much of the initial work by Robin Marx was done at the Hasselt and KU + Leuven Universities. + + Thanks to Jana Iyengar, Brian Trammell, Dmitri Tikhonov, Stephen + Petrides, Jari Arkko, Marcus Ihlar, Victor Vasiliev, Mirja Kühlewind, + Jeremy Lainé, Kazu Yamamoto, Christian Huitema and Hugo Landau for + their feedback and suggestions. + +Change Log + + This section is to be removed before publishing as an RFC. + +Since draft-ietf-quic-qlog-h3-events-06: + + * ProtocolEventBody is now called ProtocolEventData (#352) + + * Editorial changes (#402) + +Since draft-ietf-quic-qlog-h3-events-05: + + * Removed all qpack event definitions (#335) + + * Various editorial changes + +Since draft-ietf-quic-qlog-h3-events-04: + + * Renamed 'http' category to 'h3' (#300) + + * H3HTTPField.value is now optional (#296) + + * Added definitions for RFC9297 (HTTP/3 Datagram extension) (#310) + + * Added definitions for RFC9218 (HTTP Extensible Prioritizations + extension) (#312) + + * Added definitions for RFC9220 (Extended Connect extension) (#325) + + * Editorial and formatting changes (#298, #258, #299, #304, #327) + +Since draft-ietf-quic-qlog-h3-events-03: + + * Ensured consistent use of RawInfo to indicate raw wire bytes + (#243) + + * Changed HTTPStreamTypeSet:raw_stream_type to stream_type_value + (#54) + + * Changed HTTPUnknownFrame:raw_frame_type to frame_type_value (#54) + + * Renamed max_header_list_size to max_field_section_size (#282) + +Since draft-ietf-quic-qlog-h3-events-02: + + * Renamed HTTPStreamType data to request (#222) + + * Added HTTPStreamType value unknown (#227) + + * Added HTTPUnknownFrame (#224) + + * Replaced old and new fields with stream_type in HTTPStreamTypeSet + (#240) + + * Changed HTTPFrame to a CDDL plug type (#257) + + * Moved data definitions out of the appendix into separate sections + + * Added overview Table of Contents + +Since draft-ietf-quic-qlog-h3-events-01: + + * No changes - new draft to prevent expiration + +Since draft-ietf-quic-qlog-h3-events-00: + + * Change the data definition language from TypeScript to CDDL (#143) + +Since draft-marx-qlog-event-definitions-quic-h3-02: + + * These changes were done in preparation of the adoption of the + drafts by the QUIC working group (#137) + + * Split QUIC and HTTP/3 events into two separate documents + + * Moved RawInfo, Importance, Generic events and Simulation events to + the main schema document. + +Since draft-marx-qlog-event-definitions-quic-h3-01: + + Major changes: + + * Moved data_moved from http to transport. Also made the "from" and + "to" fields flexible strings instead of an enum (#111,#65) + + * Moved packet_type fields to PacketHeader. Moved packet_size field + out of PacketHeader to RawInfo:length (#40) + + * Made events that need to log packet_type and packet_number use a + header field instead of logging these fields individually + + * Added support for logging retry, stateless reset and initial + tokens (#94,#86,#117) + + * Moved separate general event categories into a single category + "generic" (#47) + + * Added "transport:connection_closed" event (#43,#85,#78,#49) + + * Added version_information and alpn_information events + (#85,#75,#28) + + * Added parameters_restored events to help clarify 0-RTT behaviour + (#88) + + Smaller changes: + + * Merged loss_timer events into one loss_timer_updated event + + * Field data types are now strongly defined (#10,#39,#36,#115) + + * Renamed qpack instruction_received and instruction_sent to + instruction_created and instruction_parsed (#114) + + * Updated qpack:dynamic_table_updated.update_type. It now has the + value "inserted" instead of "added" (#113) + + * Updated qpack:dynamic_table_updated. It now has an "owner" field + to differentiate encoder vs decoder state (#112) + + * Removed push_allowed from http:parameters_set (#110) + + * Removed explicit trigger field indications from events, since this + was moved to be a generic property of the "data" field (#80) + + * Updated transport:connection_id_updated to be more in line with + other similar events. Also dropped importance from Core to Base + (#45) + + * Added length property to PaddingFrame (#34) + + * Added packet_number field to transport:frames_processed (#74) + + * Added a way to generically log packet header flags (first 8 bits) + to PacketHeader + + * Added additional guidance on which events to log in which + situations (#53) + + * Added "simulation:scenario" event to help indicate simulation + details + + * Added "packets_acked" event (#107) + + * Added "datagram_ids" to the datagram_X and packet_X events to + allow tracking of coalesced QUIC packets (#91) + + * Extended connection_state_updated with more fine-grained states + (#49) + +Since draft-marx-qlog-event-definitions-quic-h3-00: + + * Event and category names are now all lowercase + + * Added many new events and their definitions + + * "type" fields have been made more specific (especially important + for PacketType fields, which are now called packet_type instead of + type) + + * Events are given an importance indicator (issue #22) + + * Event names are more consistent and use past tense (issue #21) + + * Triggers have been redefined as properties of the "data" field and + updated for most events (issue #23) + +Authors' Addresses + + Robin Marx (editor) + Akamai + Email: rmarx@akamai.com + + + Luca Niccolini (editor) + Meta + Email: lniccolini@meta.com + + + Marten Seemann (editor) + Email: martenseemann@gmail.com + + + Lucas Pardue (editor) + Cloudflare + Email: lucas@lucaspardue.com diff --git a/security-stuff/draft-ietf-quic-qlog-main-schema.html b/security-stuff/draft-ietf-quic-qlog-main-schema.html new file mode 100644 index 00000000..b85f9f29 --- /dev/null +++ b/security-stuff/draft-ietf-quic-qlog-main-schema.html @@ -0,0 +1,3816 @@ + + + + + + +Main logging schema for qlog + + + + + + + + + + + + + + + + + + + + + + + + + + +
Internet-DraftMain logging schema for qlogJune 2024
Marx, et al.Expires 29 December 2024[Page]
+
+
+
+
Workgroup:
+
QUIC
+
Internet-Draft:
+
draft-ietf-quic-qlog-main-schema-latest
+
Published:
+
+ +
+
Intended Status:
+
Standards Track
+
Expires:
+
+
Authors:
+
+
+
R. Marx, Ed. +
+
Akamai
+
+
+
L. Niccolini, Ed. +
+
Meta
+
+
+
M. Seemann, Ed. +
+
+
+
L. Pardue, Ed. +
+
Cloudflare
+
+
+
+
+

Main logging schema for qlog

+
+

Abstract

+

This document defines qlog, an extensible high-level schema for a standardized +logging format. It allows easy sharing of data, benefitting common debug and +analysis methods and tooling. The high-level schema is independent of protocol; +separate documents extend qlog for protocol-specific data. The schema is also +independent of serialization format, allowing logs to be represented in many +ways such as JSON, CSV, or protobuf.

+
+
+

+Note to Readers +

+
    +
  • +

    Note to RFC editor: Please remove this section before publication.

    +
    • +
+

Feedback and discussion are welcome at +https://github.com/quicwg/qlog. Readers are +advised to refer to the "editor's draft" at that URL for an up-to-date version +of this document.

+

Concrete examples of integrations of this schema in +various programming languages can be found at +https://github.com/quiclog/qlog/.

+
+
+
+

+Status of This Memo +

+

+ This Internet-Draft is submitted in full conformance with the + provisions of BCP 78 and BCP 79.

+

+ Internet-Drafts are working documents of the Internet Engineering Task + Force (IETF). Note that other groups may also distribute working + documents as Internet-Drafts. The list of current Internet-Drafts is + at https://datatracker.ietf.org/drafts/current/.

+

+ Internet-Drafts are draft documents valid for a maximum of six months + and may be updated, replaced, or obsoleted by other documents at any + time. It is inappropriate to use Internet-Drafts as reference + material or to cite them other than as "work in progress."

+

+ This Internet-Draft will expire on 29 December 2024.

+
+
+ +
+
+ ▲

+Table of Contents +

+ +
+
+
+
+

+1. Introduction +

+

Endpoint logging is a useful strategy for capturing and understanding how +applications using network protocols are behaving, particularly where protocols +have an encrypted wire image that restricts observers' ability to see what is +happening.

+

Many applications implement logging using a custom, non-standard logging format. +This has an effect on the tools and methods that are used to +analyze the logs, for example to perform root cause analysis of an +interoperability failure between distinct implementations. A lack of a common +format impedes the development of common tooling that can be used by all parties +that have access to logs.

+

This document defines qlog, an extensible high-level schema and harness that +provides a shareable, aggregatable and structured logging format. This +high-level schema is independent of protocol, with logging entries for specific +protocols and use cases being defined in other documents (see for example +[QLOG-QUIC] for QUIC and [QLOG-H3] for HTTP/3-related event definitions).

+

The goal of this high-level schema is to provide amenities and default +characteristics that each logging file should contain (or should be able to +contain), such that generic and reusable toolsets can be created that can deal +with logs from a variety of different protocols and use cases.

+

As such, qlog provides versioning, metadata inclusion, log aggregation, event +grouping and log file size reduction techniques.

+

The qlog schema can be serialized in many ways (e.g., JSON, CBOR, protobuf, +etc). This document describes only how to employ [JSON], its subset +[I-JSON], and its streamable derivative +[JSON-Text-Sequences].

+
+
+

+1.1. Notational Conventions +

+

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", +"MAY", and "OPTIONAL" in this document are to be interpreted as +described in BCP 14 [RFC2119] [RFC8174] when, and only when, they +appear in all capitals, as shown here.

+
+
+

+1.1.1. Schema definition +

+

To define events and data structures, all qlog documents use the Concise Data +Definition Language [CDDL]. This document uses the basic syntax, the +specific text, uint, float32, float64, bool, and any types, as well +as the .default, .size, and .regexp control operators, the ~ unwrapping +operator, and the $ and $$ extension points syntax from [CDDL].

+

Additionally, this document defines the following custom types for +clarity:

+
+
+
+
+; CDDL's uint is defined as being 64-bit in size
+; but for many protocol fields it is better to be restrictive
+; and explicit
+uint8 = uint .size 1
+uint16 = uint .size 2
+uint32 = uint .size 4
+uint64 = uint .size 8
+
+; an even-length lowercase string of hexadecimally encoded bytes
+; examples: 82dc, 027339, 4cdbfd9bf0
+; this is needed because the default CDDL binary string (bytes/bstr)
+; is only CBOR and not JSON compatible
+hexstring = text .regexp "([0-9a-f]{2})*"
+
+
+
Figure 1: +Additional CDDL type definitions +
+
+

All timestamps and time-related values (e.g., offsets) in qlog are +logged as float64 in the millisecond resolution.

+

Other qlog documents can define their own CDDL-compatible (struct) types +(e.g., separately for each Packet type that a protocol supports).

+

The ordering of member fields in qlog CDDL type definitions is not significant. +The ordering of member fields in the serialization formats defined in this +document, JSON (Section 10.1) and JSON Text Sequences (Section 10.2), +is not significant and qlog tools MUST NOT assume so. Other qlog serialization +formats MAY define field order significance, if they do they MUST define +requirements for qlog tools supporting those formats.

+
    +
  • +

    Note to RFC editor: Please remove the following text in this section before +publication.

    +
    • +
+

The main general CDDL syntax conventions in this document a reader +should be aware of for easy reading comprehension are:

+
    +
  • +

    ? obj : this object is optional

    +
    • +
  • +

    TypeName1 / TypeName2 : a union of these two types (object can be either type 1 OR +type 2)

    +
    • +
  • +

    obj: TypeName : this object has this concrete type

    +
    • +
  • +

    obj: [* TypeName] : this object is an array of this type with +minimum size of 0 elements

    +
    • +
  • +

    obj: [+ TypeName] : this object is an array of this type with +minimum size of 1 element

    +
    • +
  • +

    TypeName = ... : defines a new type

    +
    • +
  • +

    EnumName = "entry1" / "entry2" / entry3 / ...: defines an enum

    +
    • +
  • +

    StructName = { ... } : defines a new struct type

    +
    • +
  • +

    ; : single-line comment

    +
    • +
  • +

    * text => any : special syntax to indicate 0 or more fields that +have a string key that maps to any value. Used to indicate a generic +JSON object.

    +
    • +
+

All timestamps and time-related values (e.g., offsets) in qlog are +logged as float64 in the millisecond resolution.

+

Other qlog documents can define their own CDDL-compatible (struct) types +(e.g., separately for each Packet type that a protocol supports).

+
+
+
+
+

+1.1.2. Serialization examples +

+

Serialization examples in this document use JSON ([JSON]) unless +otherwise indicated.

+
+
+
+
+
+
+
+
+

+2. Design goals +

+

The main tenets for the qlog schema design are:

+
    +
  • +

    Streamable, event-based logging

    +
    • +
  • +

    A flexible format that can reduce log producer overhead, at the cost of +increased complexity for consumers (e.g. tools)

    +
    • +
  • +

    Extensible and pragmatic

    +
    • +
  • +

    Aggregation and transformation friendly (e.g., the top-level element +for the non-streaming format is a container for individual traces, +group_ids can be used to tag events to a particular context)

    +
    • +
  • +

    Metadata is stored together with event data

    +
    • +
+
+
+
+
+

+3. QlogFile schema +

+

A qlog using the QlogFile schema can contain several individual traces and logs +from multiple vantage points that are in some way related. The top-level element +in this schema defines only a small set of "header" fields and an array of +component traces, defined in Figure 2 as:

+
+
+
+
+QlogFile = {
+    qlog_version: text
+    ? qlog_format: text .default "JSON"
+    ? title: text
+    ? description: text
+    ? traces: [+ Trace /
+                 TraceError]
+}
+
+
+
Figure 2: +QlogFile definition +
+
+

The required "qlog_version" field MUST have the value "0.4".

+

The optional "qlog_format" field indicates the serialization format. Its value +MUST either be one of the options defined in this document (i.e., +Section 10) or the field MUST be omitted entirely. When the field is +omitted the default value of "JSON" applies.

+

The optional "title" and "description" fields provide additional free-text +information about the file.

+

The optional "traces" field contains an array of qlog traces (Section 3.2), each +of which contain metadata and an array of qlog events (Section 6).

+

In order to make it easier to parse and identify qlog files and their +serialization format, the "qlog_version" and "qlog_format" fields and their +values SHOULD be in the first 256 characters/bytes of the resulting log file.

+

Where a qlog file is serialized to a JSON format, one of the downsides is that +it is inherently a non-streamable format. Put differently, it is not possible to +simply append new qlog events to a log file without "closing" this file at the +end by appending "]}]}". Without these closing tags, most JSON parsers will be +unable to parse the file entirely. The alternative QlogFileSeq +(Section 4) is better suited to streaming.

+

JSON serialization example:

+
+
+
+
+{
+    "qlog_version": "0.4",
+    "qlog_format": "JSON",
+    "title": "Name of this particular qlog file (short)",
+    "description": "Description for this group of traces (long)",
+    "traces": [...]
+}
+
+
+
Figure 3: +QlogFile example +
+
+
+
+

+3.1. Traces +

+

It can be advantageous to group several related qlog traces together in a single +file. For example, it is possible to simultaneously perform logging on the +client, on the server, and on a single point on their common network path. For +analysis, it is useful to aggregate these three individual traces together into +a single file, so it can be uniquely stored, transferred, and annotated.

+

The QlogFile "traces" field is an array that contains a list of individual qlog +traces. When capturing a qlog at a vantage point, it is expected that the traces +field contains a single entry. Files can be aggregated, for example as part of a +post-processing operation, by copying the traces in component to files into the +combined "traces" array of a new, aggregated qlog file.

+
+
+
+
+

+3.2. Trace +

+

The exact conceptual definition of a Trace can be fluid. For example, a trace +could contain all events for a single connection, for a single endpoint, for a +single measurement interval, for a single protocol, etc. In the normal use case +however, a trace is a log of a single data flow collected at a single location +or vantage point. For example, for QUIC, a single trace only contains events for +a single logical QUIC connection for either the client or the server.

+

A Trace contains some metadata in addition to qlog events, defined in +Figure 4 as:

+
+
+
+
+Trace = {
+    ? title: text
+    ? description: text
+    ? common_fields: CommonFields
+    ? vantage_point: VantagePoint
+    events: [* Event]
+}
+
+
+
Figure 4: +Trace definition +
+
+

The optional "title" and "description" fields provide additional free-text +information about the trace.

+

The optional "common_fields" field is described in Section 6.9.

+

The optional "vantage_point" field is described in Section 5.

+

The semantics and context of the trace can mainly be deduced from the entries in +the "common_fields" list and "vantage_point" field.

+

JSON serialization example:

+
+
+
+
+{
+    "title": "Name of this particular trace (short)",
+    "description": "Description for this trace (long)",
+    "common_fields": {
+        "ODCID": "abcde1234",
+        "time_format": "absolute"
+    },
+    "vantage_point": {
+        "name": "backend-67",
+        "type": "server"
+    },
+    "events": [...]
+}
+
+
+
Figure 5: +Trace example +
+
+
+
+
+
+

+3.3. TraceError +

+

A TraceError indicates that an attempt to find/convert a file for inclusion in +the aggregated qlog was made, but there was an error during the process. Rather +than silently dropping the erroneous file, it can be explicitly included in the +qlog file as an entry in the "traces" array, defined in Figure 6 as:

+
+
+
+
+TraceError = {
+    error_description: text
+
+    ; the original URI used for attempted find of the file
+    ? uri: text
+    ? vantage_point: VantagePoint
+}
+
+
+
Figure 6: +TraceError definition +
+
+

JSON serialization example:

+
+
+
+
+{
+    "error_description": "File could not be found",
+    "uri": "/srv/traces/today/latest.qlog",
+    "vantage_point": { type: "server" }
+}
+
+
+
Figure 7: +TraceError example +
+
+

Note that another way to combine events of different traces in a single qlog file +is through the use of the "group_id" field, discussed in Section 6.7.

+
+
+
+
+
+
+

+4. QlogFileSeq schema +

+

A qlog file using the QlogFileSeq schema can be serialized to a streamable JSON +format called JSON Text Sequences (JSON-SEQ) ([RFC7464]). The top-level +element in this schema defines only a small set of "header" fields and an array +of component traces, defined in Figure 2 as:

+
+
+
+
+QlogFileSeq = {
+    qlog_format: "JSON-SEQ"
+    qlog_version: text
+    ? title: text
+    ? description: text
+    trace: TraceSeq
+}
+
+
+
Figure 8: +QlogFileSeq definition +
+
+

The required "qlog_format" field MUST have the value "JSON-SEQ".

+

The required "qlog_version" field MUST have the value "0.4".

+

The optional "title" and "description" fields provide additional free-text +information about the file.

+

The optional "trace" field contains a singular trace metadata. All qlog events +in the file are related to this trace.

+

JSON-SEQ serialization example:

+
+
+
+
+// list of qlog events, serialized in accordance with RFC 7464,
+// starting with a Record Separator character and ending with a
+// newline.
+// For display purposes, Record Separators are rendered as <RS>
+
+<RS>{
+    "qlog_version": "0.4",
+    "qlog_format": "JSON-SEQ",
+    "title": "Name of JSON Text Sequence qlog file (short)",
+    "description": "Description for this trace file (long)",
+    "trace": {
+      "common_fields": {
+        "protocol_type": ["QUIC","HTTP3"],
+        "group_id":"127ecc830d98f9d54a42c4f0842aa87e181a",
+        "time_format":"relative",
+        "reference_time": 1553986553572
+      },
+      "vantage_point": {
+        "name":"backend-67",
+        "type":"server"
+      }
+    }
+}
+<RS>{"time": 2, "name": "quic:parameters_set", "data": { ... } }
+<RS>{"time": 7, "name": "quic:packet_sent", "data": { ... } }
+...
+
+
+
Figure 9: +Top-level element +
+
+

For further information about serialization, see Section 10.2.

+
+
+

+4.1. TraceSeq +

+

TraceSeq is used with QlogFileSeq. It is conceptually similar to a Trace, with +the exception that qlog events are not contained within it, but rather appended +after it in a QlogFileSeq.

+
+
+
+
+TraceSeq = {
+    ? title: text
+    ? description: text
+    ? common_fields: CommonFields
+    ? vantage_point: VantagePoint
+}
+
+
+
Figure 10: +TraceSeq definition +
+
+
+
+
+
+
+
+

+5. VantagePoint +

+

A VantagePoint describes the vantage point from which a trace originates, +defined in Figure 11 as:

+
+
+
+
+VantagePoint = {
+    ? name: text
+    type: VantagePointType
+    ? flow: VantagePointType
+}
+
+; client = endpoint which initiates the connection
+; server = endpoint which accepts the connection
+; network = observer in between client and server
+VantagePointType = "client" /
+                   "server" /
+                   "network" /
+                   "unknown"
+
+
+
Figure 11: +VantagePoint definition +
+
+

JSON serialization examples:

+
+
+
+
+{
+    "name": "aioquic client",
+    "type": "client"
+}
+
+{
+    "name": "wireshark trace",
+    "type": "network",
+    "flow": "client"
+}
+
+
+
Figure 12: +VantagePoint example +
+
+

The flow field is only required if the type is "network" (for example, the trace +is generated from a packet capture). It is used to disambiguate events like +"packet sent" and "packet received". This is indicated explicitly because for +multiple reasons (e.g., privacy) data from which the flow direction can be +otherwise inferred (e.g., IP addresses) might not be present in the logs.

+

Meaning of the different values for the flow field: + * "client" indicates that this vantage point follows client data flow semantics (a + "packet sent" event goes in the direction of the server). + * "server" indicates that this vantage point follow server data flow semantics (a + "packet sent" event goes in the direction of the client). + * "unknown" indicates that the flow's direction is unknown.

+

Depending on the context, tools confronted with "unknown" values in the +vantage_point can either try to heuristically infer the semantics from +protocol-level domain knowledge (e.g., in QUIC, the client always sends the first +packet) or give the user the option to switch between client and server +perspectives manually.

+
+
+
+
+

+6. Events +

+

A qlog event is specified as a generic object with a number of member +fields and their associated data. Depending on the protocol and use case, the +exact member field names and their formats can differ across implementations. This +section lists the main, pre-defined and reserved field names with specific +semantics and expected corresponding value formats.

+

An Event is defined in Figure 13 as:

+
+
+
+
+Event = {
+    time: float64
+    name: text
+    data: $ProtocolEventData
+    ? path: PathID
+    ? time_format: TimeFormat
+    ? protocol_type: ProtocolTypeList
+    ? group_id: GroupID
+    ? system_info: SystemInformation
+
+    ; events can contain any amount of custom fields
+    * text => any
+}
+
+
+
Figure 13: +Event definition +
+
+

Each qlog event MUST contain the mandatory fields: "time" +(Section 6.1), "name" (Section 6.2), and "data" (Section 6.3).

+

Each qlog event MAY contain the optional fields: "time_format" +(Section 6.1), "protocol_type" (Section 6.5), "trigger" +(Section 6.6), and "group_id" (Section 6.7).

+

Multiple events can appear in a Trace or TraceSeq and they might contain fields +with identical values. It is possible to optimize out this duplication using +"common_fields" (Section 6.9).

+

The specific values for each of these fields and their semantics are defined in +separate documents, depending on protocol or use case. For example: event +definitions for QUIC and HTTP/3 can be found in [QLOG-QUIC] and [QLOG-H3].

+

Events are intended to be extended with custom fields, therefore they MAY +contain other fields not defined in this document. Custom fields may be known or +unknown to tools. Tools SHOULD allow for the presence of unknown event fields, +but their semantics depend on the context of the log usage.

+

JSON serialization:

+
+
+
+
+{
+    "time": 1553986553572,
+
+    "name": "quic:packet_sent",
+    "data": { ... },
+
+    "protocol_type":  ["QUIC","HTTP3"],
+    "group_id": "127ecc830d98f9d54a42c4f0842aa87e181a",
+
+    "time_format": "absolute",
+
+    "ODCID": "127ecc830d98f9d54a42c4f0842aa87e181a"
+}
+
+
+
Figure 14: +Event example +
+
+
+
+

+6.1. Timestamps +

+

An event's "time" field indicates the timestamp at which the event occurred. Its value is +typically the Unix timestamp since the 1970 epoch (number of milliseconds since +midnight UTC, January 1, 1970, ignoring leap seconds). However, qlog supports two +more succinct timestamps formats to allow reducing file size. The employed format +is indicated in the "time_format" field, which allows one of three values: +"absolute", "delta" or "relative".

+
+
+
+
+TimeFormat = "absolute" /
+             "delta" /
+             "relative"
+
+
+
Figure 15: +TimeFormat definition +
+
+
    +
  • +

    Absolute: Include the full absolute timestamp with each event. This approach +uses the largest amount of characters. This is also the default value of the +"time_format" field.

    +
    • +
  • +

    Delta: Delta-encode each time value on the previously logged value. The first +event in a trace typically logs the full absolute timestamp. This approach uses +the least amount of characters.

    +
    • +
  • +

    Relative: Specify a full "reference_time" timestamp (typically this is done +up-front in "common_fields", see Section 6.9) and include only +relatively-encoded values based on this reference_time with each event. The +"reference_time" value is typically the first absolute timestamp. This approach +uses a medium amount of characters.

    +
    • +
+

The first option is good for stateless loggers, the second and third for stateful +loggers. The third option is generally preferred, since it produces smaller files +while being easier to reason about. An example for each option can be seen in +Figure 16.

+
+
+
+
+The absolute approach will use:
+1500, 1505, 1522, 1588
+
+The delta approach will use:
+1500, 5, 17, 66
+
+The relative approach will:
+- set the reference_time to 1500 in "common_fields"
+- use: 0, 5, 22, 88
+
+
+
Figure 16: +Three different approaches for logging timestamps +
+
+

One of these options is typically chosen for the entire trace (put differently: +each event has the same value for the "time_format" field). Each event MUST +include a timestamp in the "time" field.

+

Events in each individual trace SHOULD be logged in strictly ascending timestamp +order (though not necessarily absolute value, for the "delta" format). Tools MAY +sort all events on the timestamp before processing them, though are not required +to (as this could impose a significant processing overhead). This can be a problem +especially for multi-threaded and/or streaming loggers, who could consider using a +separate post-processor to order qlog events in time if a tool do not provide this +feature.

+

Timestamps do not have to use the UNIX epoch timestamp as their reference. For +example for privacy considerations, any initial reference timestamps (for example +"endpoint uptime in ms" or "time since connection start in ms") can be chosen. +Tools SHOULD NOT assume the ability to derive the absolute Unix timestamp from +qlog traces, nor allow on them to relatively order events across two or more +separate traces (in this case, clock drift should also be taken into account).

+
+
+
+
+

+6.2. Names +

+

Events differ mainly in the type of metadata associated with them. The "name" +field is an identifier that parsers can use to decide how to interpret the event +metadata contained in the "data" field (see Section 6.3).

+

Event names indicate a category and type. The "name" field MUST contain a +non-empty character sequence representing a category, followed by a colon (':'), +followed by a non-empty character sequence representing a type.

+

Category allows a higher-level grouping of events per specific event type. For +example for QUIC and HTTP/3, the different categories could be "quic", "h3", +"qpack", and "recovery". Within these categories, the event type provides +additional granularity. For example for QUIC and HTTP/3, within the "quic" +category, there would be "packet_sent" and "packet_received" events.

+

JSON serialization example:

+
+
+
+
+{
+    "name": "quic:packet_sent"
+}
+
+
+
Figure 17: +An event with category "quic" and type "packet_sent". +
+
+
+
+
+
+

+6.3. Data +

+

An event's "data" field is a generic key-value map (e.g., JSON object). It +defines the per-event metadata that is to be logged. Its specific subfields and +their semantics are defined per specific event type. For example, data field +definitions for QUIC and HTTP/3 can be found in [QLOG-QUIC] and [QLOG-H3].

+

In order to keep qlog fully extensible, two separate CDDL extension points +("sockets" or "plugs") are used to fully define data fields.

+

Firstly, to allow existing data field definitions to be extended (for example by +adding an additional field needed for a new protocol feature), a CDDL "group +socket" is used. This takes the form of a subfield with a name of +* $$CATEGORY-NAME-extension. This field acts as a placeholder that can later be +replaced with newly defined fields by assigning them to the socket with the +//= operator. Multiple extensions can be assigned to the same group socket. An +example is shown in Figure 18.

+
+
+
+
+; original definition in document A
+MyCategoryEventX = {
+    field_a: uint8
+
+    * $$mycategory-eventx-extension
+}
+
+; later extension of EventX in document B
+$$mycategory-eventx-extension //= (
+  ? additional_field_b: bool
+)
+
+; another extension of EventX in document C
+$$mycategory-eventx-extension //= (
+  ? additional_field_c: text
+)
+
+; if document A, B and C are then used in conjunction,
+; the combined MyCategoryEventX CDDL is equivalent to this:
+MyCategoryEventX = {
+    field_a: uint8
+
+    ? additional_field_b: bool
+    ? additional_field_c: text
+}
+
+
+
Figure 18: +Example of using a generic CDDL group socket to extend an existing event data definition +
+
+

Secondly, to allow documents to define fully new event data field definitions +(as opposed to extend existing ones), a CDDL "type socket" is used. For this +purpose, the type of the "data" field in the qlog Event type (see Figure 13) +is the extensible $ProtocolEventData type. This field acts as an open enum of +possible types that are allowed for the data field. As such, any new event data +field is defined as its own CDDL type and later merged with the existing +$ProtocolEventData enum using the /= extension operator. Any generic +key-value map type can be assigned to $ProtocolEventData (the only common +"data" subfield defined in this document is the optional trigger field, see +Section 6.6). An example of this setup is shown in +Figure 19.

+
+
+
+
+; We define two separate events in a single document
+MyCategoryEvent1 /= {
+    field_1: uint8
+
+    ? trigger: text
+
+    * $$mycategory-event1-extension
+}
+
+MyCategoryEvent2 /= {
+    field_2: bool
+
+    ? trigger: text
+
+    * $$mycategory-event2-extension
+}
+
+; the events are both merged with the existing
+; $ProtocolEventData type enum
+$ProtocolEventData /= MyCategoryEvent1 / MyCategoryEvent2
+
+; the "data" field of a qlog event can now also be of type
+; MyCategoryEvent1 and MyCategoryEvent2
+
+
+
Figure 19: +ProtocolEventData extension +
+
+

Documents defining new qlog events MUST properly extend $ProtocolEventData +when defining data fields to enable automated validation of aggregated qlog +schemas. Furthermore, they SHOULD properly add a * $$CATEGORY-NAME-extension +extension field to newly defined event data to allow the new events to be +properly extended by other documents.

+

A combined but purely illustrative example of the use of both extension points +for a conceptual QUIC "packet_sent" event is shown in Figure 20:

+
+
+
+
+; defined in the main QUIC event document
+TransportPacketSent = {
+    ? packet_size: uint16
+    header: PacketHeader
+    ? frames:[* QuicFrame]
+    ? trigger: "pto_probe" /
+               "retransmit_timeout" /
+               "bandwidth_probe"
+
+    * $$transport-packetsent-extension
+}
+
+; Add the event to the global list of recognized qlog events
+$ProtocolEventData /= TransportPacketSent
+
+; Defined in a separate document that describes a
+; theoretical QUIC protocol extension
+$$transport-packetsent-extension //= (
+  ? additional_field: bool
+)
+
+; If both documents are utilized at the same time,
+; the following JSON serialization would pass an automated
+; CDDL schema validation check:
+
+{
+  "time": 123456,
+  "category": "transport",
+  "name": "packet_sent",
+  "data": {
+      "packet_size": 1280,
+      "header": {
+          "packet_type": "1RTT",
+          "packet_number": 123
+      },
+      "frames": [
+          {
+              "frame_type": "stream",
+              "length": 1000,
+              "offset": 456
+          },
+          {
+              "frame_type": "padding"
+          }
+      ],
+      additional_field: true
+  }
+}
+
+
+
Figure 20: +Example of an extended 'data' field for a conceptual QUIC packet_sent event +
+
+
+
+
+
+

+6.4. Path +

+

A qlog event can be associated with a single "network path" (usually, but not +always, identified by a 4-tuple of IP addresses and ports). In many cases, the +path will be the same for all events in a given trace, and does not need to be +logged explicitly with each event. In this case, the "path" field can be omitted +(in which case the default value of "" is assumed) or reflected in +"common_fields" instead (see Section 6.9).

+

However, in some situations, such as during QUIC's Connection Migration or when +using Multipath features, it is useful to be able to split events across +multiple (concurrent) paths.

+

Definition:

+
+
+
+
+PathID = text .default ""
+
+
+
Figure 21: +PathID definition +
+
+

The "path" field is an identifier that is associated with a single network path. +This document intentionally does not define further how to choose this +identifier's value per-path or how to potentially log other parameters that can +be associated with such a path. This is left for other documents. Implementers +are free to encode path information directly into the PathID or to log +associated info in a separate event. For example, QUIC has the "path_assigned" +event to couple the PathID value to a specific path configuration, see +[QLOG-QUIC].

+
+
+
+
+

+6.5. ProtocolTypeList and ProtocolType +

+

An event's "protocol_type" array field indicates to which protocols (or protocol +"stacks") this event belongs. This allows a single qlog file to aggregate traces +of different protocols (e.g., a web server offering both TCP+HTTP/2 and +QUIC+HTTP/3 connections).

+
+
+
+
+ProtocolTypeList = [+ $ProtocolType]
+
+$ProtocolType /= "UNKNOWN"
+
+
+
Figure 22: +ProtocolTypeList and ProtocolType socket definition +
+
+

For example, QUIC and HTTP/3 events have the "QUIC" and "HTTP3" protocol_type +entry values, see [QLOG-QUIC] and [QLOG-H3].

+

Typically however, all events in a single trace are of the same few protocols, and +this array field is logged once in "common_fields", see Section 6.9.

+
+
+
+
+

+6.6. Triggers +

+

Sometimes, additional information is needed in the case where a single event can +be caused by a variety of other events. In the normal case, the context of the +surrounding log messages gives a hint as to which of these other events was the +cause. However, in highly-parallel and optimized implementations, corresponding +log messages might separated in time. Another option is to explicitly indicate +these "triggers" in a high-level way per-event to get more fine-grained +information without much additional overhead.

+

In qlog, the optional "trigger" field contains a string value describing +the reason (if any) for this event instance occurring, see +Section 6.3. While this "trigger" field could be a property of the +qlog Event itself, it is instead a property of the "data" field instead. +This choice was made because many event types do not include a trigger +value, and having the field at the Event-level would cause overhead in +some serializations. Additional information on the trigger can be added +in the form of additional member fields of the "data" field value, yet +this is highly implementation-specific, as are the trigger field's +string values.

+

One purely illustrative example of some potential triggers for QUIC's +"packet_dropped" event is shown in Figure 23:

+
+
+
+
+TransportPacketDropped = {
+    ? packet_type: PacketType
+    ? raw_length: uint16
+    ? trigger: "key_unavailable" /
+               "unknown_connection_id" /
+               "decrypt_error" /
+               "unsupported_version"
+}
+
+
+
Figure 23: +Trigger example +
+
+
+
+
+
+

+6.7. Grouping +

+

As discussed in Section 3.2, a single qlog file can contain several traces taken +from different vantage points. However, a single trace from one endpoint can also +contain events from a variety of sources. For example, a server implementation +might choose to log events for all incoming connections in a single large +(streamed) qlog file. As such, a method for splitting up events belonging +to separate logical entities is required.

+

The simplest way to perform this splitting is by associating a "group id" +to each event that indicates to which conceptual "group" each event belongs. A +post-processing step can then extract events per group. However, this group +identifier can be highly protocol and context-specific. In the example above, +the QUIC "Original Destination Connection ID" could be used to uniquely identify a +connection. As such, they might add a "ODCID" field to each event. However, a +middlebox logging IP or TCP traffic might rather use four-tuples to identify +connections, and add a "four_tuple" field.

+

As such, to provide consistency and ease of tooling in cross-protocol and +cross-context setups, qlog instead defines the common "group_id" field, which +contains a string value. Implementations are free to use their preferred string +serialization for this field, so long as it contains a unique value per logical +group. Some examples can be seen in Figure 25.

+
+
+
+
+GroupID = text
+
+
+
Figure 24: +GroupID definition +
+
+

JSON serialization example for events grouped by four tuples +and QUIC connection IDs:

+
+
+
+
+"events": [
+    {
+        "time": 1553986553579,
+        "protocol_type": ["TCP", "TLS", "HTTP2"],
+        "group_id": "ip1=2001:67c:1232:144:9498:6df6:f450:110b,
+                   ip2=2001:67c:2b0:1c1::198,port1=59105,port2=80",
+        "name": "quic:packet_received",
+        "data": { ... }
+    },
+    {
+        "time": 1553986553581,
+        "protocol_type": ["QUIC","HTTP3"],
+        "group_id": "127ecc830d98f9d54a42c4f0842aa87e181a",
+        "name": "quic:packet_sent",
+        "data": { ... }
+    }
+]
+
+
+
Figure 25: +GroupID example +
+
+

Note that in some contexts (for example a Multipath transport protocol) it might +make sense to add additional contextual per-event fields (for example "path_id"), +rather than use the group_id field for that purpose.

+

Note also that, typically, a single trace only contains events belonging to a +single logical group (for example, an individual QUIC connection). As such, +instead of logging the "group_id" field with an identical value for each event +instance, this field is typically logged once in "common_fields", see +Section 6.9.

+
+
+
+
+

+6.8. SystemInformation +

+

The "system_info" field can be used to record system-specific details related to an +event. This is useful, for instance, where an application splits work across +CPUs, processes, or threads and events for a single trace occur on potentially +different combinations thereof. Each field is optional to support deployment +diversity.

+
+
+SystemInformation = {
+  ? processor_id: uint32
+  ? process_id: uint32
+  ? thread_id: uint32
+}
+
+
+
+
+
+
+

+6.9. CommonFields +

+

As discussed in the previous sections, information for a typical qlog event varies +in three main fields: "time", "name" and associated data. Additionally, there are +also several more advanced fields that allow mixing events from different +protocols and contexts inside of the same trace (for example "protocol_type" and +"group_id"). In most "normal" use cases however, the values of these advanced +fields are consistent for each event instance (for example, a single trace +contains events for a single QUIC connection).

+

To reduce file size and making logging easier, qlog uses the "common_fields" list +to indicate those fields and their values that are shared by all events in this +component trace. This prevents these fields from being logged for each individual +event. An example of this is shown in Figure 26.

+
+
+
+
+JSON serialization with repeated field values
+per-event instance:
+
+{
+    "events": [{
+            "group_id": "127ecc830d98f9d54a42c4f0842aa87e181a",
+            "protocol_type": ["QUIC","HTTP3"],
+            "time_format": "relative",
+            "reference_time": 1553986553572,
+
+            "time": 2,
+            "name": "quic:packet_received",
+            "data": { ... }
+        },{
+            "group_id": "127ecc830d98f9d54a42c4f0842aa87e181a",
+            "protocol_type": ["QUIC","HTTP3"],
+            "time_format": "relative",
+            "reference_time": 1553986553572,
+
+            "time": 7,
+            "name": "http:frame_parsed",
+            "data": { ... }
+        }
+    ]
+}
+
+JSON serialization with repeated field values instead
+extracted to common_fields:
+
+{
+    "common_fields": {
+        "group_id": "127ecc830d98f9d54a42c4f0842aa87e181a",
+        "protocol_type": ["QUIC","HTTP3"],
+        "time_format": "relative",
+        "reference_time": 1553986553572
+    },
+    "events": [
+        {
+            "time": 2,
+            "name": "quic:packet_received",
+            "data": { ... }
+        },{
+            "time": 7,
+            "name": "http:frame_parsed",
+            "data": { ... }
+        }
+    ]
+}
+
+
+
Figure 26: +CommonFields example +
+
+

An event's "common_fields" field is a generic dictionary of key-value pairs, where the +key is always a string and the value can be of any type, but is typically also a +string or number. As such, unknown entries in this dictionary MUST be disregarded +by the user and tools (i.e., the presence of an unknown field is explicitly NOT an +error).

+

The list of default qlog fields that are typically logged in common_fields (as +opposed to as individual fields per event instance) are shown in the listing +below:

+
+
+
+
+CommonFields = {
+    ? path: PathID
+    ? time_format: TimeFormat
+    ? reference_time: float64
+    ? protocol_type: ProtocolTypeList
+    ? group_id: GroupID
+    * text => any
+}
+
+
+
Figure 27: +CommonFields definition +
+
+

Tools MUST be able to deal with these fields being defined either on each event +individually or combined in common_fields. Note that if at least one event in a +trace has a different value for a given field, this field MUST NOT be added to +common_fields but instead defined on each event individually. Good example of such +fields are "time" and "data", who are divergent by nature.

+
+
+
+
+
+
+

+7. Raw packet and frame information +

+

While qlog is a high-level logging format, it also allows the inclusion of most +raw wire image information, such as byte lengths and byte values. This is useful +when for example investigating or tuning packetization behavior or determining +encoding/framing overheads. However, these fields are not always necessary, can +take up considerable space, and can have a considerable privacy and security +impact (see Section 13). Where applicable, these fields are grouped in a +separate, optional, field named "raw" of type RawInfo. The exact definition of +entities, headers, trailers and payloads depend on the protocol used.

+
+
+
+
+RawInfo = {
+
+    ; the full byte length of the entity (e.g., packet or frame),
+    ; including possible headers and trailers
+    ? length: uint64
+
+    ; the byte length of the entity's payload,
+    ; excluding possible headers or trailers
+    ? payload_length: uint64
+
+    ; the (potentially truncated) contents of the full entity,
+    ; including headers and possibly trailers
+    ? data: hexstring
+}
+
+
+
Figure 28: +RawInfo definition +
+
+

The RawInfo:data field can be truncated for privacy or security purposes, see +Section 10.4. In this case, the length and payload_length fields should +still indicate the non-truncated lengths when used for debugging purposes.

+

This document does not specify explicit header_length or trailer_length fields. +In protocols without trailers, header_length can be calculated by subtracting +the payload_length from the length. In protocols with trailers (e.g., QUIC's +AEAD tag), event definition documents SHOULD define how to support header_length +calculation.

+
+
+
+
+

+8. Common events and data classes +

+

There are some event types and data classes that are common across protocols, +applications, and use cases. This section specifies such common definitions.

+
+
+
+
+GenericEventData = GenericError /
+                GenericWarning /
+                GenericInfo /
+                GenericDebug
+
+SimulationEventData = SimulationScenario /
+                SimulationMarker
+
+$ProtocolEventData /= GenericEventData / SimulationEventData
+
+
+
Figure 29: +ProtocolEventData extension for common events +
+
+
+
+

+8.1. Generic events +

+

In typical logging setups, users utilize a discrete number of well-defined logging +categories, levels or severities to log freeform (string) data. This generic +events category replicates this approach to allow implementations to fully replace +their existing text-based logging by qlog. This is done by providing events to log +generic strings for the typical well-known logging levels (error, warning, info, +debug, verbose).

+

For the events defined below, the "category" is "generic" and their "type" is the +name of the heading in lowercase (e.g., the "name" of the error event is +"generic:error").

+
+
+

+8.1.1. error +

+

Used to log details of an internal error that might not get reflected on the +wire. It has Core importance level; see Section 9.2.

+
+
+
+
+GenericError = {
+    ? code: uint64
+    ? message: text
+
+    * $$generic-error-extension
+}
+
+
+
Figure 30: +GenericError definition +
+
+
+
+
+
+

+8.1.2. warning +

+

Used to log details of an internal warning that might not get reflected on the +wire. It has Base importance level; see Section 9.2.

+
+
+
+
+GenericWarning = {
+    ? code: uint64
+    ? message: text
+
+    * $$generic-warning-extension
+}
+
+
+
Figure 31: +GenericWarning definition +
+
+
+
+
+
+

+8.1.3. info +

+

Used mainly for implementations that want to use qlog as their one and only +logging format but still want to support unstructured string messages. The event +has Extra importance level; see Section 9.2.

+
+
+
+
+GenericInfo = {
+    message: text
+
+    * $$generic-info-extension
+}
+
+
+
Figure 32: +GenericInfo definition +
+
+
+
+
+
+

+8.1.4. debug +

+

Used mainly for implementations that want to use qlog as their one and only +logging format but still want to support unstructured string messages. The event +has Extra importance level; see Section 9.2.

+
+
+
+
+GenericDebug = {
+    message: text
+
+    * $$generic-debug-extension
+}
+
+
+
Figure 33: +GenericDebug definition +
+
+
+
+
+
+

+8.1.5. verbose +

+

Used mainly for implementations that want to use qlog as their one and only +logging format but still want to support unstructured string messages. The event +has Extra importance level; see Section 9.2.

+
+
+
+
+GenericVerbose = {
+    message: text
+
+    * $$generic-verbose-extension
+}
+
+
+
Figure 34: +GenericVerbose definition +
+
+
+
+
+
+
+
+

+8.2. Simulation events +

+

When evaluating a protocol implementation, one typically sets up a series of +interoperability or benchmarking tests, in which the test situations can change +over time. For example, the network bandwidth or latency can vary during the test, +or the network can be fully disable for a short time. In these setups, it is +useful to know when exactly these conditions are triggered, to allow for proper +correlation with other events.

+

For the events defined below, the "category" is "simulation" and their "type" is +the name of the heading in lowercase (e.g., the "name" of the scenario event is +"simulation:scenario").

+
+
+

+8.2.1. scenario +

+

Used to specify which specific scenario is being tested at this particular +instance. This supports, for example, aggregation of several simulations into +one trace (e.g., split by group_id). It has Extra importance level; see +Section 9.2.

+
+
+
+
+SimulationScenario = {
+    ? name: text
+    ? details: {* text => any }
+
+    * $$simulation-scenario-extension
+}
+
+
+
Figure 35: +SimulationScenario definition +
+
+
+
+
+
+

+8.2.2. marker +

+

Used to indicate when specific emulation conditions are triggered at set times +(e.g., at 3 seconds in 2% packet loss is introduced, at 10s a NAT rebind is +triggered). It has Extra importance level; see Section 9.2.

+
+
+
+
+SimulationMarker = {
+    ? type: text
+    ? message: text
+
+    * $$simulation-marker-extension
+}
+
+
+
Figure 36: +SimulationMarker definition +
+
+
+
+
+
+
+
+
+
+

+9. Event definition guidelines +

+

This document defines the main schema for the qlog format together with some +common events, which on their own do not provide much logging utility. It is +expected that logging is extended with specific, per-protocol event definitions +that specify the name (category + type) and data needed for each individual +event. Examples include the QUIC event definitions [QLOG-QUIC] and HTTP/3 +event definitions [QLOG-H3].

+

This section defines some basic annotations and concepts that SHOULD be used by +event definition documents. Doing so ensures a measure of consistency that makes +it easier for qlog implementers to support a wide variety of protocols.

+
+
+

+9.1. Event design +

+

There are several ways of defining qlog events. In practice, two main types of +approach have been observed: a) those that map directly to concepts seen in the +protocols (e.g., packet_sent) and b) those that act as aggregating events that +combine data from several possible protocol behaviors or code paths into one +(e.g., parameters_set). The latter are typically used as a means to reduce the +amount of unique event definitions, as reflecting each possible protocol event +as a separate qlog entity would cause an explosion of event types.

+

Additionally, logging duplicate data is typically prevented as much as possible. +For example, packet header values that remain consistent across many packets are +split into separate events (for example spin_bit_updated or +connection_id_updated for QUIC).

+

Finally, when logging additional state change events, those state changes can +often be directly inferred from data on the wire (for example flow control limit +changes). As such, if the implementation is bug-free and spec-compliant, logging +additional events is typically avoided. Exceptions have been made for common +events that benefit from being easily identifiable or individually logged (for +example packets_acked).

+
+
+
+
+

+9.2. Event importance levels +

+

Depending on how events are designed, it may be that several events allow the +logging of similar or overlapping data. For example the separate QUIC +connection_started event overlaps with the more generic +connection_state_updated. In these cases, it is not always clear which event +should be logged or used, and which event should take precedence if e.g., both are +present and provide conflicting information.

+

To aid in this decision making, qlog defines three event importance levels, in +decreasing order of importance and expected usage:

+
    +
  • +

    Core

    +
    • +
  • +

    Base

    +
    • +
  • +

    Extra

    +
    • +
+

Events definitions SHOULD assign an importance level.

+

Core-level events SHOULD be present in all qlog files for a +given protocol. These are typically tied to basic packet and frame parsing and +creation, as well as listing basic internal metrics. Tool implementers SHOULD +expect and add support for these events, though SHOULD NOT expect all Core events +to be present in each qlog trace.

+

Base-level events add additional debugging options and MAY be present in qlog +files. Most of these can be implicitly inferred from data in Core events (if +those contain all their properties), but for many it is better to log the events +explicitly as well, making it clearer how the implementation behaves. These +events are for example tied to passing data around in buffers, to how internal +state machines change, and used to help show when decisions are actually made +based on received data. Tool implementers SHOULD at least add support for +showing the contents of these events, if they do not handle them explicitly.

+

Extra-level events are considered mostly useful for low-level debugging of the +implementation, rather than the protocol. They allow more fine-grained tracking +of internal behavior. As such, they MAY be present in qlog files and tool +implementers MAY add support for these, but they are not required to.

+

Note that in some cases, implementers might not want to log for example data +content details in Core-level events due to performance or privacy considerations. +In this case, they SHOULD use (a subset of) relevant Base-level events instead to +ensure usability of the qlog output. As an example, implementations that do not +log QUIC packet_received events and thus also not which (if any) ACK frames the +packet contains, SHOULD log packets_acked events instead.

+

Finally, for event types whose data (partially) overlap with other event types' +definitions, where necessary the event definition document should include explicit +guidance on which to use in specific situations.

+
+
+
+
+

+9.3. Custom fields +

+

Event definition documents are free to define new category and event types, +top-level fields (e.g., a per-event field indicating its privacy properties or +path_id in multipath protocols), as well as values for the "trigger" property +within the "data" field, or other member fields of the "data" field, as they see +fit.

+

They however SHOULD NOT expect non-specialized tools to recognize or visualize +this custom data. However, tools SHOULD make an effort to visualize even unknown +data if possible in the specific tool's context. If they do not, they MUST ignore +these unknown fields.

+
+
+
+
+
+
+

+10. Serializing qlog +

+

qlog schema definitions in this document are intentionally agnostic to +serialization formats. The choice of format is an implementation decision.

+

Other documents related to qlog (for example event definitions for specific +protocols), SHOULD be similarly agnostic to the employed serialization format +and SHOULD clearly indicate this. If not, they MUST include an explanation on +which serialization formats are supported and on how to employ them correctly.

+

Serialization formats make certain tradeoffs between usability, flexibility, +interoperability, and efficiency. Implementations should take these into +consideration when choosing a format. Some examples of possible formats are +JSON, CBOR, CSV, protocol buffers, flatbuffers, etc. which each have their own +characteristics. For instance, a textual format like JSON can be more flexible +than a binary format but more verbose, typically making it less efficient than a +binary format. A plaintext readable (yet relatively large) format like JSON is +potentially more usable for users operating on the logs directly, while a more +optimized yet restricted format can better suit the constraints of a large scale +operation. A custom or restricted format could be more efficient for analysis +with custom tooling but might not be interoperable with general-purpose qlog +tools.

+

Considering these tradeoffs, JSON-based serialization formats provide features +that make them a good starting point for qlog flexibility and interoperability. +For these reasons, JSON is a recommended default and expanded considerations are +given to how to map qlog to JSON (Section 10.1, and its streaming counterpart +JSON Text Sequences (Section 10.2. Section 10.3 presents +interoperability considerations for both formats, and Section 10.5 presents +potential optimizations.

+

Serialization formats require appropriate deserializers/parsers. The +"qlog_format" field (Section 3) is used to indicate the chosen +serialization format.

+
+
+

+10.1. qlog to JSON mapping +

+

As described in Section 3, JSON is the default qlog serialization. When +mapping qlog to normal JSON, QlogFile (Figure 2) is used and the +"qlog_format" field MUST have the value "JSON". The file extension/suffix SHOULD +be ".qlog". The Media Type, if any, SHOULD be "application/qlog+json" per +[RFC6839].

+

In accordance with Section 8.1 of [RFC8259], JSON files are required to use +UTF-8 both for the file itself and the string values it contains. In addition, +all qlog field names MUST be lowercase when serialized to JSON.

+

In order to serialize CDDL-based qlog event and data structure +definitions to JSON, the official CDDL-to-JSON mapping defined in +Appendix E of [CDDL] SHOULD be employed.

+
+
+
+
+

+10.2. qlog to JSON Text Sequences mapping +

+

One of the downsides of using normal JSON is that it is inherently a +non-streamable format. A qlog serializer could work around this by opening a +file, writing the required opening data, streaming qlog events by appending +them, and then finalizing the log by appending appropriate closing tags e.g., +"]}]}". However, failure to append closing tags, could lead to problems because +most JSON parsers will fail if a document is malformed. Some streaming JSON +parsers are able to handle missing closing tags, however they are not widely +deployed in popular environments (e.g., Web browsers)

+

To overcome the issues related to JSON streaming, a qlog mapping to a streamable +JSON format called JSON Text Sequences (JSON-SEQ) ([RFC7464]) is provided.

+

JSON Text Sequences are very similar to JSON, except that objects are +serialized as individual records, each prefixed by an ASCII Record Separator +(<RS>, 0x1E), and each ending with an ASCII Line Feed character (\n, 0x0A). Note +that each record can also contain any amount of newlines in its body, as long as +it ends with a newline character before the next <RS> character.

+

In order to leverage the streaming capability, each qlog event is serialized and +interpreted as an individual JSON Text Sequence record, that is appended as a +new object to the back of an event stream or log file. Put differently, unlike +default JSON, it does not require a document to be wrapped as a full object with +"{ ... }" or "[... ]".

+

This alternative record streaming approach cannot be accommodated by QlogFile +(Figure 2). Instead, QlogFileSeq is defined in Figure 8, +which notably includes only a single trace (TraceSeq) and omits an explicit +"events" array. An example is provided in Figure 9. The "group_id" field +can still be used on a per-event basis to include events from conceptually +different sources in a single JSON-SEQ qlog file.

+

When mapping qlog to JSON-SEQ, the "qlog_format" field MUST have the value +"JSON-SEQ". The file extension/suffix SHOULD be ".sqlog" (for "streaming" qlog). +The Media Type, if any, SHOULD be "application/qlog+json-seq" per [RFC8091].

+

While not specifically required by the JSON-SEQ specification, all qlog field +names MUST be lowercase when serialized to JSON-SEQ.

+

In order to serialize all other CDDL-based qlog event and data structure +definitions to JSON-SEQ, the official CDDL-to-JSON mapping defined in +Appendix E of [CDDL] SHOULD be employed.

+
+
+

+10.2.1. Supporting JSON Text Sequences in tooling +

+

Note that JSON Text Sequences are not supported in most default programming +environments (unlike normal JSON). However, several custom JSON-SEQ parsing +libraries exist in most programming languages that can be used and the format is +easy enough to parse with existing implementations (i.e., by splitting the file +into its component records and feeding them to a normal JSON parser individually, +as each record by itself is a valid JSON object).

+
+
+
+
+
+
+

+10.3. JSON Interoperability +

+

Some JSON implementations have issues with the full JSON format, especially those +integrated within a JavaScript environment (e.g., Web browsers, NodeJS). I-JSON +(Internet-JSON) is a subset of JSON for such environments; see +[I-JSON]. One of the key limitations of JavaScript, and thus I-JSON, +is that it cannot represent full 64-bit integers in standard operating mode +(i.e., without using BigInt extensions), instead being limited to the range +-(253)+1 to (253)-1.

+

To accommodate such constraints in CDDL, Appendix E of [CDDL] recommends +defining new CDDL types for int64 and uint64 that limit their values to the +restricted 64-bit integer range. However, some of the protocols that qlog is +intended to support (e.g., QUIC, HTTP/3), can use the full range of uint64 +values.

+

As such, to support situations where I-JSON is in use, seralizers MAY encode +uint64 values using JSON strings. qlog parsers, therefore, SHOULD support +parsing of uint64 values from JSON strings or JSON numbers unless there is out-of-band +information indicating that neither the serializer nor parser are constrained by +I-JSON.

+
+
+
+
+

+10.4. Truncated values +

+

For some use cases (e.g., limiting file size, privacy), it can be +necessary not to log a full raw blob (using the hexstring type) but +instead a truncated value. For example, one might only store the first 100 bytes of an +HTTP response body to be able to discern which file it actually +contained. In these cases, the original byte-size length cannot be +obtained from the serialized value directly.

+

As such, all qlog schema definitions SHOULD include a separate, +length-indicating field for all fields of type hexstring they specify, +see for example Section 7. This not only ensures the original length +can always be retrieved, but also allows the omission of any raw value +bytes of the field completely (e.g., out of privacy or security +considerations).

+

To reduce overhead however and in the case the full raw value is logged, +the extra length-indicating field can be left out. As such, tools MUST +be able to deal with this situation and derive the length of the field +from the raw value if no separate length-indicating field is present. +The main possible permutations are shown by example in +Figure 37.

+
+
+
+
+// both the content's value and its length are present
+// (length is redundant)
+{
+    "content_length": 5,
+    "content": "051428abff"
+}
+
+// only the content value is present, indicating it
+// represents the content's full value. The byte
+// length is obtained by calculating content.length / 2
+{
+    "content": "051428abff"
+}
+
+// only the length is present, meaning the value
+// was omitted
+{
+    "content_length": 5,
+}
+
+// both value and length are present, but the lengths
+// do not match: the value was truncated to
+// the first three bytes.
+{
+    "content_length": 5,
+    "content": "051428"
+}
+
+
+
Figure 37: +Example for serializing truncated hexstrings +
+
+
+
+
+
+

+10.5. Optimization of serialized data +

+

Both the JSON and JSON-SEQ formatting options described above are serviceable in +general small to medium scale (debugging) setups. However, these approaches tend +to be relatively verbose, leading to larger file sizes. Additionally, generalized +JSON(-SEQ) (de)serialization performance is typically (slightly) lower than that +of more optimized and predictable formats. Both aspects present challenges to +large scale setups, though they may still be practical to deploy; see [ANRW-2020]. +JSON and JSON-SEQ compress very well using commonly-available algorithms such as +GZIP or Brotli.

+

During the development of qlog, a multitude of alternative formatting +and optimization options were assessed and the results are summarized on the qlog +github +repository.

+

Formal definition of additional qlog formats or encodings that use the +optimization techniques described here, or any other optimization technique is +left to future activity that can apply the following guidelines.

+

In order to help tools correctly parse and process serialized qlog, it is +RECOMMENDED that new formats also define suitable file extensions and media +types. This provides a clear signal and avoids the need to provide out-of-band +information or to rely on heuristic fallbacks; see Section 12.

+
+
+
+
+
+
+

+11. Methods of access and generation +

+

Different implementations will have different ways of generating and storing +qlogs. However, there is still value in defining a few default ways in which to +steer this generation and access of the results.

+
+
+

+11.1. Set file output destination via an environment variable +

+

To provide users control over where and how qlog files are created, two +environment variables are defined. The first, QLOGFILE, indicates a full path to where an +individual qlog file should be stored. This path MUST include the full file +extension. The second, QLOGDIR, sets a general directory path in which qlog files +should be placed. This path MUST include the directory separator character at the +end.

+

In general, QLOGDIR should be preferred over QLOGFILE if an endpoint is prone to +generate multiple qlog files. This can for example be the case for a QUIC server +implementation that logs each QUIC connection in a separate qlog file. An +alternative that uses QLOGFILE would be a QUIC server that logs all connections in +a single file and uses the "group_id" field (Section 6.7) to allow post-hoc +separation of events.

+

Implementations SHOULD provide support for QLOGDIR and MAY provide support for +QLOGFILE.

+

When using QLOGDIR, it is up to the implementation to choose an appropriate naming +scheme for the qlog files themselves. The chosen scheme will typically depend on +the context or protocols used. For example, for QUIC, it is recommended to use the +Original Destination Connection ID (ODCID), followed by the vantage point type of +the logging endpoint. Examples of all options for QUIC are shown in +Figure 38.

+
+
+
+
+Command: QLOGFILE=/srv/qlogs/client.qlog quicclientbinary
+
+Should result in the the quicclientbinary executable logging a
+single qlog file named client.qlog in the /srv/qlogs directory.
+This is for example useful in tests when the client sets up
+just a single connection and then exits.
+
+Command: QLOGDIR=/srv/qlogs/ quicserverbinary
+
+Should result in the quicserverbinary executable generating
+several logs files, one for each QUIC connection.
+Given two QUIC connections, with ODCID values "abcde" and
+"12345" respectively, this would result in two files:
+/srv/qlogs/abcde_server.qlog
+/srv/qlogs/12345_server.qlog
+
+Command: QLOGFILE=/srv/qlogs/server.qlog quicserverbinary
+
+Should result in the the quicserverbinary executable logging
+a single qlog file named server.qlog in the /srv/qlogs directory.
+Given that the server handled two QUIC connections before it was
+shut down, with ODCID values "abcde" and "12345" respectively,
+this would result in event instances in the qlog file being
+tagged with the "group_id" field with values "abcde" and "12345".
+
+
+
Figure 38: +Environment variable examples for a QUIC implementation +
+
+
+
+
+
+
+
+

+12. Tooling requirements +

+

Tools ingestion qlog MUST indicate which qlog version(s), qlog format(s), +compression methods and potentially other input file formats (for example .pcap) +they support. Tools SHOULD at least support .qlog files in the default JSON format +(Section 10.1). Additionally, they SHOULD indicate exactly which values for and +properties of the name (category and type) and data fields they look for to +execute their logic. Tools SHOULD perform a (high-level) check if an input qlog +file adheres to the expected qlog schema. If a tool determines a qlog file does +not contain enough supported information to correctly execute the tool's logic, it +SHOULD generate a clear error message to this effect.

+

Tools MUST NOT produce breaking errors for any field names and/or values in the +qlog format that they do not recognize. Tools SHOULD indicate even unknown event +occurrences within their context (e.g., marking unknown events on a timeline for +manual interpretation by the user).

+

Tool authors should be aware that, depending on the logging implementation, some +events will not always be present in all traces. For example, using a circular +logging buffer of a fixed size, it could be that the earliest events (e.g., +connection setup events) are later overwritten by "newer" events. Alternatively, +some events can be intentionally omitted out of privacy or file size +considerations. Tool authors are encouraged to make their tools robust enough to +still provide adequate output for incomplete logs.

+
+
+
+
+

+13. Security and privacy considerations +

+

Protocols such as TLS [RFC8446] and QUIC [RFC9000] offer secure protection +for the wire image [RFC8546]. Logging can reveal aspects of the wire image +that would ordinarily be protected, creating tension between observability, +security and privacy, especially if data can be correlated across data sources.

+

qlog permits logging of a broad and detialed range of data. Operators and +implementers are responsible for deciding what data is logged to address their +requirements and constraints. As per [RFC6973], operators must be aware that +data could be compromised, risking the privacy of all participants. Where +entities expect protocol features to ensure data privacy, logging might +unknowingly be subject to broader privacy risks, undermining their ability to +assess or respond effectively.

+
+
+

+13.1. Data at risk +

+

qlog operators and implementers need to consider security and privacy risks when +handling qlog data, including logging, storage, usage, and more. The +considerations presented in this section may pose varying risks depending on the +the data itself or its handling.

+

The following is a non-exhaustive list of example data types that could contain +sensitive information that might allow identification or correlation of +individual connections, endpoints, users or sessions across qlog or other data +sources (e.g., captures of encrypted packets):

+
    +
  • +

    IP addresses and transport protocol port numbers.

    +
    • +
  • +

    Session, Connection, or User identifiers e.g., QUIC Connection IDs Section 9.5 of [RFC9000]).

    +
    • +
  • +

    System-level information e.g., CPU, process, or thread identifiers.

    +
    • +
  • +

    Stored State e.g., QUIC address validation and retry tokens, TLS session +tickets, and HTTP cookies.

    +
    • +
  • +

    TLS decryption keys, passwords, and HTTP-level API access or authorization tokens.

    +
    • +
  • +

    High-resolution event timestamps or inter-event timings, event counts, packet +sizes, and frame sizes.

    +
    • +
  • +

    Full or partial raw packet and frame payloads that are encrypted.

    +
    • +
  • +

    Full or partial raw packet and frame payloads that are plaintext e.g., HTTP Field +values, HTTP response data, or TLS SNI field values.

    +
    • +
+
+
+
+
+

+13.2. Operational implications and recommendations +

+

Operational considerations should focus on authorizing capture and access to logs. Logging of +Internet protocols using qlog can be equivalent to the ability to store or read plaintext +communications. Without a more detailed analysis, all of the security considerations of plaintext access apply.

+

It is recommended that qlog capture is subject to access control and auditing. +These controls should support granular levels of information capture based on +role and permissions (e.g., capture of more-sensitive data requires higher +privileges).

+

It is recommended that access to stored qlogs is subject to access control and +auditing.

+

End users might not understand the implications of qlog to security or privacy, +and their environments might limit access control techniques. Implementations should +make enabling qlog conspicuous (e.g., requiring clear and explicit actions to +start a capture) and resistant to social engineering, automation, or drive-by +attacks; for example, isolation or sandboxing of capture from other activities +in the same process or component.

+

It is recommended that data retention policies are defined for the storage of +qlog files.

+

It is recommended that qlog files are encrypted in transit and at rest.

+
+
+
+
+

+13.3. Data minimization or anonymization +

+

Applying data minimization or anonymization techniques to qlog might help +address some security and privacy risks. However, removing or anonymizing data +without sufficient care might not enhance privacy or security and +could diminish the utility of qlog data.

+

Operators and implementers should balance the value of logged data with the +potential risks of voluntary or involuntary disclosure to trusted or untrusted +entities. The data requirements and definition of entities depends greatly on +the intended use cases. For example, a research project might be tightly scoped, +time bound, and require participants to explicitly opt in to having their data +collected with the intention for this to be shared in a publication. Whereas a +server administrator might desire to collect telemetry, from users whom they have +no relationship with, for continuing operational needs.

+

The most extreme form of minimization or anonymization is deleting a field, +equivalent to not logging it. qlog implementations should offer fine-grained +control for this on a per-use-case or per-connection basis.

+

Data can undergo anonymization, pseudonymization, permutation, truncation, +re-encryption, or aggregation; see Appendix B of [DNS-PRIVACY] for +techniques, especially regarding IP addresses. However, operators should be +cautious because many anonymization methods have been shown to be insufficient to safeguard +user privacy or identity, particularly with large or easily correlated data sets.

+

Operators should consider end user rights and preferences. Active user participation (as +indicated by [RFC6973]) on a per-qlog basis is challenging but aligning qlog +capture, storage, and removal with existing user preference and privacy controls +is crucial. Operators should consider agressive approaches to deletion or +aggregation.

+

The most sensitive data in qlog is typically contained in RawInfo type fields +(see Section 7). Therefore, qlog users should exercise caution and limit the +inclusion of such fields for all but the most stringent use cases.

+
+
+
+
+
+
+

+14. IANA Considerations +

+

There are no IANA considerations.

+
+
+
+

+15. References +

+
+
+

+15.1. Normative References +

+
+
[CDDL]
+
+Birkholz, H., Vigano, C., and C. Bormann, "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, , <https://www.rfc-editor.org/rfc/rfc8610>.
+
+
[DNS-PRIVACY]
+
+Dickinson, S., Overeinder, B., van Rijswijk-Deij, R., and A. Mankin, "Recommendations for DNS Privacy Service Operators", BCP 232, RFC 8932, DOI 10.17487/RFC8932, , <https://www.rfc-editor.org/rfc/rfc8932>.
+
+
[I-JSON]
+
+Bray, T., Ed., "The I-JSON Message Format", RFC 7493, DOI 10.17487/RFC7493, , <https://www.rfc-editor.org/rfc/rfc7493>.
+
+
[JSON]
+
+Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", STD 90, RFC 8259, DOI 10.17487/RFC8259, , <https://www.rfc-editor.org/rfc/rfc8259>.
+
+
[JSON-Text-Sequences]
+
+Williams, N., "JavaScript Object Notation (JSON) Text Sequences", RFC 7464, DOI 10.17487/RFC7464, , <https://www.rfc-editor.org/rfc/rfc7464>.
+
+
[RFC2119]
+
+Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/rfc/rfc2119>.
+
+
[RFC6839]
+
+Hansen, T. and A. Melnikov, "Additional Media Type Structured Syntax Suffixes", RFC 6839, DOI 10.17487/RFC6839, , <https://www.rfc-editor.org/rfc/rfc6839>.
+
+
[RFC6973]
+
+Cooper, A., Tschofenig, H., Aboba, B., Peterson, J., Morris, J., Hansen, M., and R. Smith, "Privacy Considerations for Internet Protocols", RFC 6973, DOI 10.17487/RFC6973, , <https://www.rfc-editor.org/rfc/rfc6973>.
+
+
[RFC7464]
+
+Williams, N., "JavaScript Object Notation (JSON) Text Sequences", RFC 7464, DOI 10.17487/RFC7464, , <https://www.rfc-editor.org/rfc/rfc7464>.
+
+
[RFC8091]
+
+Wilde, E., "A Media Type Structured Syntax Suffix for JSON Text Sequences", RFC 8091, DOI 10.17487/RFC8091, , <https://www.rfc-editor.org/rfc/rfc8091>.
+
+
[RFC8174]
+
+Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/rfc/rfc8174>.
+
+
[RFC8259]
+
+Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", STD 90, RFC 8259, DOI 10.17487/RFC8259, , <https://www.rfc-editor.org/rfc/rfc8259>.
+
+
[RFC9000]
+
+Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based Multiplexed and Secure Transport", RFC 9000, DOI 10.17487/RFC9000, , <https://www.rfc-editor.org/rfc/rfc9000>.
+
+
+
+
+
+
+

+15.2. Informative References +

+
+
[ANRW-2020]
+
+Marx, R., Piraux, M., Quax, P., and W. Lamotte, "Debugging QUIC and HTTP/3 with qlog and qvis", , <https://qlog.edm.uhasselt.be/anrw/>.
+
+
[QLOG-H3]
+
+Marx, R., Niccolini, L., Seemann, M., and L. Pardue, "HTTP/3 qlog event definitions", Work in Progress, Internet-Draft, draft-ietf-quic-qlog-h3-events-07, , <https://datatracker.ietf.org/doc/html/draft-ietf-quic-qlog-h3-events-07>.
+
+
[QLOG-QUIC]
+
+Marx, R., Niccolini, L., Seemann, M., and L. Pardue, "QUIC event definitions for qlog", Work in Progress, Internet-Draft, draft-ietf-quic-qlog-quic-events-07, , <https://datatracker.ietf.org/doc/html/draft-ietf-quic-qlog-quic-events-07>.
+
+
[RFC8446]
+
+Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.3", RFC 8446, DOI 10.17487/RFC8446, , <https://www.rfc-editor.org/rfc/rfc8446>.
+
+
[RFC8546]
+
+Trammell, B. and M. Kuehlewind, "The Wire Image of a Network Protocol", RFC 8546, DOI 10.17487/RFC8546, , <https://www.rfc-editor.org/rfc/rfc8546>.
+
+
+
+
+
+
+
+

+Acknowledgements +

+

Much of the initial work by Robin Marx was done at the Hasselt and KU Leuven +Universities.

+

Thanks to Jana Iyengar, Brian Trammell, Dmitri Tikhonov, Stephen Petrides, Jari +Arkko, Marcus Ihlar, Victor Vasiliev, Mirja Kühlewind, Jeremy Lainé, Kazu +Yamamoto, Christian Huitema and Hugo Landau for their feedback and suggestions.

+
+
+
+
+

+Change Log +

+

This section is to be removed before publishing as an RFC.

+
+
+

+Since draft-ietf-quic-qlog-main-schema-07: +

+
    +
  • +

    Added path and PathID (#336)

    +
    • +
  • +

    Removed custom definition of uint64 type (#360, #388)

    +
    • +
  • +

    ProtocolEventBody is now called ProtocolEventData (#352)

    +
    • +
  • +

    Editorial changes (#364, #289, #353, #361, #362)

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-main-schema-06: +

+
    +
  • +

    Editorial reworking of the document (#331, #332)

    +
    • +
  • +

    Updated IANA considerations section (#333)

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-main-schema-05: +

+
    +
  • +

    Updated qlog_version to 0.4 (due to breaking changes) (#314)

    +
    • +
  • +

    Renamed 'transport' category to 'quic' (#302)

    +
    • +
  • +

    Added 'system_info' field (#305)

    +
    • +
  • +

    Removed 'summary' and 'configuration' fields (#308)

    +
    • +
  • +

    Editorial and formatting changes (#298, #303, #304, #316, #320, #321, #322, #326, #328)

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-main-schema-04: +

+
    +
  • +

    Updated RawInfo definition and guidance (#243)

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-main-schema-03: +

+
    +
  • +

    Added security and privacy considerations discussion (#252)

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-main-schema-02: +

+
    +
  • +

    No changes - new draft to prevent expiration

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-main-schema-01: +

+
    +
  • +

    Change the data definition language from TypeScript to CDDL (#143)

    +
    • +
+
+
+
+
+

+Since draft-ietf-quic-qlog-main-schema-00: +

+
    +
  • +

    Changed the streaming serialization format from NDJSON to JSON Text Sequences +(#172)

    +
    • +
  • +

    Added Media Type definitions for various qlog formats (#158)

    +
    • +
  • +

    Changed to semantic versioning

    +
    • +
+
+
+
+
+

+Since draft-marx-qlog-main-schema-draft-02: +

+
    +
  • +

    These changes were done in preparation of the adoption of the drafts by the QUIC +working group (#137)

    +
    • +
  • +

    Moved RawInfo, Importance, Generic events and Simulation events to this document.

    +
    • +
  • +

    Added basic event definition guidelines

    +
    • +
  • +

    Made protocol_type an array instead of a string (#146)

    +
    • +
+
+
+
+
+

+Since draft-marx-qlog-main-schema-01: +

+
    +
  • +

    Decoupled qlog from the JSON format and described a mapping instead (#89)

    +
      +
    • +

      Data types are now specified in this document and proper definitions for +fields were added in this format

      +
      • +
    • +

      64-bit numbers can now be either strings or numbers, with a preference for +numbers (#10)

      +
      • +
    • +

      binary blobs are now logged as lowercase hex strings (#39, #36)

      +
      • +
    • +

      added guidance to add length-specifiers for binary blobs (#102)

      +
      • +
    +
    • +
  • +

    Removed "time_units" from Configuration. All times are now in ms instead (#95)

    +
    • +
  • +

    Removed the "event_fields" setup for a more straightforward JSON format +(#101,#89)

    +
    • +
  • +

    Added a streaming option using the NDJSON format (#109,#2,#106)

    +
    • +
  • +

    Described optional optimization options for implementers (#30)

    +
    • +
  • +

    Added QLOGDIR and QLOGFILE environment variables, clarified the .well-known URL +usage (#26,#33,#51)

    +
    • +
  • +

    Overall tightened up the text and added more examples

    +
    • +
+
+
+
+
+

+Since draft-marx-qlog-main-schema-00: +

+
    +
  • +

    All field names are now lowercase (e.g., category instead of CATEGORY)

    +
    • +
  • +

    Triggers are now properties on the "data" field value, instead of separate field +types (#23)

    +
    • +
  • +

    group_ids in common_fields is now just also group_id

    +
    • +
+
+
+
+
+
+
+

+Authors' Addresses +

+
+
Robin Marx (editor)
+
Akamai
+ +
+
+
Luca Niccolini (editor)
+
Meta
+ +
+
+
Marten Seemann (editor)
+ +
+
+
Lucas Pardue (editor)
+
Cloudflare
+ +
+
+
+ + + diff --git a/security-stuff/draft-ietf-quic-qlog-main-schema.txt b/security-stuff/draft-ietf-quic-qlog-main-schema.txt new file mode 100644 index 00000000..f018af1f --- /dev/null +++ b/security-stuff/draft-ietf-quic-qlog-main-schema.txt @@ -0,0 +1,2133 @@ + + + + +QUIC R. Marx, Ed. +Internet-Draft Akamai +Intended status: Standards Track L. Niccolini, Ed. +Expires: 29 December 2024 Meta + M. Seemann, Ed. + + L. Pardue, Ed. + Cloudflare + 27 June 2024 + + + Main logging schema for qlog + draft-ietf-quic-qlog-main-schema-latest + +Abstract + + This document defines qlog, an extensible high-level schema for a + standardized logging format. It allows easy sharing of data, + benefitting common debug and analysis methods and tooling. The high- + level schema is independent of protocol; separate documents extend + qlog for protocol-specific data. The schema is also independent of + serialization format, allowing logs to be represented in many ways + such as JSON, CSV, or protobuf. + +Note to Readers + + Note to RFC editor: Please remove this section before publication. + + Feedback and discussion are welcome at https://github.com/quicwg/qlog + (https://github.com/quicwg/qlog). Readers are advised to refer to + the "editor's draft" at that URL for an up-to-date version of this + document. + + Concrete examples of integrations of this schema in various + programming languages can be found at https://github.com/quiclog/ + qlog/ (https://github.com/quiclog/qlog/). + +Status of This Memo + + This Internet-Draft is submitted in full conformance with the + provisions of BCP 78 and BCP 79. + + Internet-Drafts are working documents of the Internet Engineering + Task Force (IETF). Note that other groups may also distribute + working documents as Internet-Drafts. The list of current Internet- + Drafts is at https://datatracker.ietf.org/drafts/current/. + + Internet-Drafts are draft documents valid for a maximum of six months + and may be updated, replaced, or obsoleted by other documents at any + time. It is inappropriate to use Internet-Drafts as reference + material or to cite them other than as "work in progress." + + This Internet-Draft will expire on 29 December 2024. + +Copyright Notice + + Copyright (c) 2024 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents (https://trustee.ietf.org/ + license-info) in effect on the date of publication of this document. + Please review these documents carefully, as they describe your rights + and restrictions with respect to this document. Code Components + extracted from this document must include Revised BSD License text as + described in Section 4.e of the Trust Legal Provisions and are + provided without warranty as described in the Revised BSD License. + +Table of Contents + + 1. Introduction + 1.1. Notational Conventions + 1.1.1. Schema definition + 1.1.2. Serialization examples + 2. Design goals + 3. QlogFile schema + 3.1. Traces + 3.2. Trace + 3.3. TraceError + 4. QlogFileSeq schema + 4.1. TraceSeq + 5. VantagePoint + 6. Events + 6.1. Timestamps + 6.2. Names + 6.3. Data + 6.4. Path + 6.5. ProtocolTypeList and ProtocolType + 6.6. Triggers + 6.7. Grouping + 6.8. SystemInformation + 6.9. CommonFields + 7. Raw packet and frame information + 8. Common events and data classes + 8.1. Generic events + 8.1.1. error + 8.1.2. warning + 8.1.3. info + 8.1.4. debug + 8.1.5. verbose + 8.2. Simulation events + 8.2.1. scenario + 8.2.2. marker + 9. Event definition guidelines + 9.1. Event design + 9.2. Event importance levels + 9.3. Custom fields + 10. Serializing qlog + 10.1. qlog to JSON mapping + 10.2. qlog to JSON Text Sequences mapping + 10.2.1. Supporting JSON Text Sequences in tooling + 10.3. JSON Interoperability + 10.4. Truncated values + 10.5. Optimization of serialized data + 11. Methods of access and generation + 11.1. Set file output destination via an environment variable + 12. Tooling requirements + 13. Security and privacy considerations + 13.1. Data at risk + 13.2. Operational implications and recommendations + 13.3. Data minimization or anonymization + 14. IANA Considerations + 15. References + 15.1. Normative References + 15.2. Informative References + Acknowledgements + Change Log + Since draft-ietf-quic-qlog-main-schema-07: + Since draft-ietf-quic-qlog-main-schema-06: + Since draft-ietf-quic-qlog-main-schema-05: + Since draft-ietf-quic-qlog-main-schema-04: + Since draft-ietf-quic-qlog-main-schema-03: + Since draft-ietf-quic-qlog-main-schema-02: + Since draft-ietf-quic-qlog-main-schema-01: + Since draft-ietf-quic-qlog-main-schema-00: + Since draft-marx-qlog-main-schema-draft-02: + Since draft-marx-qlog-main-schema-01: + Since draft-marx-qlog-main-schema-00: + Authors' Addresses + +1. Introduction + + Endpoint logging is a useful strategy for capturing and understanding + how applications using network protocols are behaving, particularly + where protocols have an encrypted wire image that restricts + observers' ability to see what is happening. + + Many applications implement logging using a custom, non-standard + logging format. This has an effect on the tools and methods that are + used to analyze the logs, for example to perform root cause analysis + of an interoperability failure between distinct implementations. A + lack of a common format impedes the development of common tooling + that can be used by all parties that have access to logs. + + This document defines qlog, an extensible high-level schema and + harness that provides a shareable, aggregatable and structured + logging format. This high-level schema is independent of protocol, + with logging entries for specific protocols and use cases being + defined in other documents (see for example [QLOG-QUIC] for QUIC and + [QLOG-H3] for HTTP/3-related event definitions). + + The goal of this high-level schema is to provide amenities and + default characteristics that each logging file should contain (or + should be able to contain), such that generic and reusable toolsets + can be created that can deal with logs from a variety of different + protocols and use cases. + + As such, qlog provides versioning, metadata inclusion, log + aggregation, event grouping and log file size reduction techniques. + + The qlog schema can be serialized in many ways (e.g., JSON, CBOR, + protobuf, etc). This document describes only how to employ [JSON], + its subset [I-JSON], and its streamable derivative + [JSON-Text-Sequences]. + +1.1. Notational Conventions + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and + "OPTIONAL" in this document are to be interpreted as described in + BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all + capitals, as shown here. + +1.1.1. Schema definition + + To define events and data structures, all qlog documents use the + Concise Data Definition Language [CDDL]. This document uses the + basic syntax, the specific text, uint, float32, float64, bool, and + any types, as well as the .default, .size, and .regexp control + operators, the ~ unwrapping operator, and the $ and $$ extension + points syntax from [CDDL]. + + Additionally, this document defines the following custom types for + clarity: + + ; CDDL's uint is defined as being 64-bit in size + ; but for many protocol fields it is better to be restrictive + ; and explicit + uint8 = uint .size 1 + uint16 = uint .size 2 + uint32 = uint .size 4 + uint64 = uint .size 8 + + ; an even-length lowercase string of hexadecimally encoded bytes + ; examples: 82dc, 027339, 4cdbfd9bf0 + ; this is needed because the default CDDL binary string (bytes/bstr) + ; is only CBOR and not JSON compatible + hexstring = text .regexp "([0-9a-f]{2})*" + + Figure 1: Additional CDDL type definitions + + All timestamps and time-related values (e.g., offsets) in qlog are + logged as float64 in the millisecond resolution. + + Other qlog documents can define their own CDDL-compatible (struct) + types (e.g., separately for each Packet type that a protocol + supports). + + The ordering of member fields in qlog CDDL type definitions is not + significant. The ordering of member fields in the serialization + formats defined in this document, JSON (Section 10.1) and JSON Text + Sequences (Section 10.2), is not significant and qlog tools MUST NOT + assume so. Other qlog serialization formats MAY define field order + significance, if they do they MUST define requirements for qlog tools + supporting those formats. + + Note to RFC editor: Please remove the following text in this + section before publication. + + The main general CDDL syntax conventions in this document a reader + should be aware of for easy reading comprehension are: + + * ? obj : this object is optional + + * TypeName1 / TypeName2 : a union of these two types (object can be + either type 1 OR type 2) + + * obj: TypeName : this object has this concrete type + + * obj: [* TypeName] : this object is an array of this type with + minimum size of 0 elements + + * obj: [+ TypeName] : this object is an array of this type with + minimum size of 1 element + + * TypeName = ... : defines a new type + + * EnumName = "entry1" / "entry2" / entry3 / ...: defines an enum + + * StructName = { ... } : defines a new struct type + + * ; : single-line comment + + * * text => any : special syntax to indicate 0 or more fields that + have a string key that maps to any value. Used to indicate a + generic JSON object. + + All timestamps and time-related values (e.g., offsets) in qlog are + logged as float64 in the millisecond resolution. + + Other qlog documents can define their own CDDL-compatible (struct) + types (e.g., separately for each Packet type that a protocol + supports). + +1.1.2. Serialization examples + + Serialization examples in this document use JSON ([JSON]) unless + otherwise indicated. + +2. Design goals + + The main tenets for the qlog schema design are: + + * Streamable, event-based logging + + * A flexible format that can reduce log producer overhead, at the + cost of increased complexity for consumers (e.g. tools) + + * Extensible and pragmatic + + * Aggregation and transformation friendly (e.g., the top-level + element for the non-streaming format is a container for individual + traces, group_ids can be used to tag events to a particular + context) + + * Metadata is stored together with event data + +3. QlogFile schema + + A qlog using the QlogFile schema can contain several individual + traces and logs from multiple vantage points that are in some way + related. The top-level element in this schema defines only a small + set of "header" fields and an array of component traces, defined in + Figure 2 as: + + QlogFile = { + qlog_version: text + ? qlog_format: text .default "JSON" + ? title: text + ? description: text + ? traces: [+ Trace / + TraceError] + } + + Figure 2: QlogFile definition + + The required "qlog_version" field MUST have the value "0.4". + + The optional "qlog_format" field indicates the serialization format. + Its value MUST either be one of the options defined in this document + (i.e., Section 10) or the field MUST be omitted entirely. When the + field is omitted the default value of "JSON" applies. + + The optional "title" and "description" fields provide additional + free-text information about the file. + + The optional "traces" field contains an array of qlog traces + (Section 3.2), each of which contain metadata and an array of qlog + events (Section 6). + + In order to make it easier to parse and identify qlog files and their + serialization format, the "qlog_version" and "qlog_format" fields and + their values SHOULD be in the first 256 characters/bytes of the + resulting log file. + + Where a qlog file is serialized to a JSON format, one of the + downsides is that it is inherently a non-streamable format. Put + differently, it is not possible to simply append new qlog events to a + log file without "closing" this file at the end by appending "]}]}". + Without these closing tags, most JSON parsers will be unable to parse + the file entirely. The alternative QlogFileSeq (Section 4) is better + suited to streaming. + + JSON serialization example: + + { + "qlog_version": "0.4", + "qlog_format": "JSON", + "title": "Name of this particular qlog file (short)", + "description": "Description for this group of traces (long)", + "traces": [...] + } + + Figure 3: QlogFile example + +3.1. Traces + + It can be advantageous to group several related qlog traces together + in a single file. For example, it is possible to simultaneously + perform logging on the client, on the server, and on a single point + on their common network path. For analysis, it is useful to + aggregate these three individual traces together into a single file, + so it can be uniquely stored, transferred, and annotated. + + The QlogFile "traces" field is an array that contains a list of + individual qlog traces. When capturing a qlog at a vantage point, it + is expected that the traces field contains a single entry. Files can + be aggregated, for example as part of a post-processing operation, by + copying the traces in component to files into the combined "traces" + array of a new, aggregated qlog file. + +3.2. Trace + + The exact conceptual definition of a Trace can be fluid. For + example, a trace could contain all events for a single connection, + for a single endpoint, for a single measurement interval, for a + single protocol, etc. In the normal use case however, a trace is a + log of a single data flow collected at a single location or vantage + point. For example, for QUIC, a single trace only contains events + for a single logical QUIC connection for either the client or the + server. + + A Trace contains some metadata in addition to qlog events, defined in + Figure 4 as: + + Trace = { + ? title: text + ? description: text + ? common_fields: CommonFields + ? vantage_point: VantagePoint + events: [* Event] + } + + Figure 4: Trace definition + + The optional "title" and "description" fields provide additional + free-text information about the trace. + + The optional "common_fields" field is described in Section 6.9. + + The optional "vantage_point" field is described in Section 5. + + The semantics and context of the trace can mainly be deduced from the + entries in the "common_fields" list and "vantage_point" field. + + JSON serialization example: + + { + "title": "Name of this particular trace (short)", + "description": "Description for this trace (long)", + "common_fields": { + "ODCID": "abcde1234", + "time_format": "absolute" + }, + "vantage_point": { + "name": "backend-67", + "type": "server" + }, + "events": [...] + } + + Figure 5: Trace example + +3.3. TraceError + + A TraceError indicates that an attempt to find/convert a file for + inclusion in the aggregated qlog was made, but there was an error + during the process. Rather than silently dropping the erroneous + file, it can be explicitly included in the qlog file as an entry in + the "traces" array, defined in Figure 6 as: + + TraceError = { + error_description: text + + ; the original URI used for attempted find of the file + ? uri: text + ? vantage_point: VantagePoint + } + + Figure 6: TraceError definition + + JSON serialization example: + + { + "error_description": "File could not be found", + "uri": "/srv/traces/today/latest.qlog", + "vantage_point": { type: "server" } + } + + Figure 7: TraceError example + + Note that another way to combine events of different traces in a + single qlog file is through the use of the "group_id" field, + discussed in Section 6.7. + +4. QlogFileSeq schema + + A qlog file using the QlogFileSeq schema can be serialized to a + streamable JSON format called JSON Text Sequences (JSON-SEQ) + ([RFC7464]). The top-level element in this schema defines only a + small set of "header" fields and an array of component traces, + defined in Figure 2 as: + + QlogFileSeq = { + qlog_format: "JSON-SEQ" + qlog_version: text + ? title: text + ? description: text + trace: TraceSeq + } + + Figure 8: QlogFileSeq definition + + The required "qlog_format" field MUST have the value "JSON-SEQ". + + The required "qlog_version" field MUST have the value "0.4". + + The optional "title" and "description" fields provide additional + free-text information about the file. + + The optional "trace" field contains a singular trace metadata. All + qlog events in the file are related to this trace. + + JSON-SEQ serialization example: + + // list of qlog events, serialized in accordance with RFC 7464, + // starting with a Record Separator character and ending with a + // newline. + // For display purposes, Record Separators are rendered as + + { + "qlog_version": "0.4", + "qlog_format": "JSON-SEQ", + "title": "Name of JSON Text Sequence qlog file (short)", + "description": "Description for this trace file (long)", + "trace": { + "common_fields": { + "protocol_type": ["QUIC","HTTP3"], + "group_id":"127ecc830d98f9d54a42c4f0842aa87e181a", + "time_format":"relative", + "reference_time": 1553986553572 + }, + "vantage_point": { + "name":"backend-67", + "type":"server" + } + } + } + {"time": 2, "name": "quic:parameters_set", "data": { ... } } + {"time": 7, "name": "quic:packet_sent", "data": { ... } } + ... + + Figure 9: Top-level element + + For further information about serialization, see Section 10.2. + +4.1. TraceSeq + + TraceSeq is used with QlogFileSeq. It is conceptually similar to a + Trace, with the exception that qlog events are not contained within + it, but rather appended after it in a QlogFileSeq. + + TraceSeq = { + ? title: text + ? description: text + ? common_fields: CommonFields + ? vantage_point: VantagePoint + } + + Figure 10: TraceSeq definition + +5. VantagePoint + + A VantagePoint describes the vantage point from which a trace + originates, defined in Figure 11 as: + + VantagePoint = { + ? name: text + type: VantagePointType + ? flow: VantagePointType + } + + ; client = endpoint which initiates the connection + ; server = endpoint which accepts the connection + ; network = observer in between client and server + VantagePointType = "client" / + "server" / + "network" / + "unknown" + + Figure 11: VantagePoint definition + + JSON serialization examples: + + { + "name": "aioquic client", + "type": "client" + } + + { + "name": "wireshark trace", + "type": "network", + "flow": "client" + } + + Figure 12: VantagePoint example + + The flow field is only required if the type is "network" (for + example, the trace is generated from a packet capture). It is used + to disambiguate events like "packet sent" and "packet received". + This is indicated explicitly because for multiple reasons (e.g., + privacy) data from which the flow direction can be otherwise inferred + (e.g., IP addresses) might not be present in the logs. + + Meaning of the different values for the flow field: * "client" + indicates that this vantage point follows client data flow semantics + (a "packet sent" event goes in the direction of the server). * + "server" indicates that this vantage point follow server data flow + semantics (a "packet sent" event goes in the direction of the + client). * "unknown" indicates that the flow's direction is unknown. + + Depending on the context, tools confronted with "unknown" values in + the vantage_point can either try to heuristically infer the semantics + from protocol-level domain knowledge (e.g., in QUIC, the client + always sends the first packet) or give the user the option to switch + between client and server perspectives manually. + +6. Events + + A qlog event is specified as a generic object with a number of member + fields and their associated data. Depending on the protocol and use + case, the exact member field names and their formats can differ + across implementations. This section lists the main, pre-defined and + reserved field names with specific semantics and expected + corresponding value formats. + + An Event is defined in Figure 13 as: + + Event = { + time: float64 + name: text + data: $ProtocolEventData + ? path: PathID + ? time_format: TimeFormat + ? protocol_type: ProtocolTypeList + ? group_id: GroupID + ? system_info: SystemInformation + + ; events can contain any amount of custom fields + * text => any + } + + Figure 13: Event definition + + Each qlog event MUST contain the mandatory fields: "time" + (Section 6.1), "name" (Section 6.2), and "data" (Section 6.3). + + Each qlog event MAY contain the optional fields: "time_format" + (Section 6.1), "protocol_type" (Section 6.5), "trigger" + (Section 6.6), and "group_id" (Section 6.7). + + Multiple events can appear in a Trace or TraceSeq and they might + contain fields with identical values. It is possible to optimize out + this duplication using "common_fields" (Section 6.9). + + The specific values for each of these fields and their semantics are + defined in separate documents, depending on protocol or use case. + For example: event definitions for QUIC and HTTP/3 can be found in + [QLOG-QUIC] and [QLOG-H3]. + + Events are intended to be extended with custom fields, therefore they + MAY contain other fields not defined in this document. Custom fields + may be known or unknown to tools. Tools SHOULD allow for the + presence of unknown event fields, but their semantics depend on the + context of the log usage. + + JSON serialization: + + { + "time": 1553986553572, + + "name": "quic:packet_sent", + "data": { ... }, + + "protocol_type": ["QUIC","HTTP3"], + "group_id": "127ecc830d98f9d54a42c4f0842aa87e181a", + + "time_format": "absolute", + + "ODCID": "127ecc830d98f9d54a42c4f0842aa87e181a" + } + + Figure 14: Event example + +6.1. Timestamps + + An event's "time" field indicates the timestamp at which the event + occurred. Its value is typically the Unix timestamp since the 1970 + epoch (number of milliseconds since midnight UTC, January 1, 1970, + ignoring leap seconds). However, qlog supports two more succinct + timestamps formats to allow reducing file size. The employed format + is indicated in the "time_format" field, which allows one of three + values: "absolute", "delta" or "relative". + + TimeFormat = "absolute" / + "delta" / + "relative" + + Figure 15: TimeFormat definition + + * Absolute: Include the full absolute timestamp with each event. + This approach uses the largest amount of characters. This is also + the default value of the "time_format" field. + + * Delta: Delta-encode each time value on the previously logged + value. The first event in a trace typically logs the full + absolute timestamp. This approach uses the least amount of + characters. + + * Relative: Specify a full "reference_time" timestamp (typically + this is done up-front in "common_fields", see Section 6.9) and + include only relatively-encoded values based on this + reference_time with each event. The "reference_time" value is + typically the first absolute timestamp. This approach uses a + medium amount of characters. + + The first option is good for stateless loggers, the second and third + for stateful loggers. The third option is generally preferred, since + it produces smaller files while being easier to reason about. An + example for each option can be seen in Figure 16. + + The absolute approach will use: + 1500, 1505, 1522, 1588 + + The delta approach will use: + 1500, 5, 17, 66 + + The relative approach will: + - set the reference_time to 1500 in "common_fields" + - use: 0, 5, 22, 88 + + Figure 16: Three different approaches for logging timestamps + + One of these options is typically chosen for the entire trace (put + differently: each event has the same value for the "time_format" + field). Each event MUST include a timestamp in the "time" field. + + Events in each individual trace SHOULD be logged in strictly + ascending timestamp order (though not necessarily absolute value, for + the "delta" format). Tools MAY sort all events on the timestamp + before processing them, though are not required to (as this could + impose a significant processing overhead). This can be a problem + especially for multi-threaded and/or streaming loggers, who could + consider using a separate post-processor to order qlog events in time + if a tool do not provide this feature. + + Timestamps do not have to use the UNIX epoch timestamp as their + reference. For example for privacy considerations, any initial + reference timestamps (for example "endpoint uptime in ms" or "time + since connection start in ms") can be chosen. Tools SHOULD NOT + assume the ability to derive the absolute Unix timestamp from qlog + traces, nor allow on them to relatively order events across two or + more separate traces (in this case, clock drift should also be taken + into account). + +6.2. Names + + Events differ mainly in the type of metadata associated with them. + The "name" field is an identifier that parsers can use to decide how + to interpret the event metadata contained in the "data" field (see + Section 6.3). + + Event names indicate a category and type. The "name" field MUST + contain a non-empty character sequence representing a category, + followed by a colon (':'), followed by a non-empty character sequence + representing a type. + + Category allows a higher-level grouping of events per specific event + type. For example for QUIC and HTTP/3, the different categories + could be "quic", "h3", "qpack", and "recovery". Within these + categories, the event type provides additional granularity. For + example for QUIC and HTTP/3, within the "quic" category, there would + be "packet_sent" and "packet_received" events. + + JSON serialization example: + + { + "name": "quic:packet_sent" + } + + Figure 17: An event with category "quic" and type "packet_sent". + +6.3. Data + + An event's "data" field is a generic key-value map (e.g., JSON + object). It defines the per-event metadata that is to be logged. + Its specific subfields and their semantics are defined per specific + event type. For example, data field definitions for QUIC and HTTP/3 + can be found in [QLOG-QUIC] and [QLOG-H3]. + + In order to keep qlog fully extensible, two separate CDDL extension + points ("sockets" or "plugs") are used to fully define data fields. + + Firstly, to allow existing data field definitions to be extended (for + example by adding an additional field needed for a new protocol + feature), a CDDL "group socket" is used. This takes the form of a + subfield with a name of * $$CATEGORY-NAME-extension. This field acts + as a placeholder that can later be replaced with newly defined fields + by assigning them to the socket with the //= operator. Multiple + extensions can be assigned to the same group socket. An example is + shown in Figure 18. + + ; original definition in document A + MyCategoryEventX = { + field_a: uint8 + + * $$mycategory-eventx-extension + } + + ; later extension of EventX in document B + $$mycategory-eventx-extension //= ( + ? additional_field_b: bool + ) + + ; another extension of EventX in document C + $$mycategory-eventx-extension //= ( + ? additional_field_c: text + ) + + ; if document A, B and C are then used in conjunction, + ; the combined MyCategoryEventX CDDL is equivalent to this: + MyCategoryEventX = { + field_a: uint8 + + ? additional_field_b: bool + ? additional_field_c: text + } + + Figure 18: Example of using a generic CDDL group socket to extend + an existing event data definition + + Secondly, to allow documents to define fully new event data field + definitions (as opposed to extend existing ones), a CDDL "type + socket" is used. For this purpose, the type of the "data" field in + the qlog Event type (see Figure 13) is the extensible + $ProtocolEventData type. This field acts as an open enum of possible + types that are allowed for the data field. As such, any new event + data field is defined as its own CDDL type and later merged with the + existing $ProtocolEventData enum using the /= extension operator. + Any generic key-value map type can be assigned to $ProtocolEventData + (the only common "data" subfield defined in this document is the + optional trigger field, see Section 6.6). An example of this setup + is shown in Figure 19. + + ; We define two separate events in a single document + MyCategoryEvent1 /= { + field_1: uint8 + + ? trigger: text + + * $$mycategory-event1-extension + } + + MyCategoryEvent2 /= { + field_2: bool + + ? trigger: text + + * $$mycategory-event2-extension + } + + ; the events are both merged with the existing + ; $ProtocolEventData type enum + $ProtocolEventData /= MyCategoryEvent1 / MyCategoryEvent2 + + ; the "data" field of a qlog event can now also be of type + ; MyCategoryEvent1 and MyCategoryEvent2 + + Figure 19: ProtocolEventData extension + + Documents defining new qlog events MUST properly extend + $ProtocolEventData when defining data fields to enable automated + validation of aggregated qlog schemas. Furthermore, they SHOULD + properly add a * $$CATEGORY-NAME-extension extension field to newly + defined event data to allow the new events to be properly extended by + other documents. + + A combined but purely illustrative example of the use of both + extension points for a conceptual QUIC "packet_sent" event is shown + in Figure 20: + + ; defined in the main QUIC event document + TransportPacketSent = { + ? packet_size: uint16 + header: PacketHeader + ? frames:[* QuicFrame] + ? trigger: "pto_probe" / + "retransmit_timeout" / + "bandwidth_probe" + + * $$transport-packetsent-extension + } + + ; Add the event to the global list of recognized qlog events + $ProtocolEventData /= TransportPacketSent + + ; Defined in a separate document that describes a + ; theoretical QUIC protocol extension + $$transport-packetsent-extension //= ( + ? additional_field: bool + ) + + ; If both documents are utilized at the same time, + ; the following JSON serialization would pass an automated + ; CDDL schema validation check: + + { + "time": 123456, + "category": "transport", + "name": "packet_sent", + "data": { + "packet_size": 1280, + "header": { + "packet_type": "1RTT", + "packet_number": 123 + }, + "frames": [ + { + "frame_type": "stream", + "length": 1000, + "offset": 456 + }, + { + "frame_type": "padding" + } + ], + additional_field: true + } + } + + Figure 20: Example of an extended 'data' field for a conceptual + QUIC packet_sent event + +6.4. Path + + A qlog event can be associated with a single "network path" (usually, + but not always, identified by a 4-tuple of IP addresses and ports). + In many cases, the path will be the same for all events in a given + trace, and does not need to be logged explicitly with each event. In + this case, the "path" field can be omitted (in which case the default + value of "" is assumed) or reflected in "common_fields" instead (see + Section 6.9). + + However, in some situations, such as during QUIC's Connection + Migration or when using Multipath features, it is useful to be able + to split events across multiple (concurrent) paths. + + Definition: + + PathID = text .default "" + + Figure 21: PathID definition + + The "path" field is an identifier that is associated with a single + network path. This document intentionally does not define further + how to choose this identifier's value per-path or how to potentially + log other parameters that can be associated with such a path. This + is left for other documents. Implementers are free to encode path + information directly into the PathID or to log associated info in a + separate event. For example, QUIC has the "path_assigned" event to + couple the PathID value to a specific path configuration, see + [QLOG-QUIC]. + +6.5. ProtocolTypeList and ProtocolType + + An event's "protocol_type" array field indicates to which protocols + (or protocol "stacks") this event belongs. This allows a single qlog + file to aggregate traces of different protocols (e.g., a web server + offering both TCP+HTTP/2 and QUIC+HTTP/3 connections). + + ProtocolTypeList = [+ $ProtocolType] + + $ProtocolType /= "UNKNOWN" + + Figure 22: ProtocolTypeList and ProtocolType socket definition + + For example, QUIC and HTTP/3 events have the "QUIC" and "HTTP3" + protocol_type entry values, see [QLOG-QUIC] and [QLOG-H3]. + + Typically however, all events in a single trace are of the same few + protocols, and this array field is logged once in "common_fields", + see Section 6.9. + +6.6. Triggers + + Sometimes, additional information is needed in the case where a + single event can be caused by a variety of other events. In the + normal case, the context of the surrounding log messages gives a hint + as to which of these other events was the cause. However, in highly- + parallel and optimized implementations, corresponding log messages + might separated in time. Another option is to explicitly indicate + these "triggers" in a high-level way per-event to get more fine- + grained information without much additional overhead. + + In qlog, the optional "trigger" field contains a string value + describing the reason (if any) for this event instance occurring, see + Section 6.3. While this "trigger" field could be a property of the + qlog Event itself, it is instead a property of the "data" field + instead. This choice was made because many event types do not + include a trigger value, and having the field at the Event-level + would cause overhead in some serializations. Additional information + on the trigger can be added in the form of additional member fields + of the "data" field value, yet this is highly implementation- + specific, as are the trigger field's string values. + + One purely illustrative example of some potential triggers for QUIC's + "packet_dropped" event is shown in Figure 23: + + TransportPacketDropped = { + ? packet_type: PacketType + ? raw_length: uint16 + ? trigger: "key_unavailable" / + "unknown_connection_id" / + "decrypt_error" / + "unsupported_version" + } + + Figure 23: Trigger example + +6.7. Grouping + + As discussed in Section 3.2, a single qlog file can contain several + traces taken from different vantage points. However, a single trace + from one endpoint can also contain events from a variety of sources. + For example, a server implementation might choose to log events for + all incoming connections in a single large (streamed) qlog file. As + such, a method for splitting up events belonging to separate logical + entities is required. + + The simplest way to perform this splitting is by associating a "group + id" to each event that indicates to which conceptual "group" each + event belongs. A post-processing step can then extract events per + group. However, this group identifier can be highly protocol and + context-specific. In the example above, the QUIC "Original + Destination Connection ID" could be used to uniquely identify a + connection. As such, they might add a "ODCID" field to each event. + However, a middlebox logging IP or TCP traffic might rather use four- + tuples to identify connections, and add a "four_tuple" field. + + As such, to provide consistency and ease of tooling in cross-protocol + and cross-context setups, qlog instead defines the common "group_id" + field, which contains a string value. Implementations are free to + use their preferred string serialization for this field, so long as + it contains a unique value per logical group. Some examples can be + seen in Figure 25. + + GroupID = text + + Figure 24: GroupID definition + + JSON serialization example for events grouped by four tuples and QUIC + connection IDs: + + "events": [ + { + "time": 1553986553579, + "protocol_type": ["TCP", "TLS", "HTTP2"], + "group_id": "ip1=2001:67c:1232:144:9498:6df6:f450:110b, + ip2=2001:67c:2b0:1c1::198,port1=59105,port2=80", + "name": "quic:packet_received", + "data": { ... } + }, + { + "time": 1553986553581, + "protocol_type": ["QUIC","HTTP3"], + "group_id": "127ecc830d98f9d54a42c4f0842aa87e181a", + "name": "quic:packet_sent", + "data": { ... } + } + ] + + Figure 25: GroupID example + + Note that in some contexts (for example a Multipath transport + protocol) it might make sense to add additional contextual per-event + fields (for example "path_id"), rather than use the group_id field + for that purpose. + + Note also that, typically, a single trace only contains events + belonging to a single logical group (for example, an individual QUIC + connection). As such, instead of logging the "group_id" field with + an identical value for each event instance, this field is typically + logged once in "common_fields", see Section 6.9. + +6.8. SystemInformation + + The "system_info" field can be used to record system-specific details + related to an event. This is useful, for instance, where an + application splits work across CPUs, processes, or threads and events + for a single trace occur on potentially different combinations + thereof. Each field is optional to support deployment diversity. + + SystemInformation = { + ? processor_id: uint32 + ? process_id: uint32 + ? thread_id: uint32 + } + +6.9. CommonFields + + As discussed in the previous sections, information for a typical qlog + event varies in three main fields: "time", "name" and associated + data. Additionally, there are also several more advanced fields that + allow mixing events from different protocols and contexts inside of + the same trace (for example "protocol_type" and "group_id"). In most + "normal" use cases however, the values of these advanced fields are + consistent for each event instance (for example, a single trace + contains events for a single QUIC connection). + + To reduce file size and making logging easier, qlog uses the + "common_fields" list to indicate those fields and their values that + are shared by all events in this component trace. This prevents + these fields from being logged for each individual event. An example + of this is shown in Figure 26. + + JSON serialization with repeated field values + per-event instance: + + { + "events": [{ + "group_id": "127ecc830d98f9d54a42c4f0842aa87e181a", + "protocol_type": ["QUIC","HTTP3"], + "time_format": "relative", + "reference_time": 1553986553572, + + "time": 2, + "name": "quic:packet_received", + "data": { ... } + },{ + "group_id": "127ecc830d98f9d54a42c4f0842aa87e181a", + "protocol_type": ["QUIC","HTTP3"], + "time_format": "relative", + "reference_time": 1553986553572, + + "time": 7, + "name": "http:frame_parsed", + "data": { ... } + } + ] + } + + JSON serialization with repeated field values instead + extracted to common_fields: + + { + "common_fields": { + "group_id": "127ecc830d98f9d54a42c4f0842aa87e181a", + "protocol_type": ["QUIC","HTTP3"], + "time_format": "relative", + "reference_time": 1553986553572 + }, + "events": [ + { + "time": 2, + "name": "quic:packet_received", + "data": { ... } + },{ + "time": 7, + "name": "http:frame_parsed", + "data": { ... } + } + ] + } + + Figure 26: CommonFields example + + An event's "common_fields" field is a generic dictionary of key-value + pairs, where the key is always a string and the value can be of any + type, but is typically also a string or number. As such, unknown + entries in this dictionary MUST be disregarded by the user and tools + (i.e., the presence of an unknown field is explicitly NOT an error). + + The list of default qlog fields that are typically logged in + common_fields (as opposed to as individual fields per event instance) + are shown in the listing below: + + CommonFields = { + ? path: PathID + ? time_format: TimeFormat + ? reference_time: float64 + ? protocol_type: ProtocolTypeList + ? group_id: GroupID + * text => any + } + + Figure 27: CommonFields definition + + Tools MUST be able to deal with these fields being defined either on + each event individually or combined in common_fields. Note that if + at least one event in a trace has a different value for a given + field, this field MUST NOT be added to common_fields but instead + defined on each event individually. Good example of such fields are + "time" and "data", who are divergent by nature. + +7. Raw packet and frame information + + While qlog is a high-level logging format, it also allows the + inclusion of most raw wire image information, such as byte lengths + and byte values. This is useful when for example investigating or + tuning packetization behavior or determining encoding/framing + overheads. However, these fields are not always necessary, can take + up considerable space, and can have a considerable privacy and + security impact (see Section 13). Where applicable, these fields are + grouped in a separate, optional, field named "raw" of type RawInfo. + The exact definition of entities, headers, trailers and payloads + depend on the protocol used. + + RawInfo = { + + ; the full byte length of the entity (e.g., packet or frame), + ; including possible headers and trailers + ? length: uint64 + + ; the byte length of the entity's payload, + ; excluding possible headers or trailers + ? payload_length: uint64 + + ; the (potentially truncated) contents of the full entity, + ; including headers and possibly trailers + ? data: hexstring + } + + Figure 28: RawInfo definition + + The RawInfo:data field can be truncated for privacy or security + purposes, see Section 10.4. In this case, the length and + payload_length fields should still indicate the non-truncated lengths + when used for debugging purposes. + + This document does not specify explicit header_length or + trailer_length fields. In protocols without trailers, header_length + can be calculated by subtracting the payload_length from the length. + In protocols with trailers (e.g., QUIC's AEAD tag), event definition + documents SHOULD define how to support header_length calculation. + +8. Common events and data classes + + There are some event types and data classes that are common across + protocols, applications, and use cases. This section specifies such + common definitions. + + GenericEventData = GenericError / + GenericWarning / + GenericInfo / + GenericDebug + + SimulationEventData = SimulationScenario / + SimulationMarker + + $ProtocolEventData /= GenericEventData / SimulationEventData + + Figure 29: ProtocolEventData extension for common events + +8.1. Generic events + + In typical logging setups, users utilize a discrete number of well- + defined logging categories, levels or severities to log freeform + (string) data. This generic events category replicates this approach + to allow implementations to fully replace their existing text-based + logging by qlog. This is done by providing events to log generic + strings for the typical well-known logging levels (error, warning, + info, debug, verbose). + + For the events defined below, the "category" is "generic" and their + "type" is the name of the heading in lowercase (e.g., the "name" of + the error event is "generic:error"). + +8.1.1. error + + Used to log details of an internal error that might not get reflected + on the wire. It has Core importance level; see Section 9.2. + + GenericError = { + ? code: uint64 + ? message: text + + * $$generic-error-extension + } + + Figure 30: GenericError definition + +8.1.2. warning + + Used to log details of an internal warning that might not get + reflected on the wire. It has Base importance level; see + Section 9.2. + + GenericWarning = { + ? code: uint64 + ? message: text + + * $$generic-warning-extension + } + + Figure 31: GenericWarning definition + +8.1.3. info + + Used mainly for implementations that want to use qlog as their one + and only logging format but still want to support unstructured string + messages. The event has Extra importance level; see Section 9.2. + + GenericInfo = { + message: text + + * $$generic-info-extension + } + + Figure 32: GenericInfo definition + +8.1.4. debug + + Used mainly for implementations that want to use qlog as their one + and only logging format but still want to support unstructured string + messages. The event has Extra importance level; see Section 9.2. + + GenericDebug = { + message: text + + * $$generic-debug-extension + } + + Figure 33: GenericDebug definition + +8.1.5. verbose + + Used mainly for implementations that want to use qlog as their one + and only logging format but still want to support unstructured string + messages. The event has Extra importance level; see Section 9.2. + + GenericVerbose = { + message: text + + * $$generic-verbose-extension + } + + Figure 34: GenericVerbose definition + +8.2. Simulation events + + When evaluating a protocol implementation, one typically sets up a + series of interoperability or benchmarking tests, in which the test + situations can change over time. For example, the network bandwidth + or latency can vary during the test, or the network can be fully + disable for a short time. In these setups, it is useful to know when + exactly these conditions are triggered, to allow for proper + correlation with other events. + + For the events defined below, the "category" is "simulation" and + their "type" is the name of the heading in lowercase (e.g., the + "name" of the scenario event is "simulation:scenario"). + +8.2.1. scenario + + Used to specify which specific scenario is being tested at this + particular instance. This supports, for example, aggregation of + several simulations into one trace (e.g., split by group_id). It has + Extra importance level; see Section 9.2. + + SimulationScenario = { + ? name: text + ? details: {* text => any } + + * $$simulation-scenario-extension + } + + Figure 35: SimulationScenario definition + +8.2.2. marker + + Used to indicate when specific emulation conditions are triggered at + set times (e.g., at 3 seconds in 2% packet loss is introduced, at 10s + a NAT rebind is triggered). It has Extra importance level; see + Section 9.2. + + SimulationMarker = { + ? type: text + ? message: text + + * $$simulation-marker-extension + } + + Figure 36: SimulationMarker definition + +9. Event definition guidelines + + This document defines the main schema for the qlog format together + with some common events, which on their own do not provide much + logging utility. It is expected that logging is extended with + specific, per-protocol event definitions that specify the name + (category + type) and data needed for each individual event. + Examples include the QUIC event definitions [QLOG-QUIC] and HTTP/3 + event definitions [QLOG-H3]. + + This section defines some basic annotations and concepts that SHOULD + be used by event definition documents. Doing so ensures a measure of + consistency that makes it easier for qlog implementers to support a + wide variety of protocols. + +9.1. Event design + + There are several ways of defining qlog events. In practice, two + main types of approach have been observed: a) those that map directly + to concepts seen in the protocols (e.g., packet_sent) and b) those + that act as aggregating events that combine data from several + possible protocol behaviors or code paths into one (e.g., + parameters_set). The latter are typically used as a means to reduce + the amount of unique event definitions, as reflecting each possible + protocol event as a separate qlog entity would cause an explosion of + event types. + + Additionally, logging duplicate data is typically prevented as much + as possible. For example, packet header values that remain + consistent across many packets are split into separate events (for + example spin_bit_updated or connection_id_updated for QUIC). + + Finally, when logging additional state change events, those state + changes can often be directly inferred from data on the wire (for + example flow control limit changes). As such, if the implementation + is bug-free and spec-compliant, logging additional events is + typically avoided. Exceptions have been made for common events that + benefit from being easily identifiable or individually logged (for + example packets_acked). + +9.2. Event importance levels + + Depending on how events are designed, it may be that several events + allow the logging of similar or overlapping data. For example the + separate QUIC connection_started event overlaps with the more generic + connection_state_updated. In these cases, it is not always clear + which event should be logged or used, and which event should take + precedence if e.g., both are present and provide conflicting + information. + + To aid in this decision making, qlog defines three event importance + levels, in decreasing order of importance and expected usage: + + * Core + + * Base + + * Extra + + Events definitions SHOULD assign an importance level. + + Core-level events SHOULD be present in all qlog files for a given + protocol. These are typically tied to basic packet and frame parsing + and creation, as well as listing basic internal metrics. Tool + implementers SHOULD expect and add support for these events, though + SHOULD NOT expect all Core events to be present in each qlog trace. + + Base-level events add additional debugging options and MAY be present + in qlog files. Most of these can be implicitly inferred from data in + Core events (if those contain all their properties), but for many it + is better to log the events explicitly as well, making it clearer how + the implementation behaves. These events are for example tied to + passing data around in buffers, to how internal state machines + change, and used to help show when decisions are actually made based + on received data. Tool implementers SHOULD at least add support for + showing the contents of these events, if they do not handle them + explicitly. + + Extra-level events are considered mostly useful for low-level + debugging of the implementation, rather than the protocol. They + allow more fine-grained tracking of internal behavior. As such, they + MAY be present in qlog files and tool implementers MAY add support + for these, but they are not required to. + + Note that in some cases, implementers might not want to log for + example data content details in Core-level events due to performance + or privacy considerations. In this case, they SHOULD use (a subset + of) relevant Base-level events instead to ensure usability of the + qlog output. As an example, implementations that do not log QUIC + packet_received events and thus also not which (if any) ACK frames + the packet contains, SHOULD log packets_acked events instead. + + Finally, for event types whose data (partially) overlap with other + event types' definitions, where necessary the event definition + document should include explicit guidance on which to use in specific + situations. + +9.3. Custom fields + + Event definition documents are free to define new category and event + types, top-level fields (e.g., a per-event field indicating its + privacy properties or path_id in multipath protocols), as well as + values for the "trigger" property within the "data" field, or other + member fields of the "data" field, as they see fit. + + They however SHOULD NOT expect non-specialized tools to recognize or + visualize this custom data. However, tools SHOULD make an effort to + visualize even unknown data if possible in the specific tool's + context. If they do not, they MUST ignore these unknown fields. + +10. Serializing qlog + + qlog schema definitions in this document are intentionally agnostic + to serialization formats. The choice of format is an implementation + decision. + + Other documents related to qlog (for example event definitions for + specific protocols), SHOULD be similarly agnostic to the employed + serialization format and SHOULD clearly indicate this. If not, they + MUST include an explanation on which serialization formats are + supported and on how to employ them correctly. + + Serialization formats make certain tradeoffs between usability, + flexibility, interoperability, and efficiency. Implementations + should take these into consideration when choosing a format. Some + examples of possible formats are JSON, CBOR, CSV, protocol buffers, + flatbuffers, etc. which each have their own characteristics. For + instance, a textual format like JSON can be more flexible than a + binary format but more verbose, typically making it less efficient + than a binary format. A plaintext readable (yet relatively large) + format like JSON is potentially more usable for users operating on + the logs directly, while a more optimized yet restricted format can + better suit the constraints of a large scale operation. A custom or + restricted format could be more efficient for analysis with custom + tooling but might not be interoperable with general-purpose qlog + tools. + + Considering these tradeoffs, JSON-based serialization formats provide + features that make them a good starting point for qlog flexibility + and interoperability. For these reasons, JSON is a recommended + default and expanded considerations are given to how to map qlog to + JSON (Section 10.1, and its streaming counterpart JSON Text Sequences + (Section 10.2. Section 10.3 presents interoperability considerations + for both formats, and Section 10.5 presents potential optimizations. + + Serialization formats require appropriate deserializers/parsers. The + "qlog_format" field (Section 3) is used to indicate the chosen + serialization format. + +10.1. qlog to JSON mapping + + As described in Section 3, JSON is the default qlog serialization. + When mapping qlog to normal JSON, QlogFile (Figure 2) is used and the + "qlog_format" field MUST have the value "JSON". The file extension/ + suffix SHOULD be ".qlog". The Media Type, if any, SHOULD be + "application/qlog+json" per [RFC6839]. + + In accordance with Section 8.1 of [RFC8259], JSON files are required + to use UTF-8 both for the file itself and the string values it + contains. In addition, all qlog field names MUST be lowercase when + serialized to JSON. + + In order to serialize CDDL-based qlog event and data structure + definitions to JSON, the official CDDL-to-JSON mapping defined in + Appendix E of [CDDL] SHOULD be employed. + +10.2. qlog to JSON Text Sequences mapping + + One of the downsides of using normal JSON is that it is inherently a + non-streamable format. A qlog serializer could work around this by + opening a file, writing the required opening data, streaming qlog + events by appending them, and then finalizing the log by appending + appropriate closing tags e.g., "]}]}". However, failure to append + closing tags, could lead to problems because most JSON parsers will + fail if a document is malformed. Some streaming JSON parsers are + able to handle missing closing tags, however they are not widely + deployed in popular environments (e.g., Web browsers) + + To overcome the issues related to JSON streaming, a qlog mapping to a + streamable JSON format called JSON Text Sequences (JSON-SEQ) + ([RFC7464]) is provided. + + JSON Text Sequences are very similar to JSON, except that objects are + serialized as individual records, each prefixed by an ASCII Record + Separator (, 0x1E), and each ending with an ASCII Line Feed + character (\n, 0x0A). Note that each record can also contain any + amount of newlines in its body, as long as it ends with a newline + character before the next character. + + In order to leverage the streaming capability, each qlog event is + serialized and interpreted as an individual JSON Text Sequence + record, that is appended as a new object to the back of an event + stream or log file. Put differently, unlike default JSON, it does + not require a document to be wrapped as a full object with "{ ... }" + or "[... ]". + + This alternative record streaming approach cannot be accommodated by + QlogFile (Figure 2). Instead, QlogFileSeq is defined in Figure 8, + which notably includes only a single trace (TraceSeq) and omits an + explicit "events" array. An example is provided in Figure 9. The + "group_id" field can still be used on a per-event basis to include + events from conceptually different sources in a single JSON-SEQ qlog + file. + + When mapping qlog to JSON-SEQ, the "qlog_format" field MUST have the + value "JSON-SEQ". The file extension/suffix SHOULD be ".sqlog" (for + "streaming" qlog). The Media Type, if any, SHOULD be "application/ + qlog+json-seq" per [RFC8091]. + + While not specifically required by the JSON-SEQ specification, all + qlog field names MUST be lowercase when serialized to JSON-SEQ. + + In order to serialize all other CDDL-based qlog event and data + structure definitions to JSON-SEQ, the official CDDL-to-JSON mapping + defined in Appendix E of [CDDL] SHOULD be employed. + +10.2.1. Supporting JSON Text Sequences in tooling + + Note that JSON Text Sequences are not supported in most default + programming environments (unlike normal JSON). However, several + custom JSON-SEQ parsing libraries exist in most programming languages + that can be used and the format is easy enough to parse with existing + implementations (i.e., by splitting the file into its component + records and feeding them to a normal JSON parser individually, as + each record by itself is a valid JSON object). + +10.3. JSON Interoperability + + Some JSON implementations have issues with the full JSON format, + especially those integrated within a JavaScript environment (e.g., + Web browsers, NodeJS). I-JSON (Internet-JSON) is a subset of JSON + for such environments; see [I-JSON]. One of the key limitations of + JavaScript, and thus I-JSON, is that it cannot represent full 64-bit + integers in standard operating mode (i.e., without using BigInt + extensions), instead being limited to the range -(2^53)+1 to (2^53)- + 1. + + To accommodate such constraints in CDDL, Appendix E of [CDDL] + recommends defining new CDDL types for int64 and uint64 that limit + their values to the restricted 64-bit integer range. However, some + of the protocols that qlog is intended to support (e.g., QUIC, + HTTP/3), can use the full range of uint64 values. + + As such, to support situations where I-JSON is in use, seralizers MAY + encode uint64 values using JSON strings. qlog parsers, therefore, + SHOULD support parsing of uint64 values from JSON strings or JSON + numbers unless there is out-of-band information indicating that + neither the serializer nor parser are constrained by I-JSON. + +10.4. Truncated values + + For some use cases (e.g., limiting file size, privacy), it can be + necessary not to log a full raw blob (using the hexstring type) but + instead a truncated value. For example, one might only store the + first 100 bytes of an HTTP response body to be able to discern which + file it actually contained. In these cases, the original byte-size + length cannot be obtained from the serialized value directly. + + As such, all qlog schema definitions SHOULD include a separate, + length-indicating field for all fields of type hexstring they + specify, see for example Section 7. This not only ensures the + original length can always be retrieved, but also allows the omission + of any raw value bytes of the field completely (e.g., out of privacy + or security considerations). + + To reduce overhead however and in the case the full raw value is + logged, the extra length-indicating field can be left out. As such, + tools MUST be able to deal with this situation and derive the length + of the field from the raw value if no separate length-indicating + field is present. The main possible permutations are shown by + example in Figure 37. + + // both the content's value and its length are present + // (length is redundant) + { + "content_length": 5, + "content": "051428abff" + } + + // only the content value is present, indicating it + // represents the content's full value. The byte + // length is obtained by calculating content.length / 2 + { + "content": "051428abff" + } + + // only the length is present, meaning the value + // was omitted + { + "content_length": 5, + } + + // both value and length are present, but the lengths + // do not match: the value was truncated to + // the first three bytes. + { + "content_length": 5, + "content": "051428" + } + + Figure 37: Example for serializing truncated hexstrings + +10.5. Optimization of serialized data + + Both the JSON and JSON-SEQ formatting options described above are + serviceable in general small to medium scale (debugging) setups. + However, these approaches tend to be relatively verbose, leading to + larger file sizes. Additionally, generalized JSON(-SEQ) + (de)serialization performance is typically (slightly) lower than that + of more optimized and predictable formats. Both aspects present + challenges to large scale setups, though they may still be practical + to deploy; see [ANRW-2020]. JSON and JSON-SEQ compress very well + using commonly-available algorithms such as GZIP or Brotli. + + During the development of qlog, a multitude of alternative formatting + and optimization options were assessed and the results are summarized + on the qlog github repository (https://github.com/quiclog/internet- + drafts/issues/30#issuecomment-617675097). + + Formal definition of additional qlog formats or encodings that use + the optimization techniques described here, or any other optimization + technique is left to future activity that can apply the following + guidelines. + + In order to help tools correctly parse and process serialized qlog, + it is RECOMMENDED that new formats also define suitable file + extensions and media types. This provides a clear signal and avoids + the need to provide out-of-band information or to rely on heuristic + fallbacks; see Section 12. + +11. Methods of access and generation + + Different implementations will have different ways of generating and + storing qlogs. However, there is still value in defining a few + default ways in which to steer this generation and access of the + results. + +11.1. Set file output destination via an environment variable + + To provide users control over where and how qlog files are created, + two environment variables are defined. The first, QLOGFILE, + indicates a full path to where an individual qlog file should be + stored. This path MUST include the full file extension. The second, + QLOGDIR, sets a general directory path in which qlog files should be + placed. This path MUST include the directory separator character at + the end. + + In general, QLOGDIR should be preferred over QLOGFILE if an endpoint + is prone to generate multiple qlog files. This can for example be + the case for a QUIC server implementation that logs each QUIC + connection in a separate qlog file. An alternative that uses + QLOGFILE would be a QUIC server that logs all connections in a single + file and uses the "group_id" field (Section 6.7) to allow post-hoc + separation of events. + + Implementations SHOULD provide support for QLOGDIR and MAY provide + support for QLOGFILE. + + When using QLOGDIR, it is up to the implementation to choose an + appropriate naming scheme for the qlog files themselves. The chosen + scheme will typically depend on the context or protocols used. For + example, for QUIC, it is recommended to use the Original Destination + Connection ID (ODCID), followed by the vantage point type of the + logging endpoint. Examples of all options for QUIC are shown in + Figure 38. + + Command: QLOGFILE=/srv/qlogs/client.qlog quicclientbinary + + Should result in the the quicclientbinary executable logging a + single qlog file named client.qlog in the /srv/qlogs directory. + This is for example useful in tests when the client sets up + just a single connection and then exits. + + Command: QLOGDIR=/srv/qlogs/ quicserverbinary + + Should result in the quicserverbinary executable generating + several logs files, one for each QUIC connection. + Given two QUIC connections, with ODCID values "abcde" and + "12345" respectively, this would result in two files: + /srv/qlogs/abcde_server.qlog + /srv/qlogs/12345_server.qlog + + Command: QLOGFILE=/srv/qlogs/server.qlog quicserverbinary + + Should result in the the quicserverbinary executable logging + a single qlog file named server.qlog in the /srv/qlogs directory. + Given that the server handled two QUIC connections before it was + shut down, with ODCID values "abcde" and "12345" respectively, + this would result in event instances in the qlog file being + tagged with the "group_id" field with values "abcde" and "12345". + + Figure 38: Environment variable examples for a QUIC implementation + +12. Tooling requirements + + Tools ingestion qlog MUST indicate which qlog version(s), qlog + format(s), compression methods and potentially other input file + formats (for example .pcap) they support. Tools SHOULD at least + support .qlog files in the default JSON format (Section 10.1). + Additionally, they SHOULD indicate exactly which values for and + properties of the name (category and type) and data fields they look + for to execute their logic. Tools SHOULD perform a (high-level) + check if an input qlog file adheres to the expected qlog schema. If + a tool determines a qlog file does not contain enough supported + information to correctly execute the tool's logic, it SHOULD generate + a clear error message to this effect. + + Tools MUST NOT produce breaking errors for any field names and/or + values in the qlog format that they do not recognize. Tools SHOULD + indicate even unknown event occurrences within their context (e.g., + marking unknown events on a timeline for manual interpretation by the + user). + + Tool authors should be aware that, depending on the logging + implementation, some events will not always be present in all traces. + For example, using a circular logging buffer of a fixed size, it + could be that the earliest events (e.g., connection setup events) are + later overwritten by "newer" events. Alternatively, some events can + be intentionally omitted out of privacy or file size considerations. + Tool authors are encouraged to make their tools robust enough to + still provide adequate output for incomplete logs. + +13. Security and privacy considerations + + Protocols such as TLS [RFC8446] and QUIC [RFC9000] offer secure + protection for the wire image [RFC8546]. Logging can reveal aspects + of the wire image that would ordinarily be protected, creating + tension between observability, security and privacy, especially if + data can be correlated across data sources. + + qlog permits logging of a broad and detialed range of data. + Operators and implementers are responsible for deciding what data is + logged to address their requirements and constraints. As per + [RFC6973], operators must be aware that data could be compromised, + risking the privacy of all participants. Where entities expect + protocol features to ensure data privacy, logging might unknowingly + be subject to broader privacy risks, undermining their ability to + assess or respond effectively. + +13.1. Data at risk + + qlog operators and implementers need to consider security and privacy + risks when handling qlog data, including logging, storage, usage, and + more. The considerations presented in this section may pose varying + risks depending on the the data itself or its handling. + + The following is a non-exhaustive list of example data types that + could contain sensitive information that might allow identification + or correlation of individual connections, endpoints, users or + sessions across qlog or other data sources (e.g., captures of + encrypted packets): + + * IP addresses and transport protocol port numbers. + + * Session, Connection, or User identifiers e.g., QUIC Connection IDs + Section 9.5 of [RFC9000]). + + * System-level information e.g., CPU, process, or thread + identifiers. + + * Stored State e.g., QUIC address validation and retry tokens, TLS + session tickets, and HTTP cookies. + + * TLS decryption keys, passwords, and HTTP-level API access or + authorization tokens. + + * High-resolution event timestamps or inter-event timings, event + counts, packet sizes, and frame sizes. + + * Full or partial raw packet and frame payloads that are encrypted. + + * Full or partial raw packet and frame payloads that are plaintext + e.g., HTTP Field values, HTTP response data, or TLS SNI field + values. + +13.2. Operational implications and recommendations + + Operational considerations should focus on authorizing capture and + access to logs. Logging of Internet protocols using qlog can be + equivalent to the ability to store or read plaintext communications. + Without a more detailed analysis, all of the security considerations + of plaintext access apply. + + It is recommended that qlog capture is subject to access control and + auditing. These controls should support granular levels of + information capture based on role and permissions (e.g., capture of + more-sensitive data requires higher privileges). + + It is recommended that access to stored qlogs is subject to access + control and auditing. + + End users might not understand the implications of qlog to security + or privacy, and their environments might limit access control + techniques. Implementations should make enabling qlog conspicuous + (e.g., requiring clear and explicit actions to start a capture) and + resistant to social engineering, automation, or drive-by attacks; for + example, isolation or sandboxing of capture from other activities in + the same process or component. + + It is recommended that data retention policies are defined for the + storage of qlog files. + + It is recommended that qlog files are encrypted in transit and at + rest. + +13.3. Data minimization or anonymization + + Applying data minimization or anonymization techniques to qlog might + help address some security and privacy risks. However, removing or + anonymizing data without sufficient care might not enhance privacy or + security and could diminish the utility of qlog data. + + Operators and implementers should balance the value of logged data + with the potential risks of voluntary or involuntary disclosure to + trusted or untrusted entities. The data requirements and definition + of entities depends greatly on the intended use cases. For example, + a research project might be tightly scoped, time bound, and require + participants to explicitly opt in to having their data collected with + the intention for this to be shared in a publication. Whereas a + server administrator might desire to collect telemetry, from users + whom they have no relationship with, for continuing operational + needs. + + The most extreme form of minimization or anonymization is deleting a + field, equivalent to not logging it. qlog implementations should + offer fine-grained control for this on a per-use-case or per- + connection basis. + + Data can undergo anonymization, pseudonymization, permutation, + truncation, re-encryption, or aggregation; see Appendix B of + [DNS-PRIVACY] for techniques, especially regarding IP addresses. + However, operators should be cautious because many anonymization + methods have been shown to be insufficient to safeguard user privacy + or identity, particularly with large or easily correlated data sets. + + Operators should consider end user rights and preferences. Active + user participation (as indicated by [RFC6973]) on a per-qlog basis is + challenging but aligning qlog capture, storage, and removal with + existing user preference and privacy controls is crucial. Operators + should consider agressive approaches to deletion or aggregation. + + The most sensitive data in qlog is typically contained in RawInfo + type fields (see Section 7). Therefore, qlog users should exercise + caution and limit the inclusion of such fields for all but the most + stringent use cases. + +14. IANA Considerations + + There are no IANA considerations. + +15. References + +15.1. Normative References + + [CDDL] Birkholz, H., Vigano, C., and C. Bormann, "Concise Data + Definition Language (CDDL): A Notational Convention to + Express Concise Binary Object Representation (CBOR) and + JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, + June 2019, . + + [DNS-PRIVACY] + Dickinson, S., Overeinder, B., van Rijswijk-Deij, R., and + A. Mankin, "Recommendations for DNS Privacy Service + Operators", BCP 232, RFC 8932, DOI 10.17487/RFC8932, + October 2020, . + + [I-JSON] Bray, T., Ed., "The I-JSON Message Format", RFC 7493, + DOI 10.17487/RFC7493, March 2015, + . + + [JSON] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data + Interchange Format", STD 90, RFC 8259, + DOI 10.17487/RFC8259, December 2017, + . + + [JSON-Text-Sequences] + Williams, N., "JavaScript Object Notation (JSON) Text + Sequences", RFC 7464, DOI 10.17487/RFC7464, February 2015, + . + + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, + DOI 10.17487/RFC2119, March 1997, + . + + [RFC6839] Hansen, T. and A. Melnikov, "Additional Media Type + Structured Syntax Suffixes", RFC 6839, + DOI 10.17487/RFC6839, January 2013, + . + + [RFC6973] Cooper, A., Tschofenig, H., Aboba, B., Peterson, J., + Morris, J., Hansen, M., and R. Smith, "Privacy + Considerations for Internet Protocols", RFC 6973, + DOI 10.17487/RFC6973, July 2013, + . + + [RFC7464] Williams, N., "JavaScript Object Notation (JSON) Text + Sequences", RFC 7464, DOI 10.17487/RFC7464, February 2015, + . + + [RFC8091] Wilde, E., "A Media Type Structured Syntax Suffix for JSON + Text Sequences", RFC 8091, DOI 10.17487/RFC8091, February + 2017, . + + [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC + 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, + May 2017, . + + [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data + Interchange Format", STD 90, RFC 8259, + DOI 10.17487/RFC8259, December 2017, + . + + [RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based + Multiplexed and Secure Transport", RFC 9000, + DOI 10.17487/RFC9000, May 2021, + . + +15.2. Informative References + + [ANRW-2020] + Marx, R., Piraux, M., Quax, P., and W. Lamotte, "Debugging + QUIC and HTTP/3 with qlog and qvis", September 2020, + . + + [QLOG-H3] Marx, R., Niccolini, L., Seemann, M., and L. Pardue, + "HTTP/3 qlog event definitions", Work in Progress, + Internet-Draft, draft-ietf-quic-qlog-h3-events-07, 4 March + 2024, . + + [QLOG-QUIC] + Marx, R., Niccolini, L., Seemann, M., and L. Pardue, "QUIC + event definitions for qlog", Work in Progress, Internet- + Draft, draft-ietf-quic-qlog-quic-events-07, 4 March 2024, + . + + [RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol + Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018, + . + + [RFC8546] Trammell, B. and M. Kuehlewind, "The Wire Image of a + Network Protocol", RFC 8546, DOI 10.17487/RFC8546, April + 2019, . + +Acknowledgements + + Much of the initial work by Robin Marx was done at the Hasselt and KU + Leuven Universities. + + Thanks to Jana Iyengar, Brian Trammell, Dmitri Tikhonov, Stephen + Petrides, Jari Arkko, Marcus Ihlar, Victor Vasiliev, Mirja Kühlewind, + Jeremy Lainé, Kazu Yamamoto, Christian Huitema and Hugo Landau for + their feedback and suggestions. + +Change Log + + This section is to be removed before publishing as an RFC. + +Since draft-ietf-quic-qlog-main-schema-07: + + * Added path and PathID (#336) + + * Removed custom definition of uint64 type (#360, #388) + + * ProtocolEventBody is now called ProtocolEventData (#352) + + * Editorial changes (#364, #289, #353, #361, #362) + +Since draft-ietf-quic-qlog-main-schema-06: + + * Editorial reworking of the document (#331, #332) + + * Updated IANA considerations section (#333) + +Since draft-ietf-quic-qlog-main-schema-05: + + * Updated qlog_version to 0.4 (due to breaking changes) (#314) + + * Renamed 'transport' category to 'quic' (#302) + + * Added 'system_info' field (#305) + + * Removed 'summary' and 'configuration' fields (#308) + + * Editorial and formatting changes (#298, #303, #304, #316, #320, + #321, #322, #326, #328) + +Since draft-ietf-quic-qlog-main-schema-04: + + * Updated RawInfo definition and guidance (#243) + +Since draft-ietf-quic-qlog-main-schema-03: + + * Added security and privacy considerations discussion (#252) + +Since draft-ietf-quic-qlog-main-schema-02: + + * No changes - new draft to prevent expiration + +Since draft-ietf-quic-qlog-main-schema-01: + + * Change the data definition language from TypeScript to CDDL (#143) + +Since draft-ietf-quic-qlog-main-schema-00: + + * Changed the streaming serialization format from NDJSON to JSON + Text Sequences (#172) + + * Added Media Type definitions for various qlog formats (#158) + + * Changed to semantic versioning + +Since draft-marx-qlog-main-schema-draft-02: + + * These changes were done in preparation of the adoption of the + drafts by the QUIC working group (#137) + + * Moved RawInfo, Importance, Generic events and Simulation events to + this document. + + * Added basic event definition guidelines + + * Made protocol_type an array instead of a string (#146) + +Since draft-marx-qlog-main-schema-01: + + * Decoupled qlog from the JSON format and described a mapping + instead (#89) + + - Data types are now specified in this document and proper + definitions for fields were added in this format + + - 64-bit numbers can now be either strings or numbers, with a + preference for numbers (#10) + + - binary blobs are now logged as lowercase hex strings (#39, #36) + + - added guidance to add length-specifiers for binary blobs (#102) + + * Removed "time_units" from Configuration. All times are now in ms + instead (#95) + + * Removed the "event_fields" setup for a more straightforward JSON + format (#101,#89) + + * Added a streaming option using the NDJSON format (#109,#2,#106) + + * Described optional optimization options for implementers (#30) + + * Added QLOGDIR and QLOGFILE environment variables, clarified the + .well-known URL usage (#26,#33,#51) + + * Overall tightened up the text and added more examples + +Since draft-marx-qlog-main-schema-00: + + * All field names are now lowercase (e.g., category instead of + CATEGORY) + + * Triggers are now properties on the "data" field value, instead of + separate field types (#23) + + * group_ids in common_fields is now just also group_id + +Authors' Addresses + + Robin Marx (editor) + Akamai + Email: rmarx@akamai.com + + + Luca Niccolini (editor) + Meta + Email: lniccolini@meta.com + + + Marten Seemann (editor) + Email: martenseemann@gmail.com + + + Lucas Pardue (editor) + Cloudflare + Email: lucas@lucaspardue.com diff --git a/security-stuff/draft-ietf-quic-qlog-quic-events.html b/security-stuff/draft-ietf-quic-qlog-quic-events.html new file mode 100644 index 00000000..c3697755 --- /dev/null +++ b/security-stuff/draft-ietf-quic-qlog-quic-events.html @@ -0,0 +1,5171 @@ + + + + + + +QUIC event definitions for qlog + + + + + + + + + + + + + + + + + + + + + + + + + + +
Internet-DraftQUIC event definitions for qlogJune 2024
Marx, et al.Expires 29 December 2024[Page]
+
+
+
+
Workgroup:
+
QUIC
+
Internet-Draft:
+
draft-ietf-quic-qlog-quic-events-latest
+
Published:
+
+ +
+
Intended Status:
+
Standards Track
+
Expires:
+
+
Authors:
+
+
+
R. Marx, Ed. +
+
Akamai
+
+
+
L. Niccolini, Ed. +
+
Meta
+
+
+
M. Seemann, Ed. +
+
+
+
L. Pardue, Ed. +
+
Cloudflare
+
+
+
+
+

QUIC event definitions for qlog

+
+

Abstract

+

This document describes concrete qlog event definitions and their metadata for +QUIC events. These events can then be embedded in the higher level schema defined +in [QLOG-MAIN].

+
+
+

+Note to Readers +

+
    +
  • +

    Note to RFC editor: Please remove this section before publication.

    +
    • +
+

Feedback and discussion are welcome at +https://github.com/quicwg/qlog. Readers are +advised to refer to the "editor's draft" at that URL for an up-to-date version +of this document.

+

Concrete examples of integrations of this schema in +various programming languages can be found at +https://github.com/quiclog/qlog/.

+
+
+
+

+Status of This Memo +

+

+ This Internet-Draft is submitted in full conformance with the + provisions of BCP 78 and BCP 79.

+

+ Internet-Drafts are working documents of the Internet Engineering Task + Force (IETF). Note that other groups may also distribute working + documents as Internet-Drafts. The list of current Internet-Drafts is + at https://datatracker.ietf.org/drafts/current/.

+

+ Internet-Drafts are draft documents valid for a maximum of six months + and may be updated, replaced, or obsoleted by other documents at any + time. It is inappropriate to use Internet-Drafts as reference + material or to cite them other than as "work in progress."

+

+ This Internet-Draft will expire on 29 December 2024.

+
+
+ +
+
+ ▲

+Table of Contents +

+ +
+
+
+
+

+1. Introduction +

+

This document describes the values of the qlog name ("category" + "event") and +"data" fields and their semantics for the QUIC protocol (see +[QUIC-TRANSPORT], [QUIC-RECOVERY], and +[QUIC-TLS]) and some of its extensions (see +[QUIC-DATAGRAM] and [GREASEBIT]).

+
+
+

+1.1. Notational Conventions +

+

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", +"MAY", and "OPTIONAL" in this document are to be interpreted as +described in BCP 14 [RFC2119] [RFC8174] when, and only when, they +appear in all capitals, as shown here.

+

The event and data structure definitions in ths document are expressed +in the Concise Data Definition Language [CDDL] and its +extensions described in [QLOG-MAIN].

+

The following fields from [QLOG-MAIN] are imported and used: name, category, +type, data, group_id, protocol_type, importance, RawInfo, and time-related +fields.

+

As is the case for [QLOG-MAIN], the qlog schema definitions in this document +are intentionally agnostic to serialization formats. The choice of format is an +implementation decision.

+
+
+
+
+
+
+

+2. Overview +

+

This document describes how the QUIC protocol can be expressed in qlog using +the schema defined in [QLOG-MAIN]. QUIC protocol events are defined with a +category, a name (the concatenation of "category" and "event"), an "importance", +an optional "trigger", and "data" fields.

+

Some data fields use complex datastructures. These are represented as enums or +re-usable definitions, which are grouped together on the bottom of this document +for clarity.

+

When any event from this document is included in a qlog trace, the +protocol_type qlog array field MUST contain an entry with the value "QUIC":

+
+
+
+
+$ProtocolType /= "QUIC"
+
+
+
Figure 1: +ProtocolType extension for QUIC +
+
+

When the qlog group_id field is used, it is recommended to use QUIC's Original +Destination Connection ID (ODCID, the CID chosen by the client when first +contacting the server), as this is the only value that does not change over the +course of the connection and can be used to link more advanced QUIC packets (e.g., +Retry, Version Negotiation) to a given connection. Similarly, the ODCID should be +used as the qlog filename or file identifier, potentially suffixed by the +vantagepoint type (For example, abcd1234_server.qlog would contain the server-side +trace of the connection with ODCID abcd1234).

+
+
+

+2.1. Raw packet and frame information +

+

QUIC packets always include an AEAD authentication tag at the end. In general, +the length of the AEAD tag depends on the TLS cipher suite, although all cipher +suites used in QUIC v1 use a 16 byte tag. For the purposes of calculating the +lengths in fields of type RawInfo (as defined in [QLOG-MAIN]) related to QUIC +packets, the AEAD tag is regarded as a trailer.

+
+
+
+
+

+2.2. Events not belonging to a single connection +

+

A single qlog event trace is typically associated with a single QUIC connection. +However, for several types of events (for example, a Section 5.7 +event with trigger value of connection_unknown), it can be impossible to tie +them to a specific QUIC connection, especially on the server.

+

There are various ways to handle these events, each making certain tradeoffs +between file size overhead, flexibility, ease of use, or ease of +implementation. Some options include:

+
    +
  • +

    Log them in a separate endpoint-wide trace (or use a special group_id value) +not associated with a single connection.

    +
    • +
  • +

    Log them in the most recently used trace.

    +
    • +
  • +

    Use additional heuristics for connection identification (for example use the +four-tuple in addition to the Connection ID).

    +
    • +
  • +

    Buffer events until they can be assigned to a connection (for example for +version negotiation and retry events).

    +
    • +
+
+
+
+
+
+
+

+3. QUIC Event Overview +

+

QUIC connections consist of different phases and interaction events. In order to +model this, QUIC event types are divided into general categories: connectivity +(Section 4), security (Section 6), quic Section 5, and recovery +Section 7.

+

As described in Section 3.4.2 of [QLOG-MAIN], the qlog name field is the +concatenation of category and type.

+

Table 1 summarizes the name value of each event type that is defined in +this specification.

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+Table 1: +QUIC Events +
Name valueImportanceDefinition
connectivity:server_listeningExtra + Section 4.1 +
connectivity:connection_startedBase + Section 4.2 +
connectivity:connection_closedBase + Section 4.3 +
connectivity:connection_id_updatedBase + Section 4.4 +
connectivity:spin_bit_updatedBase + Section 4.5 +
connectivity:connection_state_updatedBase + Section 4.6 +
connectivity:path_assignedBase + Section 4.7 +
connectivity:mtu_updatedExtra + Section 4.8 +
quic:version_informationCore + Section 5.1 +
quic:alpn_informationCore + Section 5.2 +
quic:parameters_setCore + Section 5.3 +
quic:parameters_restoredBase + Section 5.4 +
quic:packet_sentCore + Section 5.5 +
quic:packet_receivedCore + Section 5.6 +
quic:packet_droppedBase + Section 5.7 +
quic:packet_bufferedBase + Section 5.8 +
quic:packets_ackedExtra + Section 5.9 +
quic:udp_datagrams_sentExtra + Section 5.10 +
quic:udp_datagrams_receivedExtra + Section 5.11 +
quic:udp_datagram_droppedExtra + Section 5.12 +
quic:stream_state_updatedBase + Section 5.13 +
quic:frames_processedExtra + Section 5.14 +
quic:stream_data_movedBase + Section 5.15 +
quic:datagram_data_movedBase + Section 5.16 +
quic:migration_state_updatedExtra + Section 5.17 +
security:key_updatedBase + Section 6.1 +
security:key_discardedBase + Section 6.2 +
recovery:parameters_setBase + Section 7.1 +
recovery:metrics_updatedCore + Section 7.2 +
recovery:congestion_state_updatedBase + Section 7.3 +
recovery:loss_timer_updatedExtra + Section 7.4 +
recovery:packet_lostCore + Section 7.5 +
recovery:marked_for_retransmitExtra + Section 7.6 +
recovery:ecn_state_updatedExtra + Section 7.7 +
+
+

QUIC events extend the $ProtocolEventData extension point defined in +[QLOG-MAIN]. Additionally, they allow for direct extensibility by their use of +per-event extension points via the $$ CDDL "group socket" syntax, as also +described in [QLOG-MAIN].

+
+
+
+
+QuicEventData = ConnectivityServerListening /
+                ConnectivityConnectionStarted /
+                ConnectivityConnectionClosed /
+                ConnectivityConnectionIDUpdated /
+                ConnectivitySpinBitUpdated /
+                ConnectivityConnectionStateUpdated /
+                ConnectivityPathAssigned /
+                ConnectivityMTUUpdated /
+                SecurityKeyUpdated /
+                SecurityKeyDiscarded /
+                QUICVersionInformation /
+                QUICALPNInformation /
+                QUICParametersSet /
+                QUICParametersRestored /
+                QUICPacketSent /
+                QUICPacketReceived /
+                QUICPacketDropped /
+                QUICPacketBuffered /
+                QUICPacketsAcked /
+                QUICUDPDatagramsSent /
+                QUICUDPDatagramsReceived /
+                QUICUDPDatagramDropped /
+                QUICStreamStateUpdated /
+                QUICFramesProcessed /
+                QUICStreamDataMoved /
+                QUICDatagramDataMoved /
+                RecoveryParametersSet /
+                RecoveryMetricsUpdated /
+                RecoveryCongestionStateUpdated /
+                RecoveryLossTimerUpdated /
+                RecoveryPacketLost
+
+$ProtocolEventData /= QuicEventData
+
+
+
Figure 2: +QuicEventData definition and ProtocolEventData extension +
+
+
+
+
+
+

+4. Connectivity events +

+
+
+

+4.1. server_listening +

+

Emitted when the server starts accepting connections. It has Extra importance +level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+ConnectivityServerListening = {
+    ? ip_v4: IPAddress
+    ? ip_v6: IPAddress
+    ? port_v4: uint16
+    ? port_v6: uint16
+
+    ; the server will always answer client initials with a retry
+    ; (no 1-RTT connection setups by choice)
+    ? retry_required: bool
+
+    * $$connectivity-serverlistening-extension
+}
+
+
+
Figure 3: +ConnectivityServerListening definition +
+
+

Some QUIC stacks do not handle sockets directly and are thus unable to log +IP and/or port information.

+
+
+
+
+

+4.2. connection_started +

+

The connection_started event is used for both attempting (client-perspective) +and accepting (server-perspective) new connections. Note that while there is +overlap with the connection_state_updated event, this event is separate event +in order to capture additional data that can be useful to log. It has Base +importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+ConnectivityConnectionStarted = {
+    ? ip_version: IPVersion
+    src_ip: IPAddress
+    dst_ip: IPAddress
+
+    ; transport layer protocol
+    ? protocol: text .default "QUIC"
+    ? src_port: uint16
+    ? dst_port: uint16
+    ? src_cid: ConnectionID
+    ? dst_cid: ConnectionID
+
+    * $$connectivity-connectionstarted-extension
+}
+
+
+
Figure 4: +ConnectivityConnectionStarted definition +
+
+

Some QUIC stacks do not handle sockets directly and are thus unable to log +IP and/or port information.

+
+
+
+
+

+4.3. connection_closed +

+

The connection_closed event is used for logging when a connection was closed, +typically when an error or timeout occurred. It has Base importance level; see +Section 9.2 of [QLOG-MAIN].

+

Note that this event has overlap with the connection_state_updated event, as +well as the CONNECTION_CLOSE frame. However, in practice, when analyzing large +deployments, it can be useful to have a single event representing a +connection_closed event, which also includes an additional reason field to +provide more information. Furthermore, it is useful to log closures due to +timeouts, which are difficult to reflect using the other options.

+

In QUIC there are two main connection-closing error categories: connection and +application errors. They have well-defined error codes and semantics. Next to +these however, there can be internal errors that occur that may or may not get +mapped to the official error codes in implementation-specific ways. As such, +multiple error codes can be set on the same event to reflect this.

+
+
+
+
+ConnectivityConnectionClosed = {
+
+    ; which side closed the connection
+    ? owner: Owner
+    ? connection_code: $TransportError /
+                       CryptoError /
+                       uint32
+    ? application_code: $ApplicationError /
+                        uint32
+    ? internal_code: uint32
+    ? reason: text
+    ? trigger:
+        "clean" /
+        "handshake_timeout" /
+        "idle_timeout" /
+        ; this is called the "immediate close" in the QUIC RFC
+        "error" /
+        "stateless_reset" /
+        "version_mismatch" /
+        ; for example HTTP/3's GOAWAY frame
+        "application"
+
+    * $$connectivity-connectionclosed-extension
+}
+
+
+
Figure 5: +ConnectivityConnectionClosed definition +
+
+
+
+
+
+

+4.4. connection_id_updated +

+

The connection_id_updated event is emitted when either party updates their +current Connection ID. As this typically happens only sparingly over the course +of a connection, using this event is more efficient than logging the observed +CID with each and every packet_sent or packet_received events. It has Base +importance level; see Section 9.2 of [QLOG-MAIN].

+

The connection_id_updated event is viewed from the perspective of the endpoint +applying the new ID. As such, when the endpoint receives a new connection ID +from the peer, the owner field will be "remote". When the endpoint updates its +own connection ID, the owner field will be "local".

+
+
+
+
+ConnectivityConnectionIDUpdated = {
+    owner: Owner
+    ? old: ConnectionID
+    ? new: ConnectionID
+
+    * $$connectivity-connectionidupdated-extension
+}
+
+
+
Figure 6: +ConnectivityConnectionIDUpdated definition +
+
+
+
+
+
+

+4.5. spin_bit_updated +

+

The spin_bit_updated event conveys information about the QUIC latency spin +bit; see Section 17.4 of [QUIC-TRANSPORT]. The event is emitted when the spin +bit changes value, it SHOULD NOT be emitted if the spin bit is set without +changing its value. It has Base importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+ConnectivitySpinBitUpdated = {
+    state: bool
+
+    * $$connectivity-spinbitupdated-extension
+}
+
+
+
Figure 7: +ConnectivitySpinBitUpdated definition +
+
+
+
+
+
+

+4.6. connection_state_updated +

+

The connection_state_updated event is used to track progress through QUIC's +complex handshake and connection close procedures. It has Base importance +level; see Section 9.2 of [QLOG-MAIN].

+

It is intended to provide exhaustive options to log each state individually, but +also provides a more basic, simpler set for implementations less interested in +tracking each smaller state transition. As such, users should not expect to see +all these states reflected in all qlogs and implementers should focus on support +for the SimpleConnectionState set.

+
+
+
+
+ConnectivityConnectionStateUpdated = {
+    ? old: ConnectionState /
+           SimpleConnectionState
+    new: ConnectionState /
+         SimpleConnectionState
+
+    * $$connectivity-connectionstateupdated-extension
+}
+
+ConnectionState =
+    ; initial sent/received
+    "attempted" /
+    ; peer address validated by: client sent Handshake packet OR
+    ; client used CONNID chosen by the server.
+    ; RFC 9000 Section 8.1
+    "peer_validated" /
+    "handshake_started" /
+    ; 1 RTT can be sent, but handshake isn't done yet
+    "early_write" /
+    ; TLS handshake complete: Finished received and sent
+    ; RFC 9001 Section 4.1.1
+    "handshake_complete" /
+    ; HANDSHAKE_DONE sent/received (connection is now "active", 1RTT
+    ; can be sent). RFC 9001 Section 4.1.2
+    "handshake_confirmed" /
+    "closing" /
+    ; connection_close sent/received
+    "draining" /
+    ; draining period done, connection state discarded
+    "closed"
+
+SimpleConnectionState =
+    "attempted" /
+    "handshake_started" /
+    "handshake_confirmed" /
+    "closed"
+
+
+
Figure 8: +ConnectivityConnectionStateUpdated definition +
+
+

These states correspond to the following transitions for both client and server:

+

Client:

+
    +
  • +

    send initial

    +
      +
    • +

      state = attempted

      +
      • +
    +
    • +
  • +

    get initial

    +
      +
    • +

      state = validated (not really "needed" at the client, but somewhat useful to indicate progress nonetheless)

      +
      • +
    +
    • +
  • +

    get first Handshake packet

    +
      +
    • +

      state = handshake_started

      +
      • +
    +
    • +
  • +

    get Handshake packet containing ServerFinished

    +
      +
    • +

      state = handshake_complete

      +
      • +
    +
    • +
  • +

    send ClientFinished

    +
      +
    • +

      state = early_write + (1RTT can now be sent)

      +
      • +
    +
    • +
  • +

    get HANDSHAKE_DONE

    +
      +
    • +

      state = handshake_confirmed

      +
      • +
    +
    • +
+

Server:

+
    +
  • +

    get initial

    +
      +
    • +

      state = attempted

      +
      • +
    +
    • +
  • +

    send initial (TODO don't think this needs a separate state, since some handshake will always be sent in the same flight as this?)

    +
    • +
  • +

    send handshake EE, CERT, CV, ...

    +
      +
    • +

      state = handshake_started

      +
      • +
    +
    • +
  • +

    send ServerFinished

    +
      +
    • +

      state = early_write + (1RTT can now be sent)

      +
      • +
    +
    • +
  • +

    get first handshake packet / something using a server-issued CID of min length

    +
      +
    • +

      state = validated

      +
      • +
    +
    • +
  • +

    get handshake packet containing ClientFinished

    +
      +
    • +

      state = handshake_complete

      +
      • +
    +
    • +
  • +

    send HANDSHAKE_DONE

    +
      +
    • +

      state = handshake_confirmed

      +
      • +
    +
    • +
+
+
Note:
+
+

connection_state_changed with a new state of attempted is the same +conceptual event as the connection_started event above from the client's +perspective. Similarly, a state of closing or draining corresponds to the +connection_closed` event.

+
+
+
+
+
+
+
+

+4.7. path_assigned +

+

Importance: Base

+

This event is used to associate a single PathID's value with other parameters +that describe a unique network path.

+

As described in [QLOG-MAIN], each qlog event can be linked to a single network +path by means of the top-level "path" field, whose value is a PathID. However, +since it can be cumbersome to encode additional path metadata (such as IP +addresses or Connection IDs) directly into the PathID, this event allows such an +association to happen separately. As such, PathIDs can be short and unique, and +can even be updated to be associated with new metadata as the connection's state +evolves.

+

Definition:

+
+
+
+
+ConnectivityPathAssigned = {
+    path_id: PathID
+
+    ; the information for traffic going towards the remote receiver
+    ? path_remote: PathEndpointInfo
+
+    ; the information for traffic coming in at the local endpoint
+    ? path_local: PathEndpointInfo
+
+    * $$connectivity-pathassigned-extension
+}
+
+
+
Figure 9: +ConnectivityPathAssigned definition +
+
+

Choosing the different path_id values is left up to the implementation. Some +options include using a uniquely incrementing integer, using the (first) +Destination Connection ID associated with a path (or its sequence number), or +using (a hash of) the two endpoint IP addresses.

+

It is important to note that the empty string ("") is a valid PathID and that it +is the default assigned to events that do not explicitly set a "path" field. Put +differently, the initial path of a QUIC connection on which the handshake occurs +(see also Section 4.2) is implicitly associated with the +PathID with value "". Associating metadata with this default path is possible by +logging the ConnectivityPathAssigned event with a value of "" for the path_id +field.

+

As paths and their metadata can evolve over time, multiple +ConnectivityPathAssigned events can be emitted for each unique PathID. The +latest event contains the most up-to-date information for that PathID. As such, +the first time a PathID is seen in a ConnectivityPathAssigned event, it is an +indication that the path is created. Subsequent occurrences indicate the path is +updated, while a final occurrence with both path_local and path_remote +fields omitted implicitly indicates the path has been abandoned.

+
+
+
+
+

+4.8. mtu_updated +

+

The mtu_updated event indicates that the estimated Path MTU was updated. This +happens as part of the Path MTU discovery process. It has Extra importance +level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+ConnectivityMTUUpdated = {
+    ? old: uint32
+    new: uint32
+
+    ; at some point, MTU discovery stops, as a "good enough"
+    ; packet size has been found
+    ? done: bool .default false
+
+    * $$connectivity-mtuupdated-extension
+}
+
+
+
Figure 10: +ConnectivityMTUUpdated definition +
+
+
+
+
+
+
+
+

+5. QUIC events +

+
+
+

+5.1. version_information +

+

The version_information event supports QUIC version negotiation; see Section 6 of [QUIC-TRANSPORT]. It has Core importance level; see Section 9.2 of [QLOG-MAIN].

+

QUIC endpoints each have their own list of QUIC versions they support. The +client uses the most likely version in their first initial. If the server does +not support that version, it replies with a Version Negotiation packet, which +contains its supported versions. From this, the client selects a version. The +version_information event aggregates all this information in a single event +type. It also allows logging of supported versions at an endpoint without actual +version negotiation needing to happen.

+
+
+
+
+QUICVersionInformation = {
+    ? server_versions: [+ QuicVersion]
+    ? client_versions: [+ QuicVersion]
+    ? chosen_version: QuicVersion
+
+    * $$quic-versioninformation-extension
+}
+
+
+
Figure 11: +QUICVersionInformation definition +
+
+

Intended use:

+
    +
  • +

    When sending an initial, the client logs this event with client_versions and +chosen_version set

    +
    • +
  • +

    Upon receiving a client initial with a supported version, the server logs this +event with server_versions and chosen_version set

    +
    • +
  • +

    Upon receiving a client initial with an unsupported version, the server logs +this event with server_versions set and client_versions to the +single-element array containing the client's attempted version. The absence of +chosen_version implies no overlap was found

    +
    • +
  • +

    Upon receiving a version negotiation packet from the server, the client logs +this event with client_versions set and server_versions to the versions in +the version negotiation packet and chosen_version to the version it will use +for the next initial packet. If the client receives a set of server_versions +with no viable overlap with its own supported versions, this event should be +logged without the chosen_version set

    +
    • +
+
+
+
+
+

+5.2. alpn_information +

+

The alpn_information event supports Application-Layer Protocol Negotiation +(ALPN) over the QUIC transport; see [RFC7301] and Section 7.4 of [QUIC-TRANSPORT]. It has Core importance level; see Section 9.2 of [QLOG-MAIN].

+

QUIC endpoints are configured with a list of supported ALPN identifiers. Clients send the list in a TLS ClientHello, and servers match against their list. On success, a single ALPN identifier is chosen and sent back in a TLS ServerHello. If no match is found, the connection is closed.

+

ALPN identifiers are byte sequences, that may be possible to present as UTF-8. +The ALPNIdentifier` type supports either format. Implementations SHOULD log at +least one format, but MAY log both or none.

+
+
+
+
+QUICALPNInformation = {
+    ? server_alpns: [* ALPNIdentifier]
+    ? client_alpns: [* ALPNIdentifier]
+    ? chosen_alpn: ALPNIdentifier
+
+    * $$quic-alpninformation-extension
+}
+
+ALPNIdentifier = {
+  ? byte_value: hexstring
+  ? string_value: text
+}
+
+
+
Figure 12: +QUICALPNInformation definition +
+
+

Intended use:

+
    +
  • +

    When sending an initial, the client logs this event with client_alpns set

    +
    • +
  • +

    When receiving an initial with a supported alpn, the server logs this event +with server_alpns set, client_alpns equalling the client-provided list, +and chosen_alpn to the value it will send back to the client.

    +
    • +
  • +

    When receiving an initial with an alpn, the client logs this event with +chosen_alpn to the received value.

    +
    • +
  • +

    Alternatively, a client can choose to not log the first event, but wait for +the receipt of the server initial to log this event with both client_alpns +and chosen_alpn set.

    +
    • +
+
+
+
+
+

+5.3. parameters_set +

+

The parameters_set event groups settings from several different sources +(transport parameters, TLS ciphers, etc.) into a single event. This is done to +minimize the amount of events and to decouple conceptual setting impacts from +their underlying mechanism for easier high-level reasoning. The event has Core +importance level; see Section 9.2 of [QLOG-MAIN].

+

Most of these settings are typically set once and never change. However, they +are usually set at different times during the connection, so there will +regularly be several instances of this event with different fields set.

+

Note that some settings have two variations (one set locally, one requested by the +remote peer). This is reflected in the owner field. As such, this field MUST be +correct for all settings included a single event instance. If you need to log +settings from two sides, you MUST emit two separate event instances.

+

In the case of connection resumption and 0-RTT, some of the server's parameters +are stored up-front at the client and used for the initial connection startup. +They are later updated with the server's reply. In these cases, utilize the +separate parameters_restored event to indicate the initial values, and this +event to indicate the updated values, as normal.

+
+
+
+
+QUICParametersSet = {
+    ? owner: Owner
+
+    ; true if valid session ticket was received
+    ? resumption_allowed: bool
+
+    ; true if early data extension was enabled on the TLS layer
+    ? early_data_enabled: bool
+
+    ; e.g., "AES_128_GCM_SHA256"
+    ? tls_cipher: text
+
+    ; RFC9000
+    ? original_destination_connection_id: ConnectionID
+    ? initial_source_connection_id: ConnectionID
+    ? retry_source_connection_id: ConnectionID
+    ? stateless_reset_token: StatelessResetToken
+    ? disable_active_migration: bool
+    ? max_idle_timeout: uint64
+    ? max_udp_payload_size: uint32
+    ? ack_delay_exponent: uint16
+    ? max_ack_delay: uint16
+    ? active_connection_id_limit: uint32
+    ? initial_max_data: uint64
+    ? initial_max_stream_data_bidi_local: uint64
+    ? initial_max_stream_data_bidi_remote: uint64
+    ? initial_max_stream_data_uni: uint64
+    ? initial_max_streams_bidi: uint64
+    ? initial_max_streams_uni: uint64
+    ? preferred_address: PreferredAddress
+
+    ; RFC9221
+    ? max_datagram_frame_size: uint64
+
+    ; RFC9287
+    ; true if present, absent or false if extension not negotiated
+    ? grease_quic_bit: bool
+
+    * $$quic-parametersset-extension
+}
+
+PreferredAddress = {
+    ip_v4: IPAddress
+    ip_v6: IPAddress
+    port_v4: uint16
+    port_v6: uint16
+    connection_id: ConnectionID
+    stateless_reset_token: StatelessResetToken
+}
+
+
+
Figure 13: +QUICParametersSet definition +
+
+
+
+
+
+

+5.4. parameters_restored +

+

When using QUIC 0-RTT, clients are expected to remember and restore the server's +transport parameters from the previous connection. The parameters_restored +event is used to indicate which parameters were restored and to which values +when utilizing 0-RTT. It has Base importance level; see Section 9.2 of [QLOG-MAIN].

+

Note that not all transport parameters should be restored (many are even +prohibited from being re-utilized). The ones listed here are the ones expected +to be useful for correct 0-RTT usage.

+
+
+
+
+QUICParametersRestored = {
+    ? disable_active_migration: bool
+    ? max_idle_timeout: uint64
+    ? max_udp_payload_size: uint32
+    ? active_connection_id_limit: uint32
+    ? initial_max_data: uint64
+    ? initial_max_stream_data_bidi_local: uint64
+    ? initial_max_stream_data_bidi_remote: uint64,
+    ? initial_max_stream_data_uni: uint64
+    ? initial_max_streams_bidi: uint64
+    ? initial_max_streams_uni: uint64
+
+    * $$quic-parametersrestored-extension
+}
+
+
+
Figure 14: +QUICParametersRestored definition +
+
+
+
+
+
+

+5.5. packet_sent +

+

The packet_sent event indicates a QUIC-level packet was sent. It has Core +importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+QUICPacketSent = {
+    header: PacketHeader
+    ? frames: [* $QuicFrame]
+    ? is_coalesced: bool .default false
+
+    ; only if header.packet_type === "stateless_reset"
+    ; is always 128 bits in length.
+    ? stateless_reset_token: StatelessResetToken
+
+    ; only if header.packet_type === "version_negotiation"
+    ? supported_versions: [+ QuicVersion]
+    ? raw: RawInfo
+    ? datagram_id: uint32
+    ? is_mtu_probe_packet: bool .default false
+
+    ? trigger:
+      ; RFC 9002 Section 6.1.1
+      "retransmit_reordered" /
+      ; RFC 9002 Section 6.1.2
+      "retransmit_timeout" /
+      ; RFC 9002 Section 6.2.4
+      "pto_probe" /
+      ; RFC 9002 6.2.3
+      "retransmit_crypto" /
+      ; needed for some CCs to figure out bandwidth allocations
+      ; when there are no normal sends
+      "cc_bandwidth_probe"
+
+    * $$quic-packetsent-extension
+}
+
+
+
Figure 15: +QUICPacketSent definition +
+
+

The encryption_level and packet_number_space are not logged explicitly: +the header.packet_type specifies this by inference (assuming correct +implementation)

+

For more details on datagram_id, see Section 5.10. It is only needed +when keeping track of packet coalescing.

+
+
+
+
+

+5.6. packet_received +

+

The packet_received event indicates a QUIC-level packet was received. It has +Core importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+QUICPacketReceived = {
+    header: PacketHeader
+    ? frames: [* $QuicFrame]
+    ? is_coalesced: bool .default false
+
+    ; only if header.packet_type === "stateless_reset"
+    ; Is always 128 bits in length.
+    ? stateless_reset_token: StatelessResetToken
+
+    ; only if header.packet_type === "version_negotiation"
+    ? supported_versions: [+ QuicVersion]
+    ? raw: RawInfo
+    ? datagram_id: uint32
+
+    ? trigger:
+        ; if packet was buffered because it couldn't be
+        ; decrypted before
+        "keys_available"
+
+    * $$quic-packetreceived-extension
+}
+
+
+
Figure 16: +QUICPacketReceived definition +
+
+

The encryption_level and packet_number_space are not logged explicitly: the +header.packet_type specifies this by inference (assuming correct +implementation)

+

For more details on datagram_id, see Section 5.10. It is only needed +when keeping track of packet coalescing.

+
+
+
+
+

+5.7. packet_dropped +

+

The packet_dropped event indicates a QUIC-level packet was dropped. It has +Base importance level; see Section 9.2 of [QLOG-MAIN].

+

The trigger field indicates a general reason category for dropping the packet, +while the details field can contain additional implementation-specific +information.

+
+
+
+
+QUICPacketDropped = {
+
+    ; Primarily packet_type should be filled here,
+    ; as other fields might not be decrypteable or parseable
+    ? header: PacketHeader
+    ? raw: RawInfo
+    ? datagram_id: uint32
+    ? details: {* text => any}
+    ? trigger:
+        "internal_error" /
+        "rejected" /
+        "unsupported" /
+        "invalid" /
+        "duplicate" /
+        "connection_unknown" /
+        "decryption_failure" /
+        "key_unavailable" /
+        "general"
+
+    * $$quic-packetdropped-extension
+}
+
+
+
Figure 17: +QUICPacketDropped definition +
+
+

Some example situations for each of the trigger categories include:

+
    +
  • +

    internal_error: not initialized, out of memory

    +
    • +
  • +

    rejected: limits reached, DDoS protection, unwilling to track more paths, duplicate packet

    +
    • +
  • +

    unsupported: unknown or unsupported version. See also Section 2.2.

    +
    • +
  • +

    invalid: packet parsing or validation error

    +
    • +
  • +

    duplicate: duplicate packet

    +
    • +
  • +

    connection_unknown: packet does not relate to a known connection or Connection ID

    +
    • +
  • +

    decryption_failure: decryption failed

    +
    • +
  • +

    key_unavailable: decryption key was unavailable

    +
    • +
  • +

    general: situations not clearly covered in the other categories

    +
    • +
+

For more details on datagram_id, see Section 5.10.

+
+
+
+
+

+5.8. packet_buffered +

+

The packet_buffered event is emitted when a packet is buffered because it +cannot be processed yet. Typically, this is because the packet cannot be parsed +yet, and thus only the full packet contents can be logged when it was parsed in +a packet_received event. The event has Base importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+QUICPacketBuffered = {
+
+    ; primarily packet_type and possible packet_number should be
+    ; filled here as other elements might not be available yet
+    ? header: PacketHeader
+    ? raw: RawInfo
+    ? datagram_id: uint32
+    ? trigger:
+        ; indicates the parser cannot keep up, temporarily buffers
+        ; packet for later processing
+        "backpressure" /
+        ; if packet cannot be decrypted because the proper keys were
+        ; not yet available
+        "keys_unavailable"
+
+    * $$quic-packetbuffered-extension
+}
+
+
+
Figure 18: +QUICPacketBuffered definition +
+
+

For more details on datagram_id, see Section 5.10. It is only needed +when keeping track of packet coalescing.

+
+
+
+
+

+5.9. packets_acked +

+

The packets_acked event is emitted when a (group of) sent packet(s) is +acknowledged by the remote peer for the first time. It has Extra importance +level; see Section 9.2 of [QLOG-MAIN].

+

This information could also be deduced from the contents of received ACK frames. +However, ACK frames require additional processing logic to determine when a +given packet is acknowledged for the first time, as QUIC uses ACK ranges which +can include repeated ACKs. Additionally, this event can be used by +implementations that do not log frame contents.

+
+
+
+
+QUICPacketsAcked = {
+    ? packet_number_space: $PacketNumberSpace
+    ? packet_numbers: [+ uint64]
+
+    * $$quic-packetsacked-extension
+}
+
+
+
Figure 19: +QUICPacketsAcked definition +
+
+

If packet_number_space is omitted, it assumes the default value of +application_data, as this is by far the most prevalent packet +number space a typical QUIC connection will use.

+
+
+
+
+

+5.10. udp_datagrams_sent +

+

When one or more UDP-level datagrams are passed to the socket. This is useful +for determining how QUIC packet buffers are drained to the OS. The event has +Extra importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+QUICUDPDatagramsSent = {
+
+    ; to support passing multiple at once
+    ? count: uint16
+
+    ; The RawInfo fields do not include the UDP headers,
+    ; only the UDP payload
+    ? raw: [+ RawInfo]
+
+    ; ECN bits in the IP header
+    ; if not set, defaults to the value used on the last
+    ; QUICDatagramsSent event
+    ? ecn: [+ ECN]
+
+    ? datagram_ids: [+ uint32]
+
+    * $$quic-udpdatagramssent-extension
+}
+
+
+
Figure 20: +QUICUDPDatagramsSent definition +
+
+

Since QUIC implementations rarely control UDP logic directly, the raw data +excludes UDP-level headers in all fields.

+

The datagram_id is a qlog-specific concept to allow tracking of QUIC packet +coalescing inside UDP datagrams. Since QUIC generates many UDP datagrams, unique +identifiers are required to be able to track them individually in qlog traces. +However, neither UDP nor QUIC exchanges datagram identifiers on the wire. +Selecting identifier values is thus left to qlog implementations, which should +consider how to generate unique values within the scope of their created traces.

+
+
+
+
+

+5.11. udp_datagrams_received +

+

When one or more UDP-level datagrams are received from the socket. This is +useful for determining how datagrams are passed to the user space stack from the +OS. The event has Extra importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+QUICUDPDatagramsReceived = {
+
+    ; to support passing multiple at once
+    ? count: uint16
+
+    ; The RawInfo fields do not include the UDP headers,
+    ; only the UDP payload
+    ? raw: [+ RawInfo]
+
+    ; ECN bits in the IP header
+    ; if not set, defaults to the value on the last
+    ; QUICDatagramsReceived event
+    ? ecn: [+ ECN]
+
+    ? datagram_ids: [+ uint32]
+
+    * $$quic-udpdatagramsreceived-extension
+}
+
+
+
Figure 21: +QUICUDPDatagramsReceived definition +
+
+

For more details on datagram_ids, see Section 5.10.

+
+
+
+
+

+5.12. udp_datagram_dropped +

+

When a UDP-level datagram is dropped. This is typically done if it does not +contain a valid QUIC packet. If it does, but the QUIC packet is dropped for +other reasons, the packet_dropped event (Section 5.7) should be +used instead. The event has Extra importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+QUICUDPDatagramDropped = {
+
+    ; The RawInfo fields do not include the UDP headers,
+    ; only the UDP payload
+    ? raw: RawInfo
+
+    * $$quic-udpdatagramdropped-extension
+}
+
+
+
Figure 22: +QUICUDPDatagramDropped definition +
+
+
+
+
+
+

+5.13. stream_state_updated +

+

The stream_state_updated event is emitted whenever the internal state of a +QUIC stream is updated; see Section 3 of [QUIC-TRANSPORT]. Most of this can be +inferred from several types of frames going over the wire, but it's much easier +to have explicit signals for these state changes. The event has Base importance +level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+StreamType = "unidirectional" /
+             "bidirectional"
+
+QUICStreamStateUpdated = {
+    stream_id: uint64
+
+    ; mainly useful when opening the stream
+    ? stream_type: StreamType
+    ? old: $StreamState
+    new: $StreamState
+    ? stream_side: "sending" /
+                   "receiving"
+
+    * $$quic-streamstateupdated-extension
+}
+
+BaseStreamStates =  "idle" /
+                    "open" /
+                    "closed"
+
+GranularStreamStates =
+    ; bidirectional stream states, RFC 9000 Section 3.4.
+    "half_closed_local" /
+    "half_closed_remote" /
+    ; sending-side stream states, RFC 9000 Section 3.1.
+    "ready" /
+    "send" /
+    "data_sent" /
+    "reset_sent" /
+    "reset_received" /
+    ; receive-side stream states, RFC 9000 Section 3.2.
+    "receive" /
+    "size_known" /
+    "data_read" /
+    "reset_read" /
+    ; both-side states
+    "data_received" /
+    ; qlog-defined: memory actually freed
+    "destroyed"
+
+$StreamState /= BaseStreamStates / GranularStreamStates
+
+
+
Figure 23: +QUICStreamStateUpdated definition +
+
+

QUIC implementations SHOULD mainly log the simplified (HTTP/2-alike) +BaseStreamStates instead of the more fine-grained GranularStreamStates. These +latter ones are mainly for more in-depth debugging. Tools SHOULD be able to deal +with both types equally.

+
+
+
+
+

+5.14. frames_processed +

+

The frame_processed event is intended to prevent a large proliferation of +specific purpose events (e.g., packets_acknowledged, flow_control_updated, +stream_data_received). It has Extra importance level; see Section 9.2 of [QLOG-MAIN].

+

Implementations have the opportunity to (selectively) log this type of +signal without having to log packet-level details (e.g., in packet_received). +Since for almost all cases, the effects of applying a frame to the internal state +of an implementation can be inferred from that frame's contents, these events +are aggregated into this single frames_processed event.

+

The frame_processed event can be used to signal internal state change not +resulting directly from the actual "parsing" of a frame (e.g., the frame could +have been parsed, data put into a buffer, then later processed, then logged with +this event).

+

The packet_received event can convey all constituent frames. It is not +expected that the frames_processed event will also be used for a redundant +purpose. Rather, implementations can use this event to avoid having to log full +packets or to convey extra information about when frames are processed (for +example, if frame processing is deferred for any reason).

+

Note that for some events, this approach will lose some information (e.g., for which +encryption level are packets being acknowledged?). If this information is +important, the packet_received event can be used instead.

+

In some implementations, it can be difficult to log frames directly, even when +using packet_sent and packet_received events. For these cases, the +frames_processed event also contains the packet_numbers field, which can be +used to more explicitly link this event to the packet_sent/received events. +The field is an array, which supports using a single frames_processed event +for multiple frames received over multiple packets. To map between frames and +packets, the position and order of entries in the frames and packet_numbers +is used. If the optional packet_numbers field is used, each frame MUST have a +corresponding packet number at the same index.

+
+
+
+
+QUICFramesProcessed = {
+    frames: [* $QuicFrame]
+    ? packet_numbers: [* uint64]
+
+    * $$quic-framesprocessed-extension
+}
+
+
+
Figure 24: +QUICFramesProcessed definition +
+
+

For example, an instance of the frames_processed event that represents four +STREAM frames received over two packets would have the fields serialized as:

+
+
+"frames":[
+  {"frame_type":"stream","stream_id":0,"offset":0,"length":500},
+  {"frame_type":"stream","stream_id":0,"offset":500,"length":200},
+  {"frame_type":"stream","stream_id":1,"offset":0,"length":300},
+  {"frame_type":"stream","stream_id":1,"offset":300,"length":50}
+  ],
+"packet_numbers":[
+  1,
+  1,
+  2,
+  2
+]
+
+
+
+
+
+
+

+5.15. stream_data_moved +

+

The stream_data_moved event is used to indicate when QUIC stream data moves +between the different layers. This helps make clear the flow of data, how long +data remains in various buffers, and the overheads introduced by individual +layers. The event has Base importance level; see Section 9.2 of [QLOG-MAIN].

+

For example, it can be useful to understand when when data moves from an +application protocol (e.g., HTTP) to QUIC stream buffers and vice versa. +Similarly, when data moves from the application protocol layer into a +user-facing application such as a web browser.

+

The stream_data_moved event can provide insight into whether received data on +a QUIC stream is moved to the application protocol immediately (for example per +received packet) or in larger batches (for example, all QUIC packets are +processed first and afterwards the application layer reads from the streams with +newly available data). This can help identify bottlenecks, flow control issues, +or scheduling problems.

+

This event is only for data in QUIC streams. For data in QUIC Datagram Frames, +see the datagram_data_moved event defined in Section 5.16.

+
+
+
+
+QUICStreamDataMoved = {
+    ? stream_id: uint64
+    ? offset: uint64
+
+    ; byte length of the moved data
+    ? length: uint64
+    ? from: $DataLocation
+    ? to: $DataLocation
+    ? raw: RawInfo
+
+    * $$quic-streamdatamoved-extension
+}
+
+$DataLocation /=  "user" /
+                  "application" /
+                  "transport" /
+                  "network"
+
+
+
Figure 25: +QUICStreamDataMoved definition +
+
+
+
+
+
+

+5.16. datagram_data_moved +

+

The datagram_data_moved event is used to indicate when QUIC Datagram Frame +data (see [RFC9221]) moves between the different layers. This helps make +clear the flow of data, how long data remains in various buffers, and the +overheads introduced by individual layers. The event has Base importance level; +see Section 9.2 of [QLOG-MAIN].

+

For example, passing from the application protocol (e.g., WebTransport) to QUIC +Datagram Frame buffers and vice versa. Similarly, when data moves from the +application protocol layer into a user-facing application such as a web browser.

+

The datagram_data_moved event can provide insight into whether received data +in a QUIC Datagram Frame is moved to the application protocol immediately (for +example per received packet) or in larger batches (for example, all QUIC packets +are processed first and afterwards the application layer reads all Datagrams at +once). This can help identify bottlenecks, flow control issues, or scheduling +problems.

+

This event is only for data in QUIC Datagram Frames. For data in QUIC streams, +see the stream_data_moved event defined in Section 5.15.

+
+
+
+
+QUICDatagramDataMoved = {
+    ; byte length of the moved data
+    ? length: uint64
+    ? from: $DataLocation
+    ? to: $DataLocation
+    ? raw: RawInfo
+
+    * $$quic-datagramdatamoved-extension
+}
+
+
+
Figure 26: +QUICDatagramDataMoved definition +
+
+
+
+
+
+

+5.17. migration_state_updated +

+

Importance: Extra

+

Use to provide additional information when attempting (client-side) connection +migration. While most details of the QUIC connection migration process can be +inferred by observing the PATH_CHALLENGE and PATH_RESPONSE frames, in +combination with the ConnectivityPathAssigned event, it can be useful to +explicitly log the progression of the migration and potentially made decisions +in a single location/event.

+

Generally speaking, connection migration goes through two phases: a probing +phase (which is not always needed/present), and a migration phase (which can be +abandoned upon error).

+

Implementations that log per-path information in a QUICMigrationStateUpdated, +SHOULD also emit QUICPathAssigned events, to serve as a ground-truth source of +information.

+

Definition:

+
+
+
+
+QUICMigrationStateUpdated = {
+    ? old: MigrationState
+    new: MigrationState
+
+    ? path_id: PathID
+
+    ; the information for traffic going towards the remote receiver
+    ? path_remote: PathEndpointInfo
+
+    ; the information for traffic coming in at the local endpoint
+    ? path_local: PathEndpointInfo
+
+    * $$quic-migrationstateupdated-extension
+}
+
+; Note that MigrationState does not describe a full state machine
+; These entries are not necessarily chronological,
+; nor will they always all appear during
+; a connection migration attempt.
+MigrationState =
+    ; probing packets are sent, migration not initiated yet
+    "probing_started" /
+    ; did not get reply to probing packets,
+    ; discarding path as an option
+    "probing_abandoned" /
+    ; received reply to probing packets, path is migration candidate
+    "probing_successful" /
+    ; non-probing packets are sent, attempting migration
+    "migration_started" /
+    ; something went wrong during the migration, abandoning attempt
+    "migration_abandoned" /
+    ; new path is now fully used, old path is discarded
+    "migration_complete"
+
+
+
Figure 27: +QUICMigrationStateUpdated definition +
+
+
+
+
+
+
+
+

+6. Security Events +

+
+
+

+6.1. key_updated +

+

The key_updated event has Base importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+SecurityKeyUpdated = {
+    key_type: $KeyType
+    ? old: hexstring
+    ? new: hexstring
+
+    ; needed for 1RTT key updates
+    ? key_phase: uint64
+    ? trigger:
+        ; (e.g., initial, handshake and 0-RTT keys
+        ; are generated by TLS)
+        "tls" /
+        "remote_update" /
+        "local_update"
+
+    * $$quic-keyupdated-extension
+}
+
+
+
Figure 28: +SecurityKeyUpdated definition +
+
+

Note that the key_phase is the full value of the key phase (as indicated by +@M and @N in Figure 9 of [QUIC-TLS]). The key phase bit used on +the packet header is the least significant bit of the key phase.

+
+
+
+
+

+6.2. key_discarded +

+

The key_discarded event has Base importance level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+SecurityKeyDiscarded = {
+    key_type: $KeyType
+    ? key: hexstring
+
+    ; needed for 1RTT key updates
+    ? key_phase: uint64
+    ? trigger:
+        ; (e.g., initial, handshake and 0-RTT keys
+        ; are generated by TLS)
+        "tls" /
+        "remote_update" /
+        "local_update"
+
+    * $$quic-keydiscarded-extension
+}
+
+
+
Figure 29: +SecurityKeyDiscarded definition +
+
+
+
+
+
+
+
+

+7. Recovery events +

+

Most of the events in this category are kept generic to support different +recovery approaches and various congestion control algorithms. Tool creators +SHOULD make an effort to support and visualize even unknown data in these events +(e.g., plot unknown congestion states by name on a timeline visualization).

+
+
+

+7.1. parameters_set +

+

The parameters_set event groups initial parameters from both loss detection +and congestion control into a single event. It has Base importance level; see +Section 9.2 of [QLOG-MAIN].

+

All these settings are typically set once and never change. Implementation that +do, for some reason, change these parameters during execution, MAY emit the +parameters_set event more than once.

+
+
+
+
+RecoveryParametersSet = {
+
+    ; Loss detection, see RFC 9002 Appendix A.2
+    ; in amount of packets
+    ? reordering_threshold: uint16
+
+    ; as RTT multiplier
+    ? time_threshold: float32
+
+    ; in ms
+    timer_granularity: uint16
+
+    ; in ms
+    ? initial_rtt:float32
+
+    ; congestion control, see RFC 9002 Appendix B.2
+    ; in bytes. Note that this could be updated after pmtud
+    ? max_datagram_size: uint32
+
+    ; in bytes
+    ? initial_congestion_window: uint64
+
+    ; Note that this could change when max_datagram_size changes
+    ; in bytes
+    ? minimum_congestion_window: uint64
+    ? loss_reduction_factor: float32
+
+    ; as PTO multiplier
+    ? persistent_congestion_threshold: uint16
+
+    * $$recovery-parametersset-extension
+}
+
+
+
Figure 30: +RecoveryParametersSet definition +
+
+

Additionally, this event can contain any number of unspecified fields to support +different recovery approaches.

+
+
+
+
+

+7.2. metrics_updated +

+

The metrics_updated event is emitted when one or more of the observable +recovery metrics changes value. It has Core importance level; see Section 9.2 of [QLOG-MAIN].

+

This event SHOULD group all possible metric updates that happen at or around the +same time in a single event (e.g., if min_rtt and smoothed_rtt change at the +same time, they should be bundled in a single metrics_updated entry, rather +than split out into two). Consequently, a metrics_updated event is only +guaranteed to contain at least one of the listed metrics.

+
+
+
+
+RecoveryMetricsUpdated = {
+
+    ; Loss detection, see RFC 9002 Appendix A.3
+    ; all following rtt fields are expressed in ms
+    ? min_rtt: float32
+    ? smoothed_rtt: float32
+    ? latest_rtt: float32
+    ? rtt_variance: float32
+    ? pto_count: uint16
+
+    ; Congestion control, see RFC 9002 Appendix B.2.
+    ; in bytes
+    ? congestion_window: uint64
+    ? bytes_in_flight: uint64
+
+    ; in bytes
+    ? ssthresh: uint64
+
+    ; qlog defined
+    ; sum of all packet number spaces
+    ? packets_in_flight: uint64
+
+    ; in bits per second
+    ? pacing_rate: uint64
+
+    * $$recovery-metricsupdated-extension
+}
+
+
+
Figure 31: +RecoveryMetricsUpdated definition +
+
+

In order to make logging easier, implementations MAY log values even if they are the +same as previously reported values (e.g., two subsequent RecoveryMetricsUpdated entries can +both report the exact same value for min_rtt). However, applications SHOULD try to +log only actual updates to values.

+

Additionally, the metrics_updated event can contain any number of unspecified fields to support +different recovery approaches.

+
+
+
+
+

+7.3. congestion_state_updated +

+

The congestion_state_updated event indicates when the congestion controller +enters a significant new state and changes its behaviour. It has Base importance +level; see Section 9.2 of [QLOG-MAIN].

+

The values of the event's fields are intentionally unspecified here in order to +support different Congestion Control algorithms, as these typically have +different states and even different implementations of these states across +stacks. For example, for the algorithm defined in the Recovery draft ("enhanced" +New Reno), the following states are used: Slow Start, Congestion Avoidance, +Application Limited and Recovery. Similarly, states can be triggered by a +variety of events, including detection of Persistent Congestion or receipt of +ECN markings.

+
+
+
+
+RecoveryCongestionStateUpdated = {
+    ? old: text
+    new: text
+    ? trigger: text
+
+    * $$recovery-congestionstateupdated-extension
+}
+
+
+
Figure 32: +RecoveryCongestionStateUpdated definition +
+
+

The trigger field SHOULD be logged if there are multiple ways in which a state +change can occur but MAY be omitted if a given state can only be due to a single +event occurring (for example Slow Start is often exited only when ssthresh is +exceeded).

+
+
+
+
+

+7.4. loss_timer_updated +

+

The loss_timer_updated event is emitted when a recovery loss timer changes +state. It has Extra importance level; see Section 9.2 of [QLOG-MAIN].

+

The three main event types are:

+
    +
  • +

    set: the timer is set with a delta timeout for when it will trigger next

    +
    • +
  • +

    expired: when the timer effectively expires after the delta timeout

    +
    • +
  • +

    cancelled: when a timer is cancelled (e.g., all outstanding packets are +acknowledged, start idle period)

    +
    • +
+

In order to indicate an active timer's timeout update, a new set event is used.

+
+
+
+
+RecoveryLossTimerUpdated = {
+
+    ; called "mode" in RFC 9002 A.9.
+    ? timer_type: "ack" /
+                  "pto"
+    ? packet_number_space: $PacketNumberSpace
+    event_type: "set" /
+                "expired" /
+                "cancelled"
+
+    ; if event_type === "set": delta time is in ms from
+    ; this event's timestamp until when the timer will trigger
+    ? delta: float32
+
+    * $$recovery-losstimerupdated-extension
+}
+
+
+
Figure 33: +RecoveryLossTimerUpdated definition +
+
+
+
+
+
+

+7.5. packet_lost +

+

The packet_lost event is emitted when a packet is deemed lost by loss +detection. It has Core importance level; see Section 9.2 of [QLOG-MAIN].

+

It is RECOMMENDED to populate the optional trigger field in order to help +disambiguate among the various possible causes of a loss declaration.

+
+
+
+
+RecoveryPacketLost = {
+
+    ; should include at least the packet_type and packet_number
+    ? header: PacketHeader
+
+    ; not all implementations will keep track of full
+    ; packets, so these are optional
+    ? frames: [* $QuicFrame]
+    ? is_mtu_probe_packet: bool .default false
+    ? trigger:
+        "reordering_threshold" /
+        "time_threshold" /
+        ; RFC 9002 Section 6.2.4 paragraph 6, MAY
+        "pto_expired"
+
+    * $$recovery-packetlost-extension
+}
+
+
+
Figure 34: +RecoveryPacketLost definition +
+
+
+
+
+
+

+7.6. marked_for_retransmit +

+

The marked_for_retransmit event indicates which data was marked for +retransmission upon detection of packet loss (see packet_lost). It has Extra +importance level; see Section 9.2 of [QLOG-MAIN].

+

Similar to the reasoning for the frames_processed event, +in order to keep the amount of different events low, this signal is grouped into +in a single event based on existing QUIC frame definitions for all types of +retransmittable data.

+

Implementations retransmitting full packets or frames directly can just log the +constituent frames of the lost packet here (or do away with this event and use the +contents of the packet_lost event instead). Conversely, implementations that have +more complex logic (e.g., marking ranges in a stream's data buffer as in-flight), +or that do not track sent frames in full (e.g., only stream offset + length), can +translate their internal behaviour into the appropriate frame instance here even +if that frame was never or will never be put on the wire.

+

Much of this data can be inferred if implementations log packet_sent events +(e.g., looking at overlapping stream data offsets and length, one can determine +when data was retransmitted).

+
+
+
+
+RecoveryMarkedForRetransmit = {
+    frames: [+ $QuicFrame]
+
+    * $$recovery-markedforretransmit-extension
+}
+
+
+
Figure 35: +RecoveryMarkedForRetransmit definition +
+
+
+
+
+
+

+7.7. ecn_state_updated +

+

The ecn_state_updated event indicates a progression in the ECN state machine +as described in section A.4 of [QUIC-TRANSPORT]. It has Extra importance +level; see Section 9.2 of [QLOG-MAIN].

+
+
+
+
+ECNStateUpdated = {
+   ? old: ECNState
+    new: ECNState
+
+    * $$recovery-ecnstateupdated-extension
+}
+
+ECNState =
+  ; ECN testing in progress
+  "testing" /
+  ; ECN state unknown, waiting for acknowledgements
+  ; for testing packets
+  "unknown" /
+  ; ECN testing failed
+  "failed" /
+  ; testing was successful, the endpoint now
+  ; sends packets with ECT(0) marking
+  "capable"
+
+
+
Figure 36: +ECNStateUpdated definition +
+
+
+
+
+
+
+
+

+8. QUIC data field definitions +

+
+
+

+8.1. QuicVersion +

+
+
+
+
+QuicVersion = hexstring
+
+
+
Figure 37: +QuicVersion definition +
+
+
+
+
+
+

+8.2. ConnectionID +

+
+
+
+
+ConnectionID = hexstring
+
+
+
Figure 38: +ConnectionID definition +
+
+
+
+
+
+

+8.3. Owner +

+
+
+
+
+Owner = "local" /
+        "remote"
+
+
+
Figure 39: +Owner definition +
+
+
+
+
+
+

+8.4. IPAddress and IPVersion +

+
+
+
+
+; an IPAddress can either be a "human readable" form
+; (e.g., "127.0.0.1" for v4 or
+; "2001:0db8:85a3:0000:0000:8a2e:0370:7334" for v6) or
+; use a raw byte-form (as the string forms can be ambiguous).
+; Additionally, a hash-based or redacted representation
+; can be used if needed for privacy or security reasons.
+IPAddress = text /
+            hexstring
+
+
+
Figure 40: +IPAddress definition +
+
+
+
+
+
+IPVersion = "v4" /
+            "v6"
+
+
+
Figure 41: +IPVersion definition +
+
+
+
+
+
+

+8.5. PathEndpointInfo +

+

PathEndpointInfo indicates a single half/direction of a path. A full path is +comprised of two halves. Firstly: the server sends to the remote client IP ++ port using a specific destination Connection ID. Secondly: the client sends to +the remote server IP + port using a different destination Connection ID.

+

As such, structures logging path information SHOULD include two different +PathEndpointInfo instances, one for each half of the path.

+
+
+
+
+PathEndpointInfo = {
+    ? ip_v4: IPAddress
+    ? ip_v6: IPAddress
+    ? port_v4: uint16
+    ? port_v6: uint16
+
+    ; Even though usually only a single ConnectionID
+    ; is associated with a given path at a time,
+    ; there are situations where there can be an overlap
+    ; or a need to keep track of previous ConnectionIDs
+    ? connection_ids: [+ ConnectionID]
+
+    * $$quic-pathendpointinfo-extension
+}
+
+
+
Figure 42: +PathEndpointInfo definition +
+
+
+
+
+
+

+8.6. PacketType +

+
+
+
+
+$PacketType /=  "initial" /
+                "handshake" /
+                "0RTT" /
+                "1RTT" /
+                "retry" /
+                "version_negotiation" /
+                "stateless_reset" /
+                "unknown"
+
+
+
Figure 43: +PacketType definition +
+
+
+
+
+
+

+8.7. PacketNumberSpace +

+
+
+
+
+$PacketNumberSpace /= "initial" /
+                      "handshake" /
+                      "application_data"
+
+
+
Figure 44: +PacketNumberSpace definition +
+
+
+
+
+
+

+8.8. PacketHeader +

+
+
+
+
+PacketHeader = {
+    ? quic_bit: bool .default true
+    packet_type: $PacketType
+
+    ; only if packet_type === "initial" || "handshake" || "0RTT" ||
+    ;                         "1RTT"
+    ? packet_number: uint64
+
+    ; the bit flags of the packet headers (spin bit, key update bit,
+    ; etc. up to and including the packet number length bits
+    ; if present
+    ? flags: uint8
+
+    ; only if packet_type === "initial" || "retry"
+    ? token: Token
+
+    ; only if packet_type === "initial" || "handshake" || "0RTT"
+    ; Signifies length of the packet_number plus the payload
+    ? length: uint16
+
+    ; only if present in the header
+    ; if correctly using transport:connection_id_updated events,
+    ; dcid can be skipped for 1RTT packets
+    ? version: QuicVersion
+    ? scil: uint8
+    ? dcil: uint8
+    ? scid: ConnectionID
+    ? dcid: ConnectionID
+
+    * $$quic-packetheader-extension
+}
+
+
+
Figure 45: +PacketHeader definition +
+
+
+
+
+
+

+8.9. Token +

+
+
+
+
+Token = {
+    ? type: $TokenType
+
+    ; decoded fields included in the token
+    ; (typically: peer's IP address, creation time)
+    ? details: {
+      * text => any
+    }
+    ? raw: RawInfo
+
+    * $$quic-token-extension
+}
+
+$TokenType /= "retry" /
+              "resumption"
+
+
+
Figure 46: +Token definition +
+
+

The token carried in an Initial packet can either be a retry token from a Retry +packet, or one originally provided by the server in a NEW_TOKEN frame used when +resuming a connection (e.g., for address validation purposes). Retry and +resumption tokens typically contain encoded metadata to check the token's +validity when it is used, but this metadata and its format is implementation +specific. For that, Token includes a general-purpose details field.

+
+
+
+
+

+8.10. Stateless Reset Token +

+
+
+
+
+StatelessResetToken = hexstring .size 16
+
+
+
Figure 47: +Stateless Reset Token definition +
+
+

The stateless reset token is carried in stateless reset packets, in transport +parameters and in NEW_CONNECTION_ID frames.

+
+
+
+
+

+8.11. KeyType +

+
+
+
+
+$KeyType /= "server_initial_secret" /
+            "client_initial_secret" /
+            "server_handshake_secret" /
+            "client_handshake_secret" /
+            "server_0rtt_secret" /
+            "client_0rtt_secret" /
+            "server_1rtt_secret" /
+            "client_1rtt_secret"
+
+
+
Figure 48: +KeyType definition +
+
+
+
+
+
+

+8.12. ECN +

+
+
+
+
+ECN = "Not-ECT" / "ECT(1)" / "ECT(0)" / "CE"
+
+
+
Figure 49: +ECN definition +
+
+

The ECN bits carried in the IP header.

+
+
+
+
+

+8.13. QUIC Frames +

+

The generic $QuicFrame is defined here as a CDDL "type socket" extension +point. It can be extended to support additional QUIC frame types.

+
+
+
+
+; The QuicFrame is any key-value map (e.g., JSON object)
+$QuicFrame /= {
+    * text => any
+}
+
+
+
Figure 50: +QuicFrame type socket definition +
+
+

The QUIC frame types defined in this document are as follows:

+
+
+
+
+QuicBaseFrames =  PaddingFrame /
+                  PingFrame /
+                  AckFrame /
+                  ResetStreamFrame /
+                  StopSendingFrame /
+                  CryptoFrame /
+                  NewTokenFrame /
+                  StreamFrame /
+                  MaxDataFrame /
+                  MaxStreamDataFrame /
+                  MaxStreamsFrame /
+                  DataBlockedFrame /
+                  StreamDataBlockedFrame /
+                  StreamsBlockedFrame /
+                  NewConnectionIDFrame /
+                  RetireConnectionIDFrame /
+                  PathChallengeFrame /
+                  PathResponseFrame /
+                  ConnectionCloseFrame /
+                  HandshakeDoneFrame /
+                  UnknownFrame /
+                  DatagramFrame
+
+$QuicFrame /= QuicBaseFrames
+
+
+
Figure 51: +QuicBaseFrames definition +
+
+
+
+

+8.13.1. PaddingFrame +

+

In QUIC, PADDING frames are simply identified as a single byte of value 0. As +such, each padding byte could be theoretically interpreted and logged as an +individual PaddingFrame.

+

However, as this leads to heavy logging overhead, implementations SHOULD instead +emit just a single PaddingFrame and set the payload_length property to the amount +of PADDING bytes/frames included in the packet.

+
+
+
+
+PaddingFrame = {
+    frame_type: "padding"
+
+    ; total frame length, including frame header
+    ? length: uint32
+    payload_length: uint32
+}
+
+
+
Figure 52: +PaddingFrame definition +
+
+
+
+
+
+

+8.13.2. PingFrame +

+
+
+
+
+PingFrame = {
+    frame_type: "ping"
+
+    ; total frame length, including frame header
+    ? length: uint32
+    ? payload_length: uint32
+}
+
+
+
Figure 53: +PingFrame definition +
+
+
+
+
+
+

+8.13.3. AckFrame +

+
+
+
+
+; either a single number (e.g., [1]) or two numbers (e.g., [1,2]).
+; For two numbers:
+; the first number is "from": lowest packet number in interval
+; the second number is "to": up to and including the highest
+; packet number in the interval
+AckRange = [1*2 uint64]
+
+AckFrame = {
+    frame_type: "ack"
+
+    ; in ms
+    ? ack_delay: float32
+
+    ; e.g., looks like [[1,2],[4,5], [7], [10,22]] serialized
+    ? acked_ranges: [+ AckRange]
+
+    ; ECN (explicit congestion notification) related fields
+    ; (not always present)
+    ? ect1: uint64
+    ? ect0: uint64
+    ? ce: uint64
+
+    ; total frame length, including frame header
+    ? length: uint32
+    ? payload_length: uint32
+}
+
+
+
Figure 54: +AckFrame definition +
+
+

Note that the packet ranges in AckFrame.acked_ranges do not necessarily have to be +ordered (e.g., [[5,9],[1,4]] is a valid value).

+

Note that the two numbers in the packet range can be the same (e.g., [120,120] means +that packet with number 120 was ACKed). However, in that case, implementers SHOULD +log [120] instead and tools MUST be able to deal with both notations.

+
+
+
+
+

+8.13.4. ResetStreamFrame +

+
+
+
+
+ResetStreamFrame = {
+    frame_type: "reset_stream"
+    stream_id: uint64
+    error_code: $ApplicationError /
+                uint64
+
+    ; in bytes
+    final_size: uint64
+
+    ; total frame length, including frame header
+    ? length: uint32
+    ? payload_length: uint32
+}
+
+
+
Figure 55: +ResetStreamFrame definition +
+
+
+
+
+
+

+8.13.5. StopSendingFrame +

+
+
+
+
+StopSendingFrame = {
+    frame_type: "stop_sending"
+    stream_id: uint64
+    error_code: $ApplicationError /
+                uint64
+
+    ; total frame length, including frame header
+    ? length: uint32
+    ? payload_length: uint32
+}
+
+
+
Figure 56: +StopSendingFrame definition +
+
+
+
+
+
+

+8.13.6. CryptoFrame +

+
+
+
+
+CryptoFrame = {
+    frame_type: "crypto"
+    offset: uint64
+    length: uint64
+    ? payload_length: uint32
+    ? raw: RawInfo
+}
+
+
+
Figure 57: +CryptoFrame definition +
+
+
+
+
+
+

+8.13.7. NewTokenFrame +

+
+
+
+
+NewTokenFrame = {
+  frame_type: "new_token"
+  token: Token
+}
+
+
+
Figure 58: +NewTokenFrame definition +
+
+
+
+
+
+

+8.13.8. StreamFrame +

+
+
+
+
+StreamFrame = {
+    frame_type: "stream"
+    stream_id: uint64
+
+    ; These two MUST always be set
+    ; If not present in the Frame type, log their default values
+    offset: uint64
+    length: uint64
+
+    ; this MAY be set any time,
+    ; but MUST only be set if the value is true
+    ; if absent, the value MUST be assumed to be false
+    ? fin: bool .default false
+    ? raw: RawInfo
+}
+
+
+
Figure 59: +StreamFrame definition +
+
+
+
+
+
+

+8.13.9. MaxDataFrame +

+
+
+
+
+MaxDataFrame = {
+  frame_type: "max_data"
+  maximum: uint64
+}
+
+
+
Figure 60: +MaxDataFrame definition +
+
+
+
+
+
+

+8.13.10. MaxStreamDataFrame +

+
+
+
+
+MaxStreamDataFrame = {
+  frame_type: "max_stream_data"
+  stream_id: uint64
+  maximum: uint64
+}
+
+
+
Figure 61: +MaxStreamDataFrame definition +
+
+
+
+
+
+

+8.13.11. MaxStreamsFrame +

+
+
+
+
+MaxStreamsFrame = {
+  frame_type: "max_streams"
+  stream_type: StreamType
+  maximum: uint64
+}
+
+
+
Figure 62: +MaxStreamsFrame definition +
+
+
+
+
+
+

+8.13.12. DataBlockedFrame +

+
+
+
+
+DataBlockedFrame = {
+  frame_type: "data_blocked"
+  limit: uint64
+}
+
+
+
Figure 63: +DataBlockedFrame definition +
+
+
+
+
+
+

+8.13.13. StreamDataBlockedFrame +

+
+
+
+
+StreamDataBlockedFrame = {
+  frame_type: "stream_data_blocked"
+  stream_id: uint64
+  limit: uint64
+}
+
+
+
Figure 64: +StreamDataBlockedFrame definition +
+
+
+
+
+
+

+8.13.14. StreamsBlockedFrame +

+
+
+
+
+StreamsBlockedFrame = {
+  frame_type: "streams_blocked"
+  stream_type: StreamType
+  limit: uint64
+}
+
+
+
Figure 65: +StreamsBlockedFrame definition +
+
+
+
+
+
+

+8.13.15. NewConnectionIDFrame +

+
+
+
+
+NewConnectionIDFrame = {
+  frame_type: "new_connection_id"
+  sequence_number: uint32
+  retire_prior_to: uint32
+
+  ; mainly used if e.g., for privacy reasons the full
+  ; connection_id cannot be logged
+  ? connection_id_length: uint8
+  connection_id: ConnectionID
+  ? stateless_reset_token: StatelessResetToken
+}
+
+
+
Figure 66: +NewConnectionIDFrame definition +
+
+
+
+
+
+

+8.13.16. RetireConnectionIDFrame +

+
+
+
+
+RetireConnectionIDFrame = {
+  frame_type: "retire_connection_id"
+  sequence_number: uint32
+}
+
+
+
Figure 67: +RetireConnectionIDFrame definition +
+
+
+
+
+
+

+8.13.17. PathChallengeFrame +

+
+
+
+
+PathChallengeFrame = {
+  frame_type: "path_challenge"
+
+  ; always 64-bit
+  ? data: hexstring
+}
+
+
+
Figure 68: +PathChallengeFrame definition +
+
+
+
+
+
+

+8.13.18. PathResponseFrame +

+
+
+
+
+PathResponseFrame = {
+  frame_type: "path_response"
+
+  ; always 64-bit
+  ? data: hexstring
+}
+
+
+
Figure 69: +PathResponseFrame definition +
+
+
+
+
+
+

+8.13.19. ConnectionCloseFrame +

+

An endpoint that receives unknown error codes can record it in the error_code +field using the numerical value without variable-length integer encoding.

+

When the connection is closed due a connection-level error, the +trigger_frame_type field can be used to log the frame that triggered the +error. For known frame types, the appropriate string value is used. For unknown +frame types, the numerical value without variable-length integer encoding is +used.

+

The CONNECTION_CLOSE reason phrase is a byte sequences. It is likely that this +sequence is presentable as UTF-8, in which case it can be logged in the reason +field. The reason_bytes field supports logging the raw bytes, which can be useful +when the value is not UTF-8 or when an endpoint does not want to decode it. +Implementations SHOULD log at least one format, but MAY log both or none.

+
+
+
+
+ErrorSpace = "transport" /
+             "application"
+
+ConnectionCloseFrame = {
+    frame_type: "connection_close"
+    ? error_space: ErrorSpace
+    ? error_code: $TransportError /
+                  CryptoError /
+                  $ApplicationError /
+                  uint64
+    ? reason: text
+    ? reason_bytes: hexstring
+
+    ; when error_space === "transport"
+    ? trigger_frame_type: uint64 /
+                          text
+}
+
+
+
Figure 70: +ConnectionCloseFrame definition +
+
+
+
+
+
+

+8.13.20. HandshakeDoneFrame +

+
+
+
+
+HandshakeDoneFrame = {
+  frame_type: "handshake_done";
+}
+
+
+
Figure 71: +HandshakeDoneFrame definition +
+
+
+
+
+
+

+8.13.21. UnknownFrame +

+

The frame_type_bytes field is the numerical value without variable-length +integer encoding.

+
+
+
+
+UnknownFrame = {
+    frame_type: "unknown"
+    frame_type_bytes: uint64
+    ? raw: RawInfo
+}
+
+
+
Figure 72: +UnknownFrame definition +
+
+
+
+
+
+

+8.13.22. DatagramFrame +

+

The QUIC DATAGRAM frame is defined in Section 4 of [RFC9221].

+
+
+
+
+DatagramFrame = {
+    frame_type: "datagram"
+    ? length: uint64
+    ? raw: RawInfo
+}
+
+
+
Figure 73: +DatagramFrame definition +
+
+
+
+
+
+

+8.13.23. TransportError +

+

The generic $TransportError is defined here as a CDDL "type socket" extension +point. It can be extended to support additional Transport errors.

+
+
+
+
+$TransportError /= "no_error" /
+                 "internal_error" /
+                 "connection_refused" /
+                 "flow_control_error" /
+                 "stream_limit_error" /
+                 "stream_state_error" /
+                 "final_size_error" /
+                 "frame_encoding_error" /
+                 "transport_parameter_error" /
+                 "connection_id_limit_error" /
+                 "protocol_violation" /
+                 "invalid_token" /
+                 "application_error" /
+                 "crypto_buffer_exceeded" /
+                 "key_update_error" /
+                 "aead_limit_reached" /
+                 "no_viable_path"
+                 ; there is no value to reflect CRYPTO_ERROR
+                 ; use the CryptoError type instead
+
+
+
Figure 74: +TransportError definition +
+
+
+
+
+
+

+8.13.24. ApplicationError +

+

By definition, an application error is defined by the application-level +protocol running on top of QUIC (e.g., HTTP/3).

+

As such, it cannot be defined here directly. It is instead defined as an empty +CDDL "type socket" extension point.

+

Application-level qlog definitions that wish to define new ApplicationError +strings MUST do so by extending the $ApplicationError socket as such:

+
+
+$ApplicationError /= "new_error_name" /
+                     "another_new_error_name"
+
+
+
+
+
+
+

+8.13.25. CryptoError +

+

These errors are defined in the TLS document as "A TLS alert is turned into a QUIC +connection error by converting the one-byte alert description into a QUIC error +code. The alert description is added to 0x100 to produce a QUIC error code from +the range reserved for CRYPTO_ERROR."

+

This approach maps badly to a pre-defined enum. As such, the crypto_error +string is defined as having a dynamic component here, which should include the +hex-encoded and zero-padded value of the TLS alert description.

+
+
+
+
+; all strings from "crypto_error_0x100" to "crypto_error_0x1ff"
+CryptoError = text .regexp "crypto_error_0x1[0-9a-f][0-9a-f]"
+
+
+
Figure 75: +CryptoError definition +
+
+
+
+
+
+
+
+
+
+

+9. Security and Privacy Considerations +

+

The security and privacy considerations discussed in [QLOG-MAIN] apply to this +document as well.

+
+
+
+
+

+10. IANA Considerations +

+

There are no IANA considerations.

+
+
+
+

+11. References +

+
+
+

+11.1. Normative References +

+
+
[CDDL]
+
+Birkholz, H., Vigano, C., and C. Bormann, "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, , <https://www.rfc-editor.org/rfc/rfc8610>.
+
+
[GREASEBIT]
+
+Thomson, M., "Greasing the QUIC Bit", RFC 9287, DOI 10.17487/RFC9287, , <https://www.rfc-editor.org/rfc/rfc9287>.
+
+
[QLOG-MAIN]
+
+Marx, R., Niccolini, L., Seemann, M., and L. Pardue, "Main logging schema for qlog", Work in Progress, Internet-Draft, draft-ietf-quic-qlog-main-schema-08, , <https://datatracker.ietf.org/doc/html/draft-ietf-quic-qlog-main-schema-08>.
+
+
[QUIC-DATAGRAM]
+
+Pauly, T., Kinnear, E., and D. Schinazi, "An Unreliable Datagram Extension to QUIC", RFC 9221, DOI 10.17487/RFC9221, , <https://www.rfc-editor.org/rfc/rfc9221>.
+
+
[QUIC-RECOVERY]
+
+Iyengar, J., Ed. and I. Swett, Ed., "QUIC Loss Detection and Congestion Control", RFC 9002, DOI 10.17487/RFC9002, , <https://www.rfc-editor.org/rfc/rfc9002>.
+
+
[QUIC-TLS]
+
+Thomson, M., Ed. and S. Turner, Ed., "Using TLS to Secure QUIC", RFC 9001, DOI 10.17487/RFC9001, , <https://www.rfc-editor.org/rfc/rfc9001>.
+
+
[QUIC-TRANSPORT]
+
+Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based Multiplexed and Secure Transport", RFC 9000, DOI 10.17487/RFC9000, , <https://www.rfc-editor.org/rfc/rfc9000>.
+
+
[RFC2119]
+
+Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/rfc/rfc2119>.
+
+
[RFC8174]
+
+Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/rfc/rfc8174>.
+
+
[RFC9221]
+
+Pauly, T., Kinnear, E., and D. Schinazi, "An Unreliable Datagram Extension to QUIC", RFC 9221, DOI 10.17487/RFC9221, , <https://www.rfc-editor.org/rfc/rfc9221>.
+
+
+
+
+
+
+

+11.2. Informative References +

+
+
[RFC7301]
+
+Friedl, S., Popov, A., Langley, A., and E. Stephan, "Transport Layer Security (TLS) Application-Layer Protocol Negotiation Extension", RFC 7301, DOI 10.17487/RFC7301, , <https://www.rfc-editor.org/rfc/rfc7301>.
+
+
+
+
+
+
+
+

+Acknowledgements +

+

Much of the initial work by Robin Marx was done at the Hasselt and KU Leuven +Universities.

+

Thanks to Jana Iyengar, Brian Trammell, Dmitri Tikhonov, Stephen Petrides, Jari +Arkko, Marcus Ihlar, Victor Vasiliev, Mirja Kühlewind, Jeremy Lainé, Kazu +Yamamoto, Christian Huitema and Hugo Landau for their feedback and suggestions.

+
+
+
+
+

+Change Log +

+

This section is to be removed before publishing as an RFC.

+
+
+

+Since draft-ietf-qlog-quic-events-06: +

+
    +
  • +

    Added PathAssigned and MigrationStateUpdated events (#336)

    +
    • +
  • +

    Added extension points to parameters_set and parameters_restored (#400)

    +
    • +
  • +

    Removed error_code_value from connection_closed (#386, #392)

    +
    • +
  • +

    Renamed generation to key_phase for key_updated and key_discarded (#390)

    +
    • +
  • +

    Removed retry_token from packet_sent and packet_received (#389)

    +
    • +
  • +

    Updated ALPN handling (#385)

    +
    • +
  • +

    Added key_unavailable trigger to packet_dropped (#381)

    +
    • +
  • +

    Updated several uint32 to uint64

    +
    • +
  • +

    ProtocolEventBody is now called ProtocolEventData (#352)

    +
    • +
  • +

    Editorial changes (#402, #404, #394, #393)

    +
    • +
+
+
+
+
+

+Since draft-ietf-qlog-quic-events-05: +

+
    +
  • +

    SecurityKeyUpdated: the new key is no longer mandatory to log (#294)

    +
    • +
  • +

    Added ECN related events and metadata (#263)

    +
    • +
+
+
+
+
+

+Since draft-ietf-qlog-quic-events-04: +

+
    +
  • +

    Updated guidance on logging events across connections (#279)

    +
    • +
  • +

    Renamed 'transport' category to 'quic' (#302)

    +
    • +
  • +

    Added support for multiple packet numbers in 'quic:frames_processed' (#307)

    +
    • +
  • +

    Added definitions for RFC9287 (QUIC GREASE Bit extension) (#311)

    +
    • +
  • +

    Added definitions for RFC9221 (QUIC Datagram Frame extension) (#310)

    +
    • +
  • +

    (Temporarily) removed definitions for connection migration events (#317)

    +
    • +
  • +

    Editorial and formatting changes (#298, #299, #304, #306, #327)

    +
    • +
+
+
+
+
+

+Since draft-ietf-qlog-quic-events-03: +

+
    +
  • +

    Ensured consistent use of RawInfo to indicate raw wire bytes (#243)

    +
    • +
  • +

    Renamed UnknownFrame:raw_frame_type to :frame_type_value (#54)

    +
    • +
  • +

    Renamed ConnectionCloseFrame:raw_error_code to :error_code_value (#54)

    +
    • +
  • +

    Changed triggers for packet_dropped (#278)

    +
    • +
  • +

    Added entries to TransportError enum (#285)

    +
    • +
  • +

    Changed minimum_congestion_window to uint64 (#288)

    +
    • +
+
+
+
+
+

+Since draft-ietf-qlog-quic-events-02: +

+
    +
  • +

    Renamed key_retired to key_discarded (#185)

    +
    • +
  • +

    Added fields and events for DPLPMTUD (#135)

    +
    • +
  • +

    Made packet_number optional in PacketHeader (#244)

    +
    • +
  • +

    Removed connection_retried event placeholder (#255)

    +
    • +
  • +

    Changed QuicFrame to a CDDL plug type (#257)

    +
    • +
  • +

    Moved data definitions out of the appendix into separate sections

    +
    • +
  • +

    Added overview Table of Contents

    +
    • +
+
+
+
+
+

+Since draft-ietf-qlog-quic-events-01: +

+
    +
  • +

    Added Stateless Reset Token type (#122)

    +
    • +
+
+
+
+
+

+Since draft-ietf-qlog-quic-events-00: +

+
    +
  • +

    Change the data definition language from TypeScript to CDDL (#143)

    +
    • +
+
+
+
+
+

+Since draft-marx-qlog-event-definitions-quic-h3-02: +

+
    +
  • +

    These changes were done in preparation of the adoption of the drafts by the QUIC +working group (#137)

    +
    • +
  • +

    Split QUIC and HTTP/3 events into two separate documents

    +
    • +
  • +

    Moved RawInfo, Importance, Generic events and Simulation events to the main +schema document.

    +
    • +
  • +

    Changed to/from value options of the data_moved event

    +
    • +
+
+
+
+
+

+Since draft-marx-qlog-event-definitions-quic-h3-01: +

+

Major changes:

+
    +
  • +

    Moved data_moved from http to transport. Also made the "from" and "to" fields +flexible strings instead of an enum (#111,#65)

    +
    • +
  • +

    Moved packet_type fields to PacketHeader. Moved packet_size field out of +PacketHeader to RawInfo:length (#40)

    +
    • +
  • +

    Made events that need to log packet_type and packet_number use a header field +instead of logging these fields individually

    +
    • +
  • +

    Added support for logging retry, stateless reset and initial tokens (#94,#86,#117)

    +
    • +
  • +

    Moved separate general event categories into a single category "generic" (#47)

    +
    • +
  • +

    Added "transport:connection_closed" event (#43,#85,#78,#49)

    +
    • +
  • +

    Added version_information and alpn_information events (#85,#75,#28)

    +
    • +
  • +

    Added parameters_restored events to help clarify 0-RTT behaviour (#88)

    +
    • +
+

Smaller changes:

+
    +
  • +

    Merged loss_timer events into one loss_timer_updated event

    +
    • +
  • +

    Field data types are now strongly defined (#10,#39,#36,#115)

    +
    • +
  • +

    Renamed qpack instruction_received and instruction_sent to instruction_created +and instruction_parsed (#114)

    +
    • +
  • +

    Updated qpack:dynamic_table_updated.update_type. It now has the value "inserted" +instead of "added" (#113)

    +
    • +
  • +

    Updated qpack:dynamic_table_updated. It now has an "owner" field to +differentiate encoder vs decoder state (#112)

    +
    • +
  • +

    Removed push_allowed from http:parameters_set (#110)

    +
    • +
  • +

    Removed explicit trigger field indications from events, since this was moved to +be a generic property of the "data" field (#80)

    +
    • +
  • +

    Updated transport:connection_id_updated to be more in line with other similar +events. Also dropped importance from Core to Base (#45)

    +
    • +
  • +

    Added length property to PaddingFrame (#34)

    +
    • +
  • +

    Added packet_number field to transport:frames_processed (#74)

    +
    • +
  • +

    Added a way to generically log packet header flags (first 8 bits) to +PacketHeader

    +
    • +
  • +

    Added additional guidance on which events to log in which situations (#53)

    +
    • +
  • +

    Added "simulation:scenario" event to help indicate simulation details

    +
    • +
  • +

    Added "packets_acked" event (#107)

    +
    • +
  • +

    Added "datagram_ids" to the datagram_X and packet_X events to allow tracking of +coalesced QUIC packets (#91)

    +
    • +
  • +

    Extended connection_state_updated with more fine-grained states (#49)

    +
    • +
+
+
+
+
+

+Since draft-marx-qlog-event-definitions-quic-h3-00: +

+
    +
  • +

    Event and category names are now all lowercase

    +
    • +
  • +

    Added many new events and their definitions

    +
    • +
  • +

    "type" fields have been made more specific (especially important for PacketType +fields, which are now called packet_type instead of type)

    +
    • +
  • +

    Events are given an importance indicator (issue #22)

    +
    • +
  • +

    Event names are more consistent and use past tense (issue #21)

    +
    • +
  • +

    Triggers have been redefined as properties of the "data" field and updated for most events (issue #23)

    +
    • +
+
+
+
+
+
+
+

+Authors' Addresses +

+
+
Robin Marx (editor)
+
Akamai
+ +
+
+
Luca Niccolini (editor)
+
Meta
+ +
+
+
Marten Seemann (editor)
+ +
+
+
Lucas Pardue (editor)
+
Cloudflare
+ +
+
+
+ + + diff --git a/security-stuff/draft-ietf-quic-qlog-quic-events.txt b/security-stuff/draft-ietf-quic-qlog-quic-events.txt new file mode 100644 index 00000000..426f5565 --- /dev/null +++ b/security-stuff/draft-ietf-quic-qlog-quic-events.txt @@ -0,0 +1,2718 @@ + + + + +QUIC R. Marx, Ed. +Internet-Draft Akamai +Intended status: Standards Track L. Niccolini, Ed. +Expires: 29 December 2024 Meta + M. Seemann, Ed. + + L. Pardue, Ed. + Cloudflare + 27 June 2024 + + + QUIC event definitions for qlog + draft-ietf-quic-qlog-quic-events-latest + +Abstract + + This document describes concrete qlog event definitions and their + metadata for QUIC events. These events can then be embedded in the + higher level schema defined in [QLOG-MAIN]. + +Note to Readers + + Note to RFC editor: Please remove this section before publication. + + Feedback and discussion are welcome at https://github.com/quicwg/qlog + (https://github.com/quicwg/qlog). Readers are advised to refer to + the "editor's draft" at that URL for an up-to-date version of this + document. + + Concrete examples of integrations of this schema in various + programming languages can be found at https://github.com/quiclog/ + qlog/ (https://github.com/quiclog/qlog/). + +Status of This Memo + + This Internet-Draft is submitted in full conformance with the + provisions of BCP 78 and BCP 79. + + Internet-Drafts are working documents of the Internet Engineering + Task Force (IETF). Note that other groups may also distribute + working documents as Internet-Drafts. The list of current Internet- + Drafts is at https://datatracker.ietf.org/drafts/current/. + + Internet-Drafts are draft documents valid for a maximum of six months + and may be updated, replaced, or obsoleted by other documents at any + time. It is inappropriate to use Internet-Drafts as reference + material or to cite them other than as "work in progress." + + This Internet-Draft will expire on 29 December 2024. + +Copyright Notice + + Copyright (c) 2024 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents (https://trustee.ietf.org/ + license-info) in effect on the date of publication of this document. + Please review these documents carefully, as they describe your rights + and restrictions with respect to this document. Code Components + extracted from this document must include Revised BSD License text as + described in Section 4.e of the Trust Legal Provisions and are + provided without warranty as described in the Revised BSD License. + +Table of Contents + + 1. Introduction + 1.1. Notational Conventions + 2. Overview + 2.1. Raw packet and frame information + 2.2. Events not belonging to a single connection + 3. QUIC Event Overview + 4. Connectivity events + 4.1. server_listening + 4.2. connection_started + 4.3. connection_closed + 4.4. connection_id_updated + 4.5. spin_bit_updated + 4.6. connection_state_updated + 4.7. path_assigned + 4.8. mtu_updated + 5. QUIC events + 5.1. version_information + 5.2. alpn_information + 5.3. parameters_set + 5.4. parameters_restored + 5.5. packet_sent + 5.6. packet_received + 5.7. packet_dropped + 5.8. packet_buffered + 5.9. packets_acked + 5.10. udp_datagrams_sent + 5.11. udp_datagrams_received + 5.12. udp_datagram_dropped + 5.13. stream_state_updated + 5.14. frames_processed + 5.15. stream_data_moved + 5.16. datagram_data_moved + 5.17. migration_state_updated + 6. Security Events + 6.1. key_updated + 6.2. key_discarded + 7. Recovery events + 7.1. parameters_set + 7.2. metrics_updated + 7.3. congestion_state_updated + 7.4. loss_timer_updated + 7.5. packet_lost + 7.6. marked_for_retransmit + 7.7. ecn_state_updated + 8. QUIC data field definitions + 8.1. QuicVersion + 8.2. ConnectionID + 8.3. Owner + 8.4. IPAddress and IPVersion + 8.5. PathEndpointInfo + 8.6. PacketType + 8.7. PacketNumberSpace + 8.8. PacketHeader + 8.9. Token + 8.10. Stateless Reset Token + 8.11. KeyType + 8.12. ECN + 8.13. QUIC Frames + 8.13.1. PaddingFrame + 8.13.2. PingFrame + 8.13.3. AckFrame + 8.13.4. ResetStreamFrame + 8.13.5. StopSendingFrame + 8.13.6. CryptoFrame + 8.13.7. NewTokenFrame + 8.13.8. StreamFrame + 8.13.9. MaxDataFrame + 8.13.10. MaxStreamDataFrame + 8.13.11. MaxStreamsFrame + 8.13.12. DataBlockedFrame + 8.13.13. StreamDataBlockedFrame + 8.13.14. StreamsBlockedFrame + 8.13.15. NewConnectionIDFrame + 8.13.16. RetireConnectionIDFrame + 8.13.17. PathChallengeFrame + 8.13.18. PathResponseFrame + 8.13.19. ConnectionCloseFrame + 8.13.20. HandshakeDoneFrame + 8.13.21. UnknownFrame + 8.13.22. DatagramFrame + 8.13.23. TransportError + 8.13.24. ApplicationError + 8.13.25. CryptoError + 9. Security and Privacy Considerations + 10. IANA Considerations + 11. References + 11.1. Normative References + 11.2. Informative References + Acknowledgements + Change Log + Since draft-ietf-qlog-quic-events-06: + Since draft-ietf-qlog-quic-events-05: + Since draft-ietf-qlog-quic-events-04: + Since draft-ietf-qlog-quic-events-03: + Since draft-ietf-qlog-quic-events-02: + Since draft-ietf-qlog-quic-events-01: + Since draft-ietf-qlog-quic-events-00: + Since draft-marx-qlog-event-definitions-quic-h3-02: + Since draft-marx-qlog-event-definitions-quic-h3-01: + Since draft-marx-qlog-event-definitions-quic-h3-00: + Authors' Addresses + +1. Introduction + + This document describes the values of the qlog name ("category" + + "event") and "data" fields and their semantics for the QUIC protocol + (see [QUIC-TRANSPORT], [QUIC-RECOVERY], and [QUIC-TLS]) and some of + its extensions (see [QUIC-DATAGRAM] and [GREASEBIT]). + +1.1. Notational Conventions + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and + "OPTIONAL" in this document are to be interpreted as described in + BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all + capitals, as shown here. + + The event and data structure definitions in ths document are + expressed in the Concise Data Definition Language [CDDL] and its + extensions described in [QLOG-MAIN]. + + The following fields from [QLOG-MAIN] are imported and used: name, + category, type, data, group_id, protocol_type, importance, RawInfo, + and time-related fields. + + As is the case for [QLOG-MAIN], the qlog schema definitions in this + document are intentionally agnostic to serialization formats. The + choice of format is an implementation decision. + +2. Overview + + This document describes how the QUIC protocol can be expressed in + qlog using the schema defined in [QLOG-MAIN]. QUIC protocol events + are defined with a category, a name (the concatenation of "category" + and "event"), an "importance", an optional "trigger", and "data" + fields. + + Some data fields use complex datastructures. These are represented + as enums or re-usable definitions, which are grouped together on the + bottom of this document for clarity. + + When any event from this document is included in a qlog trace, the + protocol_type qlog array field MUST contain an entry with the value + "QUIC": + + $ProtocolType /= "QUIC" + + Figure 1: ProtocolType extension for QUIC + + When the qlog group_id field is used, it is recommended to use QUIC's + Original Destination Connection ID (ODCID, the CID chosen by the + client when first contacting the server), as this is the only value + that does not change over the course of the connection and can be + used to link more advanced QUIC packets (e.g., Retry, Version + Negotiation) to a given connection. Similarly, the ODCID should be + used as the qlog filename or file identifier, potentially suffixed by + the vantagepoint type (For example, abcd1234_server.qlog would + contain the server-side trace of the connection with ODCID abcd1234). + +2.1. Raw packet and frame information + + QUIC packets always include an AEAD authentication tag at the end. + In general, the length of the AEAD tag depends on the TLS cipher + suite, although all cipher suites used in QUIC v1 use a 16 byte tag. + For the purposes of calculating the lengths in fields of type RawInfo + (as defined in [QLOG-MAIN]) related to QUIC packets, the AEAD tag is + regarded as a trailer. + +2.2. Events not belonging to a single connection + + A single qlog event trace is typically associated with a single QUIC + connection. However, for several types of events (for example, a + Section 5.7 event with trigger value of connection_unknown), it can + be impossible to tie them to a specific QUIC connection, especially + on the server. + + There are various ways to handle these events, each making certain + tradeoffs between file size overhead, flexibility, ease of use, or + ease of implementation. Some options include: + + * Log them in a separate endpoint-wide trace (or use a special + group_id value) not associated with a single connection. + + * Log them in the most recently used trace. + + * Use additional heuristics for connection identification (for + example use the four-tuple in addition to the Connection ID). + + * Buffer events until they can be assigned to a connection (for + example for version negotiation and retry events). + +3. QUIC Event Overview + + QUIC connections consist of different phases and interaction events. + In order to model this, QUIC event types are divided into general + categories: connectivity (Section 4), security (Section 6), quic + Section 5, and recovery Section 7. + + As described in Section 3.4.2 of [QLOG-MAIN], the qlog name field is + the concatenation of category and type. + + Table 1 summarizes the name value of each event type that is defined + in this specification. + + +=======================================+============+==============+ + | Name value | Importance | Definition | + +=======================================+============+==============+ + | connectivity:server_listening | Extra | Section 4.1 | + +---------------------------------------+------------+--------------+ + | connectivity:connection_started | Base | Section 4.2 | + +---------------------------------------+------------+--------------+ + | connectivity:connection_closed | Base | Section 4.3 | + +---------------------------------------+------------+--------------+ + | connectivity:connection_id_updated | Base | Section 4.4 | + +---------------------------------------+------------+--------------+ + | connectivity:spin_bit_updated | Base | Section 4.5 | + +---------------------------------------+------------+--------------+ + | connectivity:connection_state_updated | Base | Section 4.6 | + +---------------------------------------+------------+--------------+ + | connectivity:path_assigned | Base | Section 4.7 | + +---------------------------------------+------------+--------------+ + | connectivity:mtu_updated | Extra | Section 4.8 | + +---------------------------------------+------------+--------------+ + | quic:version_information | Core | Section 5.1 | + +---------------------------------------+------------+--------------+ + | quic:alpn_information | Core | Section 5.2 | + +---------------------------------------+------------+--------------+ + | quic:parameters_set | Core | Section 5.3 | + +---------------------------------------+------------+--------------+ + | quic:parameters_restored | Base | Section 5.4 | + +---------------------------------------+------------+--------------+ + | quic:packet_sent | Core | Section 5.5 | + +---------------------------------------+------------+--------------+ + | quic:packet_received | Core | Section 5.6 | + +---------------------------------------+------------+--------------+ + | quic:packet_dropped | Base | Section 5.7 | + +---------------------------------------+------------+--------------+ + | quic:packet_buffered | Base | Section 5.8 | + +---------------------------------------+------------+--------------+ + | quic:packets_acked | Extra | Section 5.9 | + +---------------------------------------+------------+--------------+ + | quic:udp_datagrams_sent | Extra | Section | + | | | 5.10 | + +---------------------------------------+------------+--------------+ + | quic:udp_datagrams_received | Extra | Section | + | | | 5.11 | + +---------------------------------------+------------+--------------+ + | quic:udp_datagram_dropped | Extra | Section | + | | | 5.12 | + +---------------------------------------+------------+--------------+ + | quic:stream_state_updated | Base | Section | + | | | 5.13 | + +---------------------------------------+------------+--------------+ + | quic:frames_processed | Extra | Section | + | | | 5.14 | + +---------------------------------------+------------+--------------+ + | quic:stream_data_moved | Base | Section | + | | | 5.15 | + +---------------------------------------+------------+--------------+ + | quic:datagram_data_moved | Base | Section | + | | | 5.16 | + +---------------------------------------+------------+--------------+ + | quic:migration_state_updated | Extra | Section | + | | | 5.17 | + +---------------------------------------+------------+--------------+ + | security:key_updated | Base | Section 6.1 | + +---------------------------------------+------------+--------------+ + | security:key_discarded | Base | Section 6.2 | + +---------------------------------------+------------+--------------+ + | recovery:parameters_set | Base | Section 7.1 | + +---------------------------------------+------------+--------------+ + | recovery:metrics_updated | Core | Section 7.2 | + +---------------------------------------+------------+--------------+ + | recovery:congestion_state_updated | Base | Section 7.3 | + +---------------------------------------+------------+--------------+ + | recovery:loss_timer_updated | Extra | Section 7.4 | + +---------------------------------------+------------+--------------+ + | recovery:packet_lost | Core | Section 7.5 | + +---------------------------------------+------------+--------------+ + | recovery:marked_for_retransmit | Extra | Section 7.6 | + +---------------------------------------+------------+--------------+ + | recovery:ecn_state_updated | Extra | Section 7.7 | + +---------------------------------------+------------+--------------+ + + Table 1: QUIC Events + + QUIC events extend the $ProtocolEventData extension point defined in + [QLOG-MAIN]. Additionally, they allow for direct extensibility by + their use of per-event extension points via the $$ CDDL "group + socket" syntax, as also described in [QLOG-MAIN]. + + QuicEventData = ConnectivityServerListening / + ConnectivityConnectionStarted / + ConnectivityConnectionClosed / + ConnectivityConnectionIDUpdated / + ConnectivitySpinBitUpdated / + ConnectivityConnectionStateUpdated / + ConnectivityPathAssigned / + ConnectivityMTUUpdated / + SecurityKeyUpdated / + SecurityKeyDiscarded / + QUICVersionInformation / + QUICALPNInformation / + QUICParametersSet / + QUICParametersRestored / + QUICPacketSent / + QUICPacketReceived / + QUICPacketDropped / + QUICPacketBuffered / + QUICPacketsAcked / + QUICUDPDatagramsSent / + QUICUDPDatagramsReceived / + QUICUDPDatagramDropped / + QUICStreamStateUpdated / + QUICFramesProcessed / + QUICStreamDataMoved / + QUICDatagramDataMoved / + RecoveryParametersSet / + RecoveryMetricsUpdated / + RecoveryCongestionStateUpdated / + RecoveryLossTimerUpdated / + RecoveryPacketLost + + $ProtocolEventData /= QuicEventData + + Figure 2: QuicEventData definition and ProtocolEventData extension + +4. Connectivity events + +4.1. server_listening + + Emitted when the server starts accepting connections. It has Extra + importance level; see Section 9.2 of [QLOG-MAIN]. + + ConnectivityServerListening = { + ? ip_v4: IPAddress + ? ip_v6: IPAddress + ? port_v4: uint16 + ? port_v6: uint16 + + ; the server will always answer client initials with a retry + ; (no 1-RTT connection setups by choice) + ? retry_required: bool + + * $$connectivity-serverlistening-extension + } + + Figure 3: ConnectivityServerListening definition + + Some QUIC stacks do not handle sockets directly and are thus unable + to log IP and/or port information. + +4.2. connection_started + + The connection_started event is used for both attempting (client- + perspective) and accepting (server-perspective) new connections. + Note that while there is overlap with the connection_state_updated + event, this event is separate event in order to capture additional + data that can be useful to log. It has Base importance level; see + Section 9.2 of [QLOG-MAIN]. + + ConnectivityConnectionStarted = { + ? ip_version: IPVersion + src_ip: IPAddress + dst_ip: IPAddress + + ; transport layer protocol + ? protocol: text .default "QUIC" + ? src_port: uint16 + ? dst_port: uint16 + ? src_cid: ConnectionID + ? dst_cid: ConnectionID + + * $$connectivity-connectionstarted-extension + } + + Figure 4: ConnectivityConnectionStarted definition + + Some QUIC stacks do not handle sockets directly and are thus unable + to log IP and/or port information. + +4.3. connection_closed + + The connection_closed event is used for logging when a connection was + closed, typically when an error or timeout occurred. It has Base + importance level; see Section 9.2 of [QLOG-MAIN]. + + Note that this event has overlap with the connection_state_updated + event, as well as the CONNECTION_CLOSE frame. However, in practice, + when analyzing large deployments, it can be useful to have a single + event representing a connection_closed event, which also includes an + additional reason field to provide more information. Furthermore, it + is useful to log closures due to timeouts, which are difficult to + reflect using the other options. + + In QUIC there are two main connection-closing error categories: + connection and application errors. They have well-defined error + codes and semantics. Next to these however, there can be internal + errors that occur that may or may not get mapped to the official + error codes in implementation-specific ways. As such, multiple error + codes can be set on the same event to reflect this. + + ConnectivityConnectionClosed = { + + ; which side closed the connection + ? owner: Owner + ? connection_code: $TransportError / + CryptoError / + uint32 + ? application_code: $ApplicationError / + uint32 + ? internal_code: uint32 + ? reason: text + ? trigger: + "clean" / + "handshake_timeout" / + "idle_timeout" / + ; this is called the "immediate close" in the QUIC RFC + "error" / + "stateless_reset" / + "version_mismatch" / + ; for example HTTP/3's GOAWAY frame + "application" + + * $$connectivity-connectionclosed-extension + } + + Figure 5: ConnectivityConnectionClosed definition + +4.4. connection_id_updated + + The connection_id_updated event is emitted when either party updates + their current Connection ID. As this typically happens only + sparingly over the course of a connection, using this event is more + efficient than logging the observed CID with each and every + packet_sent or packet_received events. It has Base importance level; + see Section 9.2 of [QLOG-MAIN]. + + The connection_id_updated event is viewed from the perspective of the + endpoint applying the new ID. As such, when the endpoint receives a + new connection ID from the peer, the owner field will be "remote". + When the endpoint updates its own connection ID, the owner field will + be "local". + + ConnectivityConnectionIDUpdated = { + owner: Owner + ? old: ConnectionID + ? new: ConnectionID + + * $$connectivity-connectionidupdated-extension + } + + Figure 6: ConnectivityConnectionIDUpdated definition + +4.5. spin_bit_updated + + The spin_bit_updated event conveys information about the QUIC latency + spin bit; see Section 17.4 of [QUIC-TRANSPORT]. The event is emitted + when the spin bit changes value, it SHOULD NOT be emitted if the spin + bit is set without changing its value. It has Base importance level; + see Section 9.2 of [QLOG-MAIN]. + + ConnectivitySpinBitUpdated = { + state: bool + + * $$connectivity-spinbitupdated-extension + } + + Figure 7: ConnectivitySpinBitUpdated definition + +4.6. connection_state_updated + + The connection_state_updated event is used to track progress through + QUIC's complex handshake and connection close procedures. It has + Base importance level; see Section 9.2 of [QLOG-MAIN]. + + It is intended to provide exhaustive options to log each state + individually, but also provides a more basic, simpler set for + implementations less interested in tracking each smaller state + transition. As such, users should not expect to see all these states + reflected in all qlogs and implementers should focus on support for + the SimpleConnectionState set. + + ConnectivityConnectionStateUpdated = { + ? old: ConnectionState / + SimpleConnectionState + new: ConnectionState / + SimpleConnectionState + + * $$connectivity-connectionstateupdated-extension + } + + ConnectionState = + ; initial sent/received + "attempted" / + ; peer address validated by: client sent Handshake packet OR + ; client used CONNID chosen by the server. + ; RFC 9000 Section 8.1 + "peer_validated" / + "handshake_started" / + ; 1 RTT can be sent, but handshake isn't done yet + "early_write" / + ; TLS handshake complete: Finished received and sent + ; RFC 9001 Section 4.1.1 + "handshake_complete" / + ; HANDSHAKE_DONE sent/received (connection is now "active", 1RTT + ; can be sent). RFC 9001 Section 4.1.2 + "handshake_confirmed" / + "closing" / + ; connection_close sent/received + "draining" / + ; draining period done, connection state discarded + "closed" + + SimpleConnectionState = + "attempted" / + "handshake_started" / + "handshake_confirmed" / + "closed" + + Figure 8: ConnectivityConnectionStateUpdated definition + + These states correspond to the following transitions for both client + and server: + + *Client:* + + * send initial + + - state = attempted + + * get initial + + - state = validated _(not really "needed" at the client, but + somewhat useful to indicate progress nonetheless)_ + + * get first Handshake packet + + - state = handshake_started + + * get Handshake packet containing ServerFinished + + - state = handshake_complete + + * send ClientFinished + + - state = early_write (1RTT can now be sent) + + * get HANDSHAKE_DONE + + - state = handshake_confirmed + + *Server:* + + * get initial + + - state = attempted + + * send initial _(TODO don't think this needs a separate state, since + some handshake will always be sent in the same flight as this?)_ + + * send handshake EE, CERT, CV, ... + + - state = handshake_started + + * send ServerFinished + + - state = early_write (1RTT can now be sent) + + * get first handshake packet / something using a server-issued CID + of min length + + - state = validated + + * get handshake packet containing ClientFinished + + - state = handshake_complete + + * send HANDSHAKE_DONE + + - state = handshake_confirmed + + Note: connection_state_changed with a new state of attempted is the + same conceptual event as the connection_startedevent above from + the client's perspective. Similarly, a state + ofclosingordrainingcorresponds to theconnection_closed` event. + +4.7. path_assigned + + Importance: Base + + This event is used to associate a single PathID's value with other + parameters that describe a unique network path. + + As described in [QLOG-MAIN], each qlog event can be linked to a + single network path by means of the top-level "path" field, whose + value is a PathID. However, since it can be cumbersome to encode + additional path metadata (such as IP addresses or Connection IDs) + directly into the PathID, this event allows such an association to + happen separately. As such, PathIDs can be short and unique, and can + even be updated to be associated with new metadata as the + connection's state evolves. + + Definition: + + ConnectivityPathAssigned = { + path_id: PathID + + ; the information for traffic going towards the remote receiver + ? path_remote: PathEndpointInfo + + ; the information for traffic coming in at the local endpoint + ? path_local: PathEndpointInfo + + * $$connectivity-pathassigned-extension + } + + Figure 9: ConnectivityPathAssigned definition + + Choosing the different path_id values is left up to the + implementation. Some options include using a uniquely incrementing + integer, using the (first) Destination Connection ID associated with + a path (or its sequence number), or using (a hash of) the two + endpoint IP addresses. + + It is important to note that the empty string ("") is a valid PathID + and that it is the default assigned to events that do not explicitly + set a "path" field. Put differently, the initial path of a QUIC + connection on which the handshake occurs (see also Section 4.2) is + implicitly associated with the PathID with value "". Associating + metadata with this default path is possible by logging the + ConnectivityPathAssigned event with a value of "" for the path_id + field. + + As paths and their metadata can evolve over time, multiple + ConnectivityPathAssigned events can be emitted for each unique + PathID. The latest event contains the most up-to-date information + for that PathID. As such, the first time a PathID is seen in a + ConnectivityPathAssigned event, it is an indication that the path is + created. Subsequent occurrences indicate the path is updated, while + a final occurrence with both path_local and path_remote fields + omitted implicitly indicates the path has been abandoned. + +4.8. mtu_updated + + The mtu_updated event indicates that the estimated Path MTU was + updated. This happens as part of the Path MTU discovery process. It + has Extra importance level; see Section 9.2 of [QLOG-MAIN]. + + ConnectivityMTUUpdated = { + ? old: uint32 + new: uint32 + + ; at some point, MTU discovery stops, as a "good enough" + ; packet size has been found + ? done: bool .default false + + * $$connectivity-mtuupdated-extension + } + + Figure 10: ConnectivityMTUUpdated definition + +5. QUIC events + +5.1. version_information + + The version_information event supports QUIC version negotiation; see + Section 6 of [QUIC-TRANSPORT]. It has Core importance level; see + Section 9.2 of [QLOG-MAIN]. + + QUIC endpoints each have their own list of QUIC versions they + support. The client uses the most likely version in their first + initial. If the server does not support that version, it replies + with a Version Negotiation packet, which contains its supported + versions. From this, the client selects a version. The + version_information event aggregates all this information in a single + event type. It also allows logging of supported versions at an + endpoint without actual version negotiation needing to happen. + + QUICVersionInformation = { + ? server_versions: [+ QuicVersion] + ? client_versions: [+ QuicVersion] + ? chosen_version: QuicVersion + + * $$quic-versioninformation-extension + } + + Figure 11: QUICVersionInformation definition + + Intended use: + + * When sending an initial, the client logs this event with + client_versions and chosen_version set + + * Upon receiving a client initial with a supported version, the + server logs this event with server_versions and chosen_version set + + * Upon receiving a client initial with an unsupported version, the + server logs this event with server_versions set and + client_versions to the single-element array containing the + client's attempted version. The absence of chosen_version implies + no overlap was found + + * Upon receiving a version negotiation packet from the server, the + client logs this event with client_versions set and + server_versions to the versions in the version negotiation packet + and chosen_version to the version it will use for the next initial + packet. If the client receives a set of server_versions with no + viable overlap with its own supported versions, this event should + be logged without the chosen_version set + +5.2. alpn_information + + The alpn_information event supports Application-Layer Protocol + Negotiation (ALPN) over the QUIC transport; see [RFC7301] and + Section 7.4 of [QUIC-TRANSPORT]. It has Core importance level; see + Section 9.2 of [QLOG-MAIN]. + + QUIC endpoints are configured with a list of supported ALPN + identifiers. Clients send the list in a TLS ClientHello, and servers + match against their list. On success, a single ALPN identifier is + chosen and sent back in a TLS ServerHello. If no match is found, the + connection is closed. + + ALPN identifiers are byte sequences, that may be possible to present + as UTF-8. The ALPNIdentifier` type supports either format. + Implementations SHOULD log at least one format, but MAY log both or + none. + + QUICALPNInformation = { + ? server_alpns: [* ALPNIdentifier] + ? client_alpns: [* ALPNIdentifier] + ? chosen_alpn: ALPNIdentifier + + * $$quic-alpninformation-extension + } + + ALPNIdentifier = { + ? byte_value: hexstring + ? string_value: text + } + + Figure 12: QUICALPNInformation definition + + Intended use: + + * When sending an initial, the client logs this event with + client_alpns set + + * When receiving an initial with a supported alpn, the server logs + this event with server_alpns set, client_alpns equalling the + client-provided list, and chosen_alpn to the value it will send + back to the client. + + * When receiving an initial with an alpn, the client logs this event + with chosen_alpn to the received value. + + * Alternatively, a client can choose to not log the first event, but + wait for the receipt of the server initial to log this event with + both client_alpns and chosen_alpn set. + +5.3. parameters_set + + The parameters_set event groups settings from several different + sources (transport parameters, TLS ciphers, etc.) into a single + event. This is done to minimize the amount of events and to decouple + conceptual setting impacts from their underlying mechanism for easier + high-level reasoning. The event has Core importance level; see + Section 9.2 of [QLOG-MAIN]. + + Most of these settings are typically set once and never change. + However, they are usually set at different times during the + connection, so there will regularly be several instances of this + event with different fields set. + + Note that some settings have two variations (one set locally, one + requested by the remote peer). This is reflected in the owner field. + As such, this field MUST be correct for all settings included a + single event instance. If you need to log settings from two sides, + you MUST emit two separate event instances. + + In the case of connection resumption and 0-RTT, some of the server's + parameters are stored up-front at the client and used for the initial + connection startup. They are later updated with the server's reply. + In these cases, utilize the separate parameters_restored event to + indicate the initial values, and this event to indicate the updated + values, as normal. + + QUICParametersSet = { + ? owner: Owner + + ; true if valid session ticket was received + ? resumption_allowed: bool + + ; true if early data extension was enabled on the TLS layer + ? early_data_enabled: bool + + ; e.g., "AES_128_GCM_SHA256" + ? tls_cipher: text + + ; RFC9000 + ? original_destination_connection_id: ConnectionID + ? initial_source_connection_id: ConnectionID + ? retry_source_connection_id: ConnectionID + ? stateless_reset_token: StatelessResetToken + ? disable_active_migration: bool + ? max_idle_timeout: uint64 + ? max_udp_payload_size: uint32 + ? ack_delay_exponent: uint16 + ? max_ack_delay: uint16 + ? active_connection_id_limit: uint32 + ? initial_max_data: uint64 + ? initial_max_stream_data_bidi_local: uint64 + ? initial_max_stream_data_bidi_remote: uint64 + ? initial_max_stream_data_uni: uint64 + ? initial_max_streams_bidi: uint64 + ? initial_max_streams_uni: uint64 + ? preferred_address: PreferredAddress + + ; RFC9221 + ? max_datagram_frame_size: uint64 + + ; RFC9287 + ; true if present, absent or false if extension not negotiated + ? grease_quic_bit: bool + + * $$quic-parametersset-extension + } + + PreferredAddress = { + ip_v4: IPAddress + ip_v6: IPAddress + port_v4: uint16 + port_v6: uint16 + connection_id: ConnectionID + stateless_reset_token: StatelessResetToken + } + + Figure 13: QUICParametersSet definition + +5.4. parameters_restored + + When using QUIC 0-RTT, clients are expected to remember and restore + the server's transport parameters from the previous connection. The + parameters_restored event is used to indicate which parameters were + restored and to which values when utilizing 0-RTT. It has Base + importance level; see Section 9.2 of [QLOG-MAIN]. + + Note that not all transport parameters should be restored (many are + even prohibited from being re-utilized). The ones listed here are + the ones expected to be useful for correct 0-RTT usage. + + QUICParametersRestored = { + ? disable_active_migration: bool + ? max_idle_timeout: uint64 + ? max_udp_payload_size: uint32 + ? active_connection_id_limit: uint32 + ? initial_max_data: uint64 + ? initial_max_stream_data_bidi_local: uint64 + ? initial_max_stream_data_bidi_remote: uint64, + ? initial_max_stream_data_uni: uint64 + ? initial_max_streams_bidi: uint64 + ? initial_max_streams_uni: uint64 + + * $$quic-parametersrestored-extension + } + + Figure 14: QUICParametersRestored definition + +5.5. packet_sent + + The packet_sent event indicates a QUIC-level packet was sent. It has + Core importance level; see Section 9.2 of [QLOG-MAIN]. + + QUICPacketSent = { + header: PacketHeader + ? frames: [* $QuicFrame] + ? is_coalesced: bool .default false + + ; only if header.packet_type === "stateless_reset" + ; is always 128 bits in length. + ? stateless_reset_token: StatelessResetToken + + ; only if header.packet_type === "version_negotiation" + ? supported_versions: [+ QuicVersion] + ? raw: RawInfo + ? datagram_id: uint32 + ? is_mtu_probe_packet: bool .default false + + ? trigger: + ; RFC 9002 Section 6.1.1 + "retransmit_reordered" / + ; RFC 9002 Section 6.1.2 + "retransmit_timeout" / + ; RFC 9002 Section 6.2.4 + "pto_probe" / + ; RFC 9002 6.2.3 + "retransmit_crypto" / + ; needed for some CCs to figure out bandwidth allocations + ; when there are no normal sends + "cc_bandwidth_probe" + + * $$quic-packetsent-extension + } + + Figure 15: QUICPacketSent definition + + The encryption_level and packet_number_space are not logged + explicitly: the header.packet_type specifies this by inference + (assuming correct implementation) + + For more details on datagram_id, see Section 5.10. It is only needed + when keeping track of packet coalescing. + +5.6. packet_received + + The packet_received event indicates a QUIC-level packet was received. + It has Core importance level; see Section 9.2 of [QLOG-MAIN]. + + QUICPacketReceived = { + header: PacketHeader + ? frames: [* $QuicFrame] + ? is_coalesced: bool .default false + + ; only if header.packet_type === "stateless_reset" + ; Is always 128 bits in length. + ? stateless_reset_token: StatelessResetToken + + ; only if header.packet_type === "version_negotiation" + ? supported_versions: [+ QuicVersion] + ? raw: RawInfo + ? datagram_id: uint32 + + ? trigger: + ; if packet was buffered because it couldn't be + ; decrypted before + "keys_available" + + * $$quic-packetreceived-extension + } + + Figure 16: QUICPacketReceived definition + + The encryption_level and packet_number_space are not logged + explicitly: the header.packet_type specifies this by inference + (assuming correct implementation) + + For more details on datagram_id, see Section 5.10. It is only needed + when keeping track of packet coalescing. + +5.7. packet_dropped + + The packet_dropped event indicates a QUIC-level packet was dropped. + It has Base importance level; see Section 9.2 of [QLOG-MAIN]. + + The trigger field indicates a general reason category for dropping + the packet, while the details field can contain additional + implementation-specific information. + + QUICPacketDropped = { + + ; Primarily packet_type should be filled here, + ; as other fields might not be decrypteable or parseable + ? header: PacketHeader + ? raw: RawInfo + ? datagram_id: uint32 + ? details: {* text => any} + ? trigger: + "internal_error" / + "rejected" / + "unsupported" / + "invalid" / + "duplicate" / + "connection_unknown" / + "decryption_failure" / + "key_unavailable" / + "general" + + * $$quic-packetdropped-extension + } + + Figure 17: QUICPacketDropped definition + + Some example situations for each of the trigger categories include: + + * internal_error: not initialized, out of memory + + * rejected: limits reached, DDoS protection, unwilling to track more + paths, duplicate packet + + * unsupported: unknown or unsupported version. See also + Section 2.2. + + * invalid: packet parsing or validation error + + * duplicate: duplicate packet + + * connection_unknown: packet does not relate to a known connection + or Connection ID + + * decryption_failure: decryption failed + + * key_unavailable: decryption key was unavailable + + * general: situations not clearly covered in the other categories + + For more details on datagram_id, see Section 5.10. + +5.8. packet_buffered + + The packet_buffered event is emitted when a packet is buffered + because it cannot be processed yet. Typically, this is because the + packet cannot be parsed yet, and thus only the full packet contents + can be logged when it was parsed in a packet_received event. The + event has Base importance level; see Section 9.2 of [QLOG-MAIN]. + + QUICPacketBuffered = { + + ; primarily packet_type and possible packet_number should be + ; filled here as other elements might not be available yet + ? header: PacketHeader + ? raw: RawInfo + ? datagram_id: uint32 + ? trigger: + ; indicates the parser cannot keep up, temporarily buffers + ; packet for later processing + "backpressure" / + ; if packet cannot be decrypted because the proper keys were + ; not yet available + "keys_unavailable" + + * $$quic-packetbuffered-extension + } + + Figure 18: QUICPacketBuffered definition + + For more details on datagram_id, see Section 5.10. It is only needed + when keeping track of packet coalescing. + +5.9. packets_acked + + The packets_acked event is emitted when a (group of) sent packet(s) + is acknowledged by the remote peer _for the first time_. It has Extra + importance level; see Section 9.2 of [QLOG-MAIN]. + + This information could also be deduced from the contents of received + ACK frames. However, ACK frames require additional processing logic + to determine when a given packet is acknowledged for the first time, + as QUIC uses ACK ranges which can include repeated ACKs. + Additionally, this event can be used by implementations that do not + log frame contents. + + QUICPacketsAcked = { + ? packet_number_space: $PacketNumberSpace + ? packet_numbers: [+ uint64] + + * $$quic-packetsacked-extension + } + + Figure 19: QUICPacketsAcked definition + + If packet_number_space is omitted, it assumes the default value of + application_data, as this is by far the most prevalent packet number + space a typical QUIC connection will use. + +5.10. udp_datagrams_sent + + When one or more UDP-level datagrams are passed to the socket. This + is useful for determining how QUIC packet buffers are drained to the + OS. The event has Extra importance level; see Section 9.2 of + [QLOG-MAIN]. + + QUICUDPDatagramsSent = { + + ; to support passing multiple at once + ? count: uint16 + + ; The RawInfo fields do not include the UDP headers, + ; only the UDP payload + ? raw: [+ RawInfo] + + ; ECN bits in the IP header + ; if not set, defaults to the value used on the last + ; QUICDatagramsSent event + ? ecn: [+ ECN] + + ? datagram_ids: [+ uint32] + + * $$quic-udpdatagramssent-extension + } + + Figure 20: QUICUDPDatagramsSent definition + + Since QUIC implementations rarely control UDP logic directly, the raw + data excludes UDP-level headers in all fields. + + The datagram_id is a qlog-specific concept to allow tracking of QUIC + packet coalescing inside UDP datagrams. Since QUIC generates many + UDP datagrams, unique identifiers are required to be able to track + them individually in qlog traces. However, neither UDP nor QUIC + exchanges datagram identifiers on the wire. Selecting identifier + values is thus left to qlog implementations, which should consider + how to generate unique values within the scope of their created + traces. + +5.11. udp_datagrams_received + + When one or more UDP-level datagrams are received from the socket. + This is useful for determining how datagrams are passed to the user + space stack from the OS. The event has Extra importance level; see + Section 9.2 of [QLOG-MAIN]. + + QUICUDPDatagramsReceived = { + + ; to support passing multiple at once + ? count: uint16 + + ; The RawInfo fields do not include the UDP headers, + ; only the UDP payload + ? raw: [+ RawInfo] + + ; ECN bits in the IP header + ; if not set, defaults to the value on the last + ; QUICDatagramsReceived event + ? ecn: [+ ECN] + + ? datagram_ids: [+ uint32] + + * $$quic-udpdatagramsreceived-extension + } + + Figure 21: QUICUDPDatagramsReceived definition + + For more details on datagram_ids, see Section 5.10. + +5.12. udp_datagram_dropped + + When a UDP-level datagram is dropped. This is typically done if it + does not contain a valid QUIC packet. If it does, but the QUIC + packet is dropped for other reasons, the packet_dropped event + (Section 5.7) should be used instead. The event has Extra importance + level; see Section 9.2 of [QLOG-MAIN]. + + QUICUDPDatagramDropped = { + + ; The RawInfo fields do not include the UDP headers, + ; only the UDP payload + ? raw: RawInfo + + * $$quic-udpdatagramdropped-extension + } + + Figure 22: QUICUDPDatagramDropped definition + +5.13. stream_state_updated + + The stream_state_updated event is emitted whenever the internal state + of a QUIC stream is updated; see Section 3 of [QUIC-TRANSPORT]. Most + of this can be inferred from several types of frames going over the + wire, but it's much easier to have explicit signals for these state + changes. The event has Base importance level; see Section 9.2 of + [QLOG-MAIN]. + + StreamType = "unidirectional" / + "bidirectional" + + QUICStreamStateUpdated = { + stream_id: uint64 + + ; mainly useful when opening the stream + ? stream_type: StreamType + ? old: $StreamState + new: $StreamState + ? stream_side: "sending" / + "receiving" + + * $$quic-streamstateupdated-extension + } + + BaseStreamStates = "idle" / + "open" / + "closed" + + GranularStreamStates = + ; bidirectional stream states, RFC 9000 Section 3.4. + "half_closed_local" / + "half_closed_remote" / + ; sending-side stream states, RFC 9000 Section 3.1. + "ready" / + "send" / + "data_sent" / + "reset_sent" / + "reset_received" / + ; receive-side stream states, RFC 9000 Section 3.2. + "receive" / + "size_known" / + "data_read" / + "reset_read" / + ; both-side states + "data_received" / + ; qlog-defined: memory actually freed + "destroyed" + + $StreamState /= BaseStreamStates / GranularStreamStates + + Figure 23: QUICStreamStateUpdated definition + + QUIC implementations SHOULD mainly log the simplified (HTTP/2-alike) + BaseStreamStates instead of the more fine-grained + GranularStreamStates. These latter ones are mainly for more in-depth + debugging. Tools SHOULD be able to deal with both types equally. + +5.14. frames_processed + + The frame_processed event is intended to prevent a large + proliferation of specific purpose events (e.g., packets_acknowledged, + flow_control_updated, stream_data_received). It has Extra importance + level; see Section 9.2 of [QLOG-MAIN]. + + Implementations have the opportunity to (selectively) log this type + of signal without having to log packet-level details (e.g., in + packet_received). Since for almost all cases, the effects of + applying a frame to the internal state of an implementation can be + inferred from that frame's contents, these events are aggregated into + this single frames_processed event. + + The frame_processed event can be used to signal internal state change + not resulting directly from the actual "parsing" of a frame (e.g., + the frame could have been parsed, data put into a buffer, then later + processed, then logged with this event). + + The packet_received event can convey all constituent frames. It is + not expected that the frames_processed event will also be used for a + redundant purpose. Rather, implementations can use this event to + avoid having to log full packets or to convey extra information about + when frames are processed (for example, if frame processing is + deferred for any reason). + + Note that for some events, this approach will lose some information + (e.g., for which encryption level are packets being acknowledged?). + If this information is important, the packet_received event can be + used instead. + + In some implementations, it can be difficult to log frames directly, + even when using packet_sent and packet_received events. For these + cases, the frames_processed event also contains the packet_numbers + field, which can be used to more explicitly link this event to the + packet_sent/received events. The field is an array, which supports + using a single frames_processed event for multiple frames received + over multiple packets. To map between frames and packets, the + position and order of entries in the frames and packet_numbers is + used. If the optional packet_numbers field is used, each frame MUST + have a corresponding packet number at the same index. + + QUICFramesProcessed = { + frames: [* $QuicFrame] + ? packet_numbers: [* uint64] + + * $$quic-framesprocessed-extension + } + + Figure 24: QUICFramesProcessed definition + + For example, an instance of the frames_processed event that + represents four STREAM frames received over two packets would have + the fields serialized as: + + "frames":[ + {"frame_type":"stream","stream_id":0,"offset":0,"length":500}, + {"frame_type":"stream","stream_id":0,"offset":500,"length":200}, + {"frame_type":"stream","stream_id":1,"offset":0,"length":300}, + {"frame_type":"stream","stream_id":1,"offset":300,"length":50} + ], + "packet_numbers":[ + 1, + 1, + 2, + 2 + ] + +5.15. stream_data_moved + + The stream_data_moved event is used to indicate when QUIC stream data + moves between the different layers. This helps make clear the flow + of data, how long data remains in various buffers, and the overheads + introduced by individual layers. The event has Base importance + level; see Section 9.2 of [QLOG-MAIN]. + + For example, it can be useful to understand when when data moves from + an application protocol (e.g., HTTP) to QUIC stream buffers and vice + versa. Similarly, when data moves from the application protocol + layer into a user-facing application such as a web browser. + + The stream_data_moved event can provide insight into whether received + data on a QUIC stream is moved to the application protocol + immediately (for example per received packet) or in larger batches + (for example, all QUIC packets are processed first and afterwards the + application layer reads from the streams with newly available data). + This can help identify bottlenecks, flow control issues, or + scheduling problems. + + This event is only for data in QUIC streams. For data in QUIC + Datagram Frames, see the datagram_data_moved event defined in + Section 5.16. + + QUICStreamDataMoved = { + ? stream_id: uint64 + ? offset: uint64 + + ; byte length of the moved data + ? length: uint64 + ? from: $DataLocation + ? to: $DataLocation + ? raw: RawInfo + + * $$quic-streamdatamoved-extension + } + + $DataLocation /= "user" / + "application" / + "transport" / + "network" + + Figure 25: QUICStreamDataMoved definition + +5.16. datagram_data_moved + + The datagram_data_moved event is used to indicate when QUIC Datagram + Frame data (see [RFC9221]) moves between the different layers. This + helps make clear the flow of data, how long data remains in various + buffers, and the overheads introduced by individual layers. The + event has Base importance level; see Section 9.2 of [QLOG-MAIN]. + + For example, passing from the application protocol (e.g., + WebTransport) to QUIC Datagram Frame buffers and vice versa. + Similarly, when data moves from the application protocol layer into a + user-facing application such as a web browser. + + The datagram_data_moved event can provide insight into whether + received data in a QUIC Datagram Frame is moved to the application + protocol immediately (for example per received packet) or in larger + batches (for example, all QUIC packets are processed first and + afterwards the application layer reads all Datagrams at once). This + can help identify bottlenecks, flow control issues, or scheduling + problems. + + This event is only for data in QUIC Datagram Frames. For data in + QUIC streams, see the stream_data_moved event defined in + Section 5.15. + + QUICDatagramDataMoved = { + ; byte length of the moved data + ? length: uint64 + ? from: $DataLocation + ? to: $DataLocation + ? raw: RawInfo + + * $$quic-datagramdatamoved-extension + } + + Figure 26: QUICDatagramDataMoved definition + +5.17. migration_state_updated + + Importance: Extra + + Use to provide additional information when attempting (client-side) + connection migration. While most details of the QUIC connection + migration process can be inferred by observing the PATH_CHALLENGE and + PATH_RESPONSE frames, in combination with the + ConnectivityPathAssigned event, it can be useful to explicitly log + the progression of the migration and potentially made decisions in a + single location/event. + + Generally speaking, connection migration goes through two phases: a + probing phase (which is not always needed/present), and a migration + phase (which can be abandoned upon error). + + Implementations that log per-path information in a + QUICMigrationStateUpdated, SHOULD also emit QUICPathAssigned events, + to serve as a ground-truth source of information. + + Definition: + + QUICMigrationStateUpdated = { + ? old: MigrationState + new: MigrationState + + ? path_id: PathID + + ; the information for traffic going towards the remote receiver + ? path_remote: PathEndpointInfo + + ; the information for traffic coming in at the local endpoint + ? path_local: PathEndpointInfo + + * $$quic-migrationstateupdated-extension + } + + ; Note that MigrationState does not describe a full state machine + ; These entries are not necessarily chronological, + ; nor will they always all appear during + ; a connection migration attempt. + MigrationState = + ; probing packets are sent, migration not initiated yet + "probing_started" / + ; did not get reply to probing packets, + ; discarding path as an option + "probing_abandoned" / + ; received reply to probing packets, path is migration candidate + "probing_successful" / + ; non-probing packets are sent, attempting migration + "migration_started" / + ; something went wrong during the migration, abandoning attempt + "migration_abandoned" / + ; new path is now fully used, old path is discarded + "migration_complete" + + Figure 27: QUICMigrationStateUpdated definition + +6. Security Events + +6.1. key_updated + + The key_updated event has Base importance level; see Section 9.2 of + [QLOG-MAIN]. + + SecurityKeyUpdated = { + key_type: $KeyType + ? old: hexstring + ? new: hexstring + + ; needed for 1RTT key updates + ? key_phase: uint64 + ? trigger: + ; (e.g., initial, handshake and 0-RTT keys + ; are generated by TLS) + "tls" / + "remote_update" / + "local_update" + + * $$quic-keyupdated-extension + } + + Figure 28: SecurityKeyUpdated definition + + Note that the key_phase is the full value of the key phase (as + indicated by @M and @N in Figure 9 of [QUIC-TLS]). The key phase bit + used on the packet header is the least significant bit of the key + phase. + +6.2. key_discarded + + The key_discarded event has Base importance level; see Section 9.2 of + [QLOG-MAIN]. + + SecurityKeyDiscarded = { + key_type: $KeyType + ? key: hexstring + + ; needed for 1RTT key updates + ? key_phase: uint64 + ? trigger: + ; (e.g., initial, handshake and 0-RTT keys + ; are generated by TLS) + "tls" / + "remote_update" / + "local_update" + + * $$quic-keydiscarded-extension + } + + Figure 29: SecurityKeyDiscarded definition + +7. Recovery events + + Most of the events in this category are kept generic to support + different recovery approaches and various congestion control + algorithms. Tool creators SHOULD make an effort to support and + visualize even unknown data in these events (e.g., plot unknown + congestion states by name on a timeline visualization). + +7.1. parameters_set + + The parameters_set event groups initial parameters from both loss + detection and congestion control into a single event. It has Base + importance level; see Section 9.2 of [QLOG-MAIN]. + + All these settings are typically set once and never change. + Implementation that do, for some reason, change these parameters + during execution, MAY emit the parameters_set event more than once. + + RecoveryParametersSet = { + + ; Loss detection, see RFC 9002 Appendix A.2 + ; in amount of packets + ? reordering_threshold: uint16 + + ; as RTT multiplier + ? time_threshold: float32 + + ; in ms + timer_granularity: uint16 + + ; in ms + ? initial_rtt:float32 + + ; congestion control, see RFC 9002 Appendix B.2 + ; in bytes. Note that this could be updated after pmtud + ? max_datagram_size: uint32 + + ; in bytes + ? initial_congestion_window: uint64 + + ; Note that this could change when max_datagram_size changes + ; in bytes + ? minimum_congestion_window: uint64 + ? loss_reduction_factor: float32 + + ; as PTO multiplier + ? persistent_congestion_threshold: uint16 + + * $$recovery-parametersset-extension + } + + Figure 30: RecoveryParametersSet definition + + Additionally, this event can contain any number of unspecified fields + to support different recovery approaches. + +7.2. metrics_updated + + The metrics_updated event is emitted when one or more of the + observable recovery metrics changes value. It has Core importance + level; see Section 9.2 of [QLOG-MAIN]. + + This event SHOULD group all possible metric updates that happen at or + around the same time in a single event (e.g., if min_rtt and + smoothed_rtt change at the same time, they should be bundled in a + single metrics_updated entry, rather than split out into two). + Consequently, a metrics_updated event is only guaranteed to contain + at least one of the listed metrics. + + RecoveryMetricsUpdated = { + + ; Loss detection, see RFC 9002 Appendix A.3 + ; all following rtt fields are expressed in ms + ? min_rtt: float32 + ? smoothed_rtt: float32 + ? latest_rtt: float32 + ? rtt_variance: float32 + ? pto_count: uint16 + + ; Congestion control, see RFC 9002 Appendix B.2. + ; in bytes + ? congestion_window: uint64 + ? bytes_in_flight: uint64 + + ; in bytes + ? ssthresh: uint64 + + ; qlog defined + ; sum of all packet number spaces + ? packets_in_flight: uint64 + + ; in bits per second + ? pacing_rate: uint64 + + * $$recovery-metricsupdated-extension + } + + Figure 31: RecoveryMetricsUpdated definition + + In order to make logging easier, implementations MAY log values even + if they are the same as previously reported values (e.g., two + subsequent RecoveryMetricsUpdated entries can both report the exact + same value for min_rtt). However, applications SHOULD try to log + only actual updates to values. + + Additionally, the metrics_updated event can contain any number of + unspecified fields to support different recovery approaches. + +7.3. congestion_state_updated + + The congestion_state_updated event indicates when the congestion + controller enters a significant new state and changes its behaviour. + It has Base importance level; see Section 9.2 of [QLOG-MAIN]. + + The values of the event's fields are intentionally unspecified here + in order to support different Congestion Control algorithms, as these + typically have different states and even different implementations of + these states across stacks. For example, for the algorithm defined + in the Recovery draft ("enhanced" New Reno), the following states are + used: Slow Start, Congestion Avoidance, Application Limited and + Recovery. Similarly, states can be triggered by a variety of events, + including detection of Persistent Congestion or receipt of ECN + markings. + + RecoveryCongestionStateUpdated = { + ? old: text + new: text + ? trigger: text + + * $$recovery-congestionstateupdated-extension + } + + Figure 32: RecoveryCongestionStateUpdated definition + + The trigger field SHOULD be logged if there are multiple ways in + which a state change can occur but MAY be omitted if a given state + can only be due to a single event occurring (for example Slow Start + is often exited only when ssthresh is exceeded). + +7.4. loss_timer_updated + + The loss_timer_updated event is emitted when a recovery loss timer + changes state. It has Extra importance level; see Section 9.2 of + [QLOG-MAIN]. + + The three main event types are: + + * set: the timer is set with a delta timeout for when it will + trigger next + + * expired: when the timer effectively expires after the delta + timeout + + * cancelled: when a timer is cancelled (e.g., all outstanding + packets are acknowledged, start idle period) + + In order to indicate an active timer's timeout update, a new set + event is used. + + RecoveryLossTimerUpdated = { + + ; called "mode" in RFC 9002 A.9. + ? timer_type: "ack" / + "pto" + ? packet_number_space: $PacketNumberSpace + event_type: "set" / + "expired" / + "cancelled" + + ; if event_type === "set": delta time is in ms from + ; this event's timestamp until when the timer will trigger + ? delta: float32 + + * $$recovery-losstimerupdated-extension + } + + Figure 33: RecoveryLossTimerUpdated definition + +7.5. packet_lost + + The packet_lost event is emitted when a packet is deemed lost by loss + detection. It has Core importance level; see Section 9.2 of + [QLOG-MAIN]. + + It is RECOMMENDED to populate the optional trigger field in order to + help disambiguate among the various possible causes of a loss + declaration. + + RecoveryPacketLost = { + + ; should include at least the packet_type and packet_number + ? header: PacketHeader + + ; not all implementations will keep track of full + ; packets, so these are optional + ? frames: [* $QuicFrame] + ? is_mtu_probe_packet: bool .default false + ? trigger: + "reordering_threshold" / + "time_threshold" / + ; RFC 9002 Section 6.2.4 paragraph 6, MAY + "pto_expired" + + * $$recovery-packetlost-extension + } + + Figure 34: RecoveryPacketLost definition + +7.6. marked_for_retransmit + + The marked_for_retransmit event indicates which data was marked for + retransmission upon detection of packet loss (see packet_lost). It + has Extra importance level; see Section 9.2 of [QLOG-MAIN]. + + Similar to the reasoning for the frames_processed event, in order to + keep the amount of different events low, this signal is grouped into + in a single event based on existing QUIC frame definitions for all + types of retransmittable data. + + Implementations retransmitting full packets or frames directly can + just log the constituent frames of the lost packet here (or do away + with this event and use the contents of the packet_lost event + instead). Conversely, implementations that have more complex logic + (e.g., marking ranges in a stream's data buffer as in-flight), or + that do not track sent frames in full (e.g., only stream offset + + length), can translate their internal behaviour into the appropriate + frame instance here even if that frame was never or will never be put + on the wire. + + Much of this data can be inferred if implementations log packet_sent + events (e.g., looking at overlapping stream data offsets and length, + one can determine when data was retransmitted). + + RecoveryMarkedForRetransmit = { + frames: [+ $QuicFrame] + + * $$recovery-markedforretransmit-extension + } + + Figure 35: RecoveryMarkedForRetransmit definition + +7.7. ecn_state_updated + + The ecn_state_updated event indicates a progression in the ECN state + machine as described in section A.4 of [QUIC-TRANSPORT]. It has + Extra importance level; see Section 9.2 of [QLOG-MAIN]. + + ECNStateUpdated = { + ? old: ECNState + new: ECNState + + * $$recovery-ecnstateupdated-extension + } + + ECNState = + ; ECN testing in progress + "testing" / + ; ECN state unknown, waiting for acknowledgements + ; for testing packets + "unknown" / + ; ECN testing failed + "failed" / + ; testing was successful, the endpoint now + ; sends packets with ECT(0) marking + "capable" + + Figure 36: ECNStateUpdated definition + +8. QUIC data field definitions + +8.1. QuicVersion + + QuicVersion = hexstring + + Figure 37: QuicVersion definition + +8.2. ConnectionID + + ConnectionID = hexstring + + Figure 38: ConnectionID definition + +8.3. Owner + + Owner = "local" / + "remote" + + Figure 39: Owner definition + +8.4. IPAddress and IPVersion + + ; an IPAddress can either be a "human readable" form + ; (e.g., "127.0.0.1" for v4 or + ; "2001:0db8:85a3:0000:0000:8a2e:0370:7334" for v6) or + ; use a raw byte-form (as the string forms can be ambiguous). + ; Additionally, a hash-based or redacted representation + ; can be used if needed for privacy or security reasons. + IPAddress = text / + hexstring + + Figure 40: IPAddress definition + + IPVersion = "v4" / + "v6" + + Figure 41: IPVersion definition + +8.5. PathEndpointInfo + + PathEndpointInfo indicates a single half/direction of a path. A full + path is comprised of two halves. Firstly: the server sends to the + remote client IP + port using a specific destination Connection ID. + Secondly: the client sends to the remote server IP + port using a + different destination Connection ID. + + As such, structures logging path information SHOULD include two + different PathEndpointInfo instances, one for each half of the path. + + PathEndpointInfo = { + ? ip_v4: IPAddress + ? ip_v6: IPAddress + ? port_v4: uint16 + ? port_v6: uint16 + + ; Even though usually only a single ConnectionID + ; is associated with a given path at a time, + ; there are situations where there can be an overlap + ; or a need to keep track of previous ConnectionIDs + ? connection_ids: [+ ConnectionID] + + * $$quic-pathendpointinfo-extension + } + + Figure 42: PathEndpointInfo definition + +8.6. PacketType + + $PacketType /= "initial" / + "handshake" / + "0RTT" / + "1RTT" / + "retry" / + "version_negotiation" / + "stateless_reset" / + "unknown" + + Figure 43: PacketType definition + +8.7. PacketNumberSpace + + $PacketNumberSpace /= "initial" / + "handshake" / + "application_data" + + Figure 44: PacketNumberSpace definition + +8.8. PacketHeader + + PacketHeader = { + ? quic_bit: bool .default true + packet_type: $PacketType + + ; only if packet_type === "initial" || "handshake" || "0RTT" || + ; "1RTT" + ? packet_number: uint64 + + ; the bit flags of the packet headers (spin bit, key update bit, + ; etc. up to and including the packet number length bits + ; if present + ? flags: uint8 + + ; only if packet_type === "initial" || "retry" + ? token: Token + + ; only if packet_type === "initial" || "handshake" || "0RTT" + ; Signifies length of the packet_number plus the payload + ? length: uint16 + + ; only if present in the header + ; if correctly using transport:connection_id_updated events, + ; dcid can be skipped for 1RTT packets + ? version: QuicVersion + ? scil: uint8 + ? dcil: uint8 + ? scid: ConnectionID + ? dcid: ConnectionID + + * $$quic-packetheader-extension + } + + Figure 45: PacketHeader definition + +8.9. Token + + Token = { + ? type: $TokenType + + ; decoded fields included in the token + ; (typically: peer's IP address, creation time) + ? details: { + * text => any + } + ? raw: RawInfo + + * $$quic-token-extension + } + + $TokenType /= "retry" / + "resumption" + + Figure 46: Token definition + + The token carried in an Initial packet can either be a retry token + from a Retry packet, or one originally provided by the server in a + NEW_TOKEN frame used when resuming a connection (e.g., for address + validation purposes). Retry and resumption tokens typically contain + encoded metadata to check the token's validity when it is used, but + this metadata and its format is implementation specific. For that, + Token includes a general-purpose details field. + +8.10. Stateless Reset Token + + StatelessResetToken = hexstring .size 16 + + Figure 47: Stateless Reset Token definition + + The stateless reset token is carried in stateless reset packets, in + transport parameters and in NEW_CONNECTION_ID frames. + +8.11. KeyType + + $KeyType /= "server_initial_secret" / + "client_initial_secret" / + "server_handshake_secret" / + "client_handshake_secret" / + "server_0rtt_secret" / + "client_0rtt_secret" / + "server_1rtt_secret" / + "client_1rtt_secret" + + Figure 48: KeyType definition + +8.12. ECN + + ECN = "Not-ECT" / "ECT(1)" / "ECT(0)" / "CE" + + Figure 49: ECN definition + + The ECN bits carried in the IP header. + +8.13. QUIC Frames + + The generic $QuicFrame is defined here as a CDDL "type socket" + extension point. It can be extended to support additional QUIC frame + types. + + ; The QuicFrame is any key-value map (e.g., JSON object) + $QuicFrame /= { + * text => any + } + + Figure 50: QuicFrame type socket definition + + The QUIC frame types defined in this document are as follows: + + QuicBaseFrames = PaddingFrame / + PingFrame / + AckFrame / + ResetStreamFrame / + StopSendingFrame / + CryptoFrame / + NewTokenFrame / + StreamFrame / + MaxDataFrame / + MaxStreamDataFrame / + MaxStreamsFrame / + DataBlockedFrame / + StreamDataBlockedFrame / + StreamsBlockedFrame / + NewConnectionIDFrame / + RetireConnectionIDFrame / + PathChallengeFrame / + PathResponseFrame / + ConnectionCloseFrame / + HandshakeDoneFrame / + UnknownFrame / + DatagramFrame + + $QuicFrame /= QuicBaseFrames + + Figure 51: QuicBaseFrames definition + +8.13.1. PaddingFrame + + In QUIC, PADDING frames are simply identified as a single byte of + value 0. As such, each padding byte could be theoretically + interpreted and logged as an individual PaddingFrame. + + However, as this leads to heavy logging overhead, implementations + SHOULD instead emit just a single PaddingFrame and set the + payload_length property to the amount of PADDING bytes/frames + included in the packet. + + PaddingFrame = { + frame_type: "padding" + + ; total frame length, including frame header + ? length: uint32 + payload_length: uint32 + } + + Figure 52: PaddingFrame definition + +8.13.2. PingFrame + + PingFrame = { + frame_type: "ping" + + ; total frame length, including frame header + ? length: uint32 + ? payload_length: uint32 + } + + Figure 53: PingFrame definition + +8.13.3. AckFrame + + ; either a single number (e.g., [1]) or two numbers (e.g., [1,2]). + ; For two numbers: + ; the first number is "from": lowest packet number in interval + ; the second number is "to": up to and including the highest + ; packet number in the interval + AckRange = [1*2 uint64] + + AckFrame = { + frame_type: "ack" + + ; in ms + ? ack_delay: float32 + + ; e.g., looks like [[1,2],[4,5], [7], [10,22]] serialized + ? acked_ranges: [+ AckRange] + + ; ECN (explicit congestion notification) related fields + ; (not always present) + ? ect1: uint64 + ? ect0: uint64 + ? ce: uint64 + + ; total frame length, including frame header + ? length: uint32 + ? payload_length: uint32 + } + + Figure 54: AckFrame definition + + Note that the packet ranges in AckFrame.acked_ranges do not + necessarily have to be ordered (e.g., [[5,9],[1,4]] is a valid + value). + + Note that the two numbers in the packet range can be the same (e.g., + [120,120] means that packet with number 120 was ACKed). However, in + that case, implementers SHOULD log [120] instead and tools MUST be + able to deal with both notations. + +8.13.4. ResetStreamFrame + + ResetStreamFrame = { + frame_type: "reset_stream" + stream_id: uint64 + error_code: $ApplicationError / + uint64 + + ; in bytes + final_size: uint64 + + ; total frame length, including frame header + ? length: uint32 + ? payload_length: uint32 + } + + Figure 55: ResetStreamFrame definition + +8.13.5. StopSendingFrame + + StopSendingFrame = { + frame_type: "stop_sending" + stream_id: uint64 + error_code: $ApplicationError / + uint64 + + ; total frame length, including frame header + ? length: uint32 + ? payload_length: uint32 + } + + Figure 56: StopSendingFrame definition + +8.13.6. CryptoFrame + + CryptoFrame = { + frame_type: "crypto" + offset: uint64 + length: uint64 + ? payload_length: uint32 + ? raw: RawInfo + } + + Figure 57: CryptoFrame definition + +8.13.7. NewTokenFrame + + NewTokenFrame = { + frame_type: "new_token" + token: Token + } + + Figure 58: NewTokenFrame definition + +8.13.8. StreamFrame + + StreamFrame = { + frame_type: "stream" + stream_id: uint64 + + ; These two MUST always be set + ; If not present in the Frame type, log their default values + offset: uint64 + length: uint64 + + ; this MAY be set any time, + ; but MUST only be set if the value is true + ; if absent, the value MUST be assumed to be false + ? fin: bool .default false + ? raw: RawInfo + } + + Figure 59: StreamFrame definition + +8.13.9. MaxDataFrame + + MaxDataFrame = { + frame_type: "max_data" + maximum: uint64 + } + + Figure 60: MaxDataFrame definition + +8.13.10. MaxStreamDataFrame + + MaxStreamDataFrame = { + frame_type: "max_stream_data" + stream_id: uint64 + maximum: uint64 + } + + Figure 61: MaxStreamDataFrame definition + +8.13.11. MaxStreamsFrame + + MaxStreamsFrame = { + frame_type: "max_streams" + stream_type: StreamType + maximum: uint64 + } + + Figure 62: MaxStreamsFrame definition + +8.13.12. DataBlockedFrame + + DataBlockedFrame = { + frame_type: "data_blocked" + limit: uint64 + } + + Figure 63: DataBlockedFrame definition + +8.13.13. StreamDataBlockedFrame + + StreamDataBlockedFrame = { + frame_type: "stream_data_blocked" + stream_id: uint64 + limit: uint64 + } + + Figure 64: StreamDataBlockedFrame definition + +8.13.14. StreamsBlockedFrame + + StreamsBlockedFrame = { + frame_type: "streams_blocked" + stream_type: StreamType + limit: uint64 + } + + Figure 65: StreamsBlockedFrame definition + +8.13.15. NewConnectionIDFrame + + NewConnectionIDFrame = { + frame_type: "new_connection_id" + sequence_number: uint32 + retire_prior_to: uint32 + + ; mainly used if e.g., for privacy reasons the full + ; connection_id cannot be logged + ? connection_id_length: uint8 + connection_id: ConnectionID + ? stateless_reset_token: StatelessResetToken + } + + Figure 66: NewConnectionIDFrame definition + +8.13.16. RetireConnectionIDFrame + + RetireConnectionIDFrame = { + frame_type: "retire_connection_id" + sequence_number: uint32 + } + + Figure 67: RetireConnectionIDFrame definition + +8.13.17. PathChallengeFrame + + PathChallengeFrame = { + frame_type: "path_challenge" + + ; always 64-bit + ? data: hexstring + } + + Figure 68: PathChallengeFrame definition + +8.13.18. PathResponseFrame + + PathResponseFrame = { + frame_type: "path_response" + + ; always 64-bit + ? data: hexstring + } + + Figure 69: PathResponseFrame definition + +8.13.19. ConnectionCloseFrame + + An endpoint that receives unknown error codes can record it in the + error_code field using the numerical value without variable-length + integer encoding. + + When the connection is closed due a connection-level error, the + trigger_frame_type field can be used to log the frame that triggered + the error. For known frame types, the appropriate string value is + used. For unknown frame types, the numerical value without variable- + length integer encoding is used. + + The CONNECTION_CLOSE reason phrase is a byte sequences. It is likely + that this sequence is presentable as UTF-8, in which case it can be + logged in the reason field. The reason_bytes field supports logging + the raw bytes, which can be useful when the value is not UTF-8 or + when an endpoint does not want to decode it. Implementations SHOULD + log at least one format, but MAY log both or none. + + ErrorSpace = "transport" / + "application" + + ConnectionCloseFrame = { + frame_type: "connection_close" + ? error_space: ErrorSpace + ? error_code: $TransportError / + CryptoError / + $ApplicationError / + uint64 + ? reason: text + ? reason_bytes: hexstring + + ; when error_space === "transport" + ? trigger_frame_type: uint64 / + text + } + + Figure 70: ConnectionCloseFrame definition + +8.13.20. HandshakeDoneFrame + + HandshakeDoneFrame = { + frame_type: "handshake_done"; + } + + Figure 71: HandshakeDoneFrame definition + +8.13.21. UnknownFrame + + The frame_type_bytes field is the numerical value without variable- + length integer encoding. + + UnknownFrame = { + frame_type: "unknown" + frame_type_bytes: uint64 + ? raw: RawInfo + } + + Figure 72: UnknownFrame definition + +8.13.22. DatagramFrame + + The QUIC DATAGRAM frame is defined in Section 4 of [RFC9221]. + + DatagramFrame = { + frame_type: "datagram" + ? length: uint64 + ? raw: RawInfo + } + + Figure 73: DatagramFrame definition + +8.13.23. TransportError + + The generic $TransportError is defined here as a CDDL "type socket" + extension point. It can be extended to support additional Transport + errors. + + $TransportError /= "no_error" / + "internal_error" / + "connection_refused" / + "flow_control_error" / + "stream_limit_error" / + "stream_state_error" / + "final_size_error" / + "frame_encoding_error" / + "transport_parameter_error" / + "connection_id_limit_error" / + "protocol_violation" / + "invalid_token" / + "application_error" / + "crypto_buffer_exceeded" / + "key_update_error" / + "aead_limit_reached" / + "no_viable_path" + ; there is no value to reflect CRYPTO_ERROR + ; use the CryptoError type instead + + Figure 74: TransportError definition + +8.13.24. ApplicationError + + By definition, an application error is defined by the application- + level protocol running on top of QUIC (e.g., HTTP/3). + + As such, it cannot be defined here directly. It is instead defined + as an empty CDDL "type socket" extension point. + + Application-level qlog definitions that wish to define new + ApplicationError strings MUST do so by extending the + $ApplicationError socket as such: + + $ApplicationError /= "new_error_name" / + "another_new_error_name" + +8.13.25. CryptoError + + These errors are defined in the TLS document as "A TLS alert is + turned into a QUIC connection error by converting the one-byte alert + description into a QUIC error code. The alert description is added + to 0x100 to produce a QUIC error code from the range reserved for + CRYPTO_ERROR." + + This approach maps badly to a pre-defined enum. As such, the + crypto_error string is defined as having a dynamic component here, + which should include the hex-encoded and zero-padded value of the TLS + alert description. + + ; all strings from "crypto_error_0x100" to "crypto_error_0x1ff" + CryptoError = text .regexp "crypto_error_0x1[0-9a-f][0-9a-f]" + + Figure 75: CryptoError definition + +9. Security and Privacy Considerations + + The security and privacy considerations discussed in [QLOG-MAIN] + apply to this document as well. + +10. IANA Considerations + + There are no IANA considerations. + +11. References + +11.1. Normative References + + [CDDL] Birkholz, H., Vigano, C., and C. Bormann, "Concise Data + Definition Language (CDDL): A Notational Convention to + Express Concise Binary Object Representation (CBOR) and + JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, + June 2019, . + + [GREASEBIT] + Thomson, M., "Greasing the QUIC Bit", RFC 9287, + DOI 10.17487/RFC9287, August 2022, + . + + [QLOG-MAIN] + Marx, R., Niccolini, L., Seemann, M., and L. Pardue, "Main + logging schema for qlog", Work in Progress, Internet- + Draft, draft-ietf-quic-qlog-main-schema-08, 4 March 2024, + . + + [QUIC-DATAGRAM] + Pauly, T., Kinnear, E., and D. Schinazi, "An Unreliable + Datagram Extension to QUIC", RFC 9221, + DOI 10.17487/RFC9221, March 2022, + . + + [QUIC-RECOVERY] + Iyengar, J., Ed. and I. Swett, Ed., "QUIC Loss Detection + and Congestion Control", RFC 9002, DOI 10.17487/RFC9002, + May 2021, . + + [QUIC-TLS] Thomson, M., Ed. and S. Turner, Ed., "Using TLS to Secure + QUIC", RFC 9001, DOI 10.17487/RFC9001, May 2021, + . + + [QUIC-TRANSPORT] + Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based + Multiplexed and Secure Transport", RFC 9000, + DOI 10.17487/RFC9000, May 2021, + . + + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, + DOI 10.17487/RFC2119, March 1997, + . + + [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC + 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, + May 2017, . + + [RFC9221] Pauly, T., Kinnear, E., and D. Schinazi, "An Unreliable + Datagram Extension to QUIC", RFC 9221, + DOI 10.17487/RFC9221, March 2022, + . + +11.2. Informative References + + [RFC7301] Friedl, S., Popov, A., Langley, A., and E. Stephan, + "Transport Layer Security (TLS) Application-Layer Protocol + Negotiation Extension", RFC 7301, DOI 10.17487/RFC7301, + July 2014, . + +Acknowledgements + + Much of the initial work by Robin Marx was done at the Hasselt and KU + Leuven Universities. + + Thanks to Jana Iyengar, Brian Trammell, Dmitri Tikhonov, Stephen + Petrides, Jari Arkko, Marcus Ihlar, Victor Vasiliev, Mirja Kühlewind, + Jeremy Lainé, Kazu Yamamoto, Christian Huitema and Hugo Landau for + their feedback and suggestions. + +Change Log + + This section is to be removed before publishing as an RFC. + +Since draft-ietf-qlog-quic-events-06: + + * Added PathAssigned and MigrationStateUpdated events (#336) + + * Added extension points to parameters_set and parameters_restored + (#400) + + * Removed error_code_value from connection_closed (#386, #392) + + * Renamed generation to key_phase for key_updated and key_discarded + (#390) + + * Removed retry_token from packet_sent and packet_received (#389) + + * Updated ALPN handling (#385) + + * Added key_unavailable trigger to packet_dropped (#381) + + * Updated several uint32 to uint64 + + * ProtocolEventBody is now called ProtocolEventData (#352) + + * Editorial changes (#402, #404, #394, #393) + +Since draft-ietf-qlog-quic-events-05: + + * SecurityKeyUpdated: the new key is no longer mandatory to log + (#294) + + * Added ECN related events and metadata (#263) + +Since draft-ietf-qlog-quic-events-04: + + * Updated guidance on logging events across connections (#279) + + * Renamed 'transport' category to 'quic' (#302) + + * Added support for multiple packet numbers in + 'quic:frames_processed' (#307) + + * Added definitions for RFC9287 (QUIC GREASE Bit extension) (#311) + + * Added definitions for RFC9221 (QUIC Datagram Frame extension) + (#310) + + * (Temporarily) removed definitions for connection migration events + (#317) + + * Editorial and formatting changes (#298, #299, #304, #306, #327) + +Since draft-ietf-qlog-quic-events-03: + + * Ensured consistent use of RawInfo to indicate raw wire bytes + (#243) + + * Renamed UnknownFrame:raw_frame_type to :frame_type_value (#54) + + * Renamed ConnectionCloseFrame:raw_error_code to :error_code_value + (#54) + + * Changed triggers for packet_dropped (#278) + + * Added entries to TransportError enum (#285) + + * Changed minimum_congestion_window to uint64 (#288) + +Since draft-ietf-qlog-quic-events-02: + + * Renamed key_retired to key_discarded (#185) + + * Added fields and events for DPLPMTUD (#135) + + * Made packet_number optional in PacketHeader (#244) + + * Removed connection_retried event placeholder (#255) + + * Changed QuicFrame to a CDDL plug type (#257) + + * Moved data definitions out of the appendix into separate sections + + * Added overview Table of Contents + +Since draft-ietf-qlog-quic-events-01: + + * Added Stateless Reset Token type (#122) + +Since draft-ietf-qlog-quic-events-00: + + * Change the data definition language from TypeScript to CDDL (#143) + +Since draft-marx-qlog-event-definitions-quic-h3-02: + + * These changes were done in preparation of the adoption of the + drafts by the QUIC working group (#137) + + * Split QUIC and HTTP/3 events into two separate documents + + * Moved RawInfo, Importance, Generic events and Simulation events to + the main schema document. + + * Changed to/from value options of the data_moved event + +Since draft-marx-qlog-event-definitions-quic-h3-01: + + Major changes: + + * Moved data_moved from http to transport. Also made the "from" and + "to" fields flexible strings instead of an enum (#111,#65) + + * Moved packet_type fields to PacketHeader. Moved packet_size field + out of PacketHeader to RawInfo:length (#40) + + * Made events that need to log packet_type and packet_number use a + header field instead of logging these fields individually + + * Added support for logging retry, stateless reset and initial + tokens (#94,#86,#117) + + * Moved separate general event categories into a single category + "generic" (#47) + + * Added "transport:connection_closed" event (#43,#85,#78,#49) + + * Added version_information and alpn_information events + (#85,#75,#28) + + * Added parameters_restored events to help clarify 0-RTT behaviour + (#88) + + Smaller changes: + + * Merged loss_timer events into one loss_timer_updated event + + * Field data types are now strongly defined (#10,#39,#36,#115) + + * Renamed qpack instruction_received and instruction_sent to + instruction_created and instruction_parsed (#114) + + * Updated qpack:dynamic_table_updated.update_type. It now has the + value "inserted" instead of "added" (#113) + + * Updated qpack:dynamic_table_updated. It now has an "owner" field + to differentiate encoder vs decoder state (#112) + + * Removed push_allowed from http:parameters_set (#110) + + * Removed explicit trigger field indications from events, since this + was moved to be a generic property of the "data" field (#80) + + * Updated transport:connection_id_updated to be more in line with + other similar events. Also dropped importance from Core to Base + (#45) + + * Added length property to PaddingFrame (#34) + + * Added packet_number field to transport:frames_processed (#74) + + * Added a way to generically log packet header flags (first 8 bits) + to PacketHeader + + * Added additional guidance on which events to log in which + situations (#53) + + * Added "simulation:scenario" event to help indicate simulation + details + + * Added "packets_acked" event (#107) + + * Added "datagram_ids" to the datagram_X and packet_X events to + allow tracking of coalesced QUIC packets (#91) + + * Extended connection_state_updated with more fine-grained states + (#49) + +Since draft-marx-qlog-event-definitions-quic-h3-00: + + * Event and category names are now all lowercase + + * Added many new events and their definitions + + * "type" fields have been made more specific (especially important + for PacketType fields, which are now called packet_type instead of + type) + + * Events are given an importance indicator (issue #22) + + * Event names are more consistent and use past tense (issue #21) + + * Triggers have been redefined as properties of the "data" field and + updated for most events (issue #23) + +Authors' Addresses + + Robin Marx (editor) + Akamai + Email: rmarx@akamai.com + + + Luca Niccolini (editor) + Meta + Email: lniccolini@meta.com + + + Marten Seemann (editor) + Email: martenseemann@gmail.com + + + Lucas Pardue (editor) + Cloudflare + Email: lucas@lucaspardue.com diff --git a/security-stuff/index.html b/security-stuff/index.html new file mode 100644 index 00000000..c77201f3 --- /dev/null +++ b/security-stuff/index.html @@ -0,0 +1,55 @@ + + + + quicwg/qlog security-stuff preview + + + + +

Editor's drafts for security-stuff branch of quicwg/qlog

+ + + + + + + + + + + + + + + + +
HTTP/3 qlog event definitionsplain textsame as main
Main logging schema for qlogplain textsame as main
QUIC event definitions for qlogplain textsame as main
+ + +