Re: The future of qlog

Hello everyone,

As you may have noticed, the "qlog roadshow" suggested by Martin Duke a few
months ago actually took place last week,
with me boring people with 6 qlog presentations in various wg's and rg's
(dispatch, opsawg, saag, tsvwg, iccrg, and maprg).
In case you still missed it, the two main variations of slides are in the
datatracker [1][2].

The goal was mainly to introduce the work to the wider IETF, to gather
feedback and to gauge outside interest in expanding/generalizing qlog to
other protocols/use cases.

While I feel the presentations were generally well-received, and that
people understood the value proposition,
there did not seem to be a huge amount of interest from new people to
contribute to the effort directly,
nor did I for example hear strong feedback that qlog should be a general,
cross-protocol effort from the start (e.g., in its own wg).

As such, it seems the current proposal of adopting both the qlog documents
(the main schema and the QUIC/H3 event definitions draft)
in the QUIC wg, and to keep focusing on applying qlog to QUIC/H3 first, is
indeed the best way forward in the short term.
It was suggested however, to also bring periodic updates to other
groups/areas (e.g., via the mailing list),
as not all people interested would be willing to participate in the QUIC wg
just for the general aspects of qlog.


We also received some valuable feedback/discussion on points that I feel
are important for qlog in general and that we could/should take into
account moving forward.
I have summarized the (in my opinion) most important ones below:

a) The point was raised that "the IETF does not standardize APIs" and that
qlog might reflect too much of concrete APIs to make sense standardizing
it.
While I and several other commenters disagreed with this, I do feel it
shows that it should be stated clearly early on what exactly we're defining
with qlog
(the events, the mapping to various serialization formats, method(s) of
transferring qlog, methods of accessing/requesting qlog files,
recommendations for logging/retention of privacy-sensitive data, best
practices for logging, expected tooling behaviour,...).
A related point made was that the result should still allow for non-defined
information to be reflected in qlog (e.g., custom events)

b) As expected, much of the feedback was on the importance of the privacy
aspects of qlog and the fact that this is maybe somewhat under-defined for
logging/tracing in general (e.g., in PCAP files).
One person remarked that defining logging events would also trigger/force
people to think (more) about the privacy impact of a given approach and
about (logging) data retention policies.
Another good remark was around perception: it must be clear that while qlog
helps analyze behaviour of encrypted protocols, it does not intend to
subvert the core privacy/security properties of these protocols.

c) There was discussion about two different types of format: 1) the
serialization format of the qlog events, and 2) the format used to describe
events in the documents.
1) For the first, we currently use JSON and NDJSON, where it was remarked
that NDJSON is not an IETF standard.
The most popular suggestions for serialization formats were JSON, JSON-C
and CBOR. Interoperability with PCAP should also be considered.
Several people also mentioned the requirement to encrypt the logs
themselves, for which the COSE [3] project could be used.
It was also again remarked that there is a dichotomy between "ease of use"
and "efficiency for logging at high speeds":
qlog can potentially focus on the first, as applications can log
efficiently in a (custom) binary format that is later transformed to qlog
if needed.
2) For the second, we currently use an ad-hoc dialect of TypeScript, which
I would agree is sub-optimal.
The most suggested alternative was YANG, though I am aware that has as many
staunch opponents as it has supporters.
If JSON/CBOR is chosen as main target format, CDDL potentially seems like a
good candidate.

d) The most concrete (immediate) interest for extending qlog to other
protocols seemed to be for TCP (with DTLS also being mentioned several
times).
This led to a new effort looking into supporting qlog output for Netflix's
FreeBSD TCP "blackbox" tools [4][5], which already exfiltrate e.g.,
congestion control info from the TCP stack.
This is in parallel to another ongoing effort to extract TCP info from the
Linux kernel using eBPF [6].

e) Experience with using qlog for logging (aggregated) measurements (at
middleboxes (e.g., in the Spindump project [7])),
suggests that we might need to take a more high-level view at what exactly
constitutes a qlog "event" semantically [8].


At this point, we were not planning to change the current drafts [9] to
e.g., CBOR or CDDL prior to the adoption call in the QUIC wg, nor to
include things like privacy guidance.
The main change that is planned is splitting up the QUIC and HTTP/3 events
into their own separate documents (as they are now combined in a single
draft).
If people feel that other changes are necessary prior to considering
adoption, please let us know.

With best regards,
Robin


[1]:
https://datatracker.ietf.org/meeting/110/materials/slides-110-tsvwg-sessa-61-qlog-for-ietf-transports-00
[2]:
https://datatracker.ietf.org/meeting/110/materials/slides-110-iccrg-qlog-structured-endpoint-logging-for-encrypted-protocols-00
[3]: https://tools.ietf.org/html/rfc8152
[4]: https://github.com/Netflix/bbparse
[5]: https://github.com/uoaerg/tcpqlog
[6]: https://github.com/nrybowski/quicSim-docker
[7]: https://github.com/EricssonResearch/spindump
[8]: https://github.com/quiclog/internet-drafts/issues/139
[9]: https://github.com/quiclog/internet-drafts


On Thu, 3 Dec 2020 at 06:14, Jana Iyengar <jri.ietf@gmail.com> wrote:

>
> Thanks again, Robin, for doing this work.
>
> I strongly vote for option A -- keep it in QUIC.
>
> I think the most value for this would be in QUIC, primarily because that's
> where people are actively adopting and using it. I would be reluctant to
> make this broader yet, if only because our problem at the IETF usually
> isn't that we have too few people with opinions on data formats (and the
> color of each field).
>
> Practically, we already have multiple QUIC/H3 implementations emitting
> qlog - and some of them already have feedback on how to extend it to be
> more useful - so I would suggest that getting meaningful implementation
> experience is going to be easiest with QUIC/H3 right now. Once it's done
> for QUIC/H3, I would then do the generalization after as a next step, with
> other use cases if and where there's interest. RFC numbers are cheap, and
> you have versioning in this thing. The split you have is fine, and it
> allows you to generalize this later. But keeping this discussion local to
> the QUIC WG will be super useful as a first step.
>
> - jana
>
> On Wed, Nov 25, 2020 at 4:00 PM Martin Duke <martin.h.duke@gmail.com>
> wrote:
>
>> Splitting HTTP/3 and QUIC into separate documents, so they can evolve
>> separately makes, sense, though I would be comfortable with them both
>> remaining in QUICWG for now.
>>
>> As an AD, I will facilitate a discussion with other areas about the main
>> schema. Robin, it might be useful for you to go on a roadshow at IETF 110
>> to pitch it to other areas to see if there's wider IETF energy to pursue
>> qlog extensions. If so, the case for putting the main schema outside quicwg
>> is pretty strong.
>>
>> On Mon, Nov 23, 2020 at 12:14 PM Lucas Pardue <lucaspardue.24.7@gmail.com>
>> wrote:
>>
>>> Thanks Robin!
>>>
>>> I have some responses in line.
>>>
>>> On Mon, Nov 23, 2020 at 7:05 PM Robin MARX <robin.marx@uhasselt.be>
>>> wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> I was happy to see a large amount of support from you all for qlog at
>>>> the QUIC wg meeting last week [0]!
>>>> While it seemed there was consensus that it would benefit from a more
>>>> formal adoption, it wasn't entirely clear if this should be in the QUIC wg.
>>>>
>>>> (I am also CC'ing the HTTP wg via Lucas Pardue, who thought this would
>>>> somewhat concern them as well)
>>>>
>>>> To recap, qlog consists of two parts:
>>>> 1) the main schema, which is protocol agnostic and mostly defines the
>>>> serialization and container format [1]
>>>> 2) the QUIC and HTTP/3 specific events [2]
>>>>
>>>
>>> Yes sorry HTTP folks blame me for your additional email traffic. The
>>> reason I thought the discussion on qlog was pertinent to the HTTP WG is
>>> because one of the major places I've found benefit is in analyzing stream
>>> multiplexing using qvis. This has particularly been useful for developing
>>> an implementation of Extensible Priorities in an HTTP/3 stack. The most
>>> mature way to get qlog for HTTP/2 stacks would be to define qlog schemas
>>> for TCP, TLS and HTTP/2.
>>>
>>> There are potentially other application mappings. As was discussed at
>>> the IETF 109 session, I think congestion control is going to be something
>>> that benefits from logging and tooling. Is there an interest in making qlog
>>> work the same for a CCA across TCP and QUIC? I suspect that might drive
>>> some opinion of what to do here.
>>>
>>>
>>>> I can see roughly 3 main options and would like additional feedback on
>>>> which people want:
>>>> A) Adopt both in QUIC wg for now, bring them to ~full maturity for QUIC
>>>> and H3, and then see if/how to generalize later (it seemed to me most were
>>>> leaning to this side last week)
>>>> B) Adopt document [2] in QUIC wg and find another place for [1], either
>>>> in an existing wg or a new one via BOF by IETF 110 (what happens if that
>>>> fails though?)
>>>> C) Find an existing wg or do a BOF for both [1] and [2] together by
>>>> IETF 110 (not sure this has enough traction outside of QUIC wg?)
>>>>
>>>>
>>> Option A seems the most pragmatic given where we are. But the more I
>>> think about things, the more concerned I get that a focus on QUIC stifles
>>> the innovation of other qlog use cases simply because they fall out of the
>>> scope of permitted discussion. So far, the main schema discussion has
>>> rightly focused on more operational concerns like serialization format, we
>>> should also let people with interest and expertise in this area flourish if
>>> possible. Finally, I also have a concern that the combined QUIC & HTTP/3
>>> schema becomes difficult to manage once we hand the reigns of HTTP/3 back.
>>>
>>> So I don't have a personal strong decision yet. But I'd like to consider
>>> an Option D that splits the QUIC & HTTP/3 schema, with each WG getting
>>> first refusal for adoption. Meanwhile, we continue to discuss the main
>>> schema.
>>>
>>> Cheers
>>> Lucas
>>>
>>>

-- 

dr. Robin Marx
Postdoc researcher - Web protocols
Expertise centre for Digital Media

T +32(0)11 26 84 79 - GSM +32(0)497 72 86 94

www.uhasselt.be
Universiteit Hasselt - Campus Diepenbeek
Agoralaan Gebouw D - B-3590 Diepenbeek
Kantoor EDM-2.05