Re: New Version Notification for draft-kazuho-httpbis-selftrace-00.txt

Robin,

On the privacy question -- indeed the user doesn't know what info is being
shared but note that this is a server-side log that is being streamed to
the client. This is no different than the server logging it directly. The
primary value here is for a user to log a "problematic" connection, where
otherwise servers in general do log random connections for perf analysis.

This solves a real problem right now for us which is to be able to grab
server-side traces for specific repeatable performance problems on certain
connections. A client can tell us there's a perf issue with a particular
domain, but without much else, we have to try and repro it, and without the
client's setup, we are often unable to easily do this.

With this mechanism, we can ask the reporter to send us a self-trace for
the repeatable issue. This is quite similar to Chrome netlogs (which are
requested from users who report issues,
https://support.google.com/chrome/a/answer/6271171), except that it is
server side logs.

So, we don't stream qlog; the current format is based on what was easiest
for us to do. Importantly, this doc isn't about the format, it's about a
mechanism to spit out logs that are local to a server. We might even want
to send some server infra specific info that we can send as an encrypted
blob in this stream.

I don't think this should be in the qlog doc. Of course there is value in
emitting qlog -- one can build and use standard tools and such around this
if it were to be emitted in qlog format. But I don't see this as a reason
to limit this mechanism to emit only qlog.

- jana

On Fri, Aug 13, 2021 at 9:50 AM Matt Joras <matt.joras@gmail.com> wrote:

> Happy to see this work! Speaking as an individual:
>
> One limitation of this sort of scheme (HTTP triggering and utilizing the
> same connection) is that it won't be able to capture two classes of issues
> that can be very problematic: handshake setup issues and mid-connection
> "stalling" issues. The former I think it's pretty obvious why, as this
> trace is triggered by an HTTP signal. As an example of the latter, consider
> a bug where the server runs down its flow control or congestion control to
> zero and can't recover. This ends up with the server being perpetually
> blocked from sending anything, and both endpoints idle timing out. If a
> client wants to trigger a trace and reproduce the issue then it will end up
> not getting a trace at all, since the server is stuck. That's not to say
> this kind of tracing isn't useful for other classes of issues, but these
> are two serious ones we've had to diagnose with "out of band"
> non-HTTP-triggered qlogs, and I wonder if we should also explore a
> standardized mechanism for triggering those kind of traces.
>
> Matt
>
> On Fri, Aug 13, 2021 at 12:15 AM Kazuho Oku <kazuhooku@gmail.com> wrote:
>
>> Hello folks,
>>
>> Today Jana and I have submitted a tiny I-D called
>> draft-kazuho-httpbis-selftrace.
>>
>> The draft specifies a well-known URI to be used for providing a trace of
>> a particular HTTP/3 connection (e.g., qlog) on that same HTTP/3 connection.
>>
>> One of the biggest hurdles in analyzing HTTP/3 performance issues is
>> obtaining traces that show the symptoms. That is because clients being
>> affected by issues have to coordinate with the server operators to collect
>> the traces.
>>
>> This PR solves the problem by defining a well-known URI for serving a
>> trace to the client on the HTTP connection that the client is using. When a
>> user sees an issue, they can collect the traces themselves and provide it
>> to the server operator.
>>
>> We have already implemented the feature in h2o, and doing so was easy,
>> assuming that the underlying QUIC stack already defines callbacks for
>> collecting trace events, see lib/handler/self_trace.c of
>> https://github.com/h2o/h2o/pull/2765.
>>
>> We also have a public endpoint; to try it out, first open
>> https://ora1.kazuhooku.com/test/self-trace/video-only.html (which starts
>> streaming a video), then open
>> https://ora1.kazuhooku.com/.well-known/self-trace. While the video is
>> being served, you would see the trace flowing through the well-known URI.
>>
>> At the moment, we are using a custom JSON format for the trace, but when
>> gzip compression is applied on-the-fly, the overhead of sending a trace
>> alongside ordinary HTTP responses is less than 10%. Therefore, we tend to
>> believe that this approach would work well in practice.
>>
>> Please let us know what you think - your feedback is very welcome.
>>
>> ---------- Forwarded message ---------
>> From: <internet-drafts@ietf.org>
>> Date: 2021年8月13日(金) 14:53
>> Subject: New Version Notification for
>> draft-kazuho-httpbis-selftrace-00.txt
>> To: Jana Iyengar <jri.ietf@gmail.com>, Kazuho Oku <kazuhooku@gmail.com>
>>
>>
>>
>> A new version of I-D, draft-kazuho-httpbis-selftrace-00.txt
>> has been successfully submitted by Kazuho Oku and posted to the
>> IETF repository.
>>
>> Name:           draft-kazuho-httpbis-selftrace
>> Revision:       00
>> Title:          Self-Tracing for HTTP
>> Document date:  2021-08-13
>> Group:          Individual Submission
>> Pages:          5
>> URL:
>> https://www.ietf.org/archive/id/draft-kazuho-httpbis-selftrace-00.txt
>> Status:
>> https://datatracker.ietf.org/doc/draft-kazuho-httpbis-selftrace/
>> Htmlized:
>> https://datatracker.ietf.org/doc/html/draft-kazuho-httpbis-selftrace
>>
>>
>> Abstract:
>>    This document registers a "Well-Known URI" for exposing state of an
>>    HTTP connection to the peer using formats such as qlog schema [QLOG].
>>
>>
>>
>>
>> The IETF Secretariat
>>
>>
>>
>>
>> --
>> Kazuho Oku
>>
>