Re: Feedback on the ping="" attribute (ISSUE-1)

Ian,

thanks for the feedback.

Before I get to the details, I'd like to clarify that we have (at least) 
*two* issues here, only one of which being in ISSUE-1.

1) ISSUE-1 is about whether it's ok to use POST for this,

2) The other issue is about whether this feature is needed at all, how 
to expose it in UAs and so on.

For now, I'll just reply to what is in ISSUE-1.

> On Sat, 27 Oct 2007, Julian Reschke wrote:
>>> We're long past that. It's trivial for a page to trigger a POST 
>>> without the user knowing.
>> I consider that a bug in User Agents.
> 
> This is not a widely held opinion.

Well, it's what RFC2616 says. I would argue that if the HTML WG thinks 
there is a problem in what RFC2616 has to say about how to use unsafe 
methods, it should bring this to the attention of the newly formed HTTP WG.

>> Please do not add more of this.
> 
> While I understand that you believe that silent POSTs are somehow harmful, 
> I believe that on the balance the proposed feature is a net user benefit, 
> and that this instance of automatic POST is no more dangerous than other 
> automatic POSTs being proposed (e.g. in the cross-site XMLHttpRequest 
> specification being developed at the W3C in the WebAPI WG). Indeed, in 
> this instance I would argue the danger is significantly reduced, since no 
> POST data is sent with the request.

Agreed. But just because it's less bad than some other use cases, 
doesn't make it a good design to use it, if there's an alternative 
approach which works without POST at all.

> [Quoting HTTP:]
> 
>> "9.1.1 Safe Methods
>>
>> Implementors should be aware that the software represents the user in 
>> their interactions over the Internet, and should be careful to allow the 
>> user to be aware of any actions they might take which may have an 
>> unexpected significance to themselves or others.
> 
> I agree that ping="" should be made visible to users. Indeed, the spec 
> explicitly makes that a SHOULD, going far outside its usual boundary of 
> not specifying user interface requirements.

Currently, the standard way in HTML UAs to distinguish safe (GET) from 
unsafe (POST) is a link vs a button.

So yes, if all "audited" links turn into buttons, that concern would be 
dealt with. Somehow however I feel this is not what people have in mind.

>> In particular, the convention has been established that the GET and HEAD 
>> methods SHOULD NOT have the significance of taking an action other than 
>> retrieval. These methods ought to be considered "safe". This allows user 
>> agents to represent other methods, such as POST, PUT and DELETE, in a 
>> special way, so that the user is made aware of the fact that a possibly 
>> unsafe action is being requested.
> 
> Indeed; and in fact part of the goal here is to make the possibly unsafe 
> action (user tracking and conversion tracking, with the potential effect 
> on future performance or the potential material financial effect) be one 
> that can be explicitly brought to the user's attention if he so desires, 
> something that is not possible in legacy tracking techniques. (For 
> example, using redirects make the whole process very opaque.)

Following that, the spec should make any UA that makes an audited link 
indistinguishable from a regular link non-conforming.

>> Naturally, it is not possible to ensure that the server does not 
>> generate side-effects as a result of performing a GET request; in fact, 
>> some dynamic resources consider that a feature. The important 
>> distinction here is that the user did not request the side-effects, so 
>> therefore cannot be held accountable for them."
> 
> I think it's clear that this paragraph is trying to convey that having 
> side-effects with a GET request is a poor state of affairs, which I agree 

I disagree. A server can implement any side-effects it wants for GET, 
the important part is that it can't complain if people follow these 
links (pre-fetching, spidering, etc).

> with, and which is one of the other things that the ping="" proposal 
> attempts to address -- legacy tracking mechanisms typically abuse GET in 
> an unsafe way, which causes a number of problems for the server (mostly 
> around unpredictable caching effects like pre-caching, session history 
> navigation, and transparent cache proxies), which can then affect the user 
> in undesirable ways (e.g. if tracking is used to determine preference 
> towards one link or another, and the user's browser precaches one more 
> often than the other, then the server will act as if the user had 
> indicated a preference where in fact he had not).

That's an argument in favor of separating the link target from the ping 
target, not an argument to use an unsafe method.

As soon as the ping target lives in an attribute that is not 
automatically followed by at all, that problem would go away.

> In conclusion I think HTTP supports the design of the feature as is.

I still disagree.

Could you please clarify why the ping attribute wouldn't work equally 
well with a safe method?

>> No, I'm not suggesting that.
>>
>> In this scenario, there are three parties involved:
>>
>> A: the user
>> B: the visited site
>> C: the site being linked to
>>
>> If the link from B to C needs to be audited for the purpose of paying 
>> ads, money will be exchanged between the owners of B and C. A is not 
>> involved in that transaction.
>>
>> How the contract between B and C is implemented should be outside the 
>> scope of the stuff sent to A.
> 
> While that would be nice in practice, it is not the case today, and it is 
> not clear that it ever could be the case. We have to work within the 
> limitations we are presented with, and in this case it seems that the 
> ping="" proposal is the closest one can get to solving the problems seen 
> by both the users and the authors.

Again, this is not about safe vs unsafe, but about ping vs href.

>> Again:
>>
>> "The important distinction here is that the user did not request the
>> side-effects, so therefore cannot be held accountable for them."
>>
>> When A follows the link, he is *not* accountable for the cost of the ad, 
>> being transferred from C and B.
> 
> The HTTP specification just says that a user can never be held accountable 
> for GET side-effects. It says nothing about the user being held 
> accountable for anything else, including automatic POST requests.

If the UA decides to invoke an unsafe method *without* the user's 
consent, that *may* be a problem. With a safe method, it's guaranteed 
not to be. Thus, there's a clear advantage in using a safe method.

>>>> I personally think that the attribute in itself is a Very Bad Idea, 
>>>> but if it stays in, by all means do not use POST for it.
>>> We can't use GET... what other method would be appropriate?
>> You shouldn't do it at all. If you insist in doing it, use a safe 
>> method. Everything else is in conflict with RFC2616. And yes, you can 
>> use GET.
> 
> I think it has been explained why using GET is undesirable.

I haven't seen an explanation that was convincing to me.

>>>> BTW: I just checked, and the Google Ads on www.google.de work with 
>>>> GET and a Redirect (302). Only safe methods from the user's point of 
>>>> view. Are you saying this is a problem?
>>> Yes.
>> Interesting -- good that I asked. It seems we'll not be able to make 
>> progress on this unless we clarify this issue first.
> 
> The problems from the server-side are that it is unreliable (due to 
> pre-caching, transparent caching, and session history navigation), it 
> obfuscates the user experience (the actual target URL is hidden), it is 
> slow (there's at least one extra HTTP round-trip, possibly with an 
> additional DNS hit as well), and it uses an idempotent method for a 
> distinctly non-idempotent action.

Again, this is about ping vs href, not about safe vs unsafe.

Please keep these topics separate.

> On Sat, 27 Oct 2007, Geoffrey Sneddon wrote:
>> Having read this entire thread, I don't see why anything is actually 
>> wrong. In this context the difference between GET and POST is negligible 
>> � both can technically be used to do what is desired, though using GET 
>> would be breaking RFC 2616 (or rather, breaking a SHOULD NOT). If we 
>> disallow it to be used on external servers, people will just continue to 
>> use Javascript to achieve this, which CANNOT be disabled by a UA without 
>> breaking behaviour that sites rely upon.
> 
> Indeed.

Incorrect - a misunderstanding about what it means for method to be "safe".

> On Sat, 27 Oct 2007, Julian Reschke wrote:
>> No, sorry, that's incorrect.
>>
>> If you want to do something silently (without the user's consent), you 
>> simply have to use a safe method.
> 
> We don't want to do it without the user's consent. The whole point of 
> making ping="" explicit is to allow the user to have the final decision.

Once in the configuration, or on each navigation event? Per site?

>> And if you consider the desired effect non-safe (which I don't), then 
>> the consequence is that you just can't do it.
> 
> We can't stop tracking from occurring. We can, however, make it better for 
> users. I think we have a responsibility to do so.

No disagreement about the goal here.

> On Sun, 28 Oct 2007, Henri Sivonen wrote:
>> The ping attribute does have the same security risks that cross-domain 
>> XHR POST with empty entity body would have if the access-control 
>> Method-Check weren't there. That is, if a POST handler has been 
>> programmed to trigger stuff on mere POST without a body, a malicious 
>> ping attribute could be used to trigger that action.
> 
> (As could an empty scripted <form>.)
> 
> 
>>> And if you consider the desired effect non-safe (which I don't), then 
>>> the consequence is that you just can't do it.
>> It is about idempotent vs. non-idempotent and side effects.
>>
>> If you are counting ad impressions, clearly you don't want to
>>  a) count Google Web Accelerator (or similar) prefetches
>>  b) leave impressions uncounted due to an intermediate cache satisfying 
>>     the request.
> 
> Indeed.

Again, this is about ping vs href, not about safe vs unsafe.

> On Sun, 28 Oct 2007, Julian Reschke wrote:
>>> So would you ban XHR POST and script-initiated form submissions?
>> I would want the XHR spec to clarify that it's not OK to initiate unsafe 
>> methods without the user's consent. I would also deprecate 
>> script-initiated form submissions from something like onload().
> 
> Please bring this up with the Web API working group.

I did, and I was ignored.

>> and even if you use "ping", you still could do it with a safe method 
>> (HEAD/Cache-Control:no-cache).
> 
> Unfortunately HEAD is typically implemented in servers (e.g. Apache) 
> without running the relevant CGI scripts, which makes them hard to 
> implement at all. I also disagree that this would be a correct application 
> of the HEAD method's semantics.

HEAD and GET have the same semantics - the only difference being that 
for HEAD the response body is not transmitted. Servers that implement 
HEAD differently technically are not compliant.

For link tracking, my understanding was that there is no response body 
expected. Thus, for a server that implements a "link auditing resource", 
both GET and HEAD actually will do the same -- invoke some kind of 
tracking (minimally dumping the URI into a log file), and just return 
with an HTTP 2xx status and no body.

Thus, I would expect that GET and HEAD can be used interchangeably.


> On Sun, 28 Oct 2007, Henri Sivonen wrote:
>> That might work and could be a tad safer. It isn't in any way 
>> theoretically pure from the RFC 2616 point of view, though, to make HEAD 
>> and GET have different semantics beyond the response body presence.
> 
> Indeed.

Yes, and nobody suggested that.

> On Sun, 28 Oct 2007, Julian Reschke wrote:
>>> That might work and could be a tad safer. It isn't in any way 
>>> theoretically pure from the RFC 2616 point of view, though, to make 
>>> HEAD and GET have different semantics beyond the response body 
>>> presence.
>> I wasn't suggesting that.
> 
> You suggested that we should use HEAD for request tracking, which indeed 
> makes GET and HEAD have different semantics in a way that does not match 
> (at least my interpretation of) RFC2616.

Again, no I didn't. I suggested HEAD, but that doesn't mean I was trying 
to have different semantics for GET -- please assume for a moment that I 
have *some* expertise both with HTTP servers and the protocol definition.

>> Fundamentally, the ping being sent is not a user request of any kind at 
>> all, it is a third-party request for information about what the user is 
>> doing. This is not a transaction between a server and a client in the 
>> sense that HTTP usually offers, it is a one-way message from the client 
>> to a third party. So we are just using HTTP as a transport method of 
>> convenience since it is there. This is probably reasonable in the 
>> circumstances, but I don't yet understand how it matters which method we 
>> decide to turn into a one-way message in the absence of a mechanism for 
>> such.
> 
> Hopefully the points put forward earlier in this e-mail cover this in 
> sufficient detail.

Nope.

> On Sun, 28 Oct 2007, Charles McCathieNevile wrote:
>> You mean POST, right? As far as I am concerned, the HEAD request 
>> suggestion is the least departure from normal HTTP (since there is 
>> already llttle expectation that HEAD will pass a response to the user), 
>> but I still don't see
> 
> (I'm not sure if you meant to stop here or not.) HEAD seems even less 
> desirable that head from the point of view of HTTP -- it's only supposed 
> to get the HTTP headers of the resource, without doing anything at all!

That's incorrect. The semantics of HEAD and GET are *exactly* the same, 
except for the response body not being transmitted.

So if GET "does" something, "HEAD" will need to do so as well.

> On Mon, 29 Oct 2007, Julian Reschke wrote:
>> So the scenario is:
>>
>> 1) User A browses web site B.
>>
>> 2) A follows an HTML link to site C.
>>
>> 3) The owner of B wants to be informed of that event in order to charge 
>> the owner of C for an online ad linking to C.
> 
> That's one scenario; there are other, possibly more important ones, for 
> example: tracking results in search, so that more popular entries can have 
> subsequent rankings boosted, or usability studies tracking which links 
> users prefer on a site.

Understood. I didn't mention them here, because it seems you were mainly 
concerned about the ad issue. In *this* case, there's even less reason 
to use an unsafe method.

>> My position is that although money may be exchanged between B and C due 
>> to the notification (ping), this is a transaction between B and C, and A 
>> MUST NOT be involved. In other words, following a hyperlink MUST stay 
>> "safe" in the RFC2616 sense.
> 
> The hyperlink does stay safe; however, the ping is not idempotent, and 
> should not use an idempotent method.

I still do not understand why it needs to be unsafe. You seem to be 
concerned about the ping being executed when the user *didn't* navigate 
-- but what does this have to do with safe vs unsafe?

Just state in the spec that the GET/HEAD operation on the ping target 
MUST happen at most one time per user-initiated navigation to the href'd 
URI.

>> (emphasis on the last paragraph!)
> 
> The last paragraph actually doesn't apply -- it gives reasons not to use 
> GET, or to be careful with GET, and doesn't actually give advice on other 
> methods.

I do not understand how it "does not" apply, and I also disagree that it 
gives reasons not use GET. On the contrary, it explicitly allows servers 
to do something with GET that has side-effects -- as long as the user is 
not made accountable for it. Exactly this case, it seems to me.

> On Sun, 28 Oct 2007, Kornel Lesinski wrote:
>> OTOH ping is all about creating side-effects, and only non-safe methods 
>> should cause them.
> 
> Indeed.

And again: it depends on who is made accountable for the side effect. 
The user following the link shouldn't be.

Or are we suddenly talking about a vehicle for micro payments here?

>> The root cause why using POST is unsafe is CSRF, and there should be a 
>> separate effort dealing with that (covering all cases, not only ping).
> 
> I agree with all of the above.

I don't follow. Just because there are other problems with POST, there's 
no reason to invent a whole new feature using POST when we don't have to.

> On Sun, 28 Oct 2007, Julian Reschke wrote:
>> Following a link should not cause side effects the user (A) can be made 
>> accountable for.
> 
> Agreed. And nothing says that the user can be made accountable for POSTs 
> made for ping="" attributes. Just that the user _can't_ ever be made 
> accountable for side-effects made in response to GETs.

Wait a second.

So you are willing to state that ping-initiated HTTP method invocations 
must not cause an action the user can be made accountable for. I agree 
with that.

But then, why don't you use a safe method in the first place?

>> And, fortunately this is not the case here. The only party for which the 
>> side effect is relevant is the site owner (B), and potentially the party 
>> (C) the link points to.
> 
> And sometimes the user, e.g. when the tracking is used to improve search 
> results in future searches, or to personalise a site to the user's habbits 
> by promoting areas of a site that the user uses the most.

All of these are cases where HTTP experts will tell you that GET or HEAD 
is just fine.

> I hope this clarifies the issues surrounding the post="" attribute. I 
> understand that not everybody agrees on this, but when there are requests 
> that are mutually exclusive, we can't make everyone happy. I hope that the 
> explanations above address most of the concerns that were raised, but I 
> understand that they might not. I would ask anyone who still disagrees 
> with what the spec says to please consider the above explanations 
> carefully; simply raising the same issue that has already been raised, 
> with no new information or reasoning, is unlikely to result in a different 
> reply. I try to base the design of the spec on the balance of all input, 
> not on the volume of input.

I have to say it didn't help me.

I've seen no evidence why ping has to use an unsafe method at all.

Best regards, Julian

Received on Friday, 2 November 2007 23:10:37 UTC