Re: The Rubber meets the Road - DNT compliance code

Peter, google.com already does UA-based profiling for home page promos
(e.g. if you're a Chrome user you won't see a "Try Google Chrome" button on
the homepage, if you aren't a Chrome user we have shown such promos from
time to time in the past). I can assure you that it was far from being a
problem.

You have not posted a workable solution, nor have you actually posted any
sort of "cost of doing business with this spec". First of all, the "cost of
doing business" would be the cost of accepting a DNT:1 header from 100% of
IE users in terms of whatever revenue impact that would have. You've said
nothing about that. Secondly, your 1ms cost was not even measured properly
as I've pointed out. I've tried to explain to you that UA sniffing has been
in place on the web for years (frankly since the 90's), this is not
anything scary or problematic, people are already doing this and "making
server-side decisions" to send back appropriate content. Heck, we do this
on google apps all the time to send you javascript that's optimized for
your browser. No big deal!

Again, I'm not arguing for using this mechanism, but I think it's an option
that people have expressed that they want (or rather, that it's a deal
breaker for them), and I've seen no good arguments explaining why it would
be problematic.

And BTW, if you want data:

Here's looking for IE10 in an IE10 UA string and a Chrome UA string to see
if it matches IE10. This results in 2 _billion_ comparisons being done.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>



int main() {
  unsigned int i, chrome=0, ie=0;
  struct timeval start, end;
  int usec = 0;
  char *IE10_UA, *CHROME_UA, *IE10;
  IE10_UA = malloc(500);
  CHROME_UA = malloc(500);
  IE10 = malloc(500);
  strcpy(IE10_UA, "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1;
Trident/6.0)");
  strcpy(CHROME_UA, "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4)
AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1171.0 Safari/537.1");
  strcpy(IE10, "MSIE 10.0");
  gettimeofday(&start, NULL);
  for (i = 0; i < 10000000u; i++) {
    if (strstr(IE10_UA, IE10)) {
      ++ie;
    } else if (strstr(CHROME_UA, IE10)) {
      ++chrome;
    }
  }
  gettimeofday(&end, NULL);
  printf("Did %u comparisons, found %u IE and %u Chrome UA strings\n", i,
ie, chrome);
  usec = (end.tv_sec - start.tv_sec) * 1000000;
  usec += end.tv_usec - start.tv_usec;
  printf("It only took %d microseconds!\n", usec);
  return 0;
}

(note, I did the malloc and strcpy and compiled with -g because otherwise
the compiler seems smart enough to realize that the constants aren't
changing and optimizes the loop away, sigh.)

For me, this takes 1366907 microseconds. I.e. that's 20 million compares in
just over a second, or, about 0.07 _microseconds_ per comparison. This
really isn't that exciting.

Now it's 9pm and time to eat. Good day.

On Wed, Jun 13, 2012 at 4:58 PM, Peter Cranstone
<peter.cranstone@gmail.com>wrote:

> Ian,
>
> I don't care what the W3 does.
>
> I'm just pointing out that "no plan survives first contact" and it's
> already been proven with Microsoft. The 1ms isn't a big deal on it's own –
> it's the law of unintended consequences that follows it that is. Now I have
> to start making server side decisions and sending back pop-ups or a 400
> error message.
>
> I've already posted a solution, and we've already posted a Web page that
> tells you the cost of doing business with this spec. And that's the
> difference between people who talk about it and then people who actually
> have to implement it.
>
> How about you show us your solution to the MSIE issue. Code it up, and let
> us test it. I want to see how you handle the exception, and I want to see
> how you design the UI to inform the customer to go get another browser. And
> then I want to run some performance tests on it.
>
> Lets let the code do the talking.
>
>
>
> Peter
> ___________________________________
> Peter J. Cranstone
> 720.663.1752
>
>
> From: "Ian Fette (イアンフェッティ)" <ifette@google.com>
> Reply-To: <ifette@google.com>
> Date: Wednesday, June 13, 2012 5:48 PM
>
> To: Peter Cranstone <peter.cranstone@gmail.com>
> Cc: Kevin Smith <kevsmith@adobe.com>, W3 Tracking <public-tracking@w3.org>
> Subject: Re: The Rubber meets the Road - DNT compliance code
>
> Peter, I think I just might ignore the thread. It's quite tempting.
>
> Please realize that anyone who is likely to be impacted by DNT is probably
> already doing various checks, including checks of the UA, 50 ways to sunday
> ("Show this ad to IE users but not to Firefox users. Show this ad to people
> in Montana but not in Michigan. Oh, it's a Windows user? Show them an ad
> for this. A Mac user? Show them that.") You're trying to make up problems
> that simply don't exist.
>
> As for the response, you're either going to serve content or an error. I
> would imagine the error is probably smaller than the content. It could even
> be a net win :)
>
> As for the spec, you keep saying "it's either 1 or 0" but clearly you
> don't understand, as actually in the spec it's either "1, 0, or
> undefined/null". It's tristate, not bimodal as you suggest.
>
> I am glad that you understand web servers, but frankly you are being very
> unconvincing. I've heard a number of people say they want the ability to do
> this, none of them seem concerned by "performance issues" that you raise.
> If there is an issue, that's their problem and it's up to them to trade
> that off against everything else. No one is requiring them to take this
> route. Frankly your first reply was that it took 42 years to do UA
> matching, and then you're telling me it's 1ms but you seem to be using 1ms
> resolution times, and you haven't even accounted for the time it takes to
> get the time. You can see why I would be a bit skeptical of your claims.
>
> At the end of the day, the W3C cannot force a company to honor DNT, which
> is what you are asking this working group to do. That's not in the W3C's
> purview, so trying to argue it here really doesn't make any sense. A
> regulator might or might not be in that position, but that's a whole
> different story and if that's what you're going for, you're on the wrong
> mailing list. Indeed, if you truly want to see increased adoption of DNT,
> it would not be such a bad idea to listen when a large number of people
> bring up an issue they say would be a show-stopper for adoption and try to
> figure out how to make a good faith effort to resolve it, rather than
> inventing problems that don't exist.
>
> On Wed, Jun 13, 2012 at 4:04 PM, Peter Cranstone <
> peter.cranstone@gmail.com> wrote:
>
>> Ian,
>>
>> If you don't want to answer the questions then ignore the thread.
>>
>> We all don't have Google's resources – for some people who already have
>> loaded servers your spec adds more cost, load and complexity. And you still
>> haven't factored in the response. It seems like when anyone calls you out
>> on something you don't agree with you then it must be there fault.
>>
>> And as for not understanding how the Web works – well I do. And Google
>> and the whole Web uses stuff my partner and I invented so we know a thing
>> or two about Web servers and what it takes to improve the performance of a
>> Web site.
>>
>> The spec is brain dead simple – it's either 1 or 0. Case closed on the
>> browser side.
>>
>> All your issues remain on the server and you're not even close to
>> shipping something that can be called compliant. Heck there's not even a
>> browser out there today that can send a DNT:0 header. So how about you
>> start shipping something that works vs. calling me out.
>>
>>
>>
>> Peter
>> ___________________________________
>> Peter J. Cranstone
>> 720.663.1752
>>
>>
>> From: "Ian Fette (イアンフェッティ)" <ifette@google.com>
>> Reply-To: <ifette@google.com>
>> Date: Wednesday, June 13, 2012 4:56 PM
>> To: Peter Cranstone <peter.cranstone@gmail.com>
>> Cc: Kevin Smith <kevsmith@adobe.com>, W3 Tracking <public-tracking@w3.org
>> >
>>
>> Subject: Re: The Rubber meets the Road - DNT compliance code
>>
>> Peter, 1ms is nothing. There's 86M milliseconds in a day, not to mention
>> that that 1ms you measured is probably less than 1ms but getting counted as
>> 1ms due to the resolution of the timers available and the fact that if
>> you're actually trying to get accurate time, calling "time" is probably
>> more expensive than the operations you're timing here.
>>
>> Believe me, Google already does parsing of user agents, it has not taken
>> us down. So does virtually every large website out there. You are trying to
>> create a problem that doesn't exist. Please stop.
>>
>> You are actively detracting from this working group getting anything
>> done. You have turned this issue into a giant time sink for a number of
>> people who have tried time and time again to explain this issue to you.
>> Please either join the working group and read up on the history, or move
>> along.
>>
>> -Ian
>>
>> On Wed, Jun 13, 2012 at 3:22 PM, Peter Cranstone <
>> peter.cranstone@gmail.com> wrote:
>>
>>> Well it should be interesting to see what happens.
>>>
>>> We've now added a time section to our DNT compliance test:
>>> http://www.5o9mm.com/mod_dnt_test_1.php
>>>
>>> It's about a millisecond to complete the analysis – so you can do the
>>> math on a site that gets millions of hits a day. Plus add in the 400 error
>>> you're going to send back to the user if they have a non compliant browser.
>>>
>>> If you want to kill ad revenue this sure is the way to go about it.
>>> Sorry we see you're using MSIE 10, it's non compliant so go get another
>>> browser and please come back when you know how to set up DNT correctly.
>>>
>>> Again – you win the battle and lose the war. Plus the Ad guys are really
>>> pissed (or should be).
>>>
>>>
>>> Peter
>>> ___________________________________
>>> Peter J. Cranstone
>>> 720.663.1752
>>>
>>>
>>> From: Kevin Smith <kevsmith@adobe.com>
>>> Date: Wednesday, June 13, 2012 4:16 PM
>>>
>>> To: Peter Cranstone <peter.cranstone@gmail.com>, "ifette@google.com" <
>>> ifette@google.com>
>>> Cc: W3 Tracking <public-tracking@w3.org>
>>> Subject: RE: The Rubber meets the Road - DNT compliance code
>>>
>>> That is exactly the point of this discussion.  You are in compliance
>>> with the spec if the spec dictates that a UA is non-compliant if they set
>>> DNT on by default and that you need not comply with a non-compliant UA.*
>>> ***
>>>
>>> ** **
>>>
>>> Browser wars are unfortunate, but they have been a reality of the web
>>> since the beginning.  This is why standards exist and the way standards are
>>> effective is that wide scale adoption of a standard applies pressure to
>>> those non-compliant to become compliant – especially in cases where
>>> non-standard behavior is not supported by websites.****
>>>
>>> ** **
>>>
>>> ** **
>>>
>>> *Kevin Smith*  |  Engineering Manager  |  Adobe  |  385.221.1288 |
>>> kevsmith@adobe.com****
>>>
>>> ** **
>>>
>>> *From:* Peter Cranstone [mailto:peter.cranstone@gmail.com<peter.cranstone@gmail.com>]
>>>
>>> *Sent:* Wednesday, June 13, 2012 3:59 PM
>>> *To:* Kevin Smith; ifette@google.com
>>> *Cc:* W3 Tracking
>>> *Subject:* Re: The Rubber meets the Road - DNT compliance code****
>>>
>>> ** **
>>>
>>> Then you're not complying with the spec to honor a DNT header if your
>>> server supports it.****
>>>
>>> ** **
>>>
>>> The spec does NOT distinguish between a default setting made by the OEM
>>> and a setting made by the user. Ergo you cannot distinguish either case on
>>> the server. There should be no case where a browser that supports DNT is
>>> ever rejected because you accused the OEM of making a setting vs. the user.
>>> ****
>>>
>>> ** **
>>>
>>> It will be a PR nightmare. Can you imagine the user saying – heck the
>>> Web site rejected me because I was asking for privacy and it doesn't like
>>> the browser I'm using. Yep, I can just see all the Tier 1 Web sites
>>> adopting that approach.****
>>>
>>> ** **
>>>
>>>
>>> Peter
>>> ___________________________________
>>> Peter J. Cranstone
>>> 720.663.1752****
>>>
>>> ** **
>>>
>>> *From: *Kevin Smith <kevsmith@adobe.com>
>>> *Date: *Wednesday, June 13, 2012 3:55 PM
>>> *To: *Peter Cranstone <peter.cranstone@gmail.com>, "ifette@google.com" <
>>> ifette@google.com>
>>> *Cc: *W3 Tracking <public-tracking@w3.org>
>>> *Subject: *RE: The Rubber meets the Road - DNT compliance code****
>>>
>>> ** **
>>>
>>> No, you alert the user that you do not support the DNT setting from
>>> their browser and recommend that they change browsers if they would like to
>>> view your site in a DNT supported fashion.  I still get ‘supported browser’
>>> messages all the time.  It’s a concept nearly as old as the web.****
>>>
>>>  ****
>>>
>>> *Kevin Smith*  |  Engineering Manager  |  Adobe  |  385.221.1288 |
>>> kevsmith@adobe.com****
>>>
>>>  ****
>>>
>>> *From:* Peter Cranstone [mailto:peter.cranstone@gmail.com<peter.cranstone@gmail.com>]
>>>
>>> *Sent:* Wednesday, June 13, 2012 9:35 AM
>>> *To:* ifette@google.com
>>> *Cc:* W3 Tracking
>>> *Subject:* Re: The Rubber meets the Road - DNT compliance code****
>>>
>>>  ****
>>>
>>> Ian,****
>>>
>>>  ****
>>>
>>> It's the why that's important.****
>>>
>>>  ****
>>>
>>> So riddle me this. You're a DNT compliant server. If you see the DNT
>>> flag set to 1 on the inbound request you honor it. However if all of a
>>> sudden you see MSIE 10 UA's what do you do? Reject it because you don't
>>> like the fact that MSIE set is as the default?****
>>>
>>>  ****
>>>
>>> You see – Microsoft has won the war. If you say you honor it, then you
>>> MUST accept their header. And if you don't honor DNT then it doesn't
>>> matter. The spec just got played.****
>>>
>>>  ****
>>>
>>>
>>> Peter
>>> ___________________________________
>>> Peter J. Cranstone
>>> 720.663.1752****
>>>
>>>  ****
>>>
>>> *From: *"Ian Fette (イアンフェッティ)" <ifette@google.com>
>>> *Reply-To: *<ifette@google.com>
>>> *Date: *Wednesday, June 13, 2012 9:30 AM
>>> *To: *Peter Cranstone <peter.cranstone@gmail.com>
>>> *Cc: *W3 Tracking <public-tracking@w3.org>
>>> *Subject: *Re: The Rubber meets the Road - DNT compliance code****
>>>
>>>  ****
>>>
>>> Peter, all of DNT requires resources.****
>>>
>>>  ****
>>>
>>> I don't get where you say there are legal consequences. If you tell the
>>> user "I'm not honoring your DNT request because <X>" and you're clear about
>>> your practices then you're not breaking any promises.****
>>>
>>> On Wed, Jun 13, 2012 at 8:28 AM, Peter Cranstone <
>>> peter.cranstone@gmail.com> wrote:****
>>>
>>> Ian,****
>>>
>>>  ****
>>>
>>> This is a case of win the battle on the forum but lose the war in the
>>> real world. It doesn't matter if it's neither hard or complex. The point is
>>> that it has to be done, tested and then updated and maintained. That
>>> requires resources – not something that everyone can afford to do.****
>>>
>>>  ****
>>>
>>> Secondly, lets flog the dead horse one more time on "who set the DNT
>>> flag". If I have to write code that cannot guarantee 100% accuracy when it
>>> comes to this privacy setting AND there are legal consequences of me
>>> getting said code wrong (i.e. fines or pissed off customers) then I'm not
>>> going to do it.****
>>>
>>>  ****
>>>
>>> We all know that per the spec that MSIE is not compliant because it sets
>>> the flag by default. But what admin in his right mind is going to reject
>>> it? If the server is DNT compliant then there is NO downside to MSIE
>>> setting the default.****
>>>
>>>  ****
>>>
>>> We're back to stupid browser wars again and pissing off the customer.
>>> Not a good thing.****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>>
>>> Peter
>>> ___________________________________
>>> Peter J. Cranstone
>>> 720.663.1752****
>>>
>>>  ****
>>>
>>> *From: *"Ian Fette (イアンフェッティ)" <ifette@google.com>
>>> *Reply-To: *<ifette@google.com>
>>> *Date: *Wednesday, June 13, 2012 9:22 AM
>>> *To: *Peter Cranstone <peter.cranstone@gmail.com>
>>> *Cc: *W3 Tracking <public-tracking@w3.org>
>>> *Subject: *Re: The Rubber meets the Road - DNT compliance code****
>>>
>>>  ****
>>>
>>> Many websites already do this -- "serve this JS to this user agent". It
>>> is neither complex nor hard.****
>>>
>>> On Wed, Jun 13, 2012 at 7:44 AM, Peter Cranstone <
>>> peter.cranstone@gmail.com> wrote:****
>>>
>>> All,****
>>>
>>>  ****
>>>
>>> There's a lot of questions around a non-compliant UA sending a DNT
>>> header. There's still no definition on the forum or the spec on what
>>> constitutes a non compliant UA, or even who is going to maintain a
>>> "blacklist" of those non-compliant UA's. Finally there's no description of
>>> a message that should be sent back to the consumer indicating that he's
>>> using a non-compliant UA. ****
>>>
>>>  ****
>>>
>>> So I'm posting a link today of what something might look like running on
>>> a server. The reason this is in PHP is because there are lot of servers (in
>>> the 10's of millions) that cannot suddenly start adding server side modules
>>> that do the detection. So it will all have to be done via a script.****
>>>
>>>  ****
>>>
>>> Think about this for a moment. In the real world server side admins are
>>> going to have to add code to EVERY CGI script to do this. The performance
>>> hit is going to be HUGE.****
>>>
>>>  ****
>>>
>>> Here's the link: http://www.5o9mm.com/mod_dnt_test_1.php ****
>>>
>>>  ****
>>>
>>> We've blacklisted the following browsers:****
>>>
>>>  ****
>>>
>>> HTTP_DNT_BLACKLISTED_USER_AGENT_1 = Mozilla/4.0 (compatible; MSIE 7.0;
>>> Windows NT 6.1; Trident/5.0)****
>>>
>>> HTTP_DNT_BLACKLISTED_USER_AGENT_2 = Mozilla/5.0 (compatible; MSIE 9.0;
>>> Windows NT 6.1; Trident/5.0)****
>>>
>>> HTTP_DNT_BLACKLISTED_USER_AGENT_3 = Mozilla/4.0 (compatible; MSIE 7.0;
>>> Windows NT 6.0; Trident/5.0)****
>>>
>>> HTTP_DNT_BLACKLISTED_USER_AGENT_4 = Mozilla/5.0 (compatible; MSIE 9.0;
>>> Windows NT 6.0; Trident/5.0)****
>>>
>>> HTTP_DNT_BLACKLISTED_USER_AGENT_5 = Mozilla/5.0 (Windows NT 6.0;
>>> rv:8.0.1) Gecko/20100101 Firefox/8.0.1****
>>>
>>>  ****
>>>
>>> So every time someone hits the Web site we have to run a check. The
>>> request time for this check on our server is:****
>>>
>>>  ****
>>>
>>> REQUEST_TIME = 1339597469****
>>>
>>>  ****
>>>
>>> For that single page. Now multiply that by every page on your Web site
>>> that is scripted. Ouch.****
>>>
>>>  ****
>>>
>>> Now here's where it gets really interesting. Let's say that I'm on the
>>> blacklist. What does the server do? By rights it should abort the entire
>>> request and send a 400 invalid request response back to the user.****
>>>
>>>  ****
>>>
>>> So what the heck does the user do now?****
>>>
>>>  ****
>>>
>>> If this spec is going to be Trusted and used it has to work in the real
>>> world which is NOT 100% technical. They turn it on (or have it turned on
>>> for them) and they expect magic. They don't expect to be told that there
>>> browser is non-compliant and they can either go get another one or get
>>> tracked.****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>>
>>> Peter
>>> ___________________________________
>>> Peter J. Cranstone
>>> 720.663.1752****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>>
>>
>