- From: Dan Auerbach <dan@eff.org>
- Date: Wed, 24 Apr 2013 13:54:47 -0700
- To: ronansan@gmail.com, "public-tracking@w3.org" <public-tracking@w3.org>
Hi all, First, thanks so much to Ronan for detailing the requirements for OOBC. I want to emphasize that there is a third option on the table for how to handle OOBC, which is to not make a special exception to accommodate this. I have a concern with an extra permitted use because it may create a reason for keeping unique id cookies (I don't think there currently are permitted uses on the table that require unique id cookies and I'd like to keep it that way), there is less transparency to the user as Nick emphasized ("do these people think they have OOBC from me or not?"), and it is yet another permitted use, where we should be trying to keep these as few and narrowly scoped as possible. I have a concern about the original approach of using a separate TSV for OOBC, since I think this adds a lot of complexity, and puts burden on the user agent to appropriately report back to the user what is going on. Moreover, given added complexity, there is more room for things to break, e.g. if the tracking status resource is down for a week or a month, whose responsibility is it to demand that it go back up? Are there any repercussions for that sort of flakiness? Given that, I'd like to avoid having to make a special exception for OOBC if we can get away with it, and think it is appropriate to dive more into the statistical weeds of how these panel studies are conducted. Ronan, if this is not your area of expertise, could you ask the person in your organization responsible for handling the panel study significance tests to describe how these work to the group? I'd also be happy to talk to this person 1:1 offline. My null hypothesis is that you could easily normalize for missing DNT:1 data. A lot hinges on how panels are set up and I'd love to understand this better. But suppose you make your initial panel larger than it has to be (say, twice the size, but the multiplier will depend on total DNT:1 adoption rate), with the idea that certain individuals will be something like "alternates" whose data won't count. I sort of presume this is done already if any statistical testing is happening. Now only a subset of those participants who have given OOBC report back with data, presumably due to DNT:1 (noting again as others have that many OOBC mechanisms could set DNT:0 in-band). The question is whether you can still conduct a statistically significant study given this restriction. My contention is that this is easy, given background knowledge of the histogram of DNT usage by demographic segment. For simplicity in order to illustrate this point, let's just suppose that there is only one bit of demographic information (say, male vs female). Let's suppose women are twice as likely as men to turn DNT:1 on. Now a lot hinges on what sort of statistical tests are being used to create a balanced panel. Is it a manually curated panel where every demographic is represented (e.g. 1 man, 1 woman)? Is it a naive statistically produced panel reflecting the demographic distribution (50% men, 50% women)? Or some more complex frequentist or Bayseian statistical approach? Manual curated panels are obviously still possible without problems given a moment's thought. As for naive statistical panels, this is easy to normalize for -- just build the assumption in that the missing person is twice as likely to be a woman than a man. If there are more complex tests going on, let's hear about them! I'm optimistic that DNT:1 users should not be devastating to the creation of a fair and unbiased panel, but if I'm wrong, I'd like to understand why in detail. Finally, I think that periodically pushing static data structures with 500K records to CDNs that can be queried under 200ms is not a terribly burdensome technical ask, and I'm happy to talk about the technical challenges about how this is done. I don't want to single out OOBC in particular with this analysis. Ronan has been refreshingly forthcoming in terms of his requirements and I appreciate this very much. We need to be rigorous and examine ALL permitted uses closely, so that we can ensure that they are necessary and appropriately scoped. I think we're far from that goal now, but perhaps we can continue the constructive dialogue on OOBC and it can serve as a model for digging into other areas of industry that touch on permitted uses. Best, Dan -- Dan Auerbach Staff Technologist Electronic Frontier Foundation dan@eff.org 415 436 9333 x134
Received on Wednesday, 24 April 2013 20:55:20 UTC