Re: TPE Handling Out-of-Band Consent (including ISSUE-152)

Yes, I think it does take evidence. A few questions that spring to mind
which seem quite relevant are: what's the methodology for diversifying
the panel? And how precise is the segmentation? How do you measure panel
bias? Right now you are receiving DNT:1 headers but presumably not
honoring them, given that that we haven't yet set a standard. It seems
to me that this presents a perfect opportunity for measuring how
honoring DNT:1 might affect your panel studies, both for users who have
given OOBC, as well as for those who haven't and for whom you are
obliged to honor DNT. Have you conducted experiments along these lines,
or do you plan to? If the bias is predictable, you could potentially
correct for it in the future and gleefully ignore DNT:1 users even when
you have OOBC, and still get the same results to a high degree of
accuracy. It seems worth at least looking into this possibility, right?

As I've said, I think folks in the group -- myself included -- wouldn't
want to ask you to do something impossible or needlessly tie your hands.
But a little bit of hard data from knowledgeable software engineers in
industry like you goes a long way to help educate the group as to your
needs, and to move the conversation forward.

Dan

On 03/25/2013 04:02 PM, Ronan Heffernan wrote:
> Does it take evidence to know that if you take a carefully modeled
> panel, with known mixes of geography, demographics, etc., and allow
> some unknown number (let's say 40% at the outside) of the panel to
> opt-out of being counted (despite their signed contracts) via an
> in-band exception mechanism, that all of the tuning and extrapolation
> that was done based on that population has to go out the window? 
> Thanks to the fact that we could not collect identifiable information
> during the in-band exception process, we couldn't even estimate the
> damage or tell which demographic segments have been the most heavily
> skewed.
>
> --ronan
>
>
> On Mon, Mar 25, 2013 at 1:49 PM, Dan Auerbach <dan@eff.org
> <mailto:dan@eff.org>> wrote:
>
>     On 03/23/2013 07:45 AM, Ronan Heffernan wrote:
>     > using the in-band exception mechanism would skew research horribly,
>     > and the balanced and tuned panels constructed by our Measurement
>     > Science department would be replaced by biased and un-measurable
>     crowds.
>     Can you provide evidence for this assertion?
>
>     --
>
>


-- 
Dan Auerbach
Staff Technologist
Electronic Frontier Foundation
dan@eff.org
415 436 9333 x134