Re: discussion on FLOC performance?

On Wed, Feb 17, 2021 at 5:55 AM Wendy Seltzer <wseltzer@w3.org> wrote:


> Are there other use cases and needs we should be documenting?
>

In order to evaluate the results it looks as if we will need to understand
which sites are being used for FLoC training during the origin trial. There
are several ways this could be set up. Training could be based on...

 * Only sites enrolled in the origin trial: will not include as much
training data as would be available to FLoC in production, so would not
provide an accurate picture of FLoC capabilities

 * Sites that have opted in to FLoC or show awareness of FLoC, by setting
the Permissions Policy or by having a script on the page call a FLoC API

 * Other sites including those maintained by people who are unaware of
FLoC: Would also give an unrealistic view of FLoC because some of those
sites will later choose to opt out, or not to opt in. This level of FLoC
training would also limit participation in the origin trial, because some
cohorts would be likely to reveal protected group membership, and this risk
would cause some potential origin trial participants not to be able to look
at the cohort IDs.

The second option seems most reasonable, and the problem of getting enough
participation could be addressed by having participating companies add
optional FLoC calls to their third-party scripts. (A small site could
participate by opening an email from their ad or analytics service and
clicking a checkbox in their control panel, and would not have to modify
HTML or HTTP headers themselves.)

Thank you,
Don


> Thanks,
> --Wendy
>
> On 2/15/21 4:59 AM, Arnaud Blanchard wrote:
> > Hi Wendy and group,
> >
> > We would like to put FLoC's performance level discussion to the agenda
> to ideally get:- More details about the test objectives (metrics;
> dimensions; etc.)
> > - More technical details about the creation of the FLoC (FLoC size; FLoC
> assignment detailed methodology)
> > - External communication clarification
> >
> > 2 sessions ago, there were many interesting points raised around
> Google's recent communication pieces about FLoC performance compared to 3rd
> cookies. In particular, the idea that FLoC would retain 95% of the
> performance brought by third party cookies seemed to draw particular
> attention from the community. Then, someone from Google Ads said that they
> would share the analysis this figure was drawn from with some details.
> > If I am not mistaken, the only publication so far consists of this brief
> explainer:
> https://github.com/google/ads-privacy/blob/master/proposals/FLoC/Floc-live-experiments.md.
> It does bring some clarifications, such as the fact that the experiment
> concerned a very narrow use case ('audience targeting' based on Google
> taxonomy), with some others explicitly out of scope (remarketing, others
> vendors taxonomies).
> >
> > However, despite Google's reassurance that the AB test was conducted in
> the best conditions, and with sound analytical methodology - which we have
> no reason to doubt - we still miss a lot of details that would allow
> everyone to understand how FLoC would impact their products (flock sizes,
> proprietary taxonomy impact, etc).
> >
> > Stating that 95% of the performance is preserved without stating the
> particular use case it was measured against implies that Google Chrome's
> FLoC will be considered as good as long as it allows emulation of Google
> Ads proprietary taxonomy. I hope this is not the case, and I assume this
> was not the intention of the analysis, but that what it looks like.
> >
> > All in all, this 95% number is an overstatement that does convey a
> misleading idea to the public. In my opinion, the whole group, and the FLoC
> project itself, would seriously benefit from a broader, more detailed
> clarification.
> >
> > Thank you very much in advance,
> > Arnaud Blanchard
> >
>
>
> --
> Wendy Seltzer -- wseltzer@w3.org +1.617.715.4883 (office)
> Strategy Lead and Counsel, World Wide Web Consortium (W3C)
> https://wendy.seltzer.org/        +1.617.863.0613 (mobile)
>
>
>
>