Re: Verizon, guidelines

Thanks Jo. Counter comments in-line.

Jo Rabin wrote:
>
> On 21/12/2008 21:09, Luca Passani wrote:
>>
>> I am a user with a Nokia 7210 (WML-Only). Site www.company.com can 
>> normally recognize my UA and serve the ad-hoc WML content. The UA is 
>> spoofed. I get Mozilla-Firefox. I serve HTML with no-transform 
>> header. User-experience is totally broken because transcoder won't 
>> transcode AND 7210 is unable to read HTML.
>
> And that's why the site should include a Vary: User-Agent
>
> See 
> http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/081107#sec-receipt-of-vary-header 
>
>
> which says that the Proxy should discard a response which is in 
> response to a request with spoofed headers, and re-request with 
> original headers, if there is a relevant Vary header.

It is nice to hear that you have figured how to give practical advice to 
transcoder vendors about how to undo the bad side-effects of spoofing 
the UA, while staying within the original HTTP spec .
Unfortunately, I still have some problems with this:

- the Vary header has not been used much by many in the past (in fact, I 
had not even heard of it before the discussion about transcoders 
started). Applying this rule would require that mobile developers change 
their applications. By definition, this is not acceptable. It must be 
the transcoder responsability to figure out that an application that 
does not use vary may still be relying on the UA string (or any other 
header) to multiserve its content.

- the fact that W3C recommends respect for the Vary header does not mean 
that transcoding proxies will implement it. In fact, I would guess that 
the overwhelming majority of transcoders out there do not observe this 
behaviour.

- requesting a page multiple times is not well seen by the mobile 
developer community, because it may interfere heavily with the business 
model of a site (billing and different metrics for example).

To be totally honest, also the Manifesto admits the re-issue of a 
request with a spoofed UA in a very specific case, but only after the 
site has been declared non-mobile. The side effect of multiple requests 
on a full-web site are typically much less serious than multiple 
requests on a mobile site.


>
>>
>> A different situation. Nokia N95. Safari browser. Site 
>> www.company.com can normally recognize my UA and serve great XHTML 
>> and graphics customzed for the N95. Spoofed UA. HTML with no-trasform 
>> is served. User can see transcoded web site, but user-experience is 
>> severely impacted in a negative way (not to mention brand 
>> recognition). The mobile site's business model is to serve banners 
>> and ads. UA spoofing effectively prevents the back-end from serving 
>> the correct ads (banners of the right size. Links to ringtones for 
>> Nokia N95). Result: the mobile sites loses revenue.
>>
> ditto

ditto

>
>>>> But there is more. No-transform is an HTTP header. As such it 
>>>> cannot easily be served when using static content (and many 
>>>> resources in a web application are left static. Not everything can 
>>>> be served by a program. Some resources such as pix, javascript and 
>>>> CSS are simply static files also in complex apps).
>>>
>>> Folks wishing to affect caching of their files have been handling 
>>> this sort of situation just fine since the web first arrived. It's 
>>> why mechanisms like .htaccess files exist.
>> I recall very clearly that the period I have followed BPWG (over one 
>> year), one of the pillars has been that it MUST be possible for 
>> developers who have no access to CGI, .htaccess, nor MIME type server 
>> configuration to create MobileOK and BP compliant pages. If this has 
>> changed, then this must have been kept very well hidden.
>> On the other hand, if this has not been changed, there is a clear 
>> lack of consistency between CTG and BP-MobOK.
>> In fact, it would be cool if someone from BPWG could comment on this.
>
> See 
> http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/081107#sec-proxy-decision-to-transform 
>
>
> In the absence of a Vary or no-transform directive (or a meta 
> HTTP-Equiv element containing Cache-Control: no-transform)

So, my understanding is that BPWG still thinks that static mobile 
content should not be  transcoded, but it leaves the final decision 
about transcoding in the hands of the heuristics that a transcoder 
vendor may have decided to adopt. Do I misunderstand something?


>>>>>> As the Verizon installation has shown, CTG can be leveraged to 
>>>>>> say that an abusive installation is W3C compliant. Prohibiting UA 
>>>>>> spoofing would make this much more difficult.
>>>
>>> I'd love to see an explanation of how prohibiting UA spoofing stops 
>>> people from claiming documents say things they don't.
>>
>> I have already explained this a few times. I'll try again.
>> W3C's recommendations are not normative for anyone. The industry 
>> typically reverts to W3C for justification of how their product works 
>> and not necessarily for guidance of how they should work. This is a 
>> direct consequence of the fact that there are no sanctions for 
>> breaking W3C specs.
>> CTG has been a clear example of this. Transcoders do not need W3C to 
>> decide how they want their product to work. They only need W3C to 
>> state that their products are "standard" for some definitions of 
>> standard. Because of this, they are perfectly OK with a spec that's 
>> "misrepresentable". Novarra, for example, have shown that they are 
>> champions at this in the Verizon case.
>> My point is that CTG should be made extra tight, so that 
>> misrepresentation becomes much harder for transcoder vendors. 
>> Preventing UA spoofing would be a good move in this case. Right now 
>> transcoder vendors can spoof the UA and still say that this is what 
>> the end user wanted, so they are still compliant to CTG thanks to 
>> 4.1.5.2
>
> See 
> http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/081107#sec-user-selection 
>
>
> especially
>
> Proxies should assume that by default users will wish to receive a 
> representation prepared by the Web site. Proxies must assess whether a 
> user's expressed preference for a restructured representation is still 
> valid if a Web site changes its choice of representations (see 4.2.6 
> Receipt of Vary HTTP Header).
>
> and 
> http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/081107#sec-administrative-arrangements 
>
>
> Proxies must provide a means for users to express preferences for 
> inhibiting content transformation. Those preferences must be 
> maintained on a user by user and Web site by Web site basis. Proxies 
> must solicit re-expression of preferences in respect of a server if 
> the server starts to indicate that it offers varying responses as 
> discussed under 4.2.6 Receipt of Vary HTTP Header.

With this one I disagree deeply. I think that the choice of a 
restructured experience cannot be left to the end user alone and should 
only be made available with the agreement of the content owner. Now, the 
agreement may be "implicit" in some way (i.e. the content owner does not 
have a mobile experience in place), but this brings us back to the point 
that UA can never be spoofed or the rights of the content owner are 
infringed.
I think this is a major source of disagreement between CTG and the 
Manifesto, and one that clearly shows that Novarra has been given way 
too much say in shaping CTG.


>>>
>>> It's a good thing we're not on one of those stalinesque mailing 
>>> lists which prohibits these sort of long discussions! :)
>> We have discussed this over and over again. As long as you  keep 
>> making believe that the problem is just the wording of the CTG and 
>> not how the mobile industry works at a more general level, I will 
>> need to keep arguing about how inadequate CTG is to the protect the 
>> mobile ecosystem.
>
> It's not the job of the MWBP WG to change the face of the industry, 
> solve the credit crunch or make leopards change their spots - it is 
> our job to represent the consensus on good practice.

I am afraid you cannot get out of this so inexpensively, Jo. As you have 
seen with Verizon, CTG is referred to by the industry and used to 
influence buying decisions and transcoder configuration decisions. Those 
decisions will influence the business model of many companies and may 
also decide whether it's a neutral mobile web OR. an operators' mobile 
web which we are dealing with here.
By creating the CT task force you and the rest of the group have decided 
to influence the industry. If this is not your intention, drop CTG. If 
you think CTG is the way to go, then be ready to face criticism for 
having failed to produce a balanced spec in the interest of the whole 
ecosystem.

Luca

Received on Monday, 22 December 2008 16:34:35 UTC