Re: transcoders bad

Thanks Francois, I appreciate it. If you'll indulge me, I'd like to 
present the most articulate and complete presentation of the points I 
would like to make if I had anywhere near the grasp on the issues as E 
Casais, whose post on the WMLProgramming list last night perfectly 
represents my views.

If you're going to record any comments at all, I urge you to record this 
one below.

Best,
Terren


E Casais wrote:

Maybe I can have my turn at trying to clear things up?

 >What I personally don't understand, and I think the people behind this
 >document, is why this leads to conclusions like "don't change
 >User-Agent". That does not have to do with telling a transcoder to
 >leave you alone.

Well, follow the reasoning:

1. What is relevant is not "content", understood as some kind of
generic markup that is delivered over a neutral communication link
to a terminal; what is in place is a content delivery chain where
clients communicate their capabilities to the server, which uses
this information to deliver the most appropriate service. These
capabilities are communicated via HTTP header fields, and this has
been the case since the inception of the WWW. Thus, all elements
are relevant: content, communication link and end-point capabilities.
2. The mobile Web comprises a large number of established sites
that have been delivering content specially optimized for mobile
devices -- WAP 1 capable, WAP 2 capable, i-Mode capable, and also
WWW capable ones -- sometimes for many years. In that environment,
information from HTTP fields such as user-agent is essential
(device detection), as well as accept (supported MIME types),
accept-charset (supported encodings), and x-wap-profile (device
capabilities), plus other ones (e.g. cookies) depending on the
application.
3. The fact: transcoders deployed in the field substitute the
user-agent field, eliminate the x-wap-profile, change accept
and accept-charset to unrecognizable ones, and modify POST
requests into GET ones. The consequence: the delivery chain
no longer works. Mobile Web sites can no longer recognize the
clients and deliver mobile-optimized content to them (or any
content at all in some situations).
4. The CTG intent is to describe a (lightweight) mechanism for
Web sites to communicate to transformation proxies possibly present
between server and client that the content provisioning chain is
not to be disturbed, so that mobile optimized content can be
delivered unhindered. The proposal consists essentially in relying
upon the directive no-transform.
5. Once the no-transform directive has been sent, the provisioning
chain is supposed to be left untouched. According to the CTG (and
the discussions in the list), this is not the case: even after
receiving such a directive, the proxies can modify HTTP fields
at will. Concretely: proxies never get out of the way in the
content delivery chain, and never behave as transparent proxies.
The CTG indicates that original HTTP header fields can be found
in alternative X-fields.
6. The fact: current proxies are known to disturb the content
delivery chain in other important ways not dealt with by the CTG
(e.g. POST vs GET requests). Besides, current proxies perform
egregious content transformations, such as inserting extraneous
advertisements and content, despite origin servers loudly
signalling that they are delivering mobile-optimized content
that is not to be disturbed (via special URL, no-transform
directives, MIME types, DOCTYPE declarations, copyright meta
tags, all together). The effect of transformation across
character encodings is a question mark (remember that proxies
do actually change the accept-charset field). Finally, there
can be cascaded proxies cumulating their effect on the content
delivery chain.

This is a description of the situation. Now come the points of
disagreement.

Position of mobile Web developers:
a) There is no justification for the situation in (5); since
proxies are not supposed to play a role in the delivery
chain of original mobile-Web services, why should they be
officially allowed to disrupt it even when explicitly told
not to -- and have this stamped as a "best practice"?
b) Transformation proxies are supposed to adapt full WWW
content for non-full-WWW terminals. Why should deployed
mobile Web applications that have nothing to do with the
full-WWW be impacted and be retrofitted to proxies by
changing the way they deal with the content delivery chain,
especially HTTP header fields?
c) The CTG is silent on the aspects listed in (6). How
could such a fragmentary guideline help when it leaves many
avenues open to disable the content delivery in the mobile
Web?
d) For the past year, there has been a wide deployment of
transcoders purportedly to enhance the browsing experience
of mobile users by enabling access to the full WWW. The
actual consequence has been a dramatic degradation of the
experience in the existing mobile optimized Web. Why should
we condone officially some of the dubious practices of
these transcoding firms, thus granting them a precedent
for further deviant behaviour (streaming? Java? Flash?
Synchronization?)

 From the, ahem, lively debate going on in the W3C and
WMLProgramming lists, mobile developers interpret the
attitude of the W3C and the resulting Draft CTG as follows:
a) "We do not have any justification for this inconsistency,
but every proxy is doing it, so it is not a problem.  Why don't
you develop for the full WWW and rely upon proxies so that you
do not have to deal with those pesky HTTP fields?"
b) "It doesn't matter. Access to the full WWW is paramount. The
mobile Web must adapt to the proxies, ideally proxies should
not have to bother about the mobile Web. Why don't you just
develop for the full WWW and let proxies do their work?".
c) "We have not thought about it, and dealing with this makes
the life of proxies too complicated. Why don't you just
develop for the full WWW to avoid these issues?".
d) "The W3C is the leading organization defining the future
of the WWW, so we have to follow, and especially follow the
quirks a few proxy players have just started to do, not what
the mobile application industry at large has been doing for a
decade or what we would reasonably plan to do in a standards
body. If you just developed for the full WWW you would
understand the benefits of this approach".

That is formulated a bit sharply, but until the W3C puts
out a convincing and logical answer to a, b, c, and d,
this will remain the predominant feeling -- and I do not
see how the positions could then come closer.

 >Or why it leads to conclusions like "don't transcode
 >https".

This is another discussion, the conclusion of thoughts
about security, usability and what it means to have
end-to-end secure connections. But the basic logic applies:
if proxies are told to leave the content delivery chain
untouched, via a no-transform for instance, why should
they still insist in manipulating it?

 >1) Transcoders provide a lot of value too.
 >I personally love them since I don't have a fancy phone.

They do, but only under two conditions:
1. When accessing the full-WWW from non-full-WWW devices;
2. And when not disrupting mobile-Web content provisioning.

I have not tried recent transcoders, so I cannot comment on
the extent of the value they provide with respect to (1).
However, I have experienced what they do in (2), and
unfortunately, in their present instantiations, they do
not add any value to the existing mobile Web; rather, they
destroy it. It is just a fact, the sad reality of the
transcoding world for over a year now.

 >I would not want to be shut out of 99.9% of the web while
 >waiting for those great mobile developers to port the whole
 >web to XHTML, lovely as that would be. Without a doubt

This is another discussion, but this kind of view of the
mobile Web lost most of its relevance several years ago.
Mobile Web and Full WWW address different needs; the
applications are different, they are not really replicas
(scaled up or down) from each other. Think about this: how
many wallpapers or ringing tones or GPS mapping downloading
sites for PC are there? How many 2-D "barcodes" scanning and
browsing applications on PC? How many electronic tram tickets
do you order and pay with your PC? Even "sibling" mobile/fixed
applications from the same company are different: for instance,
mobile versions of social sites have capabilities optimized
to take advantage of direct feeds of mobile-originated
photos and videos, whereas media uploading takes place
with different modalities on the PC version.

 >I agree there are "good" and "bad" behaviors for transcoders,
 >and we should talk about that.

The broadsides you receive from some quarters has much to do
with the fact that they want you (i.e. W3C) to act, not to
talk, that they consider the draft CTG as not taking a firm
stand with respect to these practices, that they view the
guidelines as weak, and that they feel the whole affair
revolves around rubber-stamping the bad practices of some
players instead of attempting to push things in the right direction.

 >That is not the extreme conclusion that transcoders are bad.

I agree in theory. In practice, the current crop of
transcoders is bad, not for the full-WWW, but for the
mobile Web (see conditions 1 and 2 above).

 >2) Transcoders exist, dude. Sorry. They're not going away.

I do not mind about this personally. What I am concerned
about is giving a W3C stamp of approval to obnoxious practices
that destroy the value of the existing mobile Web infrastructure,
or proposing a document so vague that it implicitly allows
such misbehaviour.


E. Casais



Francois Daoust wrote:
> Hi Terren,
> 
> Just a quick message to let you know that I recorded your comment in our 
> Tracking System.
> 
> The Mobile Web Best Practices Working Group will address it as soon as 
> possible and get back to you with some official proposed resolution.
> 
> In the meantime, please note that the discussion it triggers does not 
> reflect the position of the Mobile Web Best Practices Working Group or 
> of the W3C.
> 
> Thanks,
> 
> Francois Daoust,
> W3C Staff Contact,
> Mobile Web Best Practices Working Group.
> 
> 
> Terren Suydam wrote:
>>
>> Hello,
>>
>> As the technical lead for SingleClick Systems mobile development, I'm 
>> writing to protest the W3C's failure to provide a clear rule against 
>> the modification of the User-Agent header.
>>
>> As mobile developers, my team spends a lot of time creating the mobile 
>> experience *we* want our users to see. If our users are subjected to 
>> the confusing experience of transcoding, we lose money.
>>
>> I urge the W3C to adopt the standards set forth by Luca Passani's 
>> Manifesto, of which I'm sure you're aware.
>>
>> Sincerely,
>> Terren Suydam
>>
>>
>>
>

Received on Wednesday, 6 August 2008 12:25:07 UTC