On Thu 4/26/2007 1:37 PM Sean Fraser wrote:

>The survey I propose would - simply - have Pass/Fail criteria; HTML5 would include error numbers and
>issues.

>I had perhaps mistakenly assumed that the reason we were concerned about methodology here was
>in relation to the question of

>"how much web will break if we move to a new spec?"

How much web breakage _is_ an important consideration. Though, it hasn't been defined.

>If that is the case, than determining what proportion of top 200 sites validate according to HTML4 and CSS2 specs is not so >important,

Here, I disagree. Personally, I would like to know the percentage of _valid_ Top 200 Sites. I would want to have sufficient data  that will assist in making a determination of possible web breakage, e.g., if 90% of the top sites have existing non-validating HTML code - without HTML5 involvement - I cannot see HTML5 _breaking_ that 90% of the web.

The remaining 10% may or may not break if they should use HTML5; I would be more interested in that 10% knowing that their code was once valid. It would be simpler to isolate the types of errors caused by implementation of the spec (if we make the assumption that they continue to code conscientiously). Errors on those sites may be from confusion caused by the spec's language; or, examples; or, contradicting examples and language; or, a UA bug.

Existing non-HTML5 site data is important. If historical data may be used, web developers will continue to produce non-validating HTML including HTML5. Further, if 90% of the current Top 200 sites on the web are _broken_ but benefit from graceful error-recovery, I would guess that HTML5 near-future error-recovery would need to continue as business as usual in order to meet backwards-compatibility.

>and other methodologies including which sites are sampled become important for the sake of generalizability from sample to >population. That's what I was trying to chime in about.

I understood that.

>Instead it is possible that we are discussing this survey in relationship to the "candidate requirement: market threshold".

That's another consideration.

Please send me the links to those fringe sites you mentioned earlier.

--
Sean Fraser
http://www.elementary-group-standards.com