Re: how is exactly the global score calculated?

Hi Carlos,

Apologies for the slow response, I'm just back from vacation.


Carlos wrote:
> How is exactly the global score calculated?
> 
> Can this score be got using the Java lib?

I'll start with a tiny bit of context: the mobileOK Checker at 
http://validator.w3.org/mobile is divided into two modules:
  1. the mobileOK Checker Java lib (maintained by the Mobile Web Best 
Practices Working Group)
  2. the User Interface wrapper (maintained by W3C Team)

The role of the mobileOK Checker Java lib is to be a reference 
implementation of the W3C mobileOK Basic Tests 1.0 [1] standard. Things 
that are not defined in this standard are unlikely to ever make it to 
the Java lib, simply because that's not its role.

The global score returned by the user interface is not defined by the 
standard, and I don't quite expect it will ever make it to the Java library.

The main reason why we decided to add a global score to the interface is 
that we felt the OK/not-OK binary result returned by other W3C 
validators is not rewarding: the result doesn't change unless you fix 
all the errors. See for instance Cleaning Up the Web [2] from Tim 
Berners Lee for more details on the underlying idea. With a global score 
out of 100, each time you fix something, the score improves, and you get 
a sense of achievement.

The global score is thus here to motivate people reading the report to 
improve the level of mobile-friendliness of their content. The score 
carries two somewhat contradictory notions:
  1. the level of mobile-friendliness of the content (i.e. can the 
content be rendered well on most mobile devices?)
  2. the amount of work needed to turn the content into something that 
really works well on most mobile devices.

They are a bit contradictory because, for instance, using tables for 
layout is a real problem with mobile devices (so the mobile-friendly 
score should not exceed, say, 50) but is probably easy to fix (so the 
amount-of-work score should not go below, say, 70).

The global score is thus computed as a balance between the two notions. 
It is not perfect, and is likely to evolve a bit in the future. Basically:
- we assigned a severity level from 1 to 5 to each one of the subtests 
that may fail
- the number of times a subtest fails increases its severity
- for a couple of subtests (page size, number of external resources), 
the severity depends on whether the subtest was failed by just a little 
or by more than just a little (i.e. a 25Kb page size is less of a 
problem than a 500Kb page size)

The UI wrapper is a Java servlet and the code is publicly available as well:
  http://dev.w3.org/cvsweb/2009/mobileok-servlet/

In particular, the score is computed using XSL stylesheets defined in:
http://dev.w3.org/cvsweb/2009/mobileok-servlet/src/org/w3c/mwi/mobileok/basic/servlet/xslt/

The (failed) score is returned by the "moki:compute-failed-score" 
function defined in common.xsl.

Please note that this score is not directly computed from the report 
returned by the mobileOK checker, but from a slightly different view of 
it computed by the completeResult.xsl stylesheet.

Hope this helps,
Francois.

[1] http://www.w3.org/TR/mobileOK-basic10-tests/
[2] http://www.w3.org/2008/10/22-cleaning-tbl.html

Received on Monday, 2 March 2009 09:10:04 UTC