- From: Yeliz Yesilada <yesilady@cs.man.ac.uk>
- Date: Fri, 13 Feb 2009 10:45:19 +0000
- To: Francois Daoust <fd@w3.org>
- Cc: public-mobileok-checker@w3.org, Kentarou Fukuda <KENTAROU@jp.ibm.com>
Hi Francois, Please see my comments below.` On 12 Feb 2009, at 17:43, Francois Daoust wrote: > 1/ The URI of the file resources in the moki would start with > "file://", that's probably enough to identify the file vs. http/ > https case but we may need a more dedicated attribute to ease > checks in XSLT stylesheets. I suggest we try without for the time > being. OK. We can try and if gets complicated, we can look into adding an attribute. > 2/ Not to have to create a separate set of XSLT stylesheets for > files, we might need to produce a fake HTTPResponse in the moki > representation anyway, that would actually be the result of the > file retrieval. Good idea. > > 3/ I think there is a useful distinction to be made between a > subtest that can't be run because some data is missing, and a > subtest that can't be run because it doesn't need to, i.e. if there > are no objects in the page, the OBJECT_OR_SCRIPTS subtests de facto > pass. The first possibility is what we're talking about. The second > possibility may be of some use in the future (I'm not suggesting we > implement it right now). In short, I would rather keep > NOT_APPLICABLE to the second case, and use DATA_MISSING (I can't > think of a better proposal, but the idea is to point out that the > moki representation is incomplete) for checks on files. I agree. I think what Dominique suggested is a good idea: using "cannotTell". > 4/ You seem to imply that we'll just ignore entire tests as soon as > one of the subtest cannot be run. It works with caching, but some > other subtests could be quite useful (e.g. > CHARACTER_ENCODING_SUPPORT-5 that checks whether the document is > valid UTF-8 would be useful even though > CHARACTER_ENCODING_SUPPORT-1 cannot be checked). Some other > subtests could be useful as well even though they can only apply > partially (e.g. EXTERNAL_RESOURCES-2 is still worth raising when > the document targets too many external resources, even though the > total is not 100% correct without HTTP redirections). > > In short, the possible outcomes for a subtest (the <result> element > in the XML report) would be: > - PASS, WARN, FAIL for subtests that can be run normally. > - PARTIAL_PASS, PARTIAL_WARN, PARTIAL_FAIL for subtests that can > only be applied partially. > - DATA_MISSING for subtests that simply can't be run. > I don't think that creates a real complexity or requires a lot more > changes, because we'll need to update the corresponding XSLT > stylesheets and handle these cases anyway. > > (Note I had suggested to use a different attribute to flag the > results as "partial", but I think that your approach requires fewer > changes and actually helps "seeing" that the outcome is not a > definitive response). > > The possible outcomes for a test would be: > - PASS, FAIL for tests that can be completely checked > - PARTIAL_PASS, PARTIAL_FAIL when there is a PARTIAL_* and/or > DATA_MISSING in one of the subtests > - DATA_MISSING when none of the subtests could be run (e.g. for > CACHING) > > The possible overall outcomes would be: > - PASS, FAIL when all tests can be completely checked (http/https > case) > - PARTIAL_PASS, PARTIAL_FAIL where there is a PARTIAL_* and/or > DATA_MISSING in one of the tests > [- DATA_MISSING is not going to ever occur, since it would mean > none of the tests could actually be run at all] I think we need to think "why one would like to know if a *sub-test* passed partially or not?". For example, in our application if a sub- test can only be checked partially, then we have to use the Tester (URI) version to check that again so it's enough to know that the a particular sub-test cannot be tested. I am just not sure if these partial solutions would be useful or not. I would rather prefer to keep the approach simple: subtests ======= - PASS/FAIL/WARN that can run normally - CANNOTTELL if there is missing information tests ==== - PASS/FAIL/WARN that can run normally - CANNOTTELL if any of the sub-tests return "CANNOTTELL" But do you still think it's important to have PARTIAL results? > 5/ There is one specific case we need to handle carefully: when the > primary document is using an HTTP/HTTPS scheme, then we need to > return an error when one the linked resources is using the file > scheme, whereas when the primary document is using the FILE scheme, > we should process other file resources. OK. > > If we do that correctly, the final mobileOK report returned when > the Checker is given a real http/https URI will be no different to > the one that is returned today and that is good! (i.e. no > PARTIAL_blah or DATA_MISSING in the http/https case). I agree. It's good to have something that is consistent with the current approach. Regards, Yeliz.
Received on Friday, 13 February 2009 10:46:08 UTC