W3C home > Mailing lists > Public > public-mobileok-checker@w3.org > July 2008

On the way to v1.0: some new tests/bugs/fixes/questions

From: Francois Daoust <fd@w3.org>
Date: Sun, 13 Jul 2008 19:15:13 +0200
Message-ID: <487A3821.30701@w3.org>
To: public-mobileok-checker <public-mobileok-checker@w3.org>

Hi guys,

I spent some time testing/debugging/fixing/playing with the checker.
There were a few bugs that definitely needed to be fixed before I may 
update the online checker.

I haven't listed the bugs in Bugzilla, as I only have an intermittent 
modem-like Internet connection for the time being (this allowed me to 
find a rather amusing bug btw, see the last mentioned bug below). I 
fixed most important bugs, the remaining ones being minor enough I think 
to be able to address them later on, although some of them would 
probably better be fixed as soon as possible.

Below is the report of my investigations.

Topics:
- A few remaining questions to start with
- Main changes
- Tests added/completed
- Bugs fixed
- Bugs not fixed yet


Questions that could trigger some more bugs
-------------------------------------------
1/ Should the HTTP response returned for the resource under test be 
counted in EXTERNAL_RESOURCES? The doc seems to say it should. I undid 
the change Dom made 3 weeks ago not to count it. Is it correct? On the 
one hand, that means that we only have 9 'slots' for external resources. 
On the other hand, it seems a bit strange to count the redirections to 
the primary resource, but not the last HTTP Response as an external 
resource.

2/ Suppose imgA is an image served with caching directives that say that 
the image should not be cached. If the page defines:
   <img src="imgA" alt="imgA" />
   <img src="imgA" alt="imgA" />
... should imgA be counted twice in EXTERNAL_RESOURCES and 
PAGE_SIZE_LIMIT? If so, then I think it is not handled by the checker 
right now (minor bug I would say, but not trivial to fix).

3/ Hypothetical case:
   <img src="imgA" alt="imgA" />
   <img src="imgB" alt="imgB" />
... Let's suppose a request on imgA yields a redirect to imgB (I know it 
probably never makes any sense, but that is possible). The redirect 
should be counted in EXTERNAL_RESOURCES and PAGE_SIZE_LIMIT. But the 
HTTP response on imgB should be counted only once in EXTERNAL_RESOURCES 
and PAGE_SIZE_LIMIT. Am I correct? If so, then there is another minor 
bug here, not trivial to fix.

4/ I haven't had time to check that yet: are MIME types case-sensitive?
   <object data="imgA" type="image/gif" />
   <object data="imgA" type="IMAGE/GIF" />
(same question with Content-Type headers)
Looking at HTTPObjectResource.java, I note that the Content-Type header 
is checked in a case-insensitive way, whereas the type attribute is not.

5/ Checks on type attributes and Content-Type headers are made using 
"startsWith". I suppose one may complete the definition of the 
Content-Type with charset details and the like, but that also means 
that: "image/gifted" will match "image/gif". Was the loose check done on 
purpose?

6/ If I understand things correctly, there's a bug in the 
STYLE_SHEETS_USE-4 subtest. See below. But that may be because I didn't 
understand the definition of STYLE_SHEETS_USE-4 correctly.


Main changes
------------
- I mostly removed Dom's code added a few weeks ago in 
includedResources.xsl because, unless I forgot something, changes made 
to the moki by Abel and Miguel make it useless to have to go through the 
HTML code once again to count the external resources and compute the 
total page size (except that caching directives were and are still not 
properly taken into account, I think, see one of the questions above)
- The recursive parsing on objects elements in HTTPXHTMLResource.java 
was incorrect (some objects appeared twice and the code could crash in 
some cases).
- A small 'or' in a check where an 'and' was required made all the 
images trigger a WARN on their Content-Type, no matter if their 
Content-Type was valid.


Tests
-----
I updated the moki.xml and testresults.xml where appropriate.
I think I haven't committed one or two moki updates, but it's hard to 
play with CVS when your internet connection goes up and down all the time :(
I updated/completed the test suite to match new rules:
- MeasuresTest/6: test on CSS case-insensitivity and spaces
- MeasuresTest/7: test on CSS Level 1 properties
- StyleSheetsSupportTest/6: test on CSS case-insensitivity and spaces
- StyleSheetsUseTest/6: test on deprecated elements
- StyleSheetsUseTest/7: test on style-like elements
- StyleSheetsUseTest/8: test on unknown properties/values in CSS
- ObjectsOrScriptTest/2: updated to use GIF instead of PNG not to 
trigger side errors.
- ObjectsOrScriptTest/3: updated to use an object as fallback as well.
- ExternalResourcesTest/4: updated to use 11 times the same object, to 
ensure it counts only once.
- ExternalResourcesTest/5: use 20 different objects, where only the 
first one is rendered.
- ExternalResourcesTest/6: 10 different images + the resource under test 
should trigger a WARN.
- ExternalResourcesTest/7: 11 different images defined in the CSS Style 
should trigger a WARN.
- ExternalResourcesTest/8: check CSS stylesheets are correctly counted
- ExternalResourcesTest/9: check CSS stylesheets are correctly counted
- ExternalResourcesTest/10: 9 images among which one is defined twice + 
the resource under test should PASS
- ExternalResourcesTest/11: count only objects with no type and those 
whose content-type is "image/jpeg" or "image/gif"
- ExternalResourcesTest/12: count all objects with no type and all of 
those whose content-type is "image/jpeg" or "image/gif"
- StyleSheetsUseTest/9: test on STYLE_SHEETS_USE-4 that should apply 
globally
- StyleSheetsUseTest/10: test on STYLE_SHEETS_USE-4 that should apply 
globally
- ContentFormatSupportTest/17: test on a PNG image embedded in the CSS. 
Test on CSS spaces and case-insensitivity
- ContentFormatSupportTest/18: test on an external style sheet that is 
an XHTML page
- ContentFormatSupportTest/19: test on an external style sheet not 
served as text/css
- ContentFormatSupportTest/20: test on invalid GIF images defined as an 
img element and as an object

Tests that still fail, because the underlying bugs/questions are not fixed:
  CONTENT_FORMAT_SUPPORT 18
  CONTENT_FORMAT_SUPPORT 20
  OBJECTS_OR_SCRIPT 4
  OBJECTS_OR_SCRIPT 5
  STYLE_SHEETS_USE 9
  STYLE_SHEETS_USE 10


Bugs fixed
----------
- ContentFormatSupport: images embedded in CSS stylesheets were not 
checked anymore
   see test CONTENT_FORMAT_SUPPORT 17
   -> fixed in ContentFormatSupportTest.xsl by parsing directly new 
structures

- Measures: the following valid CSS style should trigger a FAIL on '12pt'
   "font: normal small-caps 1ex/12pt sans-serif"
   see test MEASURES 7
   -> fixed in MeasuresTest.java with a new regex

- ObjectsOrScript: check for OBJECTS_OR_SCRIPT-10 is invalid. All images 
return a WARN, no matter their Content-Type.
   -> fixed in ObjectsOrScriptTest.xsl: 'or' was used instead of 'and' 
in an if

- ObjectsOrScript: the checker crashes on ObjectsOrScriptTest/3.
   Incorrect retrieval of children objects
   -> fixed in HTTPXHTMLResource.java: getTopLevelObjects updated
   -> fixed in ContentFormatSupportTest.xsl: check only first object 
type that matches a given URI
   -> fixed in includedResources.xsl: check only first object type that 
matches a given URI

- ExternalResources: the main response to the resource under test is not 
included in the count (see ExternalResourcesTest/6)
   -> undid Dom's change in ExternalResourcesTest.xsl

- ExternalResources: CSS images are not included in the count
   see test EXTERNAL_RESOURCES 7
   -> fixed in ExternalResourcesTest.xsl and includedResources.xsl

- ExternalResources: an image that appears twice may be counted twice
   see test EXTERNAL_RESOURCES 10
   -> fixed in ExternalResourcesTest.xsl and includedResources.xsl

- ExternalResources: image/png objects defined without a type attribute 
are flagged as SKIPPED instead of TASTED
   -> fixed in HTTPObjectResource.java: markupContentType may be null. 
Checks rewritten in a more logical way, inconsistency flagged so that 
someone reading the code know it's done on purpose.

- ExternalResources: an object whose content type is image/jpeg or 
image/gif should be flagged as RENDERED, no matter if its type attribute 
says otherwise
   -> fixed in HTTPObjectResource.java: markupContentType may be null. 
Checks rewritten in a more logical way, inconsistency flagged so that 
someone reading the code know it's done on purpose.
   -> fixed in HTTPXHTMLResource.java: null is different from empty when 
parsing the type attribute

- PageSizeLimit: (same bugs as above)
   -> similar fixes applied to PageSizeLimitTest.xsl

- PageSizeLimit: stylesheets were not taken into account
   -> PageSizeLimitTest.xsl: missing '/' appended

- OBJECTS_OR_SCRIPT-10: message ends with a comma
   -> messages.properties.xml: comma removed

+ I fixed the remaining CSS case-insensitivity and spaces case in 
CSSUtils.java


New bugs
--------
- ObjectsOrScript: the checker applies some of the tests to more than 
just "Included Resources": OBJECTS_OR_SCRIPT-5 and OBJECTS_OR_SCRIPT-9 
are applied to all objects, whereas OBJECTS_OR_SCRIPT-6, 
OBJECTS_OR_SCRIPT-8, and OBJECTS_OR_SCRIPT-10 seem not to be
   see tests OBJECTS_OR_SCRIPT 4 and OBJECTS_OR_SCRIPT 5

- ImagesSpecifySize: the checker checks all images, even those that are 
not "Included Resources"
   see tests OBJECTS_OR_SCRIPT 4 and OBJECTS_OR_SCRIPT 5

- ExternalResources: caching directives are not taken into account (an 
image is counted only once, even if it's served with a no cache directive)

- StyleSheetsUse: STYLE_SHEETS_USE-4 is applied to each style element.
   see test STYLE_SHEETS_USE 9
   I interpret "If all styles are restricted to presentation media types 
other than "handheld" or "all" by means of @media at-rules, warn" as 
being global, i.e. when all styles are taken together, if there are all 
restricted to presentation media other than "handheld" or "all" by means 
of @media at-rules, warn.
   In short, the following (defined in the same page) should not trigger 
any WARN IMO, because the first style element contains some style rule 
that applies to all presentations:
    <style type="text/css">body { color:green; }</style>
    <style type="text/css">@media tv { body { color: red; } }</style>

- StyleSheetsUse: STYLE_SHEETS_USE-4 is applied on style elements 
restricted to a media type different from "handheld" or "all" va the 
media attribute.
   see test STYLE_SHEETS_USE 10
   In short, the following (stupid) code should not trigger any WARN 
IMO, because the style is already restricted via the media attribute:
    <style type="text/css" media="tv">@media tv { body { color: red; } 
}</style>

- ContentFormatSupport: the CSS parser crashes when the external 
stylesheet referenced by the page is the XHTML page
   see test CONTENT_FORMAT_SUPPORT 18

- ContentFormatSupport: there is no info on the validity of objects 
within the moki. There is thus no way to check that an image defined as 
an object is a valid GIF/JPEG image.
   see test CONTENT_FORMAT_SUPPORT 20

- Using the checker without a running Internet connection has some weird 
consequences.
   run OBJECTS_OR_SCRIPT 1 without an Internet connection.
   The code defines a dummy URI at example.org when javascript links are 
detected. I'm not sure why. What is sure is that without an 
up-and-running Internet Connection, example.org cannot be resolved, and 
this yields the following error message:
   <test name="LINK_TARGET_FORMAT" outcome="FAIL">
    <result name="LINK_TARGET_FORMAT-1" outcome="WARN">
     <info>WARN: The linked resource 
http://example.org/#javascript%3Aalert%28%27javascript%3A+scripting%27%29%3B 
is in a format ("") that may not be appropriate for a mobile device</info>
    </result>
    <result name="HTTP_RESPONSE-1" outcome="FAIL">
     <info>FAIL: The request to the resource 
http://example.org/#javascript%3Aalert%28%27javascript%3A+scripting%27%29%3B 
does not result in a valid HTTP response (because of network-level 
error, DNS resolution error, or non-HTTP response) </info>
    </result>
   </test>
   -> Can we get rid of this hack somehow?


Francois.
Received on Sunday, 13 July 2008 17:16:07 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Sunday, 13 July 2008 17:16:08 GMT