Defining tests for a new checker

Hi,

As part of the work on updating the mobileOK checker [1], I've looked at
the various tests that the current checker executes and tried to assess
whether we should keep them, change them, or drop them.

In that analysis, I've chosen to ignore the fail vs warning aspects of
the tests; my current thinking is that the new checker wouldn't try to
identify failures per se, but would instead point toward possible
improvements.

I've also started to look at what new tests we would want to bring to
the checker.

I've pushed both of these to the wiki:
http://www.w3.org/wiki/Mobile_checker#New_tests_for_updated_checker
http://www.w3.org/wiki/Mobile_checker#Analysis_of_existing_mobileOK_tests

I include them below to make it easier for commenting/reacting.

New tests for updated checker 
      * viewport: is there a meta viewport declaration? if not, explain
        that page will be displayed on a ~850px wide viewport
      * media queries: are there media queries? what are the
        "breakpoints"? if no media queries, explain how they can be
        useful
      * single column layout? (not quite sure what/how to test)
      * time to load: identify bottlenecks, suggest improvements
      * compression: if not used, explain how and why
      * appcache: detect if app cache is used; if it is, look if the
        resources linked in there exists; if it isn't, explain why and
        how to use it
      * mouse vs touch: detect if JavaScript uses non-touch compatible
        mouse events (e.g. mouseover), and doesn't have touch events
        eitehr
      * new input types, datalist: try to detect with heuristics if some
        input types would deserve the new HTML5 types; suggest datalist?
      * surface for links (CSS): try to determine the hit surface of
        links, and how well separated links are; if surface too small,
        explain difficult of touch based interaction; if links too
        close, likewise
      * detect unused JS libraries: (e.g. HTML5 boilerplate with
        libraries-pre loaded but script not doing anything)
      * cookies cost: look if cookies are sent for presumably static
        resources (e.g. images, javascript, css)
      * page reflows/repaints: detect how many reflows/repaints are
        needed to display the page, and suggest possible improvements
      * network usage after page load: detect network usage after page
        load and note issues with mobile network stack
      * JS UA sniffing: if window.userAgent is used, warn about
        brittleness of ua-sniffing on client
      * incomplete prefixing: detect if CSS uses prefixes for one
        rendering engine when the properties exist in others
      * image munshing: detect embedded thumbnail in JPEG, embedded
        EXIF, etc
      *  :hover: not usable on touch based interfaces
      * <label> for input: detect if radio buttons/checkboxes are within
        a label tag (make them much easier to hit)
Analysis of existing mobileOK tests 
The following sections describe the messages currently given by the
checker for its various tests, and classifies them in 4 sections:

      * Keep: we should keep the tests, they're important
      * Keep, lower priority: we should keep the tests if it's easy, but
        they shouldn't be in our priority list
      * Drop: these tests don't provide relevant information
      * Modify: these tests highlight a relevant aspect, but they need
        to be adapted to current Web usage
Keep 
      * "Cache-Control" HTTP header is present and contains value
        "no-cache", or contains value "max-age=0"
      * The document is served without caching information ("Expires" or
        "Cache-Control" header)
      * The document's caching information is not provided in the HTTP
        headers
      * The "Pragma" header contains the value "no-cache" 
      * The "Expires" header contains a date in the past
      * A cache related header or a meta http-equiv value
        (PLACEHOLDER_1) contains an invalid value (PLACEHOLDER_2)
      * There is an error in the "Last-Modified" header 
      * There is an error in the "ETag" header
      * There are nested tables
      * Table contains only one row
      * Table contains only one column
      * A "javascript:" link is used
      * The alt attribute is missing
      * The height or width specified is greater than the corresponding
        dimension of the image
      * The height or width specified is less than the corresponding
        dimension of the image
      * A pop-up was detected
      * More than 25% of the page characters are extraneous (whitespaces
        or comments)
      * More than 10% of the page characters are extraneous (whitespaces
        or comments)
Keep, lower priority 
      * The document uses an applet
      * The document uses a markup-based redirection
      * The document uses an auto-refresh
      * There is no type attribute in <object>
      * A table cell contains only an image whose actual dimensions are
        2x2 or less
      * The Internet media type of the retrieved resource, as indicated
        by its Content-Type HTTP header does not match that stated in
        the type attribute
      * An input element with type attribute set to "image" is present
      * A usemap attribute is present
      * An ismap attribute is present
      * The character encoding is specified in more than one way, and
        not all values are the same
      * The image served as PLACEHOLDER_1 does not match that format
      * The CSS style sheet is served as "PLACEHOLDER_1" instead of
        "text/css"
      * The PLACEHOLDER_1 CSS stylesheet is not syntactically valid CSS
      * The document is not valid to its stated doctype: PLACEHOLDER_1
      * There is a bigger than 2 pixels transparent image
      * There is more than one input element in this group set to
        "checked"
      * There is more than one option selected by default
      * There is no page title
Drop 
      * The document uses a Refresh HTTP header to redirect to a
        different page
      * The document is served with a Refresh HTTP header
      * A Table element exists
      * The document contains one of basefont, bdo, center, del, dir,
        font, ins, menu, s, strike or u elements
      * The documents uses one of b, big, i, small, sub, sup or tt
        elements
      * The style attribute is used
      * All styles are restricted to CSS media types other than
        "handheld" or "all"
      * The CSS Style contains at-rules (other than the @media at-rule,
        and the presentation media type list of the @import at-rule),
        properties, or values that are not recognized as being specified
        in CSS Level 1, or the value of a recognized CSS Level 1
        property is incompatible with the property: (PLACEHOLDER_1) 
      * The CSS Style sheet contains rules referencing the position,
        display or float properties
      * The document uses scripting
      * An element uses an event attribute
      * The object element is empty
      * The content of the object element consists only of white space
      * The Internet media type indicated by the Content-Type HTTP
        Header of the retrieved resource is not "image/jpeg" or
        "image/gif"
      * The alt attribute consists only of white space
      * The height or width is not specified in pixels
      * This resource is served as "PLACEHOLDER_1" but does not specify
        UTF-8 as character encoding (PLACEHOLDER_2)
      * The document is served as "PLACEHOLDER_1", instead of one of
        "application/xhtml+xml", "application/vnd.wap.xhtml+xml", or
        "text/html"
      * The document is served as "PLACEHOLDER_1" instead of the
        recommended "application/xhtml+xml"
      * The document is not an HTML document (root element is
        PLACEHOLDER_1)
      * The root html element does not declare its namespace
      * The document does not validate against the XHTML Basic 1.1 DTD
        nor MP 1.2: PLACEHOLDER_1
      * The document uses a non-XML doctype (PLACEHOLDER_1)
      * The document uses an XHTML doctype that is not a well-known
        mobile-friendly doctype (PLACEHOLDER_1)
      * There is a small fully transparent image
      * The inputmode attribute is invalid
      * There are more than one fully transparent images
      * There is no inputmode attribute on this text entry element
      * The inputmode attribute is invalid
      * There is no inputmode attribute on this text entry element
      * There is no default selected option element
      * The linked resource PLACEHOLDER_2 is in a format (PLACEHOLDER_1)
        that may not be appropriate for a mobile device
      * The linked resource PLACEHOLDER_2 is served with a character
        encoding (PLACEHOLDER_1) that may not be appropriate for a
        mobile device
      * The internal reference PLACEHOLDER_1 has no matching target
      * The HTTP status (in response to the resource PLACEHOLDER_1)
        indicates redirection (status code 3xx) and the URI identified
        by the HTTP Location header is a relative URI 
      * The HTTP status code is 404 or 5xx (in response to the resource
        PLACEHOLDER_1) and the response relates to a request for a
        linked resource 
      * A matching HTTP response header does not exist for a meta
        http-equiv=PLACEHOLDER_1
      * A matching HTTP response header (PLACEHOLDER_1) exists but its
        value differs from the content attribute value
Modify 
      * The size of the document's markup (PLACEHOLDER_1) exceeds 10
        kilobytes

=> larger size

      * The total size of the page (PLACEHOLDER_1) exceeds 20 kilobytes
        (PLACEHOLDER_2)

=> Larger size

      * None of the nested object elements are image/gif or image/jpeg
        and the innermost nested object element is non-empty and does
        not consist of text an img element that refers to a image/gif or
        image/jpeg image

=> More accepted types

      * The document contains a frame, frameset or iframe element

=> Don't mind iframe

      * The HTTP Content-Type header specifies a character encoding
        other than UTF-8

=> check encoding is valid; maybe suggest utf-8 as better option

      * The HTTP Content-Type header does not specify a character
        encoding but it specifies an Internet media type starting with
        "text/" and there is no meta element with http-equiv attribute
        that specifies UTF-8 character encoding

=> make sure there is a declared encoding

      * The document is not valid UTF-8

=> (but done by another tool?)

      * The document does not contain a DOCTYPE declaration

=> Warn on quirks mode? Defer to HTML5 validation?

      * The embedded image or object is not of type image/gif or
        image/jpeg (PLACEHOLDER_1)

=> larger set of formats

      * There are more than 20 embedded external resources
      * There are more than 10 embedded external resources

=> tweak max #?


Dom

1. http://www.w3.org/wiki/Mobile_checker

Received on Friday, 20 April 2012 09:18:24 UTC