RE: Notes on using the CSS Mobile Profile Test Suite from a mobile device

I think I mentioned this in Boston when we met and I think actually the
major lacks of this test suite are certainly in the navigation and
description of the expected outcome. I often could not tell what the
test was supposed to do using a web browser on my Mac.

- Andrea
 

> -----Original Message-----
> From: public-mwts-request@w3.org 
> [mailto:public-mwts-request@w3.org] On Behalf Of Dominique 
> Hazael-Massieux
> Sent: Tuesday, February 27, 2007 9:54 AM
> To: public-mwts@w3.org
> Subject: Notes on using the CSS Mobile Profile Test Suite 
> from a mobile device
> 
> 
> Hi,
> 
> As per my action item, I've tried to run the CSS Mobile 
> Profile Test Suite [1] on a set of user agents/devices; I 
> have indeed run all the tests of my Treo 650 with its default 
> user agent (Blazer 4.0), as well as having run a few tests 
> cases on Opera Mini on that same device, on the Netfront 
> browser of the PSP, and on the OpenWave browser made 
> available on Sagem myX-5m.
> 
> The biggest problem of doing so at this time is the total 
> lack of usability of the test suites: the list of tests is 
> given in a big table that makes it pretty hard to use on most 
> devices, and in fact mostly impossible on the Openwave 
> browser (which sits on a much smaller configuration than the 
> other ones).
> 
> Also, to go from one test to another, one need to go back and 
> forth between the big list of tests and each individual test: 
> there is no internal navigation provided.
> 
> Beyond this, the test suite heavily relies on colors, which 
> aren't available on all devices - the mobile web best 
> practices specifically recommend not relying on colors [2]. 
> Should I have been able to run the tests suite on the 
> openwave browser, I couldn't have told any useful results 
> since it doesn't support colors on that specific device. Even 
> on devices/browsers that support colors, some choices of 
> color make it hard to distinguish whether the results matches 
> what was expected (e.g. some dark dark blue in 526-01 that is 
> hard to distinguish from black).
> 
> Some tests assume a fairly wide screen, especially the ones for float:
> and other layout properties, and make it hard or impossible 
> to assess whether the results is a pass or not.
> 
> The tests that rely on some specific fonts set up are also 
> fragile, given that most devices provide only a limited sets of fonts.
> 
> I also noticed some more casual bugs or lack of clarity in 
> the description of the expected results.
> 
> Till and Carmelo have also volunteered to provide feedback on 
> that test suite; I propose that once they have sent their own 
> feedback, we should look at ways in which these problems 
> could be solved (if they can), and send feedback to the CSS WG.
> 
> Dom
> 
> 1. http://www.w3.org/Style/CSS/Test/Mobile/1.0/current/
> 2. http://www.w3.org/TR/2006/PR-mobile-bp-20061102/#iddiv3126714184
> 
> 
> 
>

Received on Tuesday, 27 February 2007 11:29:00 UTC