Re: CSS Spec Annotations - Design Feedback Wanted

On May 22, 2011, at 2:44 AM, Patrick Garies wrote:

> On 5/21/2011 6:01 PM, Linss, Peter wrote:
>> If you have feedback, better design ideas for the annotations, or
>> ideas for different data to see in the annotations, please let me
>> know either directly or on the list. Any help would be appreciated.
> 
> Annotations Feedback:
> 
> * The annotations should note the results for HTML 4 and XHTML 1.1 
> separately. They should also note whether testing is needed for these 
> separately; if tests have been done via HTML 4, it's not evident that 
> testing still needs to be done for XHTML 1.1.

In general, we aggregate all results per test case, regardless of the format is was tested in (we do store the test format used per result so this could be computed).

My question is, is there really a need to track testing coverage per format? 

In general we're testing CSS features, not HTML or XHTML except to the extent that it interacts with CSS. For those cases where CSS is expected to interact differently, there should be HTML only or XHTML only tests. I accept that there could be an odd browser bug in HTML vs XHTML parsing but we really don't that all that much.

For those tests that are format agnostic, the only reason we present both formats is to support clients that only support on format or the other. Getting 100% coverage for each format hasn't really been a goal for us, do you really see a need for that?

As the test harness is meant to be used for other working groups, I accept that other groups may have different needs here (the harness can handle an arbitrary number of test formats). Maybe if the harness reports both the aggregated results and results per format to the annotator, then the annotator can have some UI to select all or a specific format.

> 
> * It would help if the annotations could also discriminate between 
> browser versions and release types. I don't see how the goal "quickly 
> convey to spec readers when a feature is ready to be used" is met if 
> Internet Explorer 10 Platform Preview 1 is given the same weight as 
> Internet Explorer 9 for the purpose of judging Trident support. Not 
> really sure how you would address this.

Understood, we've been debating to report results based on browser or engine. For WG needs per engine makes more sense, but it does depend on the audience. Again, I can have the harness report results both per engine and per browser (and version) and add some UI to the annotations to select the view.

> 
> * The "close all annotations" behavior of the "X" button isn't obvious. 
> I would presume that it'd just close that one annotation box. A "Close 
> All" or "Hide Annotations" button would be more obvious or the button 
> could just close that one box.

Ok. All the UI is experimental at the moment. But you're right, the "close all" should be distinctive. I've also been debating whether or not to have the collapse UI on each annotation and a "collapse all" button (or maybe a shift-click on any does the "all" function...), also if the collapsed state should collapse the summary and hide the others, or just collapse them all. Thoughts?

> 
> * If there are four boxes in an annotations box, the fourth box wraps in 
> an ugly way. On my screen "WebKit" wraps to the next line, has no border 
> on the left side, and the the WebKit box overlaps the "Presto" box above it.

Browser bug that I've seen occasionally myself, what browser were you using? I wasn't going to spend to much time on it unless we decided to keep this visual design, but it'd be good to be aware of it. (Actually, we should make sure that case is covered in the test suite :-) )

> 
> Other Feedback:
> 
> * It would be nice if there was a direct way to test both HTML and XHTML 
> at the same time.

The harness does present both formats on the test page and lets you switch back and forth. Once you make a choice it's sticky. But I'm guessing you want to be able to enter results for each (or all) formats in the same pass through the tests. This also has implications on the test sequencing if you're trying to get full coverage on each format. Again, what's the need?

> 
> * 
> http://test.csswg.org/suites/css2.1/nightly-unstable/html4/c21-pseud-link-002.htm 
> is invalid due to an apparent XHTML-to-HTML conversion issue.

Yeah, known issue in the build system. It's on my to do list, just not at the top right now.

Thanks for the input,

Peter

Received on Sunday, 22 May 2011 17:51:34 UTC