Re: Submit an Implementation report based on a nightly-unstable test suite: possible?

On Aug 16, 2015, at 2:16 PM, Gérard Talbot <css21testsuite@gtalbot.org> wrote:

> Le 2015-08-15 21:06, Linss, Peter a écrit :
>> On Aug 14, 2015, at 8:35 PM, Gérard Talbot <css21testsuite@gtalbot.org> wrote:
>>> Peter,
>>> I've read
>>> http://lists.w3.org/Archives/Public/public-css-testsuite/2010Aug/0020.html
>>> https://wiki.csswg.org/test/implementation-report
>>> I would be ready and interested to enter test results of Prince 10 rev 3 for the CSS3 Writing Modes test suite *nightly-unstable* build and submit an implementation report.
>>> 1- Is this possible?
>> Yes.
>>> 2- If so, then what would be the DATESTAMP:
>>> # http://test.csswg.org/suites/css-writing-modes-3_dev/DATESTAMP/ <http://test.csswg.org/suites/css-writing-modes-3_dev/DATESTAMP/>
>> That field isn’t really needed anymore. It was intended to be the
>> date/time (in ISO yyyy-mm-dd hh:mm:ss format) of the time the results
>> were recorded.
> 
> Isn't that DATESTAMP field associated with the alpha or beta or CR status of the test suite?
> 
> # http://test.csswg.org/suites/css2.1/20100917/
> is RC1
> 
> while
> 
> # http://test.csswg.org/suites/css2.1/20110323/
> is RC6

Sorry, you're correct, that datestamp was for the release of the test suite (this was only used back when we were trying to ship 2.1)

> If I submit an implementation report for Prince 10 rev 4, then should I remove that DATESTAMP line entirely or just leave it as
> 
> # http://test.csswg.org/suites/css-writing-modes-3_dev/DATESTAMP/ <http://test.csswg.org/suites/css-writing-modes-3_dev/DATESTAMP/>
> 
> ?

The line is just seen as a comment and ignored by the current import script, so it doesn't matter at all.

> 
>>> 3-
>>> When a test line has many characters in its 2nd (middle) column (in the column between testname and result), what does it mean?
>>> eg
>>> html/abs-pos-non-replaced-icb-vlr-003.htm	d3088b2113ff9dc88028dc89b4d030a83db59a2b	?
>>> xhtml1/abs-pos-non-replaced-icb-vlr-003.xht	d3088b2113ff9dc88028dc89b4d030a83db59a2b	?
>>> html/abs-pos-non-replaced-icb-vlr-005.htm	b959f5e9a4d09adc6784d74741b43ff0ac1ea0ca	?
>>> xhtml1/abs-pos-non-replaced-icb-vlr-005.xht	b959f5e9a4d09adc6784d74741b43ff0ac1ea0ca	?
>> The second column is the changeset ID that the test was last modified
>> in. Just leave it in place, it let’s the test harness associate the
>> result with the exact revision of the test.
> 
> Okay... it makes sense.
> 
>> There are two ways that you can submit tests for Prince, you can
>> either generate an implementation report using the template that
>> you’ve found, or you can set the proper user agent string in the
>> harness and just run the tests.
>> People with appropriate permissions (you are one of them) can upload
>> the implementation reports yourself directly into the test harness. On
>> the “Run Tests” page first click the “Change” link at the top pf the
>> page to change the user agent to the one you’re entering test for (if
>> it’s not the browser you’re using), then click the “Batch Upload”
>> button on the “Run Tests” page. Enter the date/time and upload the
>> file.
>> When you set the user agent, you can either pick the user agent you
>> want from the list (if it’s present), or you can enter the entire user
>> agent string of the agent. For Prince it should be something like:
>> Prince/10.0 (http://www.princexml.com; Linux i686)
> 
> The 'Linux i686' string is not fetchable... otherwise I do not know how you got it.

I got that string by looking up old ones in the test harness and just replacing the version number. I'm not sure where they originally came from, I probably just synthesized them so that the harness would understand them.

> 
> I have tried to query all possible navigator attributes for Prince and never got the complete infor what you indicate here:
> 
> Prince can not handle the ListAllNavigatorAttributesAndMethods() function I have in
> 
> http://www.gtalbot.org/DHTMLSection/ListAllAttributesAndMethodsOfObjects.html
> 
> So, I created
> 
> http://www.gtalbot.org/BrowserBugsSection/CSS3WritingModes/query-navigator-attributes.html
> 
> and I use in command line
> 
> ~$ prince --javascript query-navigator-attributes.html
> 
> and I still do not get os and cpu attributes or build number which would be useful.
> 
> I guess
> 
> ~$ prince --version
> 
> info is sufficient.

Yeah, just give the best info you can find. If you make it look like a "normal" UA string then the harness can parse it to get the version and OS info. So just enter something that looks like what I listed but put in the best version info you have.

> 
> 
>> Alternatively you can just email the implementation report to me and
>> I’ll upload it for you.
>> Peter
> 
> Peter, thank you for your complete response. I appreciate this.
> 
> Prince 10 rev 4 does not support 'text-orientation', it does not support 'text-combine-upright' and it only support 'writing-mode: vertical-rl' . So, the number of tests with useful+revealing test results or worth testing for Prince is about 150 tests.
> 
> Right now, the first 6 lines of the implementation report I intend to submit look like this:
> 
> # Prince 10 rev 4 Kubuntu Linux 14.04.3 LTS
> # Prince/10 (www.princexml.com)
> # http://test.csswg.org/suites/css-writing-modes-3_dev/DATESTAMP/ <http://test.csswg.org/suites/css-writing-modes-3_dev/DATESTAMP/>
> testname	revision	result	comment
> html/abs-pos-non-replaced-icb-vlr-003.htm	d3088b2113ff9dc88028dc89b4d030a83db59a2b	na	(vertical-lr not implemented)
> xhtml1/abs-pos-non-replaced-icb-vlr-003.xht	d3088b2113ff9dc88028dc89b4d030a83db59a2b	na	(vertical-lr not implemented)
> ...

You don't have to have entries for every test in the report, just whatever you have results for. Any line with a "na" result will just be ignored anyway.
Peter