Re: HTML5 RDFa Test Suite (was Re: Request to publish HTML+RDFa (draft 3) as FPWD)

On Sep 22, 2009, at 8:05 AM, Manu Sporny wrote:

> (bcc: RDFa Task Force Mailing List)
>
> Sam Ruby wrote:
>> My conclusion is that defining RDFa in HTML in terms of a DOM or an
>> Infoset are but two of the possible ways of achieving the desired
>> result, namely being precise as to what triples MUST be produced  
>> from a
>> given input.
>
> Those reading this thread should also keep in mind that we not only  
> have
> a spec to describe the processing model, but we also have a large,
> modular test suite that exercises every feature, as well as many
> potential error conditions, for RDFa processors:
>
> http://rdfa.digitalbazaar.com/test-suite/
>
> This test suite was upgraded this past weekend to cover HTML5 and
> contains 127 unit tests specifically for HTML5. There are also at  
> least
> three implementations that are close to 100% conformant with HTML+RDFa
> (the PyRDFa processor, the MarkLogic processor, and the librdfa  
> processor).

I don't think this demonstrates that the spec is unambiguous. Can you  
relate every test case back to normative conformance requirements in  
the spec? Can you relate every implementation conformance requirement  
back to a test in the test suite?

>
> There are also 7 XHTML+RDFa processors implemented in non-browser
> technologies and 2 XHTML+RDFa processors implemented in Javascript in
> the browser.

Given the serious errors in the specification for prefix mapping in  
the XHTML+RDFa spec, it's pretty clear that in that case, no one was  
coding to just the language of the spec. They either used out of band  
information about what it's really supposed to mean, or coded to the  
test suite instead of to the spec. Others (e.g. Phillip Taylor) have  
pointed out errors in the test suite itself where tests don't match  
what the spec requires, but many implementations pass the tests. This  
would seem to indicate coding to the test suite instead of to the  
spec. Or conversely, it may indicate that the test suite was written  
to match what implementations do, instead of testing what the spec  
requires.

>
> This "the RDFa spec is not precise enough" discussion should take  
> these
> facts into account, as we do have many conforming implementations by a
> diverse group of implementers. This didn't just happen by accident and
> all of these implementers were able to implement the RDFa spec.  
> Three of
> those implementations worked with the majority of the HTML5 tests
> without further modification.

Then let's make the spec accurately and precisely state the  
requirements tested by the test suite. And when people raise confusing  
edge cases, let's turn them into tests and make sure the spec  
accurately reflects the desired behavior. For example, how do  
implementations behave for Jonas's <a> inside a <table> example? How  
do browser-hosted implementations behave when xmlns: attributes in the  
DOM have been modified after parsing? Can that behavior be derived  
from the spec?

Making the spec more precise is a good thing. And it's not hard. I  
even gave examples of some more precise language to use in particular  
cases. I think the abstract tree model of XHTML+RDFa is actually a  
reasonable starting point, and HTML+RDFa just needs to make a few  
adjustments and explain how that tree model maps to HTML5 concepts. I  
don't think it is very convincing to say more precision is not  
required. It would be much more convincing to add the requested  
precision, or if it's really truly there, to walk through examples  
like Jonas's and explain what normative statements describe how to  
process it.

Regards,
Maciej

Received on Tuesday, 22 September 2009 23:09:28 UTC