- From: Eric Prud'hommeaux <eric@w3.org>
- Date: Tue, 28 Aug 2007 09:45:22 -0400
- To: Lee Feigenbaum <lee@thefigtrees.net>
- Cc: 'RDF Data Access Working Group' <public-rdf-dawg@w3.org>
- Message-ID: <20070828134521.GA15311@w3.org>
* Lee Feigenbaum <lee@thefigtrees.net> [2007-08-27 15:04-0400]
> 5. Test suite and test service and implementation report
>
> We've published the test suite and will be publishing the test service
> shortly. Where should we gather EARL results? Any progress on generating an
> implementation report?
Quick breakdown of the features passed vs. the features *tested*.
(ARQ's report included every test.)
+-----------------------------------------------------------------------------+-----------+----------+
| RDF API for PHP | SUM(pass) | COUNT(*) |
+-----------------------------------------------------------------------------+-----------+----------+
| ASK query form | 0 | 9 |
| Basic graph pattern matching. Triple pattern constructs. Blank node scoping | 64 | 171 |
| Core bits of SPARQL. Prefixed names, variables, blank nodes, graph terms | 316 | 845 |
| RDF datasets. Default and named graphs. GRAPH keyword | 0 | 7 |
| FILTER clauses and expressions | 76 | 232 |
| OPTIONAL pattern matching | 5 | 19 |
| SELECT query form | 64 | 167 |
| Sorting (ORDER BY) and slicing (LIMIT, OFFSET) | 13 | 26 |
| UNION pattern matching | 1 | 4 |
+-----------------------------------------------------------------------------+-----------+----------+
| ARQ | SUM(pass) | COUNT(*) |
+-----------------------------------------------------------------------------+-----------+----------+
| ASK query form | 102 | 102 |
| Basic graph pattern matching. Triple pattern constructs. Blank node scoping | 303 | 303 |
| CONSTRUCT query form | 15 | 15 |
| Core bits of SPARQL. Prefixed names, variables, blank nodes, graph terms | 1353 | 1353 |
| RDF datasets. Default and named graphs. GRAPH keyword | 44 | 44 |
| FILTER clauses and expressions | 545 | 545 |
| OPTIONAL pattern matching | 25 | 25 |
| SELECT query form | 236 | 236 |
| Sorting (ORDER BY) and slicing (LIMIT, OFFSET) | 92 | 92 |
| UNION pattern matching | 12 | 12 |
+-----------------------------------------------------------------------------+-----------+----------+
for a more tedious picture, we can look at RDF API for PHP by facet:
+-----------------------------------------+-----------+----------+
| facet | SUM(pass) | COUNT(*) |
+-----------------------------------------+-----------+----------+
| ASK query form |
| AskQuery | 0 | 3 |
| AskQuery-no | 0 | 3 |
| AskQuery-yes | 0 | 3 |
| Basic graph pattern matching. Triple pattern constructs. Blank node scoping
| BasicGraphPattern-spo | 24 | 58 |
| BasicGraphPattern-spoDOT | 32 | 92 |
| BasicGraphPattern-spoo | 1 | 1 |
| BasicGraphPattern-spopo | 2 | 4 |
| BasicGraphPattern-spospo | 5 | 15 |
| GroupGraphPattern-empty | 0 | 1 |
| Core bits of SPARQL. Prefixed names, variables, blank nodes, graph terms
| BaseDecl | 4 | 4 |
| BlankNodePropertyList | 0 | 2 |
| Collection-0 | 0 | 1 |
| Collection-1 | 2 | 2 |
| Collection-n | 1 | 1 |
| GraphTerm-BooleanLiteral-false | 1 | 1 |
| GraphTerm-BooleanLiteral-true | 1 | 1 |
| GraphTerm-minus | 1 | 1 |
| GraphTerm-plus | 0 | 1 |
| IRIref-rQ_IRI_REF | 2 | 3 |
| NumericLiteral-rDECIMAL | 0 | 4 |
| NumericLiteral-rDOUBLE | 0 | 2 |
| NumericLiteral-rINTEGER | 8 | 20 |
| PrefixDecl-default | 41 | 110 |
| PrefixDecl-n | 30 | 83 |
| PrefixDecl-nonDefault | 33 | 92 |
| PrefixedName-localname | 39 | 94 |
| PrefixedName-prefix-localname | 10 | 35 |
| RDFLiteral-datatype | 1 | 10 |
| RDFLiteral-plain | 11 | 32 |
| RDFLiteral-rLANGTAG | 11 | 32 |
| String-doubleQuote | 12 | 27 |
| String-singleQuote | 0 | 2 |
| String-triple-doubleQuote | 0 | 2 |
| String-triple-singleQuote | 0 | 2 |
| Var-rVAR1 | 53 | 138 |
| Var-rVAR2 | 1 | 2 |
| Verb-a | 1 | 1 |
| WhereClause-noWhere | 23 | 85 |
| WhereClause-where | 30 | 55 |
| FILTER clauses and expressions |
| BuiltInCall-bound | 1 | 3 |
| BuiltInCall-datatype | 0 | 4 |
| BuiltInCall-isBlank | 1 | 1 |
| BuiltInCall-lang | 2 | 6 |
| BuiltInCall-langMatches | 2 | 4 |
| BuiltInCall-str | 2 | 5 |
| ConditionalAndExpression-n | 0 | 1 |
| ConditionalOrExpression-n | 0 | 6 |
| IRIrefOrFunction-iri | 2 | 6 |
| PrimaryExpression-IRIref | 2 | 6 |
| PrimaryExpression-rBrackettedExpression | 0 | 1 |
| PrimaryExpression-rBuiltInCall | 5 | 18 |
| PrimaryExpression-rNumericLiteral | 6 | 16 |
| PrimaryExpression-rRDFLiteral | 8 | 23 |
| PrimaryExpression-rVar | 26 | 71 |
| RegexExpression | 2 | 3 |
| RegexExpression-flags | 0 | 1 |
| RelationalExpression-eq | 12 | 33 |
| RelationalExpression-gt | 0 | 3 |
| RelationalExpression-le | 0 | 1 |
| RelationalExpression-lt | 1 | 4 |
| RelationalExpression-ne | 2 | 10 |
| UnaryExpression-NOT | 2 | 6 |
| OPTIONAL pattern matching |
| OptionalGraphPattern | 5 | 19 |
| RDF datasets. Default and named graphs. GRAPH keyword |
| GraphGraphPattern | 0 | 7 |
| SELECT query form |
| SelectQuery-Distinct-star | 1 | 1 |
| SelectQuery-Distinct-Var | 0 | 5 |
| SelectQuery-noDistinct-star | 25 | 55 |
| SelectQuery-noDistinct-Var | 27 | 76 |
| SelectQuery-Var-n | 11 | 30 |
| Sorting (ORDER BY) and slicing (LIMIT, OFFSET) |
| OrderClause-1 | 4 | 9 |
| OrderClause-n | 1 | 1 |
| OrderCondition-ASC | 2 | 5 |
| OrderCondition-DESC | 2 | 5 |
| OrderCondition-noSC | 2 | 3 |
| OrderCondition-rVar | 2 | 3 |
| UNION pattern matching |
| GroupOrUnionGraphPattern-union | 1 | 4 |
+-----------------------------------------+-----------+----------+
I'd prefer if the passed numbers were either 0 or equal to the sum
(i.e. we had some confidence that we were isolating the parts of their
code that followed the spec or not). We could do some forensics, but I
think we should wait 'till they re-run now that we have a full test
set for them.
I did $(wget)s into a tests/reports/ directory and modified the
reports to make them uniform. I separated ARQ's assertor and
subject. RDF API for PHP had 19 mislabled tests so I commented those
out. Likewise, I commented theirs that dealt with the syntax
tests. Andy nicely separated his syntax from his semantics tests.
RDF API for PHP used a structure like this to describe the earl:subject:
<http://rdfapi-php.sf.net/> doap:name "RDF API for PHP";
doap:release [ doap:created "2007-08-13 17:09"^^<http://www.w3.org/2001/XMLSchema#date>;
doap:name "RAP SVN-2007-08-13T17:09";
a doap:Version ];
a doap:Project.
so I emulated that in ARQ's report:
_:b1
rdf:type earl:Software ;
doap:name "ARQ" ;
doap:release [ doap:name "@version@" ] ;
<http://purl.org/dc/terms/isPartOf>
[ foaf:homepage <http://jena.sf.net/>
] ;
foaf:homepage <http://jena.sf.net/ARQ> .
(The "@version@" had been on a dc:somethingorother arc.)
Should I check in the reports/ dir?
--
-eric
office: +1.617.258.5741 NE43-344, MIT, Cambridge, MA 02144 USA
mobile: +1.617.599.3509
(eric@w3.org)
Feel free to forward this message to any list for any purpose other than
email address distribution.
Received on Tuesday, 28 August 2007 13:45:34 UTC