Re: EARL based WebID tests: a first example

I'm already thinking about two cases for this page:
a) User/developer accesses the page with a browser
b) Automatic test with the test suite

And here a little picture which will make my comments hopefully more
understandable:
https://www.axolotlfarm.org/~bergi/projects/rme/tests/webid/WebID%20tests%20diagram.png
https://www.axolotlfarm.org/~bergi/projects/rme/tests/webid/WebID%20tests%20diagram.odg

We are now mainly focusing on the TestCriterion "WebIDAuthentication" in
my picture. But we should already keep the test suite in the back of our
minds.

Am 16.04.2011 06:53, schrieb Henry Story:
> Thanks Bergi, for taking the time to review this.
> 
> Your [1] is very interesting and contains a few cases I had forgotten
> to put below. It is good that we are working on the same lines. I
> went down a slightly different route following up on the hint Michael
> Hausenblas left in [3] where he mentioned the very well documented
> EARL ontology, which I have studied relatively carefully to put the
> example below together. I am new to this space, so it is really good
> to have a companion explorer.
> 
> The idea is to end up with something that can be placed into the html
> output of your [2], the output of https://auth.fcns.eu/, in what used
> to work as https://foaf.me/testSSL.php, my test cases at
> https://bblfish.net:8443/test/WebId and every other endpoint that
> people build with any of the numerous webid libraries. We have three
> example outputs already, and I think we are clearly converging
> already in the human readable output. These seem to be working pretty
> much the same way. And they have all already revealed themselves to
> be very useful.
> 
> On 15 Apr 2011, at 22:25, bergi wrote:
> 
>> It would be cool if the page for the test is not only for this
>> single purpose.
> 
> You mean not just for webid but also all other authentication
> problems? The semantic web is extensible, so unless we are doing
> something that would exclude us growing into something larger, I
> would avoid generalising too early. It makes for more work, and the
> quality is usually not that good, as one starts to develop theories
> on fields one is not so knowledgeable on. But, having said that, ...
> 

That's the extracted data from my XHTML/RDFa page:

<earl:TestResult
rdf:about="https://www.axolotlfarm.org/~bergi/projects/rme/webid/">
 <authn:Authentication>
  <rdf:Description>
   <authn:ID>
    <rdf:Description>
     <dct:description>WebID successfull verified</dct:description>
      <authn:status
rdf:resource="http://ns.bergnet.org/rdf/authentication/0.1/successful"/>
       <foaf:Agent rdf:resource="http://www.bergnet.org/webid#me"/>
      </rdf:Description>
     </authn:ID>
    </rdf:Description>
   </authn:Authentication>
  <earl:outcome rdf:resource="http://www.w3.org/ns/earl#passed"/>
</earl:TestResult>

This would be enough for the test suite and could be included in any
page. We should define what must be included for the automatic tests.

This information could be also used for others things like a browser
plugin which shows the validated WebID.

>> Therefore we should not rely on description property values.
> 
> I am not sure what you mean by this.
> 
>> I made already a proposal [1]
> 
> Yes, there are some good cases there I had not listed below. But this
> seems to repeat a lot of what EARL already offers. 
> http://www.w3.org/TR/EARL10-Guide/
> 
> @prefix bergi: <http://ns.bergnet.org/rdf/authentication/0.1/>
> 
> For example bergi:Testable seems to be a subclass of
> earl:TestRequirement . and bergi:credentialFailure seems to be a
> earl:TestRequirement . bergi:passwordWrong seems to be precise enough
> to be a earl:TestCase .
> 

You are right. I looked mostly at the output for the test suite. But we
have n TestRequirements and just one TestResult. So we have to forward
somehow the failed TestRequirement to the TestResult.

>> how the additonal information could look like and created a demo
>> [2].
> 
> yep. That works for me, and recognised my WebID. Very nice. We need
> to put a list of test endpoints together on the wiki next.
> 
>> The certificate and http informations are useful for further
>> problem analyses but I think they should not be used for verifying
>> the test cases.
> 
> I am not sure why. In the output of auth.fcns.eu or of the
> bblfish.net test suite having a good description of the cause of the
> failure is very important in helping users understand what went
> wrong. In the case of these endpoint tests there are two important
> places things can go wrong: the certificate could be broken or not
> contain enough information, or the WebID Profile could be broken, or
> the wrong profile have been downloaded.
> 

I was thinking about case b), sure the information is helpful for the user.

> In the case of Clerezza it is going to be difficult for the moment to
> show the original document to the user, as we convert everything
> directly into triples. A better test suite should keep the original
> document around, but that will be a bit inefficient for us to put
> together and take time anyway to develop. So I need to argue for
> flexibility here :-)
> 
>> The http request for the webProfile should be also included.
> 
> yes, that is what I show in the examples I posted below. See the
> :webProfile object right at the end of the original mail (and also
> copied here). The EARL documentation shows how this can be done.
> 

Just for the sake of completeness request and response should be
included in the detailed output.

>> Michael Hausenblas mentioned in this mail [3] the RDFa tests. For
>> the WebID test cases also the test vocab [4] could be used to point
>> to SPARQL queries to verify the endpoint results.
> 
> This seems to precede EARL. Is it something one needs in addition to
> EARL?
> 

mh, I can't find the property in the test schema, but the following file
uses a property "informationResourceResults" which points to a SPARQL
query. Something like that could be used for the test suite TestCriterions.

http://rdfa.digitalbazaar.com/test-suite/manifest.ttl

>> 
>> [1] http://ns.bergnet.org/rdf/authentication/0.1/ [2]
>> https://www.axolotlfarm.org/~bergi/projects/rme/webid/ [3]
>> http://lists.w3.org/Archives/Public/public-xg-webid/2011Mar/0102.html
>>
>> [4] http://www.w3.org/2006/03/test-description
>> 
>> Am 15.04.2011 13:36, schrieb Henry Story:
>>> So I have been studying the "Evaluation and Report Language
>>> (EARL) 1.0 Guide" http://www.w3.org/TR/EARL10-Guide/ ontology, to
>>> see how we could adapt our tests to produce RDF using that
>>> ontology.
>>> 
>>> Below I have tried to sketch carefully what a tester like 
>>> https://auth.fcns.eu/ could end up producing so that other robots
>>> could be created to do much deeper tests on a service, testing
>>> every part of an implementation we can think of.
>>> 
>>> The results in N3 below are in three parts: 1. declaration of the
>>> subject 2. List of test cases I could think of from my experience
>>> building a tester 3. a report example in n3
>>> 
>>> We should place the test case vocabulary somewhere with the spec,
>>> so that others can build both simple test end points, and others
>>> can produce meta test suites. This is what EARL is all about.
>>> 
>>> Feedback welcome.
>>> 
>>> ---------------------------------------------------------------------------
>>>
>>>
>>> @prefix earl: <http://www.w3.org/ns/earl#> .
>>> @prefix zz: ? . #for the clerezza agent @prefix doap:
>>> <http://usefulinc.com/ns/doap#> . @prefix rdf:
>>> <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . @prefix rdfs:
>>> <http://www.w3.org/2000/01/rdf-schema#> . @prefix wit: <?> . #Web
>>> Id Test @prefix http: <http://www.w3.org/2006/http#> .
>>> 
>>> zz:tester a earl:Software; doap:repository
>>> <https://svn.apache.org/repos/asf/incubator/clerezza/trunk/parent/platform.security.foafssl/test>;
>>>
>>> doap:programming-language "Scala";
>>> doap:developer <http://bblfish.net/#hjs>; doap:name "WebID Test
>>> suite in Clerezza" .
>>> 
>>> 
>>> <https://svn.apache.org/repos/asf/incubator/clerezza/trunk/parent/platform.security.foafssl/test>
>>> a doap:SVNRepository .
>>> 
>>> ############################################### # # Initial Test
>>> vocabulary to write somewhere # # todo: where should these be
>>> placed? # ################################################
>>> 
>>> # profile and pubkey tests
>>> 
>>> wit:profileGet a earl:TestRequirement; dct:title "WebId Profile
>>> is accessible and downloadable".
>>> 
>>> wit:profileWF a earl:TestRequirement; dct:title "WebId Profile is
>>> well formed"; dct:description "The WebId Profile is parseable
>>> Content and transformable to RDF" .
>>> 
>>> wit:profileWFKey a earl:TestRequirement; dct:title "WebIdProfile
>>> Contains well formed Keys"; dct:description "All the keys in the
>>> profile are well formed and not misleading"; skos:note "One does
>>> not need to test all keys in a profile, only those that are tied
>>> to the WebIDs found in the X509 cert. But to help users one could
>>> give them a deeper test of the profile." .
>>> 
>>> wit:pubkey a earl:TestRequirement; dct:title "Public Key is well
>>> formed" ; dct:description "A particular Public key is well
>>> formed" .
>>> 
>>> wit:pubkeyMod a earl:TestRequirement; dct:title "rsa:modulus is
>>> well formed" .
>>> 
>>> wit:pubkeyMod_func a earl:TestCase; dct:title "not more than one
>>> modulus"; dct:description "More than one modulus if they don't
>>> convert to the same number will lead to erratic behavior (one
>>> server will choose one the other server will chose the other)".
>>> 
>>> wit:pubkeyMod_lit a earl:TestCase; dct:title "rsa:modulus is a
>>> literal number"; dct:description "In the current ontology we have
>>> moved to literals as the standard way of describing modulus and
>>> exponents".
>>> 
>>> wit:pubkeyExp a earl:TestRequirement; dct:title
>>> "rsa:public_exponent is well formed" .
>>> 
>>> wit:pubkeyExp_func a earl:TestCase; dct:title "not more than one
>>> public exponent per RSA public key - or else they have to be
>>> identical" . dct:description "More than one exponent if they
>>> don't convert to the same number is very likely to create erratic
>>> behavior (one server will choose one the other server will chose
>>> the other)".
>>> 
>>> wit:pubkeyExp_lit a earl:TestCase; dct:title "rsa:exponent key is
>>> a literal"; dct:description "In the current ontology we have
>>> moved to literals as the standard way of describing modulus and
>>> exponents".
>>> 
>>> wit:pubkeyMod_old_func a earl:TestCase; dct:title "if modulus is
>>> using non literal notation, there should be only one cert:hex
>>> relation to plain literal"; skos:note "this should be a
>>> deprecated test sooner rather than later. Warn people to move to
>>> newer notation.".
>>> 
>>> wit:pubkeyExp_old_func a earl:TestCase; dct:title "if exponent is
>>> using non literal notation, there should be only one cert:decimal
>>> relation to plain literal" .
>>> 
>>> 
>>> #tying pubkey to Webid
>>> 
>>> wit:webid_verification_gt0 a earl:TestRequirement; dct:title "At
>>> least one webid mentioned in certificate has public key that
>>> verifies".
>>> 
>>> wit:webid_verification a earl:TestRequirement; dct:title "A WebId
>>> in cert's public key has public key declared in profile that
>>> matches the public key corresponding to private key used in TLS
>>> connection".
>>> 
>>> # tests on certificate internet heritage conformance # useful for
>>> test suites to help developers understand why they should build
>>> certificates # a certain way. TODO.
>>> 
>>> ############################################### # # Example test
>>> result to make sure the above ontology is at least partially
>>> correct # ################################################
>>> 
>>> 
>>> [] a earl:Assertion; earl:test wit:webid_verification; 
>>> earl:result [ a earl:TestResult; dct:description "rsa public key
>>> has two relations for modulus"; earl:outcome earl:failed ]; 
>>> earl:subject :webProfile, :x509, { [] cert:identity
>>> <http://bblfish.net/person/henry/card#me>; cert:modulus
>>> "as123123..."^^cert:hex, "dfff32093sd..."^^cert:hex; 
>>> cert:public_exponent "65537"^^cert:int; }; earl:assertedBy
>>> zz:tester .
>>> 
>>> 
>>> [] a earl:Assertion; earl:test wit:pubkeyMod_func; earl:result [
>>> a earl:TestResult; dct:description "webid http://user.example/#me
>>> does not have a matching public key in profile"; earl:outcome
>>> earl:success ]; earl:subject :webProfile, :x509; earl:assertedBy
>>> zz:tester .
>>> 
>>> :x509 a cert:X509Certificate; cert:pemFormat """-----BEGIN
>>> CERTIFICATE----- 
>>> MIIDgzCCAuygAwIBAgIQZ84ABvhjj7hqFoWqSsvBFjANBgkqhkiG9w0BAQUFADBj 
>>> MREwDwYDVQQKDAhGT0FGK1NTTDEmMCQGA1UECwwdVGhlIENvbW11bml0eSBvZiBT 
>>> ZWxmIFNpZ25lcnMxJjAkBgNVBAMMHU5vdCBhIENlcnRpZmljYXRpb24gQXV0aG9y 
>>> aXR5MB4XDTExMDMyODE0MDY1MFoXDTEyMDMxODE2MDY1MFowgYsxETAPBgNVBAoM 
>>> CEZPQUYrU1NMMSYwJAYDVQQLDB1UaGUgQ29tbXVuaXR5IE9mIFNlbGYgU2lnbmVy 
>>> czE3MDUGCgmSJomT8ixkAQEMJ2h0dHA6Ly9iYmxmaXNoLm5ldC9wZW9wbGUvaGVu 
>>> cnkvY2FyZCNtZTEVMBMGA1UEAwwMYmJsZmlzaCBjYXJkMIIBIjANBgkqhkiG9w0B 
>>> AQEFAAOCAQ8AMIIBCgKCAQEA5+kuueCGksuOuQciIrf7hjSRiahB8c3hd8hPjTH/ 
>>> 6k+NBKN+H0MRHPiSVCVwvvhstF2zmE6Ms0NwzSDWHuSOqjEwu6+CKE8tvL0Y0OHk 
>>> bkhVDhenLPQagKIWjXe0k4CDIcizyNj1L8zRwsN0TaxrYZZPlaTx2/VpMI3ApaVK 
>>> yb/4+mJ4UZDBol9TMkTfyBbPq3iISMz6rt3vsNgksXar0DCftGag2V2E1L/t8Hvu 
>>> De0UaqKajsIlVtu/iUMSYKu41dZJCVCYm/DrqcX0m1aUwHAYWKtSap9Z5p7PnJVo 
>>> wqp2/3jnsf7h6WlUN9yQtm/FeEeMp+3Mx7DokAYYTElTaQIDAQABo4GKMIGHMAwG 
>>> A1UdEwEB/wQCMAAwDgYDVR0PAQH/BAQDAgLsMBEGCWCGSAGG+EIBAQQEAwIFoDAd 
>>> BgNVHQ4EFgQUzoQy71OnKyW8qE7boOHpLrjN2aQwNQYDVR0RAQH/BCswKYYnaHR0 
>>> cDovL2JibGZpc2gubmV0L3Blb3BsZS9oZW5yeS9jYXJkI21lMA0GCSqGSIb3DQEB 
>>> BQUAA4GBAH0kxSBDYGAMah4cloznjsnglGNMCTd2zPtxnWDFUjuD2YWhc8QXd/k7 
>>> T1GlVZdLfT175/D7jYpXEVH7UyO8DTnttlAePmDqbspT+vcpV1orUrWlMTJ7hAzP 
>>> Ev9aBOHrZPyKDeUJO0JgwAWxOU/ND347Ssg3lTbFt0jrZxDLHLxC -----END
>>> CERTIFICATE-----"""; # should we also have a relation to the
>>> openssl type text format? Is that a standard? 
>>> cert:subjectAlternativeName
>>> <http://bblfish.net/people/henry/card#me>; 
>>> cert:IssuerDistinguishedName "O=FOAF+SSL, OU=The Community of
>>> Self Signers, CN=Not a Certification Authority"...
>>> 
>>> :webProfile a http:Response; http:httpVersion "1.1"; http:headers
>>> [...]; http:body "#the rdf file used. could be n3, or something.
>>> This could also point to the content?..."] .
>>> 
>>> ---------------------------------------------------------------------------
>>>
>>>
>>> Henry
>>> http://bblfish.net/
>>> 
> 
> Social Web Architect http://bblfish.net/
> 
>

Received on Sunday, 17 April 2011 16:54:48 UTC