Re: EARL based WebID tests: a first example

Thanks Bergi, for taking the time to review this. 

Your [1] is very interesting and contains a few cases I had forgotten to put below. It is good that we are working on the same lines. I went down a slightly different route following up on the hint Michael Hausenblas left in [3] where he mentioned the very well documented EARL ontology, which I have studied relatively carefully to put the example below together. I am new to this space, so it is really good to have a companion explorer.

The idea is to end up with something that can be placed into the html output of your [2], the output of
https://auth.fcns.eu/, in what used to work as https://foaf.me/testSSL.php, my test cases at https://bblfish.net:8443/test/WebId and every other endpoint that people build with any of the numerous webid libraries. We have three example outputs already, and I think we are clearly converging already in the human readable output. These seem to be working pretty much the same way. And they have all already revealed themselves to be very useful.

On 15 Apr 2011, at 22:25, bergi wrote:

> It would be cool if the page for the test is not only for this single purpose.

You mean not just for webid but also all other authentication problems? 
The semantic web is extensible, so unless we are doing something that would exclude us growing into something larger, I would avoid generalising too early. It makes for more work, and the quality is usually not that good, as one starts to develop theories on fields one is not so knowledgeable on. But, having said that, ...

> Therefore we should not rely on description property values.

I am not sure what you mean by this.

> I made already a proposal [1]

Yes, there are some good cases there I had not listed below. 
But this seems to repeat a lot of what EARL already offers.
http://www.w3.org/TR/EARL10-Guide/

@prefix bergi: <http://ns.bergnet.org/rdf/authentication/0.1/>

For example bergi:Testable seems to be a subclass of earl:TestRequirement .
and bergi:credentialFailure seems to be a earl:TestRequirement . bergi:passwordWrong seems
to be precise enough to be a earl:TestCase .

> how the additonal information could look like and created a demo [2].

yep. That works for me, and recognised my WebID. Very nice. We need to put a list of test endpoints together on the wiki next.

> The certificate and http informations are useful for further problem analyses
> but I think they should not be used
> for verifying the test cases.

I am not sure why. In the output of auth.fcns.eu or of the bblfish.net test suite having a good description of the cause of the failure is very important in helping users understand what went wrong. In the case of these endpoint tests there are two important places things can go wrong: the certificate could be broken or not contain enough information, or the WebID Profile could be broken, or the wrong profile have been downloaded. 

In the case of Clerezza it is going to be difficult for the moment to show the original document to the user, as we convert everything directly into triples. A better test suite should keep the original document around, but that will be a bit inefficient for us to put together and take time anyway to develop. So I need to argue for flexibility here :-)

> The http request for the webProfile should
> be also included.

yes, that is what I show in the examples I posted below. See the :webProfile object right at the end of the original mail (and also copied here). The EARL documentation shows how this can be done.

> Michael Hausenblas mentioned in this mail [3] the RDFa
> tests. For the WebID test cases also the test vocab [4] could be used to
> point to SPARQL queries to verify the endpoint results.

This seems to precede EARL. Is it something one needs in addition to EARL?

> 
> [1] http://ns.bergnet.org/rdf/authentication/0.1/
> [2] https://www.axolotlfarm.org/~bergi/projects/rme/webid/
> [3] http://lists.w3.org/Archives/Public/public-xg-webid/2011Mar/0102.html
> [4] http://www.w3.org/2006/03/test-description
> 
> Am 15.04.2011 13:36, schrieb Henry Story:
>> So I have been studying the "Evaluation and Report Language (EARL) 1.0 Guide"
>> http://www.w3.org/TR/EARL10-Guide/
>> ontology, to see how we could adapt our tests to produce RDF using that ontology.
>> 
>> Below I have tried to sketch carefully what a tester like 
>>   https://auth.fcns.eu/
>> could end up producing so that other robots could be created to do much deeper tests on a
>> service, testing every part of an implementation we can think of.
>> 
>> The results in N3 below are in three parts:
>> 1. declaration of the subject
>> 2. List of test cases I could think of from my experience building a tester
>> 3. a report example in n3
>> 
>> We should place the test case vocabulary somewhere with the spec, so that others can build
>> both simple test end points, and others can produce meta test suites. This is what EARL is all
>> about.
>> 
>> Feedback welcome.
>> 
>> ---------------------------------------------------------------------------
>> 
>> @prefix earl: <http://www.w3.org/ns/earl#> .
>> @prefix zz: ? . #for the clerezza agent
>> @prefix doap: <http://usefulinc.com/ns/doap#> .
>> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
>> @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
>> @prefix wit: <?> . #Web Id Test
>> @prefix http: <http://www.w3.org/2006/http#> .
>> 
>> zz:tester a earl:Software;
>>    doap:repository <https://svn.apache.org/repos/asf/incubator/clerezza/trunk/parent/platform.security.foafssl/test>;
>>    doap:programming-language "Scala";
>>    doap:developer <http://bblfish.net/#hjs>;
>>    doap:name "WebID Test suite in Clerezza" .
>> 
>> 
>> <https://svn.apache.org/repos/asf/incubator/clerezza/trunk/parent/platform.security.foafssl/test> a doap:SVNRepository .
>> 
>> ###############################################
>> #
>> # Initial Test vocabulary to write somewhere
>> #
>> # todo: where should these be placed?
>> #
>> ################################################
>> 
>> # profile and pubkey tests
>> 
>> wit:profileGet a earl:TestRequirement;
>>  dct:title "WebId Profile is accessible and downloadable".
>> 
>> wit:profileWF a earl:TestRequirement;
>>  dct:title "WebId Profile is well formed";
>>  dct:description "The WebId Profile is parseable Content and transformable to RDF" .
>> 
>> wit:profileWFKey a earl:TestRequirement;
>>   dct:title "WebIdProfile Contains well formed Keys";
>>   dct:description "All the keys in the profile are well formed and not misleading";
>>   skos:note "One does not need to test all keys in a profile, only those that are tied to the WebIDs found in the X509 cert. But to help users one could give them a deeper test of the profile." .
>> 
>> wit:pubkey a earl:TestRequirement;
>>   dct:title "Public Key is well formed" ;
>>   dct:description "A particular Public key is well formed" .
>> 
>> wit:pubkeyMod a earl:TestRequirement;
>>   dct:title "rsa:modulus is well formed" .
>> 
>> wit:pubkeyMod_func a earl:TestCase;
>>   dct:title "not more than one modulus";
>>     dct:description "More than one modulus if they don't convert to the same number will lead to erratic behavior (one server will choose one the other server will chose the other)".
>> 
>> wit:pubkeyMod_lit a earl:TestCase;
>>   dct:title "rsa:modulus is a literal number";
>>   dct:description "In the current ontology we have moved to literals as the standard way of describing modulus and exponents".
>> 
>> wit:pubkeyExp a earl:TestRequirement;
>>   dct:title "rsa:public_exponent is well formed" .
>> 
>> wit:pubkeyExp_func a earl:TestCase;
>>   dct:title "not more than one public exponent per RSA public key - or else they have to be identical" .
>>   dct:description "More than one exponent if they don't convert to the same number is very likely to create erratic behavior (one server will choose one the other server will chose the other)".
>> 
>> wit:pubkeyExp_lit a earl:TestCase;
>>   dct:title "rsa:exponent key is a literal";
>>   dct:description "In the current ontology we have moved to literals as the standard way of describing modulus and exponents".
>> 
>> wit:pubkeyMod_old_func a earl:TestCase;
>>   dct:title "if modulus is using non literal notation, there should be only one cert:hex relation to plain literal";
>>   skos:note "this should be a deprecated test sooner rather than later. Warn people to move to newer notation.".
>> 
>> wit:pubkeyExp_old_func a earl:TestCase;
>>   dct:title "if exponent is using non literal notation, there should be only one cert:decimal relation to plain literal" .
>> 
>> 
>> #tying pubkey to Webid
>> 
>> wit:webid_verification_gt0 a earl:TestRequirement;
>>   dct:title "At least one webid mentioned in certificate has public key that verifies".
>> 
>> wit:webid_verification a earl:TestRequirement;
>>   dct:title "A WebId in cert's public key has public key declared in profile that matches the public key corresponding to private key used in TLS connection".
>> 
>> # tests on certificate internet heritage conformance
>> # useful for test suites to help developers understand why they should build certificates
>> # a certain way. TODO.
>> 
>> ###############################################
>> #
>> # Example test result to make sure the above ontology is at least partially correct
>> #
>> ################################################
>> 
>> 
>> [] a earl:Assertion;
>>  earl:test wit:webid_verification;
>>  earl:result [ a earl:TestResult;
>>     dct:description "rsa public key has two relations for modulus";
>>     earl:outcome earl:failed ];
>>  earl:subject :webProfile, :x509, { 
>>    [] cert:identity <http://bblfish.net/person/henry/card#me>; 
>>       cert:modulus "as123123..."^^cert:hex, "dfff32093sd..."^^cert:hex; 
>>       cert:public_exponent "65537"^^cert:int;
>>  };
>>  earl:assertedBy zz:tester .
>> 
>> 
>> [] a earl:Assertion;
>>  earl:test wit:pubkeyMod_func;
>>  earl:result [ a earl:TestResult;
>>     dct:description "webid http://user.example/#me does not have a matching public key in profile";
>>     earl:outcome earl:success ]; 
>>  earl:subject :webProfile, :x509;
>>  earl:assertedBy zz:tester .
>> 
>> :x509 a cert:X509Certificate;
>>     cert:pemFormat """-----BEGIN CERTIFICATE-----
>> MIIDgzCCAuygAwIBAgIQZ84ABvhjj7hqFoWqSsvBFjANBgkqhkiG9w0BAQUFADBj
>> MREwDwYDVQQKDAhGT0FGK1NTTDEmMCQGA1UECwwdVGhlIENvbW11bml0eSBvZiBT
>> ZWxmIFNpZ25lcnMxJjAkBgNVBAMMHU5vdCBhIENlcnRpZmljYXRpb24gQXV0aG9y
>> aXR5MB4XDTExMDMyODE0MDY1MFoXDTEyMDMxODE2MDY1MFowgYsxETAPBgNVBAoM
>> CEZPQUYrU1NMMSYwJAYDVQQLDB1UaGUgQ29tbXVuaXR5IE9mIFNlbGYgU2lnbmVy
>> czE3MDUGCgmSJomT8ixkAQEMJ2h0dHA6Ly9iYmxmaXNoLm5ldC9wZW9wbGUvaGVu
>> cnkvY2FyZCNtZTEVMBMGA1UEAwwMYmJsZmlzaCBjYXJkMIIBIjANBgkqhkiG9w0B
>> AQEFAAOCAQ8AMIIBCgKCAQEA5+kuueCGksuOuQciIrf7hjSRiahB8c3hd8hPjTH/
>> 6k+NBKN+H0MRHPiSVCVwvvhstF2zmE6Ms0NwzSDWHuSOqjEwu6+CKE8tvL0Y0OHk
>> bkhVDhenLPQagKIWjXe0k4CDIcizyNj1L8zRwsN0TaxrYZZPlaTx2/VpMI3ApaVK
>> yb/4+mJ4UZDBol9TMkTfyBbPq3iISMz6rt3vsNgksXar0DCftGag2V2E1L/t8Hvu
>> De0UaqKajsIlVtu/iUMSYKu41dZJCVCYm/DrqcX0m1aUwHAYWKtSap9Z5p7PnJVo
>> wqp2/3jnsf7h6WlUN9yQtm/FeEeMp+3Mx7DokAYYTElTaQIDAQABo4GKMIGHMAwG
>> A1UdEwEB/wQCMAAwDgYDVR0PAQH/BAQDAgLsMBEGCWCGSAGG+EIBAQQEAwIFoDAd
>> BgNVHQ4EFgQUzoQy71OnKyW8qE7boOHpLrjN2aQwNQYDVR0RAQH/BCswKYYnaHR0
>> cDovL2JibGZpc2gubmV0L3Blb3BsZS9oZW5yeS9jYXJkI21lMA0GCSqGSIb3DQEB
>> BQUAA4GBAH0kxSBDYGAMah4cloznjsnglGNMCTd2zPtxnWDFUjuD2YWhc8QXd/k7
>> T1GlVZdLfT175/D7jYpXEVH7UyO8DTnttlAePmDqbspT+vcpV1orUrWlMTJ7hAzP
>> Ev9aBOHrZPyKDeUJO0JgwAWxOU/ND347Ssg3lTbFt0jrZxDLHLxC
>> -----END CERTIFICATE-----""";
>>  # should we also have a relation to the openssl type text format? Is that a standard?
>>  cert:subjectAlternativeName <http://bblfish.net/people/henry/card#me>;
>>  cert:IssuerDistinguishedName "O=FOAF+SSL, OU=The Community of Self Signers, CN=Not a Certification Authority"...
>> 
>> :webProfile a http:Response;
>>          http:httpVersion "1.1";
>>          http:headers [...];
>>          http:body "#the rdf file used. could be n3, or something. This could also point to the content?..."] .
>> 
>> ---------------------------------------------------------------------------
>> 
>> Henry
>> http://bblfish.net/
>> 

Social Web Architect
http://bblfish.net/

Received on Saturday, 16 April 2011 04:54:00 UTC