Re: WebID-ISSUE-9 (bblfish): Develop WebID Test Suite [WebID Spec]

On 20 Feb 2011, at 23:37, peter williams wrote:

> I disagree. The use of .p12 files is very sensible, and avoids using keygen
> tags (and similar HTMLisms that are not standardized and tend to need human
> intervention, being all about forms and human-readable documents, terms and
> conditions, copyrights etc).

I was not thinking of generating client certificates with keygen for the 
tests.  Perhaps the setup is not clearly explained. There are three actors:

1- the endpoint being tested
  that should return some special RDF when successful, such as how many 
 webids he recognised, which he had trouble with, etc...

2- a server that can pretend to be any number of agents with different WebIDs
  It could be a swarm of agents.

3- the WebId profiles
   That have corresponding public keys used in 2 above

IF 2 and 3 are run on the same server, the server can create throw away certificates
itself quite easily.

But thinking about it I realise that there may be a good reason to publish the 
test p12 files with privte keys in the end, at least to the requestor for a  
little while. The  reason is  that this can be useful for the people who ran the test, 
to verify the tests for themselves. 

   The way Toby has set it up at present is that the public keys are static,
and that seems to work less well, as my recent experience showed, with the keys
having passed their validity date.
  

> 
> As we see, .p12 files are easy to handle and fit the culture of unit
> testing. One sets an input requirements
> 
> What might make sense is that a "support" website exists that provides the
> .p12 files on demand, making the .p12 stream on the fly. Such a site merely
> exports a credential from its implementation platform to the .p12 file, and
> supplies it as a file stream resource. The site can mint keys/certs
> periodically (using any local means it finds suitable) or on the fly: being
> a simple openssl call. Henry already did 85% of the work by showing a pipe
> of two openssl commands. Just augment these with the openssl CA command...
> 
> How hard would it be to change the test client so it obtains its .p12 file
> dynamically from the web on a fixed URI, unpacking it with the likes of the
> .p12 decoder in openssl to create the keying material ready to drive a unit
> test run? If the http response indicates no change in file stream (given
> etag in request), the nth test can reuse the previously unpacked .p12 file,
> else force a new download to rebuild and rekey the test suite.

I think those are good ideas. But perhaps where we can start is with developing
a mini ontology of tests, where we describe what they should do, and mini ontology
of responses the test end point should return. We can start very small. 

Anyway, whatever works to get going.

Henry


> 
> 
> 
> -----Original Message-----
> From: public-xg-webid-request@w3.org [mailto:public-xg-webid-request@w3.org]
> On Behalf Of Henry Story
> Sent: Sunday, February 20, 2011 10:51 AM
> To: Toby Inkster
> Cc: WebID Incubator Group WG; sysbot+tracker@w3.org
> Subject: Re: WebID-ISSUE-9 (bblfish): Develop WebID Test Suite [WebID Spec]
> 
> I just put together a "servlet" in Clerezza that accepts Toby's test suite
> calls: ie it returns the WebID if it can find it or "-" if it cannot.
> 
> Then to get the rdf perl libs needed by your script  I compiled half of cpan
> I think ;-) I tried it out. But your test certificates are out of date.
> 
> Running 
> 
>    openssl pkcs12 -clcerts -nokeys -in key/0002.p12  | openssl x509 -noout
> -text
> 
> on the first two p12 files found here
> 
>   http://buzzword.org.uk/2008/foaf_ssl/tests/key/
> 
> Shows them to be valid sometime during  2009 and 2010. Well they have not
> been updated recently. :-)
> 
> This is bound to be a problem when hard coding certificates like that. I
> think to get these tests to work better one needs to create certificates and
> WebIDs on the fly. Then one can also test the https server to see if it
> correctly rejects invalid certificates. Also one can then create new WebIDs
> much more easily, and so make it possible to test caching issues too.
> 
> So perhaps the RDF will be more useful in describing the results of the
> tests, as your perl script does.  That does indeed make it easier to build
> these test implementations, as one does not need to bother with the User
> Interface aspect. Someone else can take the output of such a testing service
> and make it look nice for their purposes. 
> 
>   What we could do perhaps is immediately put together an RDF list of tests
> that we want to make. We just need to describe in english what they do
> (though german might make it look more serious and thorough). Then we can
> have different implementations of these test suites setup on different
> servers which return results using that "ontology". It would make it
> possible to compare the results of the tests from different servers, and so 
> 
> -  distribute the testing load
> -  check how things are working from  different positions around the globe
> -  test the test suites among each other
> 
> I could write a test suite that goes into Clerezza, you could do elaborate
> your perl script, and every perl instance or Clerezza instance could set up
> a test service. The we are back in the git philosophy. 
> 
>   We could also develop a better ontology for the results to be returned by
> a testing endpoint, instead of "-" and the webid, it could describe a number
> of failures that occurred, or perhaps how it got the data (from cache? did
> it fetch it?).  Then the tests could describe the type of failure expected.
> 
>  Not sure if this makes sense. Other ideas welcome. 
> 
> 
> Henry
> 
> 
> 
> On 29 Jan 2011, at 23:16, Toby Inkster wrote:
> 
>> On Sat, 29 Jan 2011 10:10:28 +0000
>> WebID Incubator Group Issue Tracker <sysbot+tracker@w3.org> wrote:
>> 
>>> There were some tests out there already. Something to place in the 
>>> mercurial repository.
>> 
>> Beginnings of a test suite is here:
>> 
>> http://buzzword.org.uk/2008/foaf_ssl/tests/manifest.ttl
>> 
>> The idea is that a test harness (and there's an example test harness 
>> provided at 
>> http://buzzword.org.uk/2008/foaf_ssl/tests/test-harness.pl)
>> would take the manifest as input and run each test against an endpoint 
>> which is what's being tested. The endpoint is a WebID-secured script 
>> which simply echoes out a text/plain document consisting of a single
>> line: a dash if authentication is unsuccessful; the WebID URI 
>> otherwise.
>> 
>> --
>> Toby A Inkster
>> <mailto:mail@tobyinkster.co.uk>
>> <http://tobyinkster.co.uk>
>> 
>> 
> 
> Social Web Architect
> http://bblfish.net/
> 
> 
> 
> 

Social Web Architect
http://bblfish.net/

Received on Tuesday, 22 February 2011 21:08:28 UTC