Re: test suite - does earl need a notion of a cause?

I am wondering if the notion of a cause for a failure would not simplify the test suite.

wit:CertificateOk could have causes of failures such as one or more of 
   - certificate Expired
   - certificate key not recognised by WebID layer (currently not an RSA key)
   - the private key negotiation did not succeed
   - ? something else

wit:WebIDClaim can fail because
  - the SAN is not recognised (or is unparseable) - now is it a WebIdClaim at that point still?
  - profile could not be fetched
  - profile could not be parsed
  - the key profile has nothing to say about that key

Somehow this seems simpler to me: we don't have to then write up tests for each of
the causal steps if they succeed. It will have to be stated that each of the causal steps
has been verified in order to assert the claim as being true. IT is only in the case of failure
that the causes of the failure would need to be stated.

This makes more sense in the WebIDClaim layer, as really there is no way one can say 
a claim passed if one was not able to get hold of the document.


On 30 Nov 2011, at 19:49, Henry Story wrote:

> I am thinking the following hierarchy seems to make a lot more sense now
> that I am implementing this in a more functional manner. In brackets '(...)'
> are the optional tests
> wit:certificateProvided
>  wit:certificateOk 
>    wit:certificateProvided,
>    wit:certificateDateOk,
>    ( wit:privateKeyNegotiationSucceeded ) //most apps won't know how to make this fail
>    wit:certificatePubkeyRecognised, //more accurately: recognised by WebId layer
>    ( wit:certificateCriticalExtensionsOk ) // requires more research
>    wit:certificateProvidedSAN // one for each SAN - tests if  SAN is URL, for ex
>       wit:webidClaim        // if SAN is good, then we have a claim
>         wit:profileOk       // we can get profile and parse it
>            wit:profileGet
>            wit:profileWellFormed
>            ( wit:profileWellFormedPubkey ) //match keys in profile
>         wit:keyMatchProfile
> Then the main test: if there is a WebId authentication succeeds if one
> wit:webidClaim succeeds.
> The above seems to follow the causal sequence more precisely.
> I'll work along these lines just to see what that gives me.
> 	Henry
> On 30 Nov 2011, at 00:53, bergi wrote:
>> Yesterday I've changed my code according the latest changes in the spec.
>> My old version contained code which wasn't required to be spec
>> compliant, but to run all test cases defined in the test vocab. This was
>> caused by different approaches of the public key verification. With the
>> ASK SPARQL query the complete verification process is now atomic. Some
>> more test cases can now be marked as optional. Optional means in our
>> case that these tests may be useful for users to check their WebIDs and
>> give them more detailed hints what could case problems. Because the term
>> optional could be a little bit confusing I propose to use the term
>> "profile". "Authentication Profile" would be the group of test cases
>> defined by the spec 3.1. "Analyze Profile" contains all test cases
>> defined in the vocab. The test suite would require the Authentication
>> Profile. If we can agree upon that I will change the vocab and the wiki
>> page.
>> Independent from these changes I will update the test suite the next days.
>> That's the new Authentication Profile ASCII diagram:
>> wit:webidAuthentication
>> 	wit:webidClaim
>> 		wit:certificateOk
>> 		wit:profileOk
>> 			wit:profileGet
>> 			wit:profileWellFormed
>> 			wit:profileWellFormedPubkey
>>> [1]
> Social Web Architect

Social Web Architect

Received on Wednesday, 30 November 2011 22:39:39 UTC