RE: DDR Simple API test class - Draft 1

The proposal to have an implementation supply its own particular recognised device as input to the test is a good one, and I shall take it on board. Ditto with the idea of letting the implementation decide which vocabularies it will use. All of this is easy to parameterise.
 
The tests have no formal status within the process of taking the API to Rec, at this point in time. The group has resolved to accept three claims of conformance as sufficient to make progress out of CR. However, on advice received from informal dialogue with W3C team, being able to offer a more robust independent support of such claims may be requested. It is not a requirement at this point in time, nor is it certain that it will be a requirement, but in anticipation of the possibility I am trying to be prepared. While the "chapter and verse" of the W3C process may come into play here, there are also people in that process who are gatekeepers of transitions and their opinions are part of the process for very good reasons. I am considering all of the hurdles between here and Rec and intend to be as prepared as possible for each, be they chapter, verse or people.
 
---Rotan

________________________________

From: Jo Rabin [mailto:jrabin@mtld.mobi]
Sent: Wed 11/06/2008 16:51
To: Rotan Hanrahan
Cc: Ignacio Marin; public-ddwg@w3.org
Subject: Re: DDR Simple API test class - Draft 1



Hi

I guess I didn't make my point clear, for which, apologies. What I am
trying to avoid is that every conforming instance of the DDR API
recognises the self same device and returns the same value for it. That
is not an aspect of API conformance as you said earlier (it would be a
test of the detection algorithm and the quality of the underlying
database, more than anything else).

So what I am suggesting is that each claim of conformance is free to
choose its own "known good device".

The idea of using the DDC goes against this in the respect that
implementations should be free to have whatever devices they like and
exclude whatever devices they like. But if we _were_ to have a single
device then that would be the one I choose, because it is a vendor
neutral choice, and because there are things you can say about it, by
definition. It does no harm for DDRs to know about it, since
increasingly we hope that authors will test their site for mobileOK ness
and recognising the DDR is usually part of conformance for mobileOK.

I do take issue with the idea that the test suite tests for elements of
the core vocabulary. As implementations are not required to support it.
DeviceAtlas does, as it happens, but that is not the point.

I'm also unsure what the status of a pass or a fail from the checker is
in respect of a claim to conform to the API. Are you required to have
run the checker? If so, at what revision? etc. etc. etc. I'm wondering
what the chapter and verse of W3C process is that mandates this and for
what purpose? (The reason for asking is that I am worried that the
conformance section of the document may have to change, or if it doesn't
then why do we have to have a checker - since it is not normatively
referred to anywhere?)

Cheers
Jo

On 11/06/2008 13:42, Rotan Hanrahan wrote:
> The only problem with a "UA known to be recognised" is that I didn't want to create any perception of favouritism towards any device manufacturer, and I wanted the test to be completely under our control.
>
> The reason for having explicit "not known" values was to test the behaviour associated with values that are not known, though like you say, this could be done on the basis of not knowing anything at all about the device (i.e. the device is unrecognised).
>
> Using the DDC as a virtual device sounds like a nice idea, though there's no particular reason why these values would have any particular benefit over any other values. It's not like the DDC is going to have a role here, and I'd be wary of implying some role for the DDC in the DDR.
>
> If we do decide to go with some "UA known to be recognised", I'd like us to agree up front the values for the properties, and also to check with the vendors (of the device and the browser) so that we don't misrepresent their products, or otherwise we might have to deal with complaints and other distractions.
>
> I'll have another draft of the code tomorrow, probably, and I'll put it into the drafts space on the main server.
>
> ---Rotan
>
>
>
> -----Original Message-----
> From: Jo Rabin [mailto:jrabin@mtld.mobi]
> Sent: 11 June 2008 10:51
> To: Ignacio Marin
> Cc: Rotan Hanrahan; public-ddwg@w3.org
> Subject: Re: DDR Simple API test class - Draft 1
>
> Thanks Rotan.
>
> Ref the C# implementation, that would be nice, but not strictly a
> conformance test, I think.
>
> I'd be happier if the test did not depend on the insertion of a virtual
> device into a particular data set - although I could see the argument
> for testing against the DDC, possibly. I'd prefer that the positive
> tests were carried out on a "UA known to be recognised" and that the
> negative tests were carried out on a device "known not to be recognised"
> by a particular implementation. It being up to those claiming
> conformance to identify what those devices were ...
>
> ... I'm also a bit concerned that it is legitimate for an implementation
> "not to know" values. If I have understood the code correctly it's
> assumed that a value exists for these properties. I'm not especially
> suggesting that two devices are tested, one of which has known values
> and the other not, but I'm not sure I know what the alternative is.
>
> That would mean in detail that we tested 3 devices. One unrecognised,
> one known for which the values are know and one known for which the
> values are not known ...
>
> Jo
>
> On 11/06/2008 08:27, Ignacio Marin wrote:
>> Once that the group agrees on the definition of these Java tests, I
>> think it would be an easy task to port them to C#.
>>
>> So C# can be counted in the set of technologies for which the test suite
>> will be available.
>>
>> 
>>
>> Regards,
>>
>> 
>>
>> Nacho
>>
>> 
>>
>> ******************************************
>>
>> Ignacio Marín Prendes
>>
>> Head of Unit of Device Independence and Mobility
>>
>> R&D Department
>>
>> ignacio.marin@fundacionctic.org
>> <blocked::BLOCKED::mailto:ignacio.marin@fundacionctic.org>
>>
>> www.fundacionctic.org
>> <blocked::BLOCKED::blocked::http://www.fundacionctic.org <http://www.fundacionctic.org/> >
>>
>> Fundación CTIC -Centro Tecnológico de la Información y la Comunicación-
>>
>> Parque Científico y Tecnológico de Gijón
>>
>> Edificio Centros Tecnológicos
>>
>> 33203 Cabueñes - Gijón - Asturias
>>
>> Teléfono: 984 29 12 12
>>
>> Fax: 984 39 06 12
>>
>> ******************************************
>>
>> Este e-mail y cualquiera de sus ficheros anexos son confidenciales y
>> pueden incluir información privilegiada. Si usted no es el destinatario
>> adecuado (o responsable de remitirlo a la persona indicada),
>> agradeceríamos lo notificase o reenviase inmediatamente al emisor. No
>> revele estos contenidos a ninguna otra persona, no los utilice para otra
>> finalidad, ni almacene y/o copie esta información en medio alguno.
>>
>> Opiniones, conclusiones y otro tipo de información relacionada con este
>> mensaje que no sean relativas a la actividad propia de CTIC deberán ser
>> entendidas como exclusivas del emisor.
>>
>> --------------------------------------------
>>
>> /This e-mail is confidential and may contain legally privileged
>> information. If you are not the intended recipient it may be unlawful
>> for you to read, copy, distribute or otherwise make use of the
>> information herein. If you have received this e-mail in error, please
>> contact us immediately. Fundación  CTIC will accept no liability for the
>> mistransmission, interference, or interception of any e-mail and you are
>> reminded that e-mail is not a secure method of communication./
>>
>> ------------------------------------------------------------------------
>>
>> *De:* public-ddwg-request@w3.org [mailto:public-ddwg-request@w3.org] *En
>> nombre de *Rotan Hanrahan
>> *Enviado el:* miércoles, 11 de junio de 2008 5:01
>> *Para:* public-ddwg@w3.org
>> *Asunto:* DDR Simple API test class - Draft 1
>>
>> 
>>
>> Attached is a simple Java class that can be used to exercise the
>> majority of the methods in a compliant Java implementation of the
>> proposed DDR Simple API.
>>
>> I have written this first draft as a contribution to next week's
>> face-to-face meeting in France in which we expect to be informed of
>> multiple implementations. Code based on the draft I am submitting can be
>> used to verify that key use cases for a Java implementation are behaving
>> in conformance with the specification. Following the recent publication
>> of what the DDWG members believe is a stable and worthy specification,
>> there is now a keen interest in seeing implementations that claim to
>> conform to this specification, so that we can make progress towards a
>> formal Recommendation. Such claims can be put to the test with the aid
>> of the attached code.
>>
>> Note, this draft does not validate the error use cases, where exceptions
>> are defined in the specification. I expect this to be in an update.
>>
>> The test currently relies only on the Core Vocabulary, and to provide
>> the necessary predictability of the underlying data, a "virtual" device
>> is being considered, whose identity can be determined solely by the User
>> Agent header. For the purpose of the test, implementations should
>> recognise this virtual device, and their underlying data should be
>> populated with the expected values (as indicated by the constants
>> present in the test source).
>>
>> This is not an exhaustive test, nor a system test, nor a performance
>> test. It is not a test of the correctness of the underlying data. All
>> such tests would be out of scope for the API itself. The purpose of the
>> test suite captured in this draft class is to exercise the API in a
>> manner consistent with the expected use cases for implementations that
>> have heeded the suggestion to support the Core Vocabulary, in order to
>> raise confidence that, from a functional point of view, the
>> implementations conform to the specification. Multiple claims that are
>> so supported may be sufficient evidence of the viability of this new
>> technology.
>>
>> This contribution also anticipates a likely expectation from observers
>> that progress towards a formal Recommendation should be accompanied,
>> insofar as possible and practical, reasonable and independent support
>> for any claims of conformance.
>>
>> Finally, the tests are implemented in Java but the API is designed to be
>> language-neutral (as far as possible). Unfortunately time constraints
>> prevent me from providing similar tests in other languages, but
>> contributions would be welcome (after the tests have been agreed by the
>> group).
>>
>> ---Rotan (chair).
>>
>> <<DDRSimpleAPITester.java>>
>>
>

Received on Wednesday, 11 June 2008 22:07:50 UTC