Re: feedback on data format for WCAG-EM reports

Hi Carlos,

On 30.6.2014 09:49, Carlos A Velasco wrote:
> Hi Shadi,
>
> I will let the decision up to you. In my opinion, an evaluation
> methodology generates results that can be represented with EARL
> TestResult. Therefore, I see the introduction of the new class unnecessary.

I think it is a similar situation to earl:Software -- it is useful to 
have to describe tools in general, plus they can be used as assertors 
and test subjects. It would be a similar case for a tbd:WebPage class.


> In regard to dct:title, dct:description and dct:date, they are part of
> the 3 EARL vocabularies, thus I do not understand your point. Static
> pages can be as well described with content-in-RDF.

Sorry, let me try to be clearer:

#1. One would need to parse content-in-RDF (or HTTP-in-RDF) every time 
to identify the title and date -- providing these as dct:title and 
dct:Date in the tbd:WebPage class would help processing this class.

#2. The dct:description could contain additional information in human 
readable form like "content after completing the form with wrong data" 
or such. It could also provide instructions to regenerate the content.

#3. Yes, all these attributes are already part of earl:TestCase so the 
proposed tbd:WebPage would be fully compatible with it. No existing 
tools would need to change their implementation. It is a refinement.

Thoughts?

Best,
   Shadi


> On 06/29/2014 07:20 PM, Shadi Abou-Zahra wrote:
>> Hi Carlos,
>>
>> The problem is, what is the best way to describe a "web page" (and web
>> page state) that is *independent* of any test subject. That is, I want
>> to describe a web page on its own. This *could* then be the subject in
>> an EARL report but could also be a web page description outside EARL.
>>
>> The basis for such a class should probably be http:ConnectionClass:
>>   - http://www.w3.org/TR/HTTP-in-RDF10/#ConnectionClass
>>
>> Ideally we would also have dct:title, dct:description, and dct:date to
>> make the data easier to handle. Also to allow people to leave out such
>> HTTP recordings when there is no need for it (eg. static web page).
>>
>> In other words, tbd:WebPage would be the same as earl:TestSubject:
>>   - http://www.w3.org/TR/EARL10/#TestSubject
>>
>> This class would be fully compatible with EARL because it can be used as
>> a test subject. It can also be used outside the context of EARL.
>>
>> Thoughts?
>>
>> Best,
>>    Shadi
>>
>>
>> On 29.6.2014 14:51, Carlos A Velasco wrote:
>>> Hi Shadi, Wilco,
>>>
>>> Sorry to jump late on this. *I do not think that there is a need for any
>>> new class in EARL*. This use case is already covered within EARL via the
>>> *TestSubject combined with HTTP-in-RDF and Content-in-RDF*. You can see
>>> examples in the EARL guide:
>>>
>>> http://www.w3.org/TR/EARL10-Guide/#report-negot
>>>
>>> There you can include any representation of a HTTP resource with a
>>> series of requests and responses. Therefore, after a quick review of the
>>> format for EM, I think it needs a better mapping to EARL.
>>>
>>> On 06/26/2014 10:35 AM, Shadi Abou-Zahra wrote:
>>>> Hi Wilco,
>>>>
>>>> On 25.6.2014 16:25, Wilco Fiers wrote:
>>>>> Hi Shadi,
>>>>>
>>>>> Thanks for the fast response! Great stuff! I'll address the key issues
>>>>> below:
>>>>>
>>>>>> # webPage
>>>>>>    - EARL does not provide such a class -- might be good to add here;
>>>>>>    - also needs to consider the notion of "web page states";
>>>>> Wilco: I think this should maybe be a subclass of http:Response with
>>>>> an optional 'state' property added to it. I think that addresses both
>>>>> issues.
>>>>
>>>> I think a web page is much more than an HTTP response. Consider a
>>>> series
>>>> of redirect and authorization exchanges until you actually receive any
>>>> content from the server. Also a web page could pull additional content,
>>>> for example for dynamic (AJAX) applications.
>>>>
>>>> So, I think we need to think about a "webPage" resource. Maybe this
>>>> needs to become part of EARL. I will try to propose something interim.
>>>>
>>>>
>>>>>> # reliedUponTechnology
>>>>>>    - maybe this should be a class too? with dct:title and a URI to
>>>>>> any publicly available specification for the technology;
>>>>> Wilco: Excellent idea. I'll work that into the file
>>>>>
>>>>>
>>>>>> # dct:conformsTo
>>>>>>    - actually WCAG-EM results do not include conformance claims;
>>>>> Wilco: You are absolutely correct. I'll change this.
>>>>>
>>>>>
>>>>>> # additionalRequirement
>>>>>>    - could also be a list; maybe rename to additionalEvalRequirement?
>>>>> Wilco: All of these are properties of the evaluation class, so I
>>>>> didn't add 'eval' to any of the names. Though admittedly that makes
>>>>> them a bit vague here and there. I guess it's a question of
>>>>> consistency. I'm not bothered either way. I will leave this up to your
>>>>> best judgement.
>>>>
>>>> Good point regarding naming.
>>>>
>>>> But did you model that there may be multiple additional requirements?
>>>>
>>>>
>>>>>> # dct:date
>>>>>>    - what does this date stand for? is it different from dct:date in
>>>>>> earl:TestResult class (provided for each assertion individually)?
>>>>> Wilco: This is date is the property of the evaluation. In WCAG-EM it's
>>>>> described as "Date for the evaluation (completion date or duration
>>>>> period)". I think dct:date fits this sufficiently. The date on each
>>>>> individual earl result in my thinking was going to be a time stamp of
>>>>> when it was entered. It helps to account for issues in a report that
>>>>> might have already been fixed before the report was completed. This
>>>>> occasionally happens to me and colleges.
>>>>
>>>> Thanks for clarifying. Maybe good to add to the schema description too.
>>>>
>>>>
>>>>>> # Specifics
>>>>>>    - what is that?
>>>>> Wilco: Evaluation specifics is one of the things WCAG-EM has as an
>>>>> optional part of reporting. It seems to be a bit of a catch all option
>>>>> for evaluators to record any miscellaneous details. I could rename it
>>>>> to evalSpecifics, like you suggested for additionalRequirement. I am
>>>>> not sure if that makes all that much of a difference.
>>>>
>>>> Yeah, this one seems ambiguous. Please add comments to the schema too.
>>>>
>>>>
>>>>>> # dct:abstract
>>>>>>    - I like reusing existing terms but I need to re-read the
>>>>>> definition;
>>>>> Wilco: This is a summary of the evaluation for in the report.
>>>>
>>>> Yup, I get that. But is DC "abstract" a good mapping? I need to check.
>>>>
>>>>
>>>>>> # commonPages
>>>>>>    - think this should be a list of webPage;
>>>>>> # essentialFunctionality
>>>>>>    - also could be a list;
>>>>>> # otherRelevantPages
>>>>>>    - think this should be a list of webPage;
>>>>> Wilco: The list question is an interesting one. I do not think these
>>>>> should be web pages. This will be part of the exploration, and I think
>>>>> it would be sufficient for users to provide titles or names of
>>>>> sections or any other kind of textual indication they can use during
>>>>> this step. We could allow this in webPage, but I think that wouldn't
>>>>> be sufficient once an auditor is defining their sample.
>>>>
>>>> In WCAG-EM "common web pages" and "other relevant web pages" are pretty
>>>> much web pages or web page states. Do evaluators typically take
>>>> notes of
>>>> such pages, then include them later on, or should we combine this?
>>>>
>>>> As to essentialFunctionality, yes, this is a description ... but a list
>>>> of such descriptions, I imagine.
>>>>
>>>>
>>>>> As for the lists, I think this makes sense from a data perspective. I
>>>>> did not do this because I don't think I want to force auditors into
>>>>> using some sort of widget to define lists. I think a single textarea
>>>>> for any of these things would be better as it would give them more
>>>>> flexibility.
>>>>> I suggest leaving this issue open until we figure out what we want
>>>>> these fields to look like in the tool and then match the output from
>>>>> that to the data format.
>>>>
>>>> Agree. By the way, this goes back to my "scratchpad" idea -- an open
>>>> text field where evaluators can enter notes along the evaluation...
>>>>
>>>>
>>>>>> # structuredSample and randomSample
>>>>>>    - maybe create an abstract "webPageSample" class for the two?
>>>>> Wilco: Makes sense to me. I haven't really looked at subclasses or
>>>>> anything like that. I'll add this to the file.
>>>>>
>>>>>
>>>>>> # Misc other comments
>>>>>>    - maybe also need an overall "wcagem:Report" class?
>>>>> Wilco: By report, do you mean that this would be a pointer to where
>>>>> the report could be found or what it's HTML would be? I can see some
>>>>> advantages, but I could also see some problems arising from that as it
>>>>> is basically a duplicate of the data in a different format. It's worth
>>>>> exploring though.
>>>>>
>>>>>>    - maybe package conformanceTarget, commissioner, and
>>>>>> additionalRequirement into an abstract wcagem:EvaluationScope class?
>>>>> Wilco: Good point. I just realized there are actually two different
>>>>> ways the word 'scope' is used in WCAG-EM. The evaluation scope (step
>>>>> 1) and the website scope (step 1a). I'll better distinguish between
>>>>> the two. Should I then also add the website to the evaluationScope?
>>>>> That way you would go: evaluation > evaluationScope > website >
>>>>> websiteScope
>>>>
>>>> Yes, we use the term twice so we need some differentiation.
>>>>
>>>> Looking good...
>>>>
>>>> Best,
>>>>    Shadi
>>>>
>>>>
>>>>> Regards,
>>>>>
>>>>> Wilco
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

Received on Monday, 30 June 2014 11:58:36 UTC