How to support aggregation requirement

ERT WG:
I took a pseudo-action during our last meeting to start a thread on what 
support for aggregation looks like in the requirements document.

http://www.w3.org/2009/04/22-er-minutes

The relevant requirement is:

F04. EARL 1.0 will support aggregation of test results according to 
different criteria (for example with respect to the subject). 

http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Requirements-20090421.html

This requirement, in my mind, has at least three interpretations:
1. combination - EARL 1.0 will support the combining of test results into 
a single EARL report. That is, it will be possible that EARL reports 
generated by different tools or authored by different persons can be 
combined to form a single report that is in accord with the relevant 
specifications. (From the EARL consumer's perspective, does this also mean 
that EARL affords reports to be distributed over several distinct 
documents? I believe we address this iether in the schema or in the 
guide.)
2. composition - EARL 1.0 will support the composition of reports into a 
single report. That is, EARL 1.0 will support a deterministic and 
efficient way of deriving a single test outcome for a given test subject 
tested against a specific test criterion from multiple outcomes generated 
by that same or "closely related" test subject tested against the same or 
"closely related" test criteria. We understand 'closely related' (for lack 
of a better term) to indicate differing dates for the same resource, for 
example, or similar criteria (e.g. WCAG2 criteria for alt on img elements 
and that given by Sec 508 in the U.S.).
3. aggregation - EARL 1.0 will support the aggregation of reports. That 
is, the EARL framework makes it possible to produce a synopsis or 
evaluation of an entire collection of test subjects based on the test 
outcomes associated with those subjects in multiple reports. (Think, for 
instance, of an evaluation about the accessibility of an entire site based 
on a subset of test outcomes for a random collection of pages.)

IMHO, (2) and (3) would be nice, but they introduce unnecessary complexity 
into our suite of documents. What EARL-consuming tools do with reports is 
an application-specific question. If we stick with just (1), I think (as I 
said) that this is covered elsewhere, though it certainly won't hurt to 
identify it as a requirement.

Please let me know your thoughts.

--> Mike Squillace
IBM Human Ability and Accessibility Center

W:512.286.8694
M:512.970.0066

External: http://www.ibm.com/able
Internal: http://w3.ibm.com/able