Re: Functional Outcomes MUST cover all benefits OR must be duplicated

 from the link Test Development Frequently Asked Questions (FAQ) Question 8 - What makes a good test?: 

 https://www.w3.org/QA/WG/2005/01/test-faq#good
"A good test is:   
   - Mappable to the specification (you must know what portion of the specification it tests)
   - Atomic (tests a single feature rather than multiple features)
   - Self-documenting (explains what it is testing and what output it expects)
   - Focused on the technology under test rather than on ancillary technologies
   - Correct"

Other useful information pertaining to testing may be found in other questions in the Test FAQ link:
https://www.w3.org/QA/WG/2005/01/test-faq






    On Monday, September 7, 2020, 9:01:17 AM EDT, John Foliot <john.foliot@deque.com> wrote:  
 
 Sticking my head up from the move for a minute...
I agree with Jake: Functional Outcomes are "compound" goals, consisting of multiple unit tests. 
So, for (rudimentary) example:   
   - Does the content convey a sense of hierarchy? (T/F)
   - Can users navigate content programmatically? (T/F)
   - Can users locate key information blocks? (T/F)
I keep hearing in my head, over and over, "testable, measurable, repeatable".
JF

(Sent from my mobile, apologies for any spelling mistakes)
On Mon, Sep 7, 2020, 2:47 AM jake abma <jake.abma@gmail.com> wrote:


Hi all,
Just another issue we must have correct or discuss at least before publication I think.
--------------------
As Guidelines are not normative but (Functional) Outcomes are, they must cover all benefits for all Functional Groups and Functional Needs we try to tackle.
This means the "so... bla bla" statement should be broad enough to cover all benefits OR a bulleted list might be needed with the benefits (and are the benefits normative then?).
--------------------

On the other hand, if we use bulleted lists for Benefits, then all methods and the scoring / tests MUST cover all benefits also otherwise they are not compatible (Charles Hall commented on this in the functional needs subgroup).
--------------------

If this is not a "Catch All" for (Functional) Outcomes, we might need to split / duplicate Outcomes covering different Benefits (?!)
--------------------
EXAMPLE 1--------------------

"Provides semantic structure So can convey a sense of hierarchy" 
In this case the benefits of navigating or locating are not mentioned, also the Functional Needs are not covered as it's not in the normative text.
Three options for this example:
1. (long sentence, covering all benefits)
"Provides semantic structure So can convey a sense of hierarchy AND/OR users can navigate AND/OR users can locate" 
2. (use of bulleted list)
"Provides semantic structure   
   - So can convey a sense of hierarchy
   - So users can navigate
   - So users can locate" 
3. (split in 3 Functional Outcomes)

"Provides semantic structure so can convey a sense of hierarchy""Provides semantic structure so users can navigate""Provides semantic structure so users can locate"
--------------------

This is just an example of the challenge with the Functional Outcome texts being normative, more examples are not difficult to think of.
Another option would be to separate the Benefits from the functional outcome and mention them as something like: " Benefits might be but not limited to: bla, bla and bla"
--------------------

At the moment I think the Functional Outcomes as we have now are to open to interpretation and probably will not make it as normative text to be tested and scored.
Of course happy to illustrate of dsicus.
Cheers,Jake