- From: Gregg Vanderheiden <gregg@vanderheiden.us>
- Date: Tue, 30 Aug 2022 10:16:17 -0700
- To: GLWAI Guidelines WG org <w3c-wai-gl@w3.org>
- Message-Id: <C913F97E-A465-46A8-9F47-44F47BE7D4CB@vanderheiden.us>
Referring to https://docs.google.com/presentation/d/1KhuTdTemRjjJIFViftKYPXc1nKhcJ4LqrQum0qOJLBI/edit#slide=id.gfe3effe5fb_0_16 <https://docs.google.com/presentation/d/1KhuTdTemRjjJIFViftKYPXc1nKhcJ4LqrQum0qOJLBI/edit#slide=id.gfe3effe5fb_0_16> If I combine the definitions of old terms with the new terms for them I get Computational: tests where the results will not vary based on the tester or approach. Qualitative: informed qualitative evaluations based on a set of criteria. The test results may vary slightly between experienced testers. Adaptive: evaluate the results within a particular context. The context dictates which unconditional or conditional tests are used, and what baseline is tested against. Extensible: (new) -no definition but from call it looks like: different tools can be used to test for pass Procedural tests: evaluate whether an acceptable process was adequately followed to improve accessibility. Examples include usability and plain language testing. Adding my comments (and ideas from others that I remember from the meeting) under each of the above: Computational: tests where the results will not vary based on the tester or approach. GOOD I might add to end of definition "(this includes numeric and logical tests)" Objective I think is better term but may be controversial. Definition should include "good inter-rater reliability that is close to unity for non borderline cases" Qualitative: informed qualitative evaluations based on a set of criteria. The test results may vary slightly between experienced testers. Good term but definition is ambiguous. Computational can also vary slightly between experienced testers at the borders. I would remove ’slightly’ here to differentiate between this and computational Adaptive: evaluate the results within a particular context. The context dictates which unconditional or conditional tests are used, and what baseline is tested against. If this is about context then maybe "context based" is better name. Adaptive sounds like an active process and not something that would be the same from moment to moment of from use to users. Unless we are going to allow sites to say "this site is only for these users" or only intended to be used in these contexts" so we don’t need to meet the guidelines for other users or contexts — then I’m not sure what these would do HAVING SAID THAT — I think that Context-based guidelines would have a place with non-required guidelines or criteria. For example — if you are doing this for children we SUGGEST that you also do the following as well. Extensible: (new) (no old definition but from call it looks like:) different tools can be used to test for pass I don’t see this as usable with anything that is required. Again- ok for things that are advisory only but you cannot confirm that something conforms if there are multiple or an open list of things to test with. It can pass with some tool other than the one you are using to test it. AND a company can always create a tool internally and claim conformance to it that I cannot get hold of to confirm. OR they can create one that is easy to pass and then release it publicly. Too easy to game. Again - OK for recommendations but not for requirements. Extensible seems like the wrong word but not sure of the right word since I am not entirely sure what this one means. I suggest that we ask what it means and then pick the most salient word out of the explanation to name it. Would the word "extensible" be used in explaining this? Maybe "Flexible" ? Procedural tests: evaluate whether an acceptable process was adequately followed to improve accessibility. Examples include usability and plain language testing. I note that here it is called PROCEDURAL TEST — but in use (see below) it is not used as a test type -but a recommendation type It is actually not possible to test whether a processes or protocol was carried out unless you were present at the time in the company. So you can’t test protocols or processes. The only thing you CAN test is the presence of an affirmation of conformance. That is, you can check to see if they affirm that they carried it out. This could be done by a) a claim or b) by documentation/minutes of meetings/sessions (though these are unlikely to be provided except in a legal case). So if being used to label a requirement - then Procedural would be correct. But for a test type - it would be affirmation. I also note that the next slide has (removing old terms) has the following — which sheds some light and is useful since it shows that some terms are for test types and some are for requirement types Test types Computational: Qualitative: Requirement types Prescriptive (new) Adaptive (Conventional) Extensible (new) Procedural I suggest the definitions always be listed in this grouping to better understand them So I have done that below - adding my suggestions AND ideas from others at the meeting I also added a "recommendation types" so that provisions that have looser requirements on them (than something would that is required and must therefore be cleanly testable) can be included Also changed Prescriptive (which was not defined above but was defined at meeting to be outcome oriented via examples) to Outcome Test types Computational - tests where the results will not vary based on the tester or approach and that have near unity inter-rater reliability except at borders where it is still good. Qualitative - informed qualitative evaluations based on a set of criteria. The test/evaluation results may vary between experienced testers. Affirmation - where the only thing tested is whether the author affirms that a process or protocol has been followed. Requirement types Outcome (was prescriptive) - requirements as to what should be true of the web content Procedural - requiems on what should be true regarding the process used to create the web contents Recommendation types Contextual (was Adaptable) - additional recommendations that are provided for particular contexts (e.g. a children’s site, a technical site, a site only designed for viewing in a particular culture or language) Extensible (Flexible?) - recommendations that can be measured in different ways (which will result in different scores) but all of which move content in the right direction. For example different measures of plain language. Best Gregg Vanderheiden gregg@vanderheiden.us
Received on Tuesday, 30 August 2022 17:16:33 UTC