- From: <detlev.fischer@testkreis.de>
- Date: Tue, 13 Mar 2012 09:54:12 +0100 (CET)
- To: public-wai-evaltf@w3c.org
Hi all, Following Shadi's suggestion I have strucured my methodology draft comments posted yesterday in the survey a bit better. Unfiortunately, the system curently does not allow me to update them, so I post them here. As I note at the end, even if I personally would hold on for a bit longer before publishing, especially to flesh out step 4 and 5, if the rest of you think it makes sense to go puiblic with what we have, I don't want to be in the way. Best regards, Detlev ------------------------------------------ My (revised) comments: -------- Issue 1 -------- Priority: low Location: 1. Introduction Current wording :"web applications and websites for mobile devices" Suggested changes: Not sure this is beneficial. We don't know what UA will be used to view content. For touch devices, some WCAG criteria are not testable (keyboard functionality). Leave out? (The same applies to section 1.1 Scope of this Document - mention of mobile devices) -------- Issue 2 -------- Priority: low Location: 1.4 Terms and Definitions: Current wording :"Elemental Web Pages": Suggested changes: not quite clear what "elemental" conveys. Wouldn't be "Typical Web Pages" more understandable? Applies also to other instances where the term is used. -------- Issue 3 -------- Priority: high, if possible to be addressed before publication Location: 2.1 Scope of Applicability Current wording: "the methodology always applies to a full website without exclusions or omissions of website parts", followed by an example of a university website where it would not be permitted to exclude the library section. Suggested changes: I find this unneccessarily restrictive. I would scrap this requirement. Rationale: First of all, I think this is in contradiction to the WCAG section on conformance claims which states: "However, a conformance claim may be made to cover one page, a series of pages, or multiple related Web pages." http://www.w3.org/TR/WCAG/#conformance-claims But my real concern is not of a formal nature. I think this requirement will severely limit the applicability of the methodology and thereby, its practical usefulness and application in the field. Clearly defining the scope of conformance including, if applicable, the earmarking of sections that are excluded from the scope will mean that the methodology can be used in many cases where a conformance evaluation might otherwise be dropped simply because some part of the site cannot be brought into the scope of conformance (and this would be a real shame). It should be up to the particular evaluation scheme employed by the evaluating expert of company or up to another set of policies what additional requirements are imposed on setting the scope. If a national regulation mandates that the entire website has to conform, this is the point of reference for a more restricted way of setting the scope. If a certifying organization refuses to evaluate sections of websites, they can tell their customers that they won't do the job. I agree however that complete processes should be included fully (in the score, not necessarily with every step in the sample, see comment further down), so the aforementioned exemptions should not be parts of complete processes. The scope of the evaluation should make it crystal clear what is being evaluated and what isn't (by providing the URI's of sections), and any conformance claim placed on the site must make this equally clear. >From practical evaluation cases, I know that there are often situations where legacy or third-party content cannot be made fully conformant - often in areas where lack of conformance may have only a minor impact. Typical are website that in some sections draw on unstrucured data entries from elsewhere as content, for example, violating SC 1.3.1 by having page breaks instead of p elements or some list content not marked up as list. In all other aspects, such web sites may be well designed, well structured, and very accessible overall. If a binary evaluation that is favoured in this group, this means that such a site would never even achieve level A conformance. The message sent is “Mend your ways and come back another time”. A good part of potential clients will simply think of the methodology as remote from reality and leave it alone. -------- Issue 4 -------- Priority: low Location: 3. Evaluation Procedure Typo: "Requirement 1: he evaluation scope" -------- Issue 5 -------- Priority: low Location: 3. Evaluation Procedure, Step 1, Requirement 1.b Current wording: “In the case of "Other" the exact goal must be clearly defined.” Suggested changes: This forth option should be scrapped or one or two examples be given, as for the other three options Rationale: It is not clear what "Other" might be and when this would apply,, so it should be explained with examples. -------- Issue 6 -------- Priority: medium Location: 3. Evaluation Procedure, Step 1.b: Define the Goal of the Evaluation Current wording: N/A Suggested Changes: One useful goal of the evaluation might be to provide an accessibility score in addition to the binary conformance statement per page and SC, as a means of giving a tangible measure of the distance to full conformance. Rationale: I know that many clients find an accessibility score helpful. Putting it in the text as an optional element would give us a broader feedback whether such a score is widely perceived as useful addition, and what kind of objections to it exist. The accessibility score has surfaced several times in our discussions but there seems to be no trace of it in the current draft. The methodology may in future revisions elaborate how such a score should be built up, or abstain from defining it. In either case, it can be labelled as 'merely informative'. -------- Issue 7 -------- Priority: medium Location: 3. Evaluation Procedure, Step 1, Requirement 1e Current wording: “Requirement 1.e: The WCAG 2.0 techniques to be used during the evaluation should be specified.” “While optional, it is usually quite important to define the WCAG 2.0 Techniques that will be used to carry out the evaluation, to ensure consistent expectation between the evaluation commissioner and the evaluator.” Suggested changes: “Requirement 1.e: The WCAG 2.0 techniques successfully or unsuccessfully used to implement web content in an accessible manner should be specified where possible. When WCAG failures are found to apply, these should also be specified.” “Wherever possible, It is useful to point to the WCAG 2.0 Techniques that have been used (successfully or unsuccessfully) to implement the content. Where WCAG techniques have been used, the test at their end provides a clear documentation of success or failure, supporting the replicability of the evaluation procedure.” “When a WCAG failure applies to content, listing it gives a clear and replicable proof of the failure for the Success Criterion in question.” “Listing and referring to applicable techniques and failures also helps in targeting remedial actions.” -------- Issue 8 -------- Priority: medium Location: 3. Evaluation Procedure, Step 3.a: Include Elemental Web Pages of the Website, Requirement 3.a Current wording: “All elemental web pages shall be part of the selected sample of web pages” Suggested changes: “Instances of all elemental web pages shall be part of the selected sample of web pages” Rationale: If elemental pages are of the same design / based on the same template, selecting one of this type should be sufficient. The statement of "all" seems of little value here since the definition referred to is rather vague. -------- Issue 9 -------- Priority: medium Location: 3. Evaluation Procedure, Step 3.b: Include Exemplar Instances of Web Pages, Requirement 3.b Current wording: “Requirement 3.b: At least two distinct web pages (where applicable) of each (1) key functionality, (2) content, design, and functionality, and (3) web technologies shall be part of the selected sample of web pages.” Suggested changes: “Requirement 3.b: The range of different web pages (templates) used, the key functionality evaluated, and web technologies employed shall all be part of the selected sample of web pages.” Rationale: I think the current text is unneccessarily specific and possibly misleading. On the one hand, if two instances selected are essentially the same the work would be redundant. If, on the other hand, just two web pages "from distinct types of web pages" are selected, this may be insufficient. At least for the reader / user of the methodology, the impression could arise that, say, for a site with seven distinct web page templates, selecting just two of the seven would be sufficient. And this would be bad. This is probably not the intention of this section, but it can easily be misunderstood. Essentially, sampling will need to fully reflect the complexity of the site under review. Any quantification is bound to lead to misunderstandings. -------- Issue 10 -------- Priority: medium Location: 3. Evaluation Procedure, Step 3.d Current wording: “Requirement 3.d: All web pages that are part of a process for each web page selected through Requirement 3.a, Requirement 3.b, and Requirement 3.c shall be part of the selected sample” Suggested changes: “Requirement 3.d: All complete processes selected must be covered fully by ensuring that all different (non-repetitive) steps are represented in the page sample.” (not sure about this, perhaps it is better to be still less specific here to cover the variations of processes out there) Rationale: I do not read into the WCAG statement that it is mandatory to evaluate each and every page or page state that is part of a complete process provided that the exploration of complete processes in Step 2 has concluded that the basic setup is accessible. If, for example, a step-by-step process uses the same template and just changes a few instructions and form elements (as in a test with yes/no options used to narrow down a problem space in some expert system), it might be enough to select a few key steps of the entire process. So the stipulation above seems too restrictive and cumbersome to me, especially if each page must be evaluated fully (and not just for selected elements). -------- Issue 11 -------- Priority: high, if at all possible resolve before publication Location: Step 4 and 5 Current wording: Unfortunately, only Editors' notes up to now. Suggested changes: Flesh out step 4 and 5 to draft stage before publication. Rationale: With step 4 and 5 just being editor notes at the moment, it is plainly obvious to readers that our methodology is not just a draft, but an *unfinished draft*. Sorry but I think this will come across as a rather sloppy approach. If we expect readers to spend some time to read and comment on our draft, we would be well advised to present the *entire* approach, even at a draft stage. If there is no really urgent need to publish right now, I think we should spend a few more weeks fleshing out steps 4 and 5 before publication of the draft. If there are really pressing reasons to publish this now in an incomplete state, I don't want to be in the way. But I'd much prefer having all sections fleshed out at least a little. -- testkreis c/o feld.wald.wiese Borselstraße 3-7 (im Hof), 22765 Hamburg Mobil +49 (0)1577 170 73 84 Tel +49 (0)40 439 10 68-3 Fax +49 (0)40 439 10 68-5 http://www.testkreis.de Beratung, Tests und Schulungen für barrierefreie Websites
Received on Tuesday, 13 March 2012 09:50:31 UTC