Re: WCAG-EM Report Tool comments

Hi Shawn,

Thank you for these excellent comments. Please consider using Github 
next time. Please find below responses to your comments:

Wilco, some items for you below as well.


On 26.2.2015 07:50, Shawn Henry wrote:
> All of these comments are for editor's discretion now, or for
> consideration for the next version. There are some comments marked
> [high] that I think should be considered for this version.
>
> *Bug*
>
> *[high] Step 2: Explore the Target Website and Step 3: Select a
> Representative Sample. The (x)s don't seem to be working to delete a line.

I also had this weird problem before. Try restarting your browser (as 
odd as this sounds). Wilco, there seems to be something strange here 
that we need to try and locate. Not sure what we can do right now. We 
cannot reproduce it. Wilco, any updated thoughts on this?


> *Wording Tweaks: Start page*
>
> * [medium-high] Start page: "Note: This tool does not record or
> automatically save the information that you enter. Use the 'Save' link
> (Ctrl + S or ⌘ S) that is at the top of all pages to save your data in a
> file locally on your computer." -> Note: This tool does not
> automatically save the information that you enter. To save your data in
> a file locally on your computer, use Windows shortcut keys Ctrl+S or Mac
> shortcut keys ⌘S to open the Save dialog. (Or the "Save" link at the top
> of the page will open the Save Evaluation Report page and from there the
> "Save data file locally to your computer" link will open the Save dialog.)

Done. But quite long. How about just "Or use the 'Save' link at the top 
of the page" (without parenthesis)? The instructions are there anyway.


> * [medium] Start page: "Trick: To have all WCAG 2.0 Success Criteria
> listed in your final report, select the conformance target 'Level AAA'
> in the tool and edit the conformance target text in your final HTML
> report." -> You can include in your report WCAG 2.0 success criteria
> beyond the conformance target. For example, the website is only required
> to meet Level AA, yet you want to also include Level AAA success
> criteria in the report. In Step 1, select the "Conformance target" as
> "Level AAA". Then after you download the report, manually change the
> Conformance target to Level AA in the downloaded HTML file.

Done. Also quite long...


> * [medium-low] Start page: " You can download the report with your input
> as an HTML file and customize it." -> The tool provides your report as
> HTML and CSS files. You can download the files from the "View Report"
> page. You can then change the report content and visual design.
>
> * [low] Start page: "You can save partially-complete reports and work on
> them later. Use the 'Open' link at the top and find the file that you
> previously saved to reload it. If you have already entered data, then
> 'Open' will load the file in a new report." -> You can save
> partially-complete reports and work on them later. To reload a saved
> file, use the "Open" link at the top and find the file that you
> previously saved.
> [SLH note: I don't understand "If you have already entered data, then
> 'Open' will load the file in a new report."]
>
> * [typo] Start page. "It is designed for experienced evaluators who know
> Web Content Accessibility Guidelines (WCAG) 2.0 and somewhat familiar
> with WCAG-EM." -> It is designed for experienced evaluators who know Web
> Content Accessibility Guidelines (WCAG) 2.0 and are somewhat familiar
> with WCAG-EM.

All done.


> *Wording Tweaks: Step 1 page*
>
> * [high] Step 1. Conformance target (i) box: "Select a target WCAG 2.0
> conformance level ('A', 'AA', or 'AAA') for the evaluation. For more
> information, see WCAG-EM Step 1.b: Define the Conformance Target." ->
> Select a target WCAG 2.0 conformance level ('A', 'AA', or 'AAA') for the
> evaluation. For more information, see WCAG-EM Step 1.b: Define the
> Conformance Target.</p><p>This selection filters the success criteria
> that are included in "Step 4: Audit the Selected Sample" and in the
> report. For example, if you select Level A, only the Level A success
> criteria will be included. If you want the report to include all of the
> success criteria, select Level AAA here. If the conformance target for
> the website is less, then after you download the report, manually change
> the Conformance target to the appropriate Level in the downloaded HTML
> file.

Done but quite long.


> *Wording Tweaks: Step 2 page*
>
> * [high] Step 2: Explore the Target Website. Web Technologies Relied
> Upon (i) box. "… Note: The input field 'Web Technology' is also a
> drop-down list with a selection of web technologies that you can use, or
> add to." -> "</p><p><strong>Note:</strong> The "Web Technology" and
> "Specification Address (URL)" input fields provide drop-down lists that
> you can pick from. You can also type different text into these fields.
>
> * [medium-low] Step 2. "(Optional) Note Taking" -> Optional Exploration
> Notes
>
> * [typo] Step 2. "…This steps helps you determine…-> …This step helps
> you determine…
>
> * [low] Step 2. "Identify the web technologies used to provide the
> website and optionally take notes on other aspects of the website." ->
> Identify the web technologies used to provide the website and, if you
> want, take notes on other aspects of the website. OR Identify the web
> technologies used to provide the website. You can also take notes on
> other aspects of the website.
> [1. typo "This steps helps…"; 2. Take out "optionally" – it's awkward in
> the sentence and I think not necessary given "optional" is below)]
>
> * [low] Step 2. (Optional) Note Taking (i) box: "This section allows you
> to take optional notes during your website exploration. These notes will
> be provided to you again in the next step, to help you select a sample
> of web pages for evaluation. The notes are not part of the final report,
> but they will be kept along with all your other input in the data for
> saving and reloading." -> "This section lets you record information
> during your website exploration. These notes are provided in the next
> step, to help you select a sample of web pages for evaluation.
> <strong>These notes are not included in the final report.</strong> They
> are saved when you save the file, and available when you later reload
> the file.

All done.

Wilco, note the <strong> in the locale file for this last comment.


> * [low] Step 2. Last 2 (i) boxes: "Use this field to take notes about"
> -> You can use this field to take notes about…
> [rationale: to re-iterate that this is optional]
>
> * [low] Step 2. Variety of web page types (i) box. "This includes notes
> about varying styles, layouts, structures, and functionality provided on
> the website." -> This includes notes about different styles, layouts,
> structures, and functionality used on the website.
>
> *Wording Tweaks: Step 3 page*
>
> * [medium] Step 3. "Select web pages for evaluation. The web page
> recordings and notes that you took during website exploration in the
> previous step are provided here, to help you make the selection. It is
> important to ensure that your selection of web pages is representative
> of the target website. " -> Select web pages for evaluation. The notes
> that you took during exploration in the previous step are provided here,
> to help you select the pages. Ensure that the web pages selected are
> representative of the target website."
> [SLH note: medium priority because "web page recordings" is confusing]
>
> * [low-medium] Step 3. 2 (i) boxes: "Note: Web pages include different
> web page states; see definition of web page states." -> Note: "Web
> pages" include different "web page states"; see the definition of web
> page states.
>
> *Wording Tweaks: Step 4*
>
> * [typo] Step 4 intro. "This steps helps you…" -> This step helps you…
> [although text edit suggestion below removes that word to better match
> the approach of the other pages]
>
> * [medium-high] Step 4 intro. "This steps helps you record the outcome
> from evaluating the web pages selected in the previous step. This
> evaluation includes comparing the results between structured and
> randomly selected web pages, and adjusting the selected sample as
> needed. More guidance on this step is provided in WCAG-EM Step 4: Audit
> the Selected Sample. Note: For each WCAG 2.0 Success Criteria listed you
> can record aggregated results for the entire sample, or, optionally, you
> can record results for each web page by activating 'Show web pages to
> enter individual results'. Use the 'sample to evaluate' filters to
> control which web pages get displayed when 'Show web pages to enter
> individual results' is activated. ->
> Record the outcome from evaluating the web pages selected in the
> previous step. Compare the results between the structured page and
> randomly selected pages, and if needed, adjust the selected sample in
> the previous step. More guidance on this step is provided in WCAG-EM
> Step 4: Audit the Selected Sample.</p><p>Note: For each WCAG 2.0 success
> criteria, you can enter "Results for the entire sample" and you can
> enter results for individual web pages. You can choose to enter either
> or both. To enter individual results, select the page(s) under "Sample
> to Evaluate" (in the left column); then under the specific success
> criteria, select "Show web pages to enter individual results".
>
> * [medium-low] Sample to Evaluate (i) box. "This column lists all web
> pages selected in the previous step. You can select items to show them
> when 'Show web pages to enter individual results' is activated in the
> next column. This allows you show and hide individual web pages that you
> are evaluating, or several web pages at the same time." -> "This section
> lists the web pages you selected in the previous step. The web pages
> that are checked in this section are listed when "Show web pages to
> enter individual results" is activated in the "Success Criteria to
> Evaluate" section. You can show an individual web page that you are
> evaluating, or show several web pages at the same time.</p><p>The
> chainlink icon opens the web page in a separate browser window.
> [SLH note: medium because "columns" is visual only]
>
> * [medium-low] Success Criteria to Evaluate (i) box. "This column lists
> all WCAG 2.0 Success Criteria applicable, based on the conformance
> target defined in Step 1. In addition to the nominal results 'Not
> checked', 'Passed', 'Failed', 'Not present', and 'Cannot tell', you can
> optionally provide more details, comments, or observations in the
> accompanying text box." -> This section lists the WCAG 2.0 success
> criteria at the level (A, AA, or AAA) that you set in the "Conformance
> target" in Step 1. You can select results as: 'Not checked', 'Passed',
> 'Failed', 'Not present', and 'Cannot tell'; and you can provide details,
> comments, or other observations made during evaluation in the
> accompanying text box."
> [SLH note: medium because "columns" is visual only. I'm not really sure
> that last bit is necessary – it's pretty obvious from the UI – but I
> guess it's OK to include since this is on-demand-only info.]
>
> *Wording Tweaks: Step 5*
>
> * [high-medium] Recordings of any evaluation specifics. -> Record of
> evaluation specifics (optional)
> [SLH: I didn't have a clue what this was until I looked it up.
> "Recordings" to me means audio recording.]
> (i) box: "Optionally, archive the web pages audited and use this text
> field to record the evaluation tools, web browsers, assistive
> technologies, other software, and methods used to audit them." ->
> WCAG-EM suggests that you archive the web pages audited. You can use
> this text field to record the evaluation tools, web browsers, assistive
> technologies, other software, and methods used for the evaluation. What
> you enter here will be included in the generated report. After you
> download the report, you could delete or edit this information in the
> HTML file before submitting the report.
> [SLH note: WCAG-EM says "…is typically kept internal and not shared by
> the evaluator"]
>
> * [low-medium] Step 5 intro. "Provide concluding information about the
> evaluation, such as the evaluation date or period and a summary of the
> overall evaluation findings. The detailed evaluation findings provided
> in the previous step are also included below for review and to help you
> write the executive summary. " -> Provide additional information about
> the evaluation that you want included in the report. The evaluation
> findings that you entered in the previous step are included below under
> "Detailed audit results" for review and to help you write the executive
> summary.
>
> * [low] Evaluation date label and (i) box. "Provide the completion date
> or duration period for carrying out this evaluation." -> Provide the
> completion date or duration dates of this evaluation. You can use any
> date format.
> [maybe also change the label from "Evaluation date" to "Evaluation
> date(s)"]

All done.


> *Wording Tweaks: View Report*
>
> * [medium-low] Related WCAG 2.0 resources. The list wording and links
> should be updated to match what's in the nav bar now -- "How to meet
> WCAG 2.0" -> "How to meet WCAG 2.0 Quick Reference" and links to the
> overview pages.

Wilco, please make this change (use the same link texts and destination 
in the 'view report', as we already have in the 'related resources').


> [low] Actually, will this often be a printed report and so we should
> list the URIs here? If so, we could list both the overview and the /TR/:
> - Web Content Accessibility Guidelines (WCAG) 2.0 – Overview: www.@@
> Technical Standard: www.@@
> - How to meet WCAG 2.0 Quick Reference www.@@
> - WCAG Evaluation Methodology (WCAG-EM) – Overview: www.@@ Note: www.@@

Agree. Wilco, please make the links visible beside the link texts.


> * [low] View Report. "Below is the sample evaluation report based on the
> input that you provided in previous steps. You can go back and adjust
> any of this information as needed. You can download the evaluation
> report as HTML to further customize it to your liking, and you can save
> the data locally to your own computer to retrieve it at a later time."
> -> Below is the evaluation report based on the input that you provided
> in previous steps. You can go back to previous steps now and change any
> of this information. You can download the evaluation data so that you
> can work on it later. (This is the same as the "Save"
> functionality.)</p><p>This report is intended to be downloaded. You can
> download the Evaluation Report HTML and CSS files below. You can then
> change the report content and visual design to meet your needs.

Done.


> * [low] Should we change "Download the evaluation data (JSON)" to match
> the wording of "Save" since it's the same functionality?

Changed "Download" to "Save".


> * [low] Reconsider calling this "Sample Website Accessibility Evaluation
> Report". Why "Sample"? Perhaps just "Website Accessibility Evaluation
> Report" or "Generated Website Accessibility Evaluation Report".
>
> * [low] "Sample of Audited Web Pages" is all headline-style
> capitalization and the other headings are sentence-style capitalization
>
> *Wording Tweaks: Save Evaluation Report*
>
> * [medium] Save page. "Use the link below to save your data in a JSON
> data file locally on your computer. You can save partially-complete
> reports and work on them later. You can also use save to transfer the
> data to another computer or web browser. You can Save periodically as
> you work to avoid losing data if your web browser closes.
> [Save data file locally to your computer]
> Use Ctrl + S or ⌘ S on any page as a shortcut to save the your data as
> JSON data file. This shortcut initiates the file save dialog of your web
> browser, or it may automatically save it in your downloads folder
> depending on your web browser settings."
> ->
>
> Use the link below to save the evaluation report information that you
> have entered in a JSON data file locally on your computer.</p>
> [Save data file locally to your computer]
> You can save partially-complete reports and work on them later. You can
> open them from any web browser and you can transfer the data file to
> another computer.</p>
> You can Save periodically as you work to avoid losing data if your web
> browser closes. You can use Windows shortcut keys Ctrl+S or Mac shortcut
> keys ⌘S to open the Save dialog (or automatically save it in your
> downloads folder depending on your web browser settings).</p>
> When you are done entering your evaluation report information, you can
> download an HTML file of the completed report from the <a href>View
> Report</a> page.
>
> *Wording Tweaks: Open/Load page*
>
> * [medium-high] "Load Evaluation Report" title & h1 -> maybe "Open
> Evaluation Report" or "Open/Load Evaluation Report File" or "Open
> Existing Report File"…
> [rationale: connect the "Open" in the toolbar with the title of this page]
>
> * [medium-low] Load Evaluation Report page: "Find and select the JSON
> data file that your previously saved to reload it. If you have already
> entered data, then data file will be loaded in a new report." -> To
> reload a saved file, "Browse" to find the JSON data file that you
> previously saved, then select the "Load data from selected file" button.
> [SLH note: I don't understand "If you have already entered data, then
> data file will be loaded in a new report."]

All done.


> *UI Tweaks: Step 5*
>
> * [low] Left align the field labels.

Wilco, please make that change.


> *Overall*
>
> * [low] Please re-open
> https://github.com/w3c/wcag-em-report-tool/issues/122

Responded there. We cannot maintain a list of supported browsers. We 
hope this list is quickly diminishing anyway.


> * [low] From the Save page and the Open page, provide a link to the
> Start page.

Wilco, if this is not an easy change then please create an issue for the 
next version of the tool.


> * [low] Reconsider where we say "audit" and if it should say "evaluate"
> instead.

Wording is usually taken from WCAG-EM. Let's keep this for usability 
testing, please.

Thanks a lot for this input!

Best,
   Shadi

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

Received on Thursday, 26 February 2015 09:51:58 UTC