Re: WCAG-EM Report Tool comments

Hi Shawn,

Some updated responses on the previously open issues:


On 26.2.2015 10:51, Shadi Abou-Zahra wrote:
> Hi Shawn,
>
> Thank you for these excellent comments. Please consider using Github
> next time. Please find below responses to your comments:
>
> Wilco, some items for you below as well.
>
>
> On 26.2.2015 07:50, Shawn Henry wrote:
>> All of these comments are for editor's discretion now, or for
>> consideration for the next version. There are some comments marked
>> [high] that I think should be considered for this version.
>>
>> *Bug*
>>
>> *[high] Step 2: Explore the Target Website and Step 3: Select a
>> Representative Sample. The (x)s don't seem to be working to delete a
>> line.
>
> I also had this weird problem before. Try restarting your browser (as
> odd as this sounds). Wilco, there seems to be something strange here
> that we need to try and locate. Not sure what we can do right now. We
> cannot reproduce it. Wilco, any updated thoughts on this?

Wilco seems to have located and fixed the bug.


>> *Wording Tweaks: Start page*
>>
>> * [medium-high] Start page: "Note: This tool does not record or
>> automatically save the information that you enter. Use the 'Save' link
>> (Ctrl + S or ⌘ S) that is at the top of all pages to save your data in a
>> file locally on your computer." -> Note: This tool does not
>> automatically save the information that you enter. To save your data in
>> a file locally on your computer, use Windows shortcut keys Ctrl+S or Mac
>> shortcut keys ⌘S to open the Save dialog. (Or the "Save" link at the top
>> of the page will open the Save Evaluation Report page and from there the
>> "Save data file locally to your computer" link will open the Save
>> dialog.)
>
> Done. But quite long. How about just "Or use the 'Save' link at the top
> of the page" (without parenthesis)? The instructions are there anyway.
>
>
>> * [medium] Start page: "Trick: To have all WCAG 2.0 Success Criteria
>> listed in your final report, select the conformance target 'Level AAA'
>> in the tool and edit the conformance target text in your final HTML
>> report." -> You can include in your report WCAG 2.0 success criteria
>> beyond the conformance target. For example, the website is only required
>> to meet Level AA, yet you want to also include Level AAA success
>> criteria in the report. In Step 1, select the "Conformance target" as
>> "Level AAA". Then after you download the report, manually change the
>> Conformance target to Level AA in the downloaded HTML file.
>
> Done. Also quite long...
>
>
>> * [medium-low] Start page: " You can download the report with your input
>> as an HTML file and customize it." -> The tool provides your report as
>> HTML and CSS files. You can download the files from the "View Report"
>> page. You can then change the report content and visual design.
>>
>> * [low] Start page: "You can save partially-complete reports and work on
>> them later. Use the 'Open' link at the top and find the file that you
>> previously saved to reload it. If you have already entered data, then
>> 'Open' will load the file in a new report." -> You can save
>> partially-complete reports and work on them later. To reload a saved
>> file, use the "Open" link at the top and find the file that you
>> previously saved.
>> [SLH note: I don't understand "If you have already entered data, then
>> 'Open' will load the file in a new report."]
>>
>> * [typo] Start page. "It is designed for experienced evaluators who know
>> Web Content Accessibility Guidelines (WCAG) 2.0 and somewhat familiar
>> with WCAG-EM." -> It is designed for experienced evaluators who know Web
>> Content Accessibility Guidelines (WCAG) 2.0 and are somewhat familiar
>> with WCAG-EM.
>
> All done.
>
>
>> *Wording Tweaks: Step 1 page*
>>
>> * [high] Step 1. Conformance target (i) box: "Select a target WCAG 2.0
>> conformance level ('A', 'AA', or 'AAA') for the evaluation. For more
>> information, see WCAG-EM Step 1.b: Define the Conformance Target." ->
>> Select a target WCAG 2.0 conformance level ('A', 'AA', or 'AAA') for the
>> evaluation. For more information, see WCAG-EM Step 1.b: Define the
>> Conformance Target.</p><p>This selection filters the success criteria
>> that are included in "Step 4: Audit the Selected Sample" and in the
>> report. For example, if you select Level A, only the Level A success
>> criteria will be included. If you want the report to include all of the
>> success criteria, select Level AAA here. If the conformance target for
>> the website is less, then after you download the report, manually change
>> the Conformance target to the appropriate Level in the downloaded HTML
>> file.
>
> Done but quite long.
>
>
>> *Wording Tweaks: Step 2 page*
>>
>> * [high] Step 2: Explore the Target Website. Web Technologies Relied
>> Upon (i) box. "… Note: The input field 'Web Technology' is also a
>> drop-down list with a selection of web technologies that you can use, or
>> add to." -> "</p><p><strong>Note:</strong> The "Web Technology" and
>> "Specification Address (URL)" input fields provide drop-down lists that
>> you can pick from. You can also type different text into these fields.
>>
>> * [medium-low] Step 2. "(Optional) Note Taking" -> Optional Exploration
>> Notes
>>
>> * [typo] Step 2. "…This steps helps you determine…-> …This step helps
>> you determine…
>>
>> * [low] Step 2. "Identify the web technologies used to provide the
>> website and optionally take notes on other aspects of the website." ->
>> Identify the web technologies used to provide the website and, if you
>> want, take notes on other aspects of the website. OR Identify the web
>> technologies used to provide the website. You can also take notes on
>> other aspects of the website.
>> [1. typo "This steps helps…"; 2. Take out "optionally" – it's awkward in
>> the sentence and I think not necessary given "optional" is below)]
>>
>> * [low] Step 2. (Optional) Note Taking (i) box: "This section allows you
>> to take optional notes during your website exploration. These notes will
>> be provided to you again in the next step, to help you select a sample
>> of web pages for evaluation. The notes are not part of the final report,
>> but they will be kept along with all your other input in the data for
>> saving and reloading." -> "This section lets you record information
>> during your website exploration. These notes are provided in the next
>> step, to help you select a sample of web pages for evaluation.
>> <strong>These notes are not included in the final report.</strong> They
>> are saved when you save the file, and available when you later reload
>> the file.
>
> All done.
>
> Wilco, note the <strong> in the locale file for this last comment.
>
>
>> * [low] Step 2. Last 2 (i) boxes: "Use this field to take notes about"
>> -> You can use this field to take notes about…
>> [rationale: to re-iterate that this is optional]
>>
>> * [low] Step 2. Variety of web page types (i) box. "This includes notes
>> about varying styles, layouts, structures, and functionality provided on
>> the website." -> This includes notes about different styles, layouts,
>> structures, and functionality used on the website.
>>
>> *Wording Tweaks: Step 3 page*
>>
>> * [medium] Step 3. "Select web pages for evaluation. The web page
>> recordings and notes that you took during website exploration in the
>> previous step are provided here, to help you make the selection. It is
>> important to ensure that your selection of web pages is representative
>> of the target website. " -> Select web pages for evaluation. The notes
>> that you took during exploration in the previous step are provided here,
>> to help you select the pages. Ensure that the web pages selected are
>> representative of the target website."
>> [SLH note: medium priority because "web page recordings" is confusing]
>>
>> * [low-medium] Step 3. 2 (i) boxes: "Note: Web pages include different
>> web page states; see definition of web page states." -> Note: "Web
>> pages" include different "web page states"; see the definition of web
>> page states.
>>
>> *Wording Tweaks: Step 4*
>>
>> * [typo] Step 4 intro. "This steps helps you…" -> This step helps you…
>> [although text edit suggestion below removes that word to better match
>> the approach of the other pages]
>>
>> * [medium-high] Step 4 intro. "This steps helps you record the outcome
>> from evaluating the web pages selected in the previous step. This
>> evaluation includes comparing the results between structured and
>> randomly selected web pages, and adjusting the selected sample as
>> needed. More guidance on this step is provided in WCAG-EM Step 4: Audit
>> the Selected Sample. Note: For each WCAG 2.0 Success Criteria listed you
>> can record aggregated results for the entire sample, or, optionally, you
>> can record results for each web page by activating 'Show web pages to
>> enter individual results'. Use the 'sample to evaluate' filters to
>> control which web pages get displayed when 'Show web pages to enter
>> individual results' is activated. ->
>> Record the outcome from evaluating the web pages selected in the
>> previous step. Compare the results between the structured page and
>> randomly selected pages, and if needed, adjust the selected sample in
>> the previous step. More guidance on this step is provided in WCAG-EM
>> Step 4: Audit the Selected Sample.</p><p>Note: For each WCAG 2.0 success
>> criteria, you can enter "Results for the entire sample" and you can
>> enter results for individual web pages. You can choose to enter either
>> or both. To enter individual results, select the page(s) under "Sample
>> to Evaluate" (in the left column); then under the specific success
>> criteria, select "Show web pages to enter individual results".
>>
>> * [medium-low] Sample to Evaluate (i) box. "This column lists all web
>> pages selected in the previous step. You can select items to show them
>> when 'Show web pages to enter individual results' is activated in the
>> next column. This allows you show and hide individual web pages that you
>> are evaluating, or several web pages at the same time." -> "This section
>> lists the web pages you selected in the previous step. The web pages
>> that are checked in this section are listed when "Show web pages to
>> enter individual results" is activated in the "Success Criteria to
>> Evaluate" section. You can show an individual web page that you are
>> evaluating, or show several web pages at the same time.</p><p>The
>> chainlink icon opens the web page in a separate browser window.
>> [SLH note: medium because "columns" is visual only]
>>
>> * [medium-low] Success Criteria to Evaluate (i) box. "This column lists
>> all WCAG 2.0 Success Criteria applicable, based on the conformance
>> target defined in Step 1. In addition to the nominal results 'Not
>> checked', 'Passed', 'Failed', 'Not present', and 'Cannot tell', you can
>> optionally provide more details, comments, or observations in the
>> accompanying text box." -> This section lists the WCAG 2.0 success
>> criteria at the level (A, AA, or AAA) that you set in the "Conformance
>> target" in Step 1. You can select results as: 'Not checked', 'Passed',
>> 'Failed', 'Not present', and 'Cannot tell'; and you can provide details,
>> comments, or other observations made during evaluation in the
>> accompanying text box."
>> [SLH note: medium because "columns" is visual only. I'm not really sure
>> that last bit is necessary – it's pretty obvious from the UI – but I
>> guess it's OK to include since this is on-demand-only info.]
>>
>> *Wording Tweaks: Step 5*
>>
>> * [high-medium] Recordings of any evaluation specifics. -> Record of
>> evaluation specifics (optional)
>> [SLH: I didn't have a clue what this was until I looked it up.
>> "Recordings" to me means audio recording.]
>> (i) box: "Optionally, archive the web pages audited and use this text
>> field to record the evaluation tools, web browsers, assistive
>> technologies, other software, and methods used to audit them." ->
>> WCAG-EM suggests that you archive the web pages audited. You can use
>> this text field to record the evaluation tools, web browsers, assistive
>> technologies, other software, and methods used for the evaluation. What
>> you enter here will be included in the generated report. After you
>> download the report, you could delete or edit this information in the
>> HTML file before submitting the report.
>> [SLH note: WCAG-EM says "…is typically kept internal and not shared by
>> the evaluator"]
>>
>> * [low-medium] Step 5 intro. "Provide concluding information about the
>> evaluation, such as the evaluation date or period and a summary of the
>> overall evaluation findings. The detailed evaluation findings provided
>> in the previous step are also included below for review and to help you
>> write the executive summary. " -> Provide additional information about
>> the evaluation that you want included in the report. The evaluation
>> findings that you entered in the previous step are included below under
>> "Detailed audit results" for review and to help you write the executive
>> summary.
>>
>> * [low] Evaluation date label and (i) box. "Provide the completion date
>> or duration period for carrying out this evaluation." -> Provide the
>> completion date or duration dates of this evaluation. You can use any
>> date format.
>> [maybe also change the label from "Evaluation date" to "Evaluation
>> date(s)"]
>
> All done.
>
>
>> *Wording Tweaks: View Report*
>>
>> * [medium-low] Related WCAG 2.0 resources. The list wording and links
>> should be updated to match what's in the nav bar now -- "How to meet
>> WCAG 2.0" -> "How to meet WCAG 2.0 Quick Reference" and links to the
>> overview pages.
>
> Wilco, please make this change (use the same link texts and destination
> in the 'view report', as we already have in the 'related resources').
>
>
>> [low] Actually, will this often be a printed report and so we should
>> list the URIs here? If so, we could list both the overview and the /TR/:
>> - Web Content Accessibility Guidelines (WCAG) 2.0 – Overview: www.@@
>> Technical Standard: www.@@
>> - How to meet WCAG 2.0 Quick Reference www.@@
>> - WCAG Evaluation Methodology (WCAG-EM) – Overview: www.@@ Note: www.@@
>
> Agree. Wilco, please make the links visible beside the link texts.
>
>
>> * [low] View Report. "Below is the sample evaluation report based on the
>> input that you provided in previous steps. You can go back and adjust
>> any of this information as needed. You can download the evaluation
>> report as HTML to further customize it to your liking, and you can save
>> the data locally to your own computer to retrieve it at a later time."
>> -> Below is the evaluation report based on the input that you provided
>> in previous steps. You can go back to previous steps now and change any
>> of this information. You can download the evaluation data so that you
>> can work on it later. (This is the same as the "Save"
>> functionality.)</p><p>This report is intended to be downloaded. You can
>> download the Evaluation Report HTML and CSS files below. You can then
>> change the report content and visual design to meet your needs.
>
> Done.
>
>
>> * [low] Should we change "Download the evaluation data (JSON)" to match
>> the wording of "Save" since it's the same functionality?
>
> Changed "Download" to "Save".
>
>
>> * [low] Reconsider calling this "Sample Website Accessibility Evaluation
>> Report". Why "Sample"? Perhaps just "Website Accessibility Evaluation
>> Report" or "Generated Website Accessibility Evaluation Report".
>>
>> * [low] "Sample of Audited Web Pages" is all headline-style
>> capitalization and the other headings are sentence-style capitalization
>>
>> *Wording Tweaks: Save Evaluation Report*
>>
>> * [medium] Save page. "Use the link below to save your data in a JSON
>> data file locally on your computer. You can save partially-complete
>> reports and work on them later. You can also use save to transfer the
>> data to another computer or web browser. You can Save periodically as
>> you work to avoid losing data if your web browser closes.
>> [Save data file locally to your computer]
>> Use Ctrl + S or ⌘ S on any page as a shortcut to save the your data as
>> JSON data file. This shortcut initiates the file save dialog of your web
>> browser, or it may automatically save it in your downloads folder
>> depending on your web browser settings."
>> ->
>>
>> Use the link below to save the evaluation report information that you
>> have entered in a JSON data file locally on your computer.</p>
>> [Save data file locally to your computer]
>> You can save partially-complete reports and work on them later. You can
>> open them from any web browser and you can transfer the data file to
>> another computer.</p>
>> You can Save periodically as you work to avoid losing data if your web
>> browser closes. You can use Windows shortcut keys Ctrl+S or Mac shortcut
>> keys ⌘S to open the Save dialog (or automatically save it in your
>> downloads folder depending on your web browser settings).</p>
>> When you are done entering your evaluation report information, you can
>> download an HTML file of the completed report from the <a href>View
>> Report</a> page.
>>
>> *Wording Tweaks: Open/Load page*
>>
>> * [medium-high] "Load Evaluation Report" title & h1 -> maybe "Open
>> Evaluation Report" or "Open/Load Evaluation Report File" or "Open
>> Existing Report File"…
>> [rationale: connect the "Open" in the toolbar with the title of this
>> page]
>>
>> * [medium-low] Load Evaluation Report page: "Find and select the JSON
>> data file that your previously saved to reload it. If you have already
>> entered data, then data file will be loaded in a new report." -> To
>> reload a saved file, "Browse" to find the JSON data file that you
>> previously saved, then select the "Load data from selected file" button.
>> [SLH note: I don't understand "If you have already entered data, then
>> data file will be loaded in a new report."]
>
> All done.
>
>
>> *UI Tweaks: Step 5*
>>
>> * [low] Left align the field labels.
>
> Wilco, please make that change.
>
>
>> *Overall*
>>
>> * [low] Please re-open
>> https://github.com/w3c/wcag-em-report-tool/issues/122
>
> Responded there. We cannot maintain a list of supported browsers. We
> hope this list is quickly diminishing anyway.
>
>
>> * [low] From the Save page and the Open page, provide a link to the
>> Start page.
>
> Wilco, if this is not an easy change then please create an issue for the
> next version of the tool.

This turns out to be a bigger change and the use-case is unclear. We 
have added an issue (#207) to address in the next revision.


>> * [low] Reconsider where we say "audit" and if it should say "evaluate"
>> instead.
>
> Wording is usually taken from WCAG-EM. Let's keep this for usability
> testing, please.
>
> Thanks a lot for this input!
>
> Best,
>    Shadi
>

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

Received on Thursday, 26 February 2015 17:08:58 UTC