Re: Minutes Auto-WCAG retrospective

Great meeting minutes!
My only added comment is that I think we need to be very clear on what our relationship to the ACT Group.
Oh and I do find it very hard to use Github which has definitely constrained my participation.
Cheers
Gian

Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Wilco Fiers <wilco.fiers@deque.com>
Sent: Thursday, July 5, 2018 2:04:50 AM
To: Auto-WCAG List
Subject: Minutes Auto-WCAG retrospective

For those of you who missed it, here are the minutes:

Present: Wilco, Jean, Jey, Leon, Shadi, Emma, Anne

[16:06] <Wilco> https://auto-wcag.github.io/auto-wcag/pages/design/review.html
[16:06] <Wilco> https://github.com/auto-wcag/auto-wcag/pulls
[16:07] <anne_thyme> https://github.com/auto-wcag/auto-wcag/issues/162
[16:07] <JeanKaplansky> Review process isn't helping us the way we had hoped. What should we be doing better? We have 13 PR's right now that are sitting without action.
[16:07] <JeanKaplansky> Also attending - Anne.
[16:08] <JeanKaplansky> GitHub is not the easiest thing to understand with all the different stages and terms involved between GitHub tech. and W3C workflow process. Very hard to track specific rules in the workflow.
[16:09] <anne_thyme> https://github.com/auto-wcag/auto-wcag/projects/1
[16:09] <JeanKaplansky> Suggestion - 1 PR per rule, use GitHub projects to track the rule via kanban boards ^^^
[16:10] <JeanKaplansky> Nothing can be automated because each stage requires a new pull request as a gate to go forward. Too many gates in current workflow - issue, draft stage, current stage, review, final. Bottom line, simplification is a good idea.
[16:10] <JeanKaplansky> Wilco: What should a simplified workflow look like.
[16:11] <JeanKaplansky> See Anne's links.
[16:12] <JeanKaplansky> Essentially removing draft stage, and PR's are being filed before all the comments in a draft are integrated. Final stage to Review stage loop is not working as originally intended. Too many iterations and stuff is still missing from final iteration.
[16:13] <JeanKaplansky> I _think_ I just set myself to scribe.
[16:14] <JeanKaplansky> Jey shows Kanban screen in GitHub...
[16:14] <JeanKaplansky> We may be able to do some things with the columns to enable automation. Jey live demos potentials - new project in GH. GH starts with a default 3-col process.
[16:16] <JeanKaplansky> To-do, In-Progress,  - automation can be managed for these 3 columns via presets. If we can marry the presets to the new process... we should be able to solve this. The nice thing about going in this direction means that we can sort cards within a stage, and use labels for other forms of sort.
[16:17] <JeanKaplansky> Jey: We should try this with automation - it's possible. New columns can be automated through the automation presets.
[16:18] <JeanKaplansky> Automation rules don't preclude more than 3 columns. We need to reconfigure our existing Kanban to watch the status and use the automation presets. The current process requires whomever is working on the ticket to manually move cards around (and that's not happening consistently).
[16:18] <JeanKaplansky> Understand what we want and reverse engineer into our workflow.
[16:19] <JeanKaplansky> Wilco: Based on this - no problem with dropping the draft stage. The reason we have a draft stage is we found that relying on PR's mean issues stay open for a very long time. However, with the rate things are moving at right now, it's not as much of a problem anymore because we're pushing rules through the process more efficiently.
[16:20] <JeanKaplansky> The current process did speed up the rate at which we handle rules (note).
[16:21] <JeanKaplansky> Wilco added a "reviewer wanted" label to tag issues that are missing 2-3 reviewers in order to move forward.
[16:22] <JeanKaplansky> Automation on GH boards do not offer printing the labels - some sort of customization script is required to make a custom label move forward. We need at least 3 people are reviewers in order to properly assign reviewers which enables the group to see the status of stuff in review.
[16:23] <JeanKaplansky> We want 2-3 reviewers to look at an issue even if the first reviewer requests changes.
[16:23] <JeanKaplansky> Do we lose reviewers when changes are made?
[16:23] <JeanKaplansky> What if changes are made that invalidate either the original rule, or comments from other people? This requires another round of review to shake everything out.
[16:24] <JeanKaplansky> If you sign up to review a rule, you're a reviewer on that rule until it's done.
[16:24] <JeanKaplansky> Everything should loop from Work In Progress/In Review until the rule author has incorporated ALL of the comments to finalize state to a PR.
[16:25] <JeanKaplansky> "Reviewer Wanted" we want 3 people to commit to following through.
[16:25] <JeanKaplansky> "Review Required" means that we need the reviewers to revisit.
[16:26] <JeanKaplansky> Should labels exist for "Review Again" or "Back to the Drawing Board"?
[16:26] <JeanKaplansky> There is a "Changes requested" label.
[16:26] <JeanKaplansky> The automation trigger for getting something out of review is when someone addresses a comment and acknowledges it within GH.
[16:27] <JeanKaplansky> Wilco: Alright. Rough idea of where we want to go. Hopefully this resolves the difference in process between definitions and rules. So... Who's going to take this and implement it?
[16:27] <JeanKaplansky> Jey: Volunteers. Anne to work with Jey to build prototype.
[16:28] <JeanKaplansky> Wilco: now that we're starting to use more advanced GH features, we need better documentation.
[16:28] <JeanKaplansky> Wilco: Jean - can you REVIEW documentation? Jean: Yes.
[16:28] <JeanKaplansky> Shadi - I can help... Not sure the best way...
[16:29] <JeanKaplansky> We lost Shadi's audio.
[16:29] <JeanKaplansky> Emma has joined.
[16:29] <JeanKaplansky> (note to self. Look up the Zakim commands if I do this again.)
[16:30] <JeanKaplansky> We need to move on to next topic until Shadi rejoins.
[16:31] <JeanKaplansky> Emma - what's the difference between definitions and rules? Wilco: algorithms used to describe how to share process between rules. We want to rename stuff that is shared between rules "definitions".
[16:31] <JeanKaplansky> Moving on... REBRANDING!!!
[16:32] <JeanKaplansky> Email conversation in progress. Auto-WCAG is no longer as focused on automation as we used to be. Rules format no longer cares if something is automated or manual or hybrid. We're no longer writing explicitly writing steps for a tool developer to implement re: what should fail and what should pass.
[16:33] * shadi agreed to helping with documentation -- was that audible before my network kicked me out again?
[16:33] <JeanKaplansky> We keep walking into this problem in which people outside the group think that we are entirely focused on automation. That's limiting because the rules we're putting out is more of a harmonization effort that can help anyone implement a rule in a harmonized manner, not necessarily an automated manner.
[16:34] <JeanKaplansky> Shadi returns: Happy to help with GH documentation.
[16:35] <JeanKaplansky> Wilco: Suggestions for potential name changes... Testing, some people were thinking a11y-testing or WCAG-testing. Someone suggested QA. Makoto's WCAG-Testing CG got a lot of votes. Shadi made the point that WCAG is kind of implied. Emma makes point that WCAG is a specification and accessibility may encompass far more than WCAG.
[16:36] <JeanKaplansky> Wilco: Are we truly limited to WCAG? Do we even want to rename the group? (Going by the emails pretty much everyone said yes... except Emma)
[16:36] <JeanKaplansky> Emma was thinking in context of the long-term goal - if it's automation then we should stick with automation.
[16:37] <JeanKaplansky> However, if it's no longer automation, then clarifying the name is not necessarily a bad thing to do.
[16:38] <JeanKaplansky> W3C small print: We cannot change the name. We would have to close down the Auto-WCAG group officially and start a whole new group with the new name. Do we really want to change? What's the real benefit? Is it worth the work? We could potentially lose people...
[16:39] <JeanKaplansky> Wilco: Do we REALLY need to start a new group, or can we rebrand while keeping the same web page on the W3?
[16:39] <JeanKaplansky> (most people lost to such a change may not have been fully engaged in participation)
[16:40] <JeanKaplansky> Shadi needs to think about the rebranding. Wilco doesn't want to close the group - seems like a painful process and a lot of work and a lot of cat-herding to get people to move to the new group.
[16:40] <JeanKaplansky> Shadi - if you're OK with keeping the URLs and the mailing list names, then Shadi can ask if we can change the headings and descriptions of the groups in place.
[16:41] <JeanKaplansky> BUT... it will be confusing to go to auto-wcag group, and then seeing something that describes something else. Emma wants to keep WCAG in the name.
[16:46] <JeanKaplansky> Keeping WCAG in the group name means we stick to WCAG only. Does this exclude the W3 EPUB contextual descriptions? Jean: No. W3 Publishing Group is supposed to be WCAG-compliant. Just WCAG-compliant with a particular context in mind.
[16:47] <EmmaJPR> Doing something like changing the name online to Automated WCAG Conformance Testing CG ... could allow keeping the existing URL and CG website, but clarify purpose and connect with ACT TF name.
[16:48] <JeanKaplansky> Wilco: Sounds like we want auto-wcag name to go away. Shadi to look into the "rebranding" question. we don't currently know if rebranding is possible. a 3rd option is to close this group and start a new group and take on all the work that goes with opening a new group.
[16:48] <JeanKaplansky> Wilco lists potential names for a survey that he will build with Shadi to send out to the group.
[16:49] <JeanKaplansky> (See public email list for a collection of potential names suggested.)
[16:49] <Wilco> A) Auto-WCAG (unchanged) B) ACT Rules CG C) WCAG Testing CG D) WCAG ACT Rules CG E) Accessibility Testing CG
[16:52] <JeanKaplansky> Do we want to allow for other groups that test accessibility to have names that include accessibility? Will using the word "accessibility" preclude something like DPUB W3 efforts to create context-specific testing? Shadi - WCAG-ACT-Rules CG. This means that we can have a DPUG-ACT-Rules CG...
[16:52] <JeanKaplansky> Wilco: are you objecting to some of the suggestions?
[16:54] <JeanKaplansky> Shadi: I heard objections to ACT-rules and other various names. We want to make it clear in some way that this is WCAG-ACT... Wilco: This narrows the scope significantly though. This puts us out of Silver. Ties us specifically to writing rules? Doesn't this mean that we would need a new Silver-Testing group later on? Does ACT have agreement on how narrow or broad this group should be?
[16:55] <JeanKaplansky> Also include in survey -  potential risks including name confusion and scope of the group.
[16:55] <JeanKaplansky> Wilco: Publishing Rules to W3.
[16:56] <JeanKaplansky> Auto-WCAG rules were published by W3. It's Wilco's hope to move the work to W3 to make this part of the standard W3 ecosystem - as in take the work coming out of the CG and making it more "official".
[16:58] <JeanKaplansky> HOWEVER... going in this direction means new requirements. More W3 process. Techniques may be affected - they may need to become part of the rule. None of this can be normative. Jean: What's the point of taking the rules out of the CG if none of this can be normative. Wilco: It makes it more official and part of WCAG instead of in addition to WCAG.
[17:00] <JeanKaplansky> Emma - Auto-WCAG - trying to standardize the way testing is done against a set of guidelines. We are focused on WCAG right now. We are trying to standardize the way testing is done to conform to those guidelines. Does the rationale or approach - why something passes or fails - shouldn't this be in the guidelines? The reason that something does or doesn't conform to a guideline exist in the rules, or is it OK to leave guidelines-like infor[CUT]
[17:02] <JeanKaplansky> Shadi - Auto-wcag - or any CG - can do whatever they want. Tomorrow, this group can meet and change the process, etc. This is the difference between a WG and a CG. The WG has a very set process. However, this group is turning out high quality rules. The WG can definitely "bless" any of our rules - however, from that point on, the rule enters the WG process realm. Approved rules must be frozen and marked "approved by the WG."
[17:03] <JeanKaplansky> The rules do reference stuff from the WG guidelines though.
[17:04] <JeanKaplansky> Wilco wants to explore replacing the "failure techniques" in WCAG with rules. The rules are very different though.
[17:04] <JeanKaplansky> Jean: The scribe has a hard-stop...
[17:04] <JeanKaplansky> Wilco ends meeting.

--
Wilco Fiers
Senior Accessibility Engineer - Co-facilitator WCAG-ACT - Chair Auto-WCAG
[cid:BCBD7D4B-677E-4B95-AE3F-60005DBD9EE4]