RE: UIA platform testing

Thanks, Bogdan.

We have started testing, and I was actually going to send mail today regarding some issues we've found with the tests that make it a bit more challenging that I think it needs to be. I'm curious how the automation system worked around these:


  1.  Simple typo inconsistency: "Control Type", "control type", "ControlType" and "Localized Control Type", "localizedControlType", "LocalizedControlType".
     *   This is manageable, but makes understanding the tests challenging
Q: Is there anyone going through to make these consistent? I guess this is the result of multiple people writing test statements.


  1.  TestCases looking for aria-properties, ask in lots of different ways
     *   Some tests say

                                                               i.      <testelement> PROPERTY "AriaProperties" IS "hidden:true"

                                                             ii.      <testelement> PROPERTY "AriaProperties.hidden" IS "true"

                                                           iii.      <testelement> PROPERTY "hidden" IS "true" (and just assumes you know it wants ariaproperties)

                                                           iv.      Might be more.

     *   Some more ambiguous tests ask

                                                               i.      <testelement> PROPERTY "DataGrid.ColumnCount" IS "4" and seems to really be wanting "AriaProperties.ColCount == 4"

           *   The element in inspect.exe shows no other fields showing value "4" so it must be AriaProperties.ColCount
           *   I don't know what DataGrid has to do with anything
     *   This is hard to handle... and probably should be fixed on the test side.
Q: Should we just log bugs in the repo?


  1.  There is an entire family of tests asking for Pattern support, in different ways seemingly:
     *   <testelement> PROPERTY "controlPatterns" CONTAINS "ExpandCollapse"
     *   <testelement> CONTROL PATTERN "ExpandCollapse" EXISTS "false"
Q: is anyone working on this?


  1.  Some I cannot figure out what it's trying to verify at all, inspect.exe shows nothing with a similar name, nothing with a value matching the assertion value.
     *   <testelement> PROPERTY "StyleId_Heading2" IS "present" - inspect shows nothing similar to StyleId_heading2.
     *   Some are more ambiguous, some are less ambiguous.
Q: Not sure what to do with these types of issues. What are your thoughts?




From: Bogdan Brinza
Sent: Monday, May 22, 2017 10:02 PM
To: Rich Schwerdtfeger <richschwer@gmail.com>; John Jansen <John.Jansen@microsoft.com>
Cc: Shane McCarron <shane@spec-ops.io>; Joanmarie Diggs <jdiggs@igalia.com>; ARIA Working Group <public-aria@w3.org>
Subject: Re: UIA platform testing


Adding John who drives UIA testing on our end.

________________________________
From: Rich Schwerdtfeger <richschwer@gmail.com<mailto:richschwer@gmail.com>>
Sent: Monday, May 22, 2017 12:28:01 PM
To: Bogdan Brinza
Cc: Shane McCarron; Joanmarie Diggs; ARIA Working Group
Subject: UIA platform testing

Hi Bogdan,

Shane fixed the bug in the ATTA to enable manual testing to proceed.

How close are you to being able to start manual testing?

Have you updated the wiki to reflect the UIA Core AAM changes you made?

Regards,
Rich

Rich Schwerdtfeger