Re: UIA platform testing

Hi Bogdan.

Regarding DPub AAM: Are you planning to implement them any time soon?
Before you say "no"...

My guess (having done the implementation for WebKitGtk) is that the
implementation will be pretty quick and easy to do in Edge. It appears
the mappings for the roles on your platform are all:

* Control Type: Text (with the exception of doc-cover, which is Image)
* Localized Control Type: <rolename with the "doc-" removed>
* If a landmark, also:
  - Landmark Type: Custom
  - Localized Landmark Type: <rolename with the "doc-" removed>

So while officially -- and for the publishing industry -- it's 39 roles,
for implementations it's more like a handful of roles and 39 localizable
strings. Low-hanging fruit, in other words....

If you think your team might be able to pull off the implementation
before too long, we can hold off on adding results for Edge. Otherwise,
I can add an Edge column to
https://w3c.github.io/test-results/dpub-aam/all.html, showing all the
mappings as "FAIL." If you'd like me to do the latter, what address(es)
would you like me to use in
https://w3c.github.io/test-results/dpub-aam/#index-of-implementations-in-reports?

--joanie

On 05/24/2017 01:33 AM, Bogdan Brinza wrote:
> Few notes:
> 
> - DPUB AAM - we currently don't implement those mappings, so this part can be skipped for EdgeHTML
> - Things similar to "StyleId_Heading2" - this is about TextPattern and can be verified in Inspect.exe using Text Pattern Explorer (available on the <body> element).
> 
> -----Original Message-----
> From: Joanmarie Diggs [mailto:jdiggs@igalia.com] 
> Sent: Tuesday, May 23, 2017 10:24 AM
> To: John Jansen <John.Jansen@microsoft.com>; Bogdan Brinza <bbrinza@microsoft.com>; Rich Schwerdtfeger <richschwer@gmail.com>
> Cc: Shane McCarron <shane@spec-ops.io>; ARIA Working Group <public-aria@w3.org>
> Subject: Re: UIA platform testing
> 
> Hi John.
> 
> <cough>Welcome to my world</cough>. ;)
> 
> If you go through the wiki history, you will find quite a few instances of my correcting statements. Automated tools have proven handy in this
> regard: They flag a failure, you look into it and realize the bug is not in the implementation or your ATTA, but in the testable statement itself. :-/
> 
> Before you reach this as your definitive conclusion, it's important to check what is stated in the Core AAM, because I've also discovered bugs there (i.e. the testable statement is correct given what is in the Core AAM, which is wrong -- or at least highly suspect). If the bug is in the Core AAM, please file an issue against the spec. Bonus points for providing a patch which fixes it, should time permit and the fix be obvious.
> 
> But if in the end you are sure the fault lies 100% in the testable statement, and it's a statement for UIA, please just edit the wiki to reflect the correct assertion.
> 
> Thanks!
> --joanie
> 
> On 05/23/2017 07:05 PM, John Jansen wrote:
>> Thanks, Bogdan.
>>
>>  
>>
>> We have started testing, and I was actually going to send mail today 
>> regarding some issues we've found with the tests that make it a bit 
>> more challenging that I think it needs to be. I'm curious how the 
>> automation system worked around these:
>>
>>  
>>
>>  1. Simple typo inconsistency: "Control Type", "control type",
>>     "ControlType" and "Localized Control Type", "localizedControlType",
>>     "LocalizedControlType".
>>      1. This is manageable, but makes understanding the tests 
>> challenging
>>
>> *Q: Is there anyone going through to make these consistent? I guess 
>> this is the result of multiple people writing test statements.*
>>
>>  
>>
>>  2. TestCases looking for aria-properties, ask in lots of different ways
>>      1. Some tests say
>>
>>                                                                i.     
>> <testelement> /PROPERTY/ "AriaProperties" /IS/ "*_hidden:_*true"
>>
>>                                                              ii.     
>> <testelement> /PROPERTY /"AriaProperties*_.hidden_*" /IS/ "true"
>>
>>                                                            iii.     
>> <testelement> /PROPERTY/ "hidden" /IS /"true" (and just assumes you 
>> know it wants ariaproperties)
>>
>>                                                            iv.     
>> Might be more.
>>
>>      2. Some more ambiguous tests ask
>>
>>                                                                i.     
>> <testelement> /PROPERTY /"DataGrid.ColumnCount" /IS /"4" and seems to 
>> really be wanting "AriaProperties.ColCount == 4"
>>
>>              1. The element in inspect.exe shows no other fields showing
>>                 value "4" so it must be AriaProperties.ColCount
>>              2. I don't know what DataGrid has to do with anything
>>      2. This is hard to handle... and probably should be fixed on the test
>>         side. 
>>
>> *Q: Should we just log bugs in the repo?*
>>
>>  
>>
>>  3. There is an entire family of tests asking for Pattern support, in
>>     different ways seemingly:
>>      1. <testelement> /PROPERTY /"controlPatterns" /CONTAINS
>>         /"*ExpandCollapse*"
>>      2. <testelement> /CONTROL PATTERN /"*ExpandCollapse*" /EXISTS/ "false"
>>
>> *Q: is anyone working on this?*
>>
>>  
>>
>>  4. Some I cannot figure out what it's trying to verify at all,
>>     inspect.exe shows nothing with a similar name, nothing with a value
>>     matching the assertion value.
>>      1. <testelement> /PROPERTY /"StyleId_Heading2" /IS/ "present" -
>>         inspect shows nothing similar to StyleId_heading2.
>>      2. Some are more ambiguous, some are less ambiguous.
>>
>> *Q: Not sure what to do with these types of issues. What are your 
>> thoughts?*
>>
>>  
>>
>>  
>>
>>  
>>
>>  
>>
>> *From:* Bogdan Brinza
>> *Sent:* Monday, May 22, 2017 10:02 PM
>> *To:* Rich Schwerdtfeger <richschwer@gmail.com>; John Jansen 
>> <John.Jansen@microsoft.com>
>> *Cc:* Shane McCarron <shane@spec-ops.io>; Joanmarie Diggs 
>> <jdiggs@igalia.com>; ARIA Working Group <public-aria@w3.org>
>> *Subject:* Re: UIA platform testing
>>
>>  
>>
>> Adding John who drives UIA testing on our end.
>>
>> ----------------------------------------------------------------------
>> --
>>
>> *From:*Rich Schwerdtfeger <richschwer@gmail.com 
>> <mailto:richschwer@gmail.com>>
>> *Sent:* Monday, May 22, 2017 12:28:01 PM
>> *To:* Bogdan Brinza
>> *Cc:* Shane McCarron; Joanmarie Diggs; ARIA Working Group
>> *Subject:* UIA platform testing
>>
>>  
>>
>> Hi Bogdan,
>>
>>  
>>
>> Shane fixed the bug in the ATTA to enable manual testing to proceed. 
>>
>>  
>>
>> How close are you to being able to start manual testing?
>>
>>  
>>
>> Have you updated the wiki to reflect the UIA Core AAM changes you made?
>>
>>  
>>
>> Regards,
>>
>> Rich
>>
>>  
>>
>> Rich Schwerdtfeger
>>
>>  
>>
>>  
>>
>>  
>>
> 
> 

Received on Wednesday, 24 May 2017 11:14:11 UTC