RE: UIA platform testing from Siegman, Tzviya - Hoboken on 2017-05-24 (public-aria@w3.org from May 2017)

From: Siegman, Tzviya - Hoboken <tsiegman@wiley.com>
Date: Wed, 24 May 2017 17:02:57 +0000
To: Rich Schwerdtfeger <richschwer@gmail.com>, Bogdan Brinza <bbrinza@microsoft.com>
CC: Joanmarie Diggs <jdiggs@igalia.com>, John Jansen <John.Jansen@microsoft.com>, ARIA Working Group <public-aria@w3.org>
Message-ID: <CY1PR0201MB1611916C6B213B7C010F933CD5FE0@CY1PR0201MB1611.namprd02.prod.outlook.>
Just a point of clarification about EPUB. DPUB-ARIA is not actually part of the EPUB specs because it was not finished in time for the last release of EPUB.  However, DPUB-ARIA will be a part of future releases of EPUB and Web Publications.



Tzviya Siegman
Information Standards Lead
Wiley
201-748-6884
tsiegman@wiley.com<mailto:tsiegman@wiley.com>

From: Rich Schwerdtfeger [mailto:richschwer@gmail.com]
Sent: Wednesday, May 24, 2017 12:51 PM
To: Bogdan Brinza
Cc: Joanmarie Diggs; John Jansen; ARIA Working Group; Siegman, Tzviya - Hoboken
Subject: Re: UIA platform testing

Hi Bogdan,

I believe that your browser platform now supports EPUB documents. Given that this is part of EPUB would you not want to fully support the specification or is partial support acceptable for Microsoft?

Incidentally, this is the current publisher support for ARIA-EPUB: https://github.com/mattgarrish/test-results/blob/ccff0e25121d8c928b890a23c38aece0a1377c28/dpub-aria/README.md


Rich



On May 24, 2017, at 12:10 PM, Bogdan Brinza <bbrinza@microsoft.com<mailto:bbrinza@microsoft.com>> wrote:

Appreciate the input - we currently have no plans to implement this mapping. I would agree with you that mapping itself doesn't sound too costly, however additional costs we'll need to drive for proper implementation like internal validations, regression protection, localization start to add up quickly.

More importantly, while cost is an important factor in planning discussions, this is not the only factor. We're typically driven by partner, developer, customer and internal teams' requirements more than just cost and at this point there is very low demand for this feature.

We'll keep this under consideration, but I won't expect we'll land this in time to justify holding off adding Edge results.

If you'd want to list any address in the implementation report - please use my address: Bogdan Brinza, bbrinza@microsoft.com<mailto:bbrinza@microsoft.com>.

Hope this helps!

-----Original Message-----
From: Joanmarie Diggs [mailto:jdiggs@igalia.com]
Sent: Wednesday, May 24, 2017 4:14 AM
To: Bogdan Brinza <bbrinza@microsoft.com<mailto:bbrinza@microsoft.com>>
Cc: John Jansen <John.Jansen@microsoft.com<mailto:John.Jansen@microsoft.com>>; Rich Schwerdtfeger <richschwer@gmail.com<mailto:richschwer@gmail.com>>; ARIA Working Group <public-aria@w3.org<mailto:public-aria@w3.org>>; Siegman, Tzviya - Hoboken <tsiegman@wiley.com<mailto:tsiegman@wiley.com>>
Subject: Re: UIA platform testing

Hi Bogdan.

Regarding DPub AAM: Are you planning to implement them any time soon?
Before you say "no"...

My guess (having done the implementation for WebKitGtk) is that the implementation will be pretty quick and easy to do in Edge. It appears the mappings for the roles on your platform are all:

* Control Type: Text (with the exception of doc-cover, which is Image)
* Localized Control Type: <rolename with the "doc-" removed>
* If a landmark, also:
 - Landmark Type: Custom
 - Localized Landmark Type: <rolename with the "doc-" removed>

So while officially -- and for the publishing industry -- it's 39 roles, for implementations it's more like a handful of roles and 39 localizable strings. Low-hanging fruit, in other words....

If you think your team might be able to pull off the implementation before too long, we can hold off on adding results for Edge. Otherwise, I can add an Edge column to https://w3c.github.io/test-results/dpub-aam/all.html, showing all the mappings as "FAIL." If you'd like me to do the latter, what address(es) would you like me to use in https://w3c.github.io/test-results/dpub-aam/#index-of-implementations-in-reports?

--joanie

On 05/24/2017 01:33 AM, Bogdan Brinza wrote:

Few notes:

- DPUB AAM - we currently don't implement those mappings, so this part
can be skipped for EdgeHTML
- Things similar to "StyleId_Heading2" - this is about TextPattern and can be verified in Inspect.exe using Text Pattern Explorer (available on the <body> element).

-----Original Message-----
From: Joanmarie Diggs [mailto:jdiggs@igalia.com]
Sent: Tuesday, May 23, 2017 10:24 AM
To: John Jansen <John.Jansen@microsoft.com<mailto:John.Jansen@microsoft.com>>; Bogdan Brinza
<bbrinza@microsoft.com<mailto:bbrinza@microsoft.com>>; Rich Schwerdtfeger <richschwer@gmail.com<mailto:richschwer@gmail.com>>
Cc: Shane McCarron <shane@spec-ops.io<mailto:shane@spec-ops.io>>; ARIA Working Group
<public-aria@w3.org<mailto:public-aria@w3.org>>
Subject: Re: UIA platform testing

Hi John.

<cough>Welcome to my world</cough>. ;)

If you go through the wiki history, you will find quite a few
instances of my correcting statements. Automated tools have proven
handy in this
regard: They flag a failure, you look into it and realize the bug is
not in the implementation or your ATTA, but in the testable statement
itself. :-/

Before you reach this as your definitive conclusion, it's important to check what is stated in the Core AAM, because I've also discovered bugs there (i.e. the testable statement is correct given what is in the Core AAM, which is wrong -- or at least highly suspect). If the bug is in the Core AAM, please file an issue against the spec. Bonus points for providing a patch which fixes it, should time permit and the fix be obvious.

But if in the end you are sure the fault lies 100% in the testable statement, and it's a statement for UIA, please just edit the wiki to reflect the correct assertion.

Thanks!
--joanie

On 05/23/2017 07:05 PM, John Jansen wrote:

Thanks, Bogdan.



We have started testing, and I was actually going to send mail today
regarding some issues we've found with the tests that make it a bit
more challenging that I think it needs to be. I'm curious how the
automation system worked around these:



1. Simple typo inconsistency: "Control Type", "control type",
   "ControlType" and "Localized Control Type", "localizedControlType",
   "LocalizedControlType".
    1. This is manageable, but makes understanding the tests
challenging

*Q: Is there anyone going through to make these consistent? I guess
this is the result of multiple people writing test statements.*



2. TestCases looking for aria-properties, ask in lots of different ways
    1. Some tests say

                                                              i.
<testelement> /PROPERTY/ "AriaProperties" /IS/ "*_hidden:_*true"

                                                            ii.
<testelement> /PROPERTY /"AriaProperties*_.hidden_*" /IS/ "true"

                                                          iii.
<testelement> /PROPERTY/ "hidden" /IS /"true" (and just assumes you
know it wants ariaproperties)

                                                          iv.
Might be more.

    2. Some more ambiguous tests ask

                                                              i.
<testelement> /PROPERTY /"DataGrid.ColumnCount" /IS /"4" and seems to
really be wanting "AriaProperties.ColCount == 4"

            1. The element in inspect.exe shows no other fields showing
               value "4" so it must be AriaProperties.ColCount
            2. I don't know what DataGrid has to do with anything
    2. This is hard to handle... and probably should be fixed on the test
       side.

*Q: Should we just log bugs in the repo?*



3. There is an entire family of tests asking for Pattern support, in
   different ways seemingly:
    1. <testelement> /PROPERTY /"controlPatterns" /CONTAINS
       /"*ExpandCollapse*"
    2. <testelement> /CONTROL PATTERN /"*ExpandCollapse*" /EXISTS/ "false"

*Q: is anyone working on this?*



4. Some I cannot figure out what it's trying to verify at all,
   inspect.exe shows nothing with a similar name, nothing with a value
   matching the assertion value.
    1. <testelement> /PROPERTY /"StyleId_Heading2" /IS/ "present" -
       inspect shows nothing similar to StyleId_heading2.
    2. Some are more ambiguous, some are less ambiguous.

*Q: Not sure what to do with these types of issues. What are your
thoughts?*









*From:* Bogdan Brinza
*Sent:* Monday, May 22, 2017 10:02 PM
*To:* Rich Schwerdtfeger <richschwer@gmail.com<mailto:richschwer@gmail.com>>; John Jansen
<John.Jansen@microsoft.com<mailto:John.Jansen@microsoft.com>>
*Cc:* Shane McCarron <shane@spec-ops.io<mailto:shane@spec-ops.io>>; Joanmarie Diggs
<jdiggs@igalia.com<mailto:jdiggs@igalia.com>>; ARIA Working Group <public-aria@w3.org<mailto:public-aria@w3.org>>
*Subject:* Re: UIA platform testing



Adding John who drives UIA testing on our end.

---------------------------------------------------------------------
-
--

*From:*Rich Schwerdtfeger <richschwer@gmail.com<mailto:richschwer@gmail.com>
<mailto:richschwer@gmail.com>>
*Sent:* Monday, May 22, 2017 12:28:01 PM
*To:* Bogdan Brinza
*Cc:* Shane McCarron; Joanmarie Diggs; ARIA Working Group
*Subject:* UIA platform testing



Hi Bogdan,



Shane fixed the bug in the ATTA to enable manual testing to proceed.



How close are you to being able to start manual testing?



Have you updated the wiki to reflect the UIA Core AAM changes you made?



Regards,

Rich



Rich Schwerdtfeger
Received on Wednesday, 24 May 2017 17:03:39 UTC