Re: Rendered content vs source content.

Hans Riesebos wrote:
> 
> Rendered content vs source content.
> 
> I have been doing some thinking and in in my humble opinion I find the rendered content of real importance: Here are some reasons.

Thanks for sending these, Hans. By the way, I verified with Philippe Le
Hegaret that
content generated by style sheets does not appear in the DOM tree.
 
> I believe MSAA gives you rendered content.
> 
> If the AT relies (sort of piggybags) on a UA one can assume the AT wants to make 
> this UA accessible.

Do you want to make the UA accessible or the content accessible (or,
most
likely, both)?

> Therefore the AT wants to be aware of the closed-loop interaction between User and > UA of action and feedback. (What is the current point of regard, what is
> selected). To this purpose the AT needs to know about rendered content.

Yes, for the selection and focus in particular, I agree. 

> The AT might want to control some of the rendering in order to get the information
> the UA-AT user wants. This could be done by controlling the UA and also by 
> controlling style sheets.

What control is necessary? Do you need to change style sheet values or
just read them?

> Without access to stylesheets I don't believe the AT can synchronise different 
> output modalities. (I'm not sure of all the possibilities).

I think that's partially true. But there is no guarantee, for example,
that style sheets are used by the browser to style the selection/focus.
So, even if you give access to style sheets, unless you give access
to the rendering structure, you still may not have the control that you
seek. 

Furthermore, I think the selection and focus are considered part of the
user interface (though they can designate content), and here we are
specifically talking about access to content, not the user interface
(covered by checkpoint 5.4, which mentions focus and selection
explicitly).

> As said in the teleconference, the AT historically uses an Off Screen Model (OSM). > The making of an OSM is difficult because it involves reverse engineering the
> rendering proces. The value of the DOM for the AT is that it does not need to
> reverse engineer rendering. However, if the AT has to work with the pure
> structured content without any rendering taken place, the AT needs to be able to
> render itself in order to obtain information on visibility, position, etc.. This
> seems redundant, because the UA it works with does the same rendering.

To paraphrase:
 1) The AT must not be forced to rely on the rendering structure.
 2) The AT may want to use some information provided by the rendering
structure.

> Now what the UAGL currently says is that new/future AT is much helped with access
> to the DOM. I agree. But is seems unnatural for an AT to use a graphical desktop
> browser to provide an aural-only rendering of internet-content. It would seem
> natural for  an AT that wants graphical stuff to use a graphical desktop browser.
> In this last situation the UA does more service by providing rendered info to this
> AT.

> Concluding:
> - To me it seems that the rendered sturcture is of real interest to the AT.
> - Access to stylesheets is priority one.

Though I understand the value of being able to synchronize views, and
of an AT wanting to know what's on the screen, I'm still not convinced
that access to style sheets + the source tree will suffice. Suppose the 
author has used style sheets to reposition content (with the 'float' or 
'position' properties). In this case, the graphical layout may have
nothing to
do with the tree order. So an AT walking the tree and reading computed
property values would basically have to implement CSS floats and
positioning to figure out the screen layout. While an AT providing
a non-graphical rendering could ignore many of the CSS visual properties
(e.g., font-family), there are many it could not (e.g., font-size)
since they have an effect on layout, and that seems to interest ATs.

An attempt to summarize scenarios and difficulties:

1) If an AT works with a source tree and applies style sheets itself,
   it's basically a stand-alone user agent. In this case, for example,
   the AT should respect the media types described in the CSS2 spec [1]
   (e.g., braille, aural). In this case, the user agent doesn't
   need to support the DOM CSS module since the AT is implementing
   CSS anyway.

2) If an AT wants the rendering of a "host" user agent, it can 
   compute some useful information from the DOM tree in conjunction
   with the DOM CSS module (which offers computed CSS property
   values, I believe, for each node in the DOM tree). This obviously
   requires ATs to be CSS-aware, even if they don't implement
   all of CSS. There are at least two limitations to this approach,
   however:

        a) The host user agent may not base its rendering entirely
           on CSS (e.g., for selection and focus and possibly more),
           therefore, the CSS information will not suffice.

        b) CSS positioning would require that the AT calculate
           positions itself in order to know the layout of
           rendered content. This includes z-index, which 
           specifies how to layer content on the screen.
           Other properties, such as 'clip' and 'overflow' will
           also affect what's finally rendered.

        c) You could even imagine other cases such as text with
           the same background and foreground color (thus, invisible
           to the user). The AT would have to take into account
           color values as well.

    My conclusion from these examples, which are surely not
    exhaustive, is that for an AT to really know what the host
    UA has rendered, it will have to recognize/implement most
    CSS properties.

3) If an AT wants the rendering of a "host" user agent, it can
   ask for the UA's rendering structure. However, there are no
   APIs that I know of for accessing the rendering structure of
   a user agent. That's why ATs have had to rely on offscreen models,
   I believe. 

   In my opinion, adding a requirement that a user agent expose
   its (possibly proprietary) internal representation of the
   rendering structure is an additional requirement in these guidelines
   that should cause us to return to Working Draft status (e.g.,
   last call). 

I do not deny that there may be cases where access to the
rendered content of the UA will interest the AT. I also agree with
Hans that access to source style sheets is priority 1 WHEN the AT 
implements the style sheet language (scenario 1 above). Access to
style sheets may not be priority 2 in scenario 2, where the 
property values have already been computed (and where media types
are not significant since the AT is relying on the media type
of the host user agent, not its own). 

I am very relieved that we have identified this issue during
Candidate Recommendation. I would phrase this issue: "Should the UA
be required to provide access to its rendering structure?" I envision
a couple of possible solutions:

  1) We answer "no" for this version of the Guidelines and move on.
  2) We answer "yes" and limit ourselves to providing access
     to the DOM core + DOM CSS module (which was, in essence, our
     resolution yesterday [2]) and we acknowledge that this might
     not satisfy all needs of an AT.
  3) We answer "yes" and impose a new requirement to provide
     access to the internal rendering structure, and (in my opinion)
     we return to Working Draft since this is a significant new
     requirement.

 - Ian

[1] http://www.w3.org/TR/CSS2/media.html
[2] http://lists.w3.org/Archives/Public/w3c-wai-ua/2000JanMar/0374.html
-- 
Ian Jacobs (jacobs@w3.org)   http://www.w3.org/People/Jacobs
Tel/Fax:                     +1 212 684-1814 or 212 532-4767
Cell:                        +1 917 450-8783

Received on Friday, 25 February 2000 10:30:30 UTC