6 Loading assistive technologies for access to User Agent DOMs

There are several methods for developers to accomplish this. Most of these methods fall into four categories:

1. Launch the entire AT inside the address space of the (UA) browser
2. Launch some part of the AT, a piece of stub code, a DLL, a Browser Helper Object [special DLL], etc., inside the address space of the (UA) browser
3. Write your own combined UA/AT (e.g., pwWebSpeak)
4. Out-of-process access to the DOM

These methods are ordered as developments within a rapidly changing technology with the most recent advances/methods listed first.

6.1.1 Loading assistive technologies for direct access to User Agent DOMs

(Note: Method 6.1.1 and 6.1.2 are very similar. What differs is the amount of, or capability of, the AT that actually gets loaded in the same process or address space as the User Agent.)

Access to application specific data across process boundaries might be costly in terms of performance. Therefore, user agents may wish to provide a mechanism to load the entire assistive technology (AT), into the process space of the application as a separate thread with direct access to the DOM.

Determining the Assistive Technologies to load

A technique for accomplishing this would be to store a reference to an assistive technology in a system registry file or properties file in the case of Java. Registry files are common among many operating system platforms.

In Windows you have the system registry file. On OS/2 you have the system.ini file and on distributed network client networks you often have a system registry server that can be querried by an application running on the network client computer.

In Java 2, the existence of an accessibility.properties causes the system event queue to check the file for assistive technologies required for loading. If the file contains a property called assistive_technologes, it will load all registered assistive technologies and start them on there own thread in the Java Virtual Machine that is a single process. An example entry for Java is as follows:

    assistive_technologies=com.ibm.sns.svk.AccessEngine

In Windows, a similar technique could be followed by storing the name of a Dynamic Link Library (DLL) for an assistive technology in a designated assistive technology key name, AT pair. An example entry for Windows could be as follows:

    HKEY_LOCAL_MACHINE\Software\Accessibility\DOM      "ScreenReader, VoiceNavigation"

Attaching the Assistive Technologies to the DOM

Once the assistive technology is determined from the registry, any user agent on the given operating system can now determine if an assistive technology needs to be loaded with their application and load it.

On a non-Java platform, a technique to do this would be to create a separate thread with a reference to the User Agent's DOM using a Dynamic Link Library (DLL). This new thread will load the DLL and call a specified DLL entry name with a pointer to the DOM interface. The assistive technology's task will then run until such time as is necessary to end communication with the DOM.

Once loaded, the assistive technology can monitor the DOM as needed. The assistive technology has the option of communicating with a main assistive technology of its own and process the DOM as a caching mechanism for the main AT application or be used as a bridge to the DOM for the main assistive technology.

In the future, it will be necessary to provide a more comprehensive reference to the application that not only provides direct access to it's client area DOM, but also multiple DOM's that it is processing and an event model for monitoring them.

Example Technique: Java's Direct Access

Java is a working example where the direct access to application components is performed in a timely manner. Here, an assistive technology running on a separate thread monitors GUI events such as focus changes. Focus changes give the AT notification of which component object has focus. The AT can communicate directly with all components in the application by walking the parent/child hierarchy and connecting to each component's methods and monitor events directly. In this case an AT has direct access to component specific methods as well as those provided for by the Java Accessibility API. There is no reason that a DOM interface to UA components could not be provided

In Java 1.1.x, Sun's Java access utilities load an assistive by monitoring the Java awt.properties file for the presence of assistive technologies and loads them as shown in the folowing code example:

   import java.awt.*;
   import java.util.*;
       
   String atNames = Toolkit.getProperty("AWT.assistive_technologies",null);
   if (atNames != null) {
       StringTokenizer parser = new StringTokenizer(atNames," ,");
       String atName;
       while (parser.hasMoreTokens()) {
          atName = parser.nextToken();
          try {
             Class.forName(atName).newInstance();
          } 
          catch (ClassNotFoundException e) {
             throw new AWTError("Assistive Technology not found: " + atName);
          } 
          catch (InstantiationException e) {
             throw new AWTError("Could not instantiate Assistive" + " Technology: " + atName);
          } 
          catch (IllegalAccessException e) {
             throw new AWTError("Could not access Assistive" + " Technology: " + atName);
          } catch (Exception e) {
             throw new AWTError("Error trying to install Assistive" + " Technology: " + atName + " " + e);
          }
       }
   }

In the above code example, the function Class.forName(atName).newInstance() creates a new instance of the assistive technology. The constructor for the assistive technology will then be responsible for monitoring application component objects by monitoring system events.

In the following code example, the constructor for the the assistive technology "Access Engine," adds a focus change listener using Java accessibility utilities. When the assistive technology is notified of an objects gaining focus it has direct access to that object. If the Object, o, implemented a DOM interface the assistive technology would now have direct access to the DOM in the same process space as the application.

   import java.awt.*;
   import javax.accessibility.*;
   import com.sun.java.accessibility.util.*;
   import java.awt.event.FocusListener;

   class AccessEngine implements FocusListener {
      public AccessEngine() {
         //Add the AccessEngine as a focus change listener
         SwingEventMonitor.addFocusListener((FocusListener)this);
      }
      
      public void focusGained(FocusEvent theEvent) {
         // get the component object source
         Object o = theEvent.getSource();
         // check to see if this is a DOM component
         if (o instanceof DOM) {
            ...
         }
      }
      public void focusLost(FocusEvent theEvent) {
         // Do Nothing
      }
   }

In this example, the assistive technology has the option of running standalone or acting as a cache for a bridge that communicates with a main assistive technology running outside the Java virtual machine.

6.1.2 Loading part of the assistive technologies for direct access to User Agent DOMs

(See note as part of 6.1.1.)

Access to application specific data across process boundaries might be costly in terms of performance. Therefore, user agents may wish to provide a mechanism to load part of the assistive technology (AT) into the process space of the application as a separate thread, with direct access to the DOM, to provide the specific functionality they require. This could consist of a piece of stub code, a DLL, a Browser Helper Object, etc. An example of how to do this follows.

Browser Helper Objects:

In order to attach to a running instance of Internet Explorer 4.0, you can use a "Browser Helper Object." A "Browser Helper Object" is a DLL that will attach itself to every new instance of Internet Explorer 4.0 (only if you explicitly run iexplore.exe). You can use this feature to gain access to the object model of a particular running instance of Internet Explorer. You can also use this feature to get events from an instance of Internet Explorer 4.0. This can be tremendously helpful when many method calls need to be made to IE, as each call will be performed much more quickly than the out of process case.

There are some requirements when creating a Browser Helper Object

• The application that you create must be an in-proc server (that is, DLL).
• This DLL must implement IObjectWithSite.
• The IObjectWithSite::SetSite() method must be implemented. It is through this method that your application receives a pointer to Internet Explorer's IUnknown. (Internet Explorer actually passes a pointer to IWebBrowser2 but the implementation of SetSite() receives a pointer to IUnknown.) You can use this IUnknown pointer to automate Internet Explorer or to sink events from Internet Explorer.
• It must be registered as a Browser Helper Object as described above.

For more information, please check out:

Java Access Bridge:

In order for native Windows ATs to gain access to Java applications without the creating a Java native solution Sun Microsystems provides the "Java Access Bridge." This bridge is loaded as an AT as described in section 6.1.3. The bridge uses a Java Native Invocation (JNI) to Dynamic Link Library) (DLL) communication and caching mechanism that allows a native assistive technology to gather and monitor accessibility information in the Java environment. In this environment, the AT determines that a Java application or applet is running and communicates with the Java Access Bridge DLL to process accessibility information about the application/applet running in the Java Virtual Machine.

6.1.3 Loading assistive technologies "as" the User Agent with access to the DOMs

Specialized user agents might also include the necessary assistive technology as part of their interface, and thus provide possibly the best of both worlds. An example would be pwWebSpeak, from The Productivity Works.

(Have The Productivity Works provide short description.)

6.1.4 Loading assistive technologies for indirect access to User Agent DOMs

Access to application specific data across process boundaries or address space might be costly in terms of performance. However, there are other reasons to consider when accessing the User Agent DOM that might lead a developer to wish to access the DOM from their own process or memory address space. One obvious protection this method provides, is that if the User Agent application fails, it doesn't disable the user's AT as well. Another consideration would be legacy systems, where the user relies on their AT for access to other applications as well as the User Agent, and thus would have their AT loaded all the time, not just for accessing the User Agent.

There are several ways to gain access to the User Agent's DOM. Most User Agents support some kind of external interface, or act as a mini-server to other applications running on the desktop. Internet Explorer is a good example of this, as IE can behave as a component object model (COM) server to other applications. Mozilla, the open source release of Navigator also supports cross platform COM (XPCOM).

An example of using COM to access the IE Object Model can be seen in the code snippet below. This is an example of how to use COM to get a pointer to the WebBrowser2 module, which in turn allows you to get a interface/pointer to the document object, or IE DOM for the web page in view.

   /* first, get a pointer to the WebBrowser2 control */
   if (m_pIE == NULL) {
      hr = CoCreateInstance(CLSID_InternetExplorer, NULL, CLSCTX_LOCAL_SERVER, IID_IWebBrowser2, (void**)&m_pIE);


      /* next, get a interface/pointer to the document in view, this is an interface to the document object model (DOM)*/

      void CHelpdbDlg::Digest_Document() {
         HRESULT hr;
         if (m_pIE != NULL) {
            IDispatch* pDisp;
            hr = m_pIE->QueryInterface(IID_IDispatch, (void**) &pDisp);
            if (SUCCEEDED(hr)) {
        
               IDispatch* lDisp;
               hr = m_pIE->get_Document(&lDisp);
               if (SUCCEEDED(hr)) {
           
                   IHTMLDocument2* pHTMLDocument2;
                   hr = lDisp->QueryInterface(IID_IHTMLDocument2, (void**) &pHTMLDocument2);
                   if (SUCCEEDED(hr)) {
            
                      /* with this interface/pointer, IHTMLDocument2*, you can then work on the document */
                      IHTMLElementCollection* pColl;
                      hr = pHTMLDocument2->get_all(&pColl);
                      if (SUCCEEDED(hr)) {
                    
                         LONG c_elem;
                         hr = pColl->get_length(&c_elem);
                         if (SUCCEEDED(hr)) {
                            FindElements(c_elem, pColl);
                         }
                         pColl->Release();
                      }
                      pHTMLDocument2->Release();
                   }
                   lDisp->Release();
               }
               pDisp->Release();
            }
         }
      }
   }   

For more information on using COM with IE, please visit the Microsoft web site:

For more information on using XPCOM with Mozilla, please visit the Mozilla web site:

For a working example of the method described in 6.1.4, please visit the following web site and review HelpDB, developed as a testing tool for web table navigation.