Introduction - rich user interaction patterns and platforms

Web users today seek ever increasing interactivity and responsiveness in Web applications, particularly as those applications expand in scope and function to provide increasing levels of capability ("richness') in presentation, control, and data management. The drive for richness stems from a desire for authors to address greater functional goals in their applications -- moving from web content to commerce to collaborative services. The means for achieving richness stem largely from migrating function back to the client that historically has been resident on the server -- we do not seek to re-establish a world of client-server computing but rather to increasingly leverage client capabilities in an era of greatly expanded distributed computing and online services.

The Rich Web Application Backplane Incubator Group (XG) has had as its goal understanding, demonstrating, and documenting two areas of work: (1) authoring patterns for increased client-side capabilities in not only rich presentation or graphics but also in navigation control and data capture and validation for high-function web applications, and (2) techniques for extending current browser runtimes to support these authoring patterns across a variety of markup formats beyond the varients of HTML supported "natively" by existing browsers.

In this XG report, be begin in the first section by presenting an example scenario and application focused on expense tracking. This application will be used in the following section to to illustrate the manner in which a variety of markup formats including XHTML, SVG, XForms, SMIL, and other emerging formats such as the Open Document Format (ODF) come together in a coherent working whole. In section two, we illustrate a set of relevant authoring patterns for similar high function extensible web applications, including a separation of concerns between data and presentation and corresponding single and multiple node data binding, an approach to loose coupling between separately authored components using XML events, coordination of client-side state across the page using shared and transformed data models, and computed metadata over that client-side state to drive UI behavior such as conditional display of content and conditional triggering of control flow. In many cases, these patterns may already be supported today in several of the formats used in the example scenario. In other cases, we identify extensions to those vocubularies where increased function or simplified authoring could be possible.

In section three, we go behind these application patterns to describe their implementation in current browser platforms. We have come to focus on this point significantly in the XG's work since we believe that in the evolution of the web today it is not sufficient only to propose improved formats without corresponding emphasis on pragmatic routes to adoption. Indeed, in many cases the patterns we describe have been known for some time but have not reached critical mass in the developer community due to lack of convenient runtime support rather than lack of consensus as to their utility. We are excited at this point as to the potential for greater cross-format adoption of concepts such as MVC data binding, event-based coordination, implicit creation of the UI through repeats over queries on the active client data, and so on pricisely since we now see and are actively involved in the creation of practical, performant, and deployable techniques for their use in current browser technologies.

Example: Introducing the Expense Tracking Application

End-user experience supported by the scenario

Figure 1 is the main page of a simple expense tracking web application that we will use to illustrate authoring patterns and runtime integration of multiple markup formats. A list of expenses can be entered with a category, description, date, and amount for each. Expenses are stored in an XForms data model bound to this list. The list is coordinated with an entry area which provides extended editing controls in a master-detail relationship with the expense list. Once created, expense entries may be edited in place using the UI controls in the master list. Entries may be deleted by clicking on the "X" at the right of each record.

Figure 2 shows a graphical view of expenses aggregated by category and drawn using an SVG piechart. The expense categories included in the piechart are determined dynamically by a data-driven aggregation of all non-zero categories from the underlying expense list as described and illustrated in markup in the next section. Figure 3 similarly shows a projection of expense entries by date again using SVG.

A set of expenses, once entered, may be submitted for re-imbursement by a report-generation tab implemented using the Open Document Format (ODF) text format as shown in Figure 4. Open Document text (ODT) elements are used to define a template for expense reimbursement with the actual expense document containing only those categories with non-zero expenses. ODF documents in the OASIS 1.0 specification may include XForms data models. As shown in Figure 4 we use this capability to bind the expense report directly to data in the underlying expense tracker's data model. Further, the table of contents entries are computed dynamically based again on those expense categories having non-zero entries and thus needing to appear in the running document. Of particular note in this example is this enablement of ODF as a web-centric markup format participating directly in the overall page lifecycle with other web formats such as XHTML, XForms, and SVG.

Schedules permitting, our goal with this scenario has been to augment it with time-based controller logic implemented using SMIL 3.0 markup running in the browser. Section X, below, describes an AJAX implementation of SMIL following the same platform extension mechanisms as used for XForms, XML Events, and ODF elsewhere in this scenario. We describe below the use of this library in a separate demo of an interactive tour of cities including New York and Amsterdam [cite]. In the detailed discussion of the authoring and runtime behavior of our finance example in the following section, however, we outline our current thoughts as to the use of SMIL as a general controller for web applications having some element of time-based behavior -- and how this might be applied to the expense tracking scenario.

Scenario as illustration of infrastructure requirements

While Section 4, below, treats the implementation of each format used in our expense tracking example in detail, we provide first a roadmap to understanding the document structure and runtime behavior of this example.

Figure X shows a cut-away view of the "stack" of a running document. The base browser provides core HTML and XHTML parsing capabilities along with required styling (CSS) and network support as usual. Pages are authored using standards-compliant formats (XHTML, SVG, XForms, XML Events, SMIL, ODF) and delivered to the browser directly. In contrast to other approaches to browser extension we do not view these markups as authoring-time only formats but rather support them directly in the browser by attaching their executable "behaviors" to the original source document elements as shown in the figure. The technique used for this behavioral "decoration" varies by browser (see the discussion below in Section X) but such differences are localized to page loading time and are not visible to page authors as they continue to see a common document structure consistent with its originally authored structure (view source is what is running not transcoded into HTML) and consistent behavior in all browsers supported by this library (currently Internet Explorer 7, FireFox 3, and the Safari/Chrome WebKit-based rendering engines).

Indeed, we see this approach to using Javascript as an important implementation technique for extensible markup elements whether they are XML or HTML-based. The examples we have given are from XML vocabularies (others may include State Chart XML -- SCXML, and ...) but a script-based "tag library" mechanism would be interesting for supporting incremental modules of HTML5, for example, as they appear in working drafts or perhaps separable Recommendations -- and also as a means to accelerate their implementation and adoption in a broad developer community before "native" implementations in each browser are fully available.

It's also important to track emerging markup formats from sources other than the W3C and ask whether they might be of interest as components of web-based applications. Our example shows one such format, from OASIS -- the ODT text format used for expense reimbursement. There are potentially other emerging formats, particularly in the space of industry vertical standards, which may similarly be of interest for use client-side with first-class implementation of their behaviors. Examples might include HL7 and Clinical Document Architecture (CDA) from the healthcare industry, ACORD from insurance, and XBRL from the finance industry. These formats are of potential interest not just as data-exchange formats but to the extent they have interactive behaviors also as components of client-side web applications.

Application patterns supporting RIAs

Patterns for client-side rich data management

Data binding

Transparant composition -- event based patterns

Extension with new controller formats

SMIL

SCXML

Extension with new presentation or content formats

ODF

Leveraging existing AJAX libraries as custom controls

Overview of approaches to language extensibility

Server-side transcoding

Client-side transcoding, e.g. dojo.E, "Ample" sdk (www.amplesdk.com)

Progressive enhancement and unobtrusive javascript, e.g. Dojo

XBL

Cross-platform client-side behaviors, e.g. Ubiquity-xforms

Requirements for standardization

Appendix: Implementation of formats used in the financial services application

XForms and XML Events

SMIL

Open Document Format Text (ODT)

Other possibilities: SCXML, Acord, HL7 -- many have behaviors not just data formats.

HTML and its extensions through HTML5/6/custom modules