Summary of the MMI TPAC face to face meeting and related meetings

W3C Technical Plenary and Advisory Council Meeting

(http://www.w3.org/2015/10/TPAC/)

 

Multimodal Interaction Working Group Meeting

Group page http://www.w3.org/2002/mmi/

 

This is a summary of the Multimodal Interaction Working Group's meetings at
the 2015 W3C Technical Plenary and Advisory Council Meeting (TPAC) in
Sapporo, Japan, October 26-30. We had a number of productive discussions
with other Working Groups, including Geolocation, Spatial Data and Timed
Text (getting feedback on new EMMA 2.0 features), Automotive (discussion of
the MMI Architecture) as well as discussions with the Web of Things Interest
Group, on discovery and thing description. 

The minutes of the meeting will be published soon.

 

October 26

Monday: 

1. We held a meeting with the Geolocation WG [1]  to review the
<emma:location> element [2]. Most of the attributes in <emma:location> are
based on attributes in the Geolocation API; however there are two additional
higher level attributes in <emma:location> (<emma:address> and
<emma:description>) that aren't included in the Geolocation API. These are
related to the work of the Spatial Data on the Web WG, so we scheduled a
meeting with them on Tuesday. We were also referred to some non-W3C specs
which might be relevant to locations in EMMA, specifically, "A
Presence-based GEOPRIV Location Object Format" [3] and "Civic Addresses"
[4].

 

October 27

Tuesday:

1. Meeting with Spatial Data on the Web WG [5]. This group is just getting
started, but they may have specs relevant to the <emma:description>
attribute of <emma:location> in a few months. 

2. Meeting with the Automotive Working Group [6] to review the Multimodal
Architecture [7] for automotive applications, and to talk about location in
EMMA 2.0.

 

Technical Plenary Day

October 28

Wednesday:

1. The Wednesday TPAC agenda consisted primarily of breakout sessions. We
held a session on "Standards for Personal Assistants". One interesting
suggestion that came out of this session was that virtual personal
assistants should be more knowledgeable -- that is, they should be able to
intelligently analyze web pages or other information on the web, and present
the information to the user according to the user's preferences. Typically,
the "intelligence" of intelligent personal assistants is thought of as
having to do with the ability to engage in a natural spoken conversation,
but this discussion also considered the issue of how the assistant actually
acquires knowledge and determines how to present it to the user. See the
summary and minutes [8] for details of the session.

 

October 29

Thursday:

MMI Meeting

1. Review of EMMA 1.0 in preparation for EMMA 2.0 discussion in the
afternoon [9]. 

2. We joined the Web of Things Interest Group [10], specifically the task
forces on API's and Discovery. Helena Rodriguez (invited Expert)  presented
her work on discovery of multimodal components to the WoT discovery task
force. 

3. Review of EMMA 2.0 features [9] and discussion of a JSON version of EMMA.
>From the discussions with the meeting observers, there did not seem to be
much value in officially standardizing a JSON format for EMMA, since there
are many tools that can automatically generate JSON from XML. 

4. We reviewed several MMI use cases [11], specifically UC-1, 2, 3, 7 and 8.
We also discussed two new EMMA use cases. One was using EMMA to represent
user input from gestures in a Virtual Reality enviroment, and the second was
to record user input, including speech and gesture, in order to be able to
replay interactions for testing. Deborah Dahl (Invited Expert) also
presented a demo of EMMA 2.0 including the <emma:output> element.

 

October 30 

Friday:

MMI Meeting

1. Discovery: Helena Rodriguez (Invited Expert) presented initial work on
Discovery Vocabulary, which provides a vocabulary for describing Modality
Components. This work follows on from Discovery Use Cases and Requirements
[12] and Discovery State Handling [13] This will form the basis of the next
MMI WG publication on Discovery. The overall approach is based on concepts
from the Web Services Description Language (WSDL) 2.0 [14] except that in
this case the description is of modality components rather than Web
Services. The vocabularies are used to describe aspects of the component
such as its type, interaction modalities available, and operations. 

It was pointed out that some work in accessibility, such as Universal Remote
Console (ISO 24752) [15] and also some work on individual needs and
preferences for accessibility is relevant to this topic.

2. Joint meeting with the Timed Text WG [16]  to present our example of EMMA
outputing Timed Text [17]:

We presented the example of EMMA 2.0 containing Timed Text as an output
format. Use cases include showing caption text for a video in a browser,
presenting machine-translated captions, or video descriptions. The main
comment we received was that if we wanted to show the final rendering for
the user, the timed text should be in a container, for example, an HTML
page. However, the timed text within EMMA would not need to be in a
container if the EMMA document represents an intermediate stage prior to
final rendering, and the exact presentation for the user hasn't yet been
specified. Other feedback was that the section would be clearer if we added
discussion of specific use cases.

3.  Use cases:

Our final agenda topic was continuing to discuss use cases and brainstorm
new use cases. We added  UC-17: User interface as a sensor, UC-18: smart
remote for appliances and UC-19: Handling millions of components [11].

 

Links

[1] Geolocation Working Group http://www.w3.org/2008/geolocation/

[2] <emma:location> element http://www.w3.org/TR/emma20/#s4.1.10

[3] A Presence-based GEOPRIV Location Object Format
http://tools.ietf.org/html/rfc4119

[4] Civic Addresses: http://tools.ietf.org/html/rfc5139#section-4

[5] Spatial Data on the Web WG http://www.w3.org/2015/spatial/wiki/Main_Page

[6] Automotive Working Group http://www.w3.org/auto/wg/

[7] Multimodal Architecture and Interfaces: http://www.w3.org/TR/mmi-arch/

[8] Standards for Personal Assistants
https://www.w3.org/wiki/TPAC/2015/SessionIdeas#Standards_for_Personal_Assist
ants

[9] EMMA slides (prepared by Michael Johnston, Ivited
Expert):https://lists.w3.org/Archives/Member/w3c-mmi-wg/2015Oct/0020.html
(to be replaced with public link)

[10] Web of Things Interest Group http://www.w3.org/WoT/IG/

[11] MMI use cases https://www.w3.org/wiki/MMI/Use_Cases

[12] Discovery Use cases and Requirements
http://www.w3.org/TR/mmi-discovery/

[13] Discovery State Handling http://www.w3.org/TR/mmi-mc-discovery/

[14] Web Services Description Language (WSDL .20)
http://www.w3.org/TR/wsdl20/

[15] Universal Remote Console
http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber
=60821

[16] Timed Text Working Group http://www.w3.org/AudioVideo/TT/

[17] Timed Text in EMMA Timed Text in EMMA 2.0
http://www.w3.org/TR/emma20/#s6.2