AT&T Labs-Research
Jennifer L. Beckham, beckham@research.att.com
Nils Klarlund, klarlund@research.att.com
This document presents an event flow and Mealy machine semantics for SMIL. We clarify the role of events in a SMIL document by making a distinction between the event based timing and the scheduling properties of a SMIL element. A Mealy machine is a finite state machine whose current state and input determine an output for every transition. In our semantics, we associate a scheduler and Mealy machine (which we call a SMILey Machine) with each element; the automaton is driven by input events sent both from the sync-arc environment and from its scheduler. The environment enforces sync-arcs among elements. The scheduler produces events that control the simple and active duration of an element. The fundamental machine that models the state of a SMIL element has four states: dormant, idle, active, and finished; its input events are partitioned into duration events, parent events, and timing events; and its outputs are synchronization events. The duration eventsare emitted by the scheduler of a SMILey Machine; they are activate, repeat, and finish. Parent events (wake-up and reset) and timing events (beginElement and endElement) are emitted by the environment in order to enforce sync-arcs among elements. Output events from the Mealy machine are synchronization events (begin, end, reset children, and wake children) that are used by the environment to maintain sync-arcs.
Figure 1: Event flow in a SMIL interpreter.
Figure 1 illustrates the event flow for a specific element in a SMIL document. Events that control the state of the SMILey machine originate from two sources: the scheduler for the element and the sync-arc environment. Through events sent from these sources, an element transitions through a dormant state and the awake states which are idle, active, and finished. An element propagates environment events to its scheduler and sends output events to the environment. The purpose of the scheduler is to control the temporal behavior of an element. It determines transitions to and from the active state, therefore establishing the simple and active durations of the element. The scheduler cannot force a dormant element into an awake state and is not aware of any other SMIL elements.
The sync-arc environment is a machine that receives interactive events from outside sources and enforces sync-arcs, the relationships among nodes. It controls the state of an element by issuing parent events and timing events. The parent of an element controls whether it is able to be scheduled through wake-up and reset events. In this respect, the parent has sole control over when the element is in an awake state. Timing events can request that an element be scheduled to activate by issuing a beginElement event or scheduled to finish by issuing endElement event. These events result from sync-arcs that result from relationships between nodes as well as from the relationships to interactive user events, such as user click events that schedule an element for activation. We have used the SMIL DOM function names, beginElement() and endElement(), because the timing events embody the same semantics as these SMIL DOM functions.
An element sends synchronization events to the environment. Synchronization events allow the environment to maintain the sync-arc with other elements in the document. Wake children and reset children are parental synchronization events that force child elements into idle and dormant states, respectively. The begin and end events allow the environment to preserve the sync-arc of an element with other elements.
@@SMIL documentation refers to a begin event as something that schedules the start of a particular element, as well as, something an element sends to other elements. These two distinct events may happen at different times, but are referred to with the same name. We distinguish clearly between beginElement, a timing event akin to SMIL DOM's beginElement() and beginElementAt(), and begin, a synchronization event sent to and handled in the environment. BeginElement is an event that schedules the element for a transition to the active state.
Figure 2: A SMIL element's Mealy Machine. The figure gives the input values on the left and output values on the right (i.e. input/output). events sent from the scheduler are in blue, from the sync-arc environment in red, and from the parent in purple.
Figure 2 shows the state diagram of a SMILely machine. In this machine, an element may wait in any state for no time, a known positive time, or an indeterminate time. The parent controls the time spent in the dormant state and the scheduler determines the time spent in the idle, active, and finished states.
When an element is in the dormant state, it is only able to receive one specific event--the wake-up event that corresponds to the parent's transition to the active state. The significance is that the element ignores events from external sources, specifically a beginElement event from the environment. At any time, a parent may force an element to reset, placing it back in the dormant state.
An element transitions to the idle state after waking up. An element idles until a specified time or event prompts the scheduler to activate the element. The scheduler sends an activate event to move the element from the idle to the active state. On that transition, the machine sends a wake children and begin event to the environment. Also, a move to the active state initiates the active timeline of the element in the scheduler.
Once in the active state, an element presents the media or waits out its active timeline. The active duration is the time spent in the active state before a transition out of the state and a simple duration is the time spent in the active state before a repeat or finish event occurs. Repeat is a simple transition from the active to active state that forces all children to reset then wake-up, and throws a repeat event to other elements. The scheduler controls this transition when the element has a repeatDur, repeatCount or repeat attribute specified. An element may receive an endElement event while it is in the active state. This type of event occurs through the DOM via endElement() and endElementAt() or can be triggered by an event on another element’s timeline.
The scheduler sends a finish event when the element's active timeline ends and the machine transitions to the finished state. If it is a video element and fill="frozen" or fill="hold", the player stops its presentation and the last defined state of the media remains on the screen. The element sends a reset children and end event to the environment on the transition to the finished state.
An element may receive a beginElement event from the DOM via beginElement() and beginElementAt() or triggered by an event on the timeline of another element. These can either force an element back to the idle state or keep an element in its current state, depending on the restart attribute of the attribute. The element propagates environment events to the scheduler when the environment forces an element transition to a new state or sends an endElementevent.
The table below presents a transition table for the Mealy machine.
from\to |
Dormant | Idle | Active | Finished |
| Dormant |
Input: Wake-up |
|||
| Idle |
Input: Reset |
Input: beginElement |
Input: Activate |
|
| Active |
Input: Reset |
When:restart="always" |
Input: Repeat Input: endElement When: restart="whenNotActive" or restart="never
|
Input: Finish Output: end; reset children |
| Finished |
Input: Reset |
When:restart="whenNotActive" or restart="always" |
When:restart="never" |
Table 1: A transition table for a SMILey Machine.
We explain the event communication for the example below.
<par dur="60s">
<audio id= "song1" begin="button.click" src="song1.au" dur= "2min"/>
<img id="img1" begin= "song1.begin+2s" src= "img1.jpg"dur="30s"/>
</par>
Assume a user clicks on the button at time one second. The high-level behavior of the SMIL document is: the par container activates at time zero; one second later the audio plays on the button click; two seconds pass and the image is displayed for 30 seconds; and finally the par container finishes at 60 seconds consequently stopping the audio container.
Each time step involves an intricate communication among the environment, elements, and schedulers. At the very start, time zero, the environment wakes the par element and it transitions to the idle state then its scheduler makes it transition to active. This causes the par children, audio and img, to both wake-up and transition to idle. When user clicks on the button at one second, the environment sends a beginElement to song1 and it transitions to active. The scheduler for img1 is notified that the img needs to activate in two seconds. After two seconds pass, at time three seconds, the scheduler for img1 activates the element. The image has duration 30 seconds, so at time 33 seconds the scheduler for img1 places it in the finished state by telling it to finish. At 60 seconds, the par duration concludes and the scheduler for the par tells it to finish. This forces song1 to transition from active to dormant and img1 from finish to dormant. Communication at each time step happens instantaneously. The following table gives the complete event flow for this example.
| Time 0s | Time 1s | Time 3s |
|
Before: par, img1, and song1 are in dormant state After: par is in active state img1 and song1 are in idle state Event Flow: env to par: wake-up (par to idle) par to sched: wake-up sched to par: activate par to env: begin par to env: wake-up children env to img1: wake-up (img1 to idle) img1 to sched: wake-up env to song1: wake-up (song1 to idle) song1 to sched: wake-up |
Before: par is in active state After: par and song1 are in active state img1 is in idle state Event Flow: env receives click on button env to song1: beginElement (song1 to idle) song1 to sched: beginElement sched to song1: activate (song1 to active) song1 to env: wake-up children song1 to env: begin env to img1: beginElement at 2s (img1 to idle) img1 to sched: beginElement at 2s |
Before: par and song1 are in active state img1 is in idle state After: par, song1, and img1 are in active state Event Flow: sched to img1: activate (img1 to active) img1 to env: wake-up children img1 to env: begin |
| Time 33s | Time 60s | |
|
Before: par, song1, and img1 are in active state After: par and song1 are in active state img1 is in finished state Event Flow: sched to img1: finish (img1 to finished) img1 to env: end img1 to env: reset children |
Before: par and song1 are in active state img1 is in finished state After: par is in finished state img1 and song1 are in dormant state Event Flow: sched to par: finish (par to finished) par to env: end par to env: rest children env to img1: reset (img1 to dormant) env to song1: reset (song1 to dormant) song1 to env: end song1 to env: reset children |
Table 2: The event flow for the example. States and transitions
are indicated with the following abbreviations.
-- The Sync-arc Environment sends a wake-up event to the par SMILey Machine. --
env to par: wake-up
-- The par SMILey Machine transitions to idle. --
(par to idle)
-- The par SMILey Machine sends the wake-up event to its scheduler. --
par to sched: wake-up