W3C home > Mailing lists > Public > public-device-apis@w3.org > March 2017

In defence of a motion sensor spec

From: Tobie Langel <tobie@sensors.codespeaks.com>
Date: Tue, 07 Mar 2017 01:09:24 +0100
Message-Id: <1488845364.1111719.902465936.7AA09DB1@webmail.messagingengine.com>
To: public-device-apis@w3.org
So I've been going on for a little while about grouping the motion
sensors specs together rather them have them in different specs.

There are multiple reasons to this. Here are a few:

### Sensor fusion is the key that unlocks the value of motion sensors
###
Taken separately, motion sensors aren't very useful. The data they
provide isn't meaningful to most common use cases. Furthermore, each
ones has characteristics which makes it unfit for certain use cases and
specific weaknesses. Accelerometers and magnetometers are noisy.
Gyroscopes tend to drift and consume more power.
 
Combined, however, they shine. You can use the output of the gyroscope
to smooth out that of the gyroscope. Remove the gravity from the
accelerometer by using the gyroscope, or correct the horizontal drift if
the gyroscope by using the magnetometer.

Combined, motion sensors gives you the fused sensors which power virtual
reality, indoor location, or game controllers. 

The use cases Web developers *really* care about.

Add to that different data representations (quaternions, matrixes, Euler
angles) and you're looking at a dozen different combinations which
probably need distilling in about as many types. Each with their own
weaknesses and strengths. Choosing a fused sensors is making a tradeoff.
Do you need low latency? Pick something gyro-driven, poll it at the
highest frequency you can afford so you have the freshest data possible
on each new animation frame, and deal with the drift when you can. Need
to save on the battery? An accelerometer with a low-pass filter might
do. Need a bearing? Use an magnetometer that's tilt-compensated by the
accelerometers. Etc. 

Platforms like Android provide close to a dozen ready made types which
cover all common use cases. As the fusion is generally done directly in
the chip, they're much more efficient and avoid a lot of the issues that
arise when the fusion is done in software.

Nothing will give you a better idea of the subtleties of combining these
three sensors than the following video. If you can spare 45 minutes,
you'll be rewarded tenfolds:

https://www.youtube.com/watch?v=C7JQ7Rpwn2k

### Staying close to the metal enables new use cases ###
Historically, the Web platform has done a really bad job of exposing the
different combinations of motion sensors, bundling the data together in
high-level constructs, hiding away their complexity, the strength and
the weaknesses of each, and in the process, preventing key use cases.

In contrast, and following the precept of the Extensible Web Manifesto,
we devised the generic sensor API with the idea that we needed to expose
these sensor APIs (both for fused and non fused sensors) without hiding
away their complexity or artificially limiting them.

You need low latency? There is no { latency: "low" } option. You just
increase the polling rate as much as you can so that whenever you look
at the data (generally from within requestAnimationFramerate), the
latest available sensor reading is as close as possible to you. For
context, the DeviceOrientation Event currently clocks at 60 Hz, so a
sample can easily be 16ms old when you're using it, not accounting for
the delay in reading the sample, fusing it, converting it to JavaScript,
etc. In contrast, Oculus is polling it's gyroscope at 1,000 Hz [1] so
the latency between a head movement and application code being aware of
it stays below 2ms. Yes. 2ms. You've read that right.

You want to avoid garbage collection pauses? You bring your own buffer,
(that does mean you need to learn about typed arrays,) set its size
depending on the size of the sensor readings your collecting and their
number. Etc.

### Helping web developers deal with low-level APIs ### 
While these APIs enable use cases which weren't possible so far, they
unquestionably require more domain expertise than ones which were
exposed previously. Perhaps, we'll see JavaScript libraries emerge that
abstract some of these, if so, we'll always be able to pave the cowpath,
in the true spirit of the Extensible Web Manifesto.

But until then, it's critical to distill information about how theses
sensors work, how they get combined, what each combination's strengths
and weaknesses are, and how they fit different uses cases in a single
document so Web developers can get the most out of them.

Combining these sensors in a single spec will also make naming and APIs
a lot more consistent, reduce boilerplate, and make sure implementors
consider this space holistically.

>From a security and privacy perspective, this also seems beneficial.
Motion sensors are pretty much always used together, represent similar
threats, and are equally poorly known and understood by end-users.

Finally, nothing is more consistent with the approach we've taken with
the generic sensor API, than combining the motion sensors specs
together.

### In practice, what does it mean? ###
First of all, as mentioned above, and similar to what's found in the
generic sensor API, include an important introductory section to the
spec that details key concepts and gives a high-level overview.

Secondly, provide a list of all primitive and fused sensors, probably at
least partially in the form of a table, listing what primitive sensors
fused sensors are composed of, the kind of data they're exposing, their
strengths and weaknesses, and their use cases.

Thirdly, define a Sensor subclass these sensors would inherit from.
Would it be specific to motion sensors? E.g. a MotionSensor interface?
Or would instead sit in the Generic Sensor API and be defined by certain
characteristics? E.g. a HighFrequencySensor interface. That's still TBD.
But the intent is clearly to expose some dedicated APIs that are
relevant to motion sensors, that meet the requirements we've been
gathering in the different GitHub repositories (and that we hope to
complete with more developer input), that can seamlessly handle BYOB
(bring your own buffer) scenarios, multiple data formats (matrixes,
quaternions, etc.) were relevant, and fine tune things like polling
frequency, buffering, etc.
Note this might also allow moving some of the features which seem not so
relevant to motion sensors, and even potential foot guns, into a
dedicated interface which other, non-motion sensors could inherit from.

Fourthly, fold in the gyro, accelerometer and magnetometer spec,
expanding them in multiple types where necessary. For example, splitting
up accelerometer (which included gravity) and linear acceleration (which
doesn't because it filtered out on old devices, or fused out when a
gyroscope is present) into two distinct types. Figuring out how to
expose uncalibrated data where it opens up new use cases, etc.

Fifthly, specify all the key motion sensor fusions listed in 2) above.

Lastly, and perhaps most importantly, sprinkle all of the above with
copious examples.

I hope the above has convinced you of the value and necessity to
consider motion sensors holistically. I'm happy to discuss this further
either in writing here on the list or on our call this Thursday.

Thanks for your time reading the above.

Best,

--tobie

---
[1]:
https://www3.oculus.com/en-us/blog/building-a-sensor-for-low-latency-vr/
Received on Tuesday, 7 March 2017 00:09:51 UTC

This archive was generated by hypermail 2.3.1 : Monday, 23 October 2017 14:54:09 UTC