Re: ISSUE-1: Mandatory algorithms (was Re: ISSUE-3: Algorithm discovery)

On Tue, Jul 10, 2012 at 1:52 PM, Harry Halpin <hhalpin@w3.org> wrote:
> On 07/10/2012 10:36 PM, Ryan Sleevi wrote:
>>> I think that in general, we should have some subset (the JOSE subset
>>> seems
>>> the obvious and ideal candidate) as a SHOULD implement. If all we have is
>>> a
>>> discovery algorithm, then I can not see how we will create test-cases
>>> that
>>> are meanginful and that Web developers can rely on. We need to be able to
>>> say, for a given browser X, it supports this functionality as embodied in
>>> test-cases. Now, if a browser *only* throws errors, then obviously that
>>> is
>>> useless, but we don't want that technically passing the test-cases. We
>>> want
>>> to say that's non-conforming.
>>>
>>> On the same-hand, I can see real value in having some generic extensible
>>> framework, of which I see this discovery mechanism as one way of
>>> approaching. I'm wondering if there any other alternative approaches?
>>
>> Harry,
>>
>> I'm not sure I understand why this is required. For example, how are
>> test cases for the <video> tag covered, or <object>, or <img> or any
>> of the other hyper-media tags?
>
>
> Note that HTML is a bit special, due to its legacy status. However, the W3C
> is working on test-cases to get HTML5 to the next level of W3C Process. In
> general, we have to imagine "what is that we will test"?
>
> Again, this is a hard question. The WebApps WG is using this process [1],
> and now there is the WebDriver work at W3C which I suggest we use [2].

Ok, I'll use a set of different examples then that may be more
reflective of the modern HTML5 design.

Gamepad APIs:
- window.gamepads.length == 0 if there is no hardware support for a
Gamepad. Thus, conformance to the API cannot be tested without
supporting a physical gamepad. However, an implementation could fully
support what the spec says - it just doesn't have any hardware devices
connected.

Geolocation API
- It does not require that implementations support GPS over, say, Wifi
sourced data - or even user input. It simply specifies how the API
would behave if position information was available.
- If geolocation is not supported, it will invoke the errorCallback
with a positionError. Further, there is no requirement that the
geolocation API provide meaningful information in the information it
returns.

Device APIs:
Vibration API
 - An implementation can fully implement the Vibration API, raising
NotSupportedErrors whenever window.vibrate is called. It will be a
conforming implementation to do so.
Contacts API
- An implementation may fully implement the contacts API without
exposing any contacts.
MediaStream API
- Places no requirements on the interpretation of the binary data
provided by the MediaStream/MediaStreamTrack object

>
>>
>> For example, I'm not sure why we cannot detach the "API specification"
>> (these are the state machines, these are the error handling routines)
>> from "Algorithm specification" (this is how RSA-OAEP behaves, this is
>> how AES-GCM behaves).
>
>
> In general, with specs is better to have everything in one document unless
> document becomes unwieldy (for example, many people complained about XML
> namespaces not being part of the XML the spec). However, there are examples
> of W3C specs that *should* have been split into multiple documents (XML
> Schema Structures comes to mind).

Sure, but as you've discussed on past calls, it's perfectly acceptable
to think of them as logically separate.

Note that I'm absolutely in support of the WG's work product including
definitions of how 'common' algorithms should and will behave -
RSA-OAEP, RSA-PSS, HMAC, AES, etc . I'm absolutely in support of
saying "If you are going to implement this algorithm, this is how it
MUST behave.". What I'm concerned about is saying either of "You may
not implement any algorithm that is not specified" or "You MUST
implement these algorithms"

>
>>
>> Test cases for the API specification can focus on the objects having
>> the correct types / methods, the exception types existing, and any
>> user interaction.
>
>
> I agree with the first two, although if we have correct types/methods its
> generally useful to have a result we can check outside of an error message
> for some (possibly minimal) part of the spec. Generally, user *interface* is
> outside of test-cases, but kinds of interaction may be inside test-cases if
> necessary to test a feature of the API.
>
>>
>> For algorithm specifications, it can test individual algorithm handling.
>>
>> However, for error handling, it seems like some tests will not be able
>> to be programatically simulating by a test suite, and must be
>> manually/synthetically simulated. For example, how might you test a
>> system failure between .processData() and .complete(), to ensure that
>> onerror is raised appropriately.
>
>
> Usually the test-cases are not as rigorous as say, production test-cases or
> full-scale state simulations done in a formal manner. W3C test-cases assume
> a normal operating environment without systems failures. We will likely not
> to simulations.
>
>>
>> Beyond ensuring IDL conformance, I would think all tests can belong to
>> the algorithms - that is, IF a user agent implements RSA, here's tests
>> 1-15, to ensure it implements the "correct" form of RSA. IF a user
>> agent does not, it automatically passes that test suite/that suite is
>> not-applicable.
>
>
> I still am worried a browser can implement basically nothing and still pass
> the test-cases. The conformance is judged as regards the entire spec.
> However, we can clearly delineate which parts of the spec are SHOULD
> implement and which parts are OPTIONAL. That is quite useful. I am worried
> about all algorithms being OPTIONAL in theory, even if that seems to be an
> extremer. Again, I think having a subset of guaranteed algorithms (with
> *perhaps* a well-agreed upon deprecation method) and an extensible framework
> (of which the discovery algorithm the only proposal standing so far) makes
> sense.

Ah, this may be where the disconnect is. I think it should be
perfectly acceptable for a browser to do this - that is, implement the
API without supporting a SINGLE algorithm - returning errors for all
attempts.

Here's the reasoning for why I take this position:

Cryptography, unlike many other software implementations, is something
that is very frequently controlled by regulatory bodies. While things
are nowhere near as problematic as they were in the 90s, any vendor in
the US that wishes to ship a product with a cryptographic
implementation will, at the least, be required to go through EAR
requirements via the US Department of Commerce. Similar requirements
exist for other countries participating in the Wassenaar Agreement.

Mandating a MUST implement means that any conforming user-agent MUST
now be subject to these requirements and appropriate declarations. I
believe that such stipulation is akin (but not identical) to requiring
that implementations MUST implement some functionality that is
governed by patents - something that the W3C rightly avoids via its
patent policy.

Today (that is, outside of the discussion of this particular API), and
keeping in mind this is far from legal advice, applications may be
able to avoid the EAR requirements by making use of cryptography
either part of the operating environment or via some other interface
(for example, hardware). Depending on how they use it, they may not be
subject to the requirements.

Additionally, cryptographic implementations may be subject to various
government or business regulatory concerns. For example, FIPS 140-2
governs the use of cryptography within the US Government (along with
various other regulations requiring an implementation MUST be FIPS
140-2 certified). A FIPS 140-2 compliant implementation MUST NOT use
certain algorithms / key sizes / etc. These regulations vary from
country to country - and some countries may mandate MUST IMPLEMENT for
other additional algorithms (GOST, SEED).

Again, applications can often avoid these potentially onerous
requirements by delegating the use of crypto to operating system APIs
or hardware.

Finally, implementing crypto is hard. Really hard. We've already seen
examples on the list where subtle issues such as timing can undermine
the entire implementation. For security reasons, it's very much
preferable to delegate out - again, to the operating system,
third-party libraries, or hardware. Something that is stable, well
tested, and centrally updatable.


If you look at the State of the Art of modern cryptographic APIs, they
are designed around a "module" concept. The API specifications
/simply/ provide a definition of how the APIs will behave and /may/
specify how specific algorithms (AES, RSA, OAEP, etc) may behave.
These APIs do NOT typically mandate innate support for any particular
algorithms. Conceptually, these APIs think of everything as "hardware"
(or "module", or "token", or "library", but same aggregation).

Now, for utility, vendors /may/ ship software implementations - either
with the OS or with their library. But they /regularly/ give
administrators the ability to disable parts of such software
implementations - or even outright remove or replace them.

In the straw-man, rather than expose this concept of "modules", which
varies in detail and functionality from implementation, I proposed
that the API be associated based on the concept of "keys". That is,
there is no requirement in which "module" a key is stored, nor is even
the concept of modules exposed to the consumer of the API. Instead,
the API exposes keys - of which the total set of available keys may
span multiple modules.

For these reasons, any User Agent that wishes to operate in these
environments has NO guarantee about what the environment may or may
not support, short of re-implementing an entire cryptographic stack
for all of the MUST implements. I believe that, categorically, such a
requirement would REDUCE the security of users of this API, because
again, crypto is "hard".

Thus, when I think of this API, I think of it as working with a
particular type of hardware device, much like Gamepad, Geolocation, or
MediaStream. There MAY be ways to emulate these devices via software
(A virtual gamepad might be emulated via a keyboard, a virtual GPS may
be emulated via Wifi location settings, a virtual video capture device
may be emulated by reading from a file on disk), but there is no
requirement that implementations MUST do so.

This is part of why I'm so opposed to MUST-IMPLEMENT. User Agents that
wish to defer cryptographic implementations to their operating
environment (which itself may be backed by hardware) have no ability
to know what the operating environment may provide. As such, they
CANNOT guarantee that they're able to implement all of it, short of
shipping with their own cryptographic implementations, which then
opens a huge morass of governmental and regulatory concerns.

For example, when running on Linux, Google Chrome and Mozilla Firefox
have no guarantees what cryptographic primitives will be available to
it. This can affect everything using crypto - most notably, SSL/TLS -
but that's perfectly acceptable and is something that is the
responsibility of the user. Chrome & Firefox simply consume the
standard cryptographic APIs (in this case, we're both using PKCS#11).
As such, it's impossible to fairly test that we implement all of the
MUST SUPPORT algorithms, if there were any.

>
>>
>> I was also hoping you could explain the statement: " Now, if a browser
>> *only* throws errors, then obviously that is useless, but we don't
>> want that technically passing the test-cases. We want to say that's
>> non-conforming."
>>
>> Why?
>
>
> As that browser would have no functionality that a web developer could use
> as regards this API other than produce the correct "not supported" errors.
> Its an edge-case, but could happen!
>
> [1] http://www.w3.org/2008/webapps/wiki/Harness
> [2] http://www.w3.org/TR/webdriver/

Received on Tuesday, 10 July 2012 22:03:50 UTC