Re: Interesting notes on designing Web standards

Hi Ricardo,

On Apr 16, 2010, at 17:40 , Ricardo Varela wrote:
> For me, API design goes hand in hand with testing. Most people are not
> going to read the specs, ever. Most designs will never cover all
> corner cases, ever. But forcing you to write tests to your API puts
> you in the point of view of the implementer (good) and at the same
> time provides something to the implementer that allows them to test if
> they are done EVEN if they have never read the specs (double good)
> 
> I think we have to strengthen up the position of producing tests
> alongside with the specifications, even in early phases. Now, I
> realize it may be a question of getting people to help (if we don't
> have editors, even less test writers) but for this specific item,
> maybe we could leverage some help from the OEMs and other "early
> implementors"?

As you say, given how difficult it is to get people to edit specifications, getting testers is likely to be even harder. That being said I think that there's a useful middle ground to be found here.

Getting *unit* tests is a painful, work-intensive process that is difficult to get going during specification development for two reasons: 1) a lack of work-force; and 2) in-development specifications can change very radically, which will invalidate a lot of your unit tests. The latter point is particularly annoying because you never want to hit a point at which someone says "yes, it would be better if we rearchitected this specification but we've invested so much time in writing the tests that we really don't want it to change" (I've seen that happen).

However getting scenario tests written is a lot more fun as they look a lot more like real code in which you get to do cool stuff. They still need to change when the specification changes, but that involves a lot less work and is generally far more interesting. More importantly, scenario tests can easily become what drives the specification, and they're also much better at discovering architectural issues. Once the specification is stabilised (LC/CR) we can then turn to unit testing (also note that thanks to WebIDL we can generate quite a lot of those automatically too).

I think that we should get more scenario tests, but OEMs are probably not the best people to go to for these. The best people would be Web hackers. So to everyone out there: want to shape what the next generation in sexy Web APIs will look like? Don't be shy, pick whichever one of the APIs we're working on that you fancy and send us wicked cool code samples of how you'd like to see them work.

-- 
Robin Berjon - http://berjon.com/

Received on Tuesday, 20 April 2010 11:37:42 UTC