Re: Ringmark, Core Mobile, and the Financial Times

On Jun 12, 2012, at 23:32 , Arthur Barstow wrote:
> On 6/12/12 6:52 AM, ext Robin Berjon wrote:
>> So we have resources on one side and work that needs to be done on testing on the other, but something is not happening — I'd like to know why. 
> 
> Please ignore the following if your question is purely rhetorical ...

No, it's not meant to be rhetorical. I think that the issue you point out is true, but is only part of the story. The reasons you point to are part of individual incentives — and those are real — but I'm also concerned about why companies that have indicated they want to commit resources to testing don't — that's also part of the picture.

> Testing is `hard` and has no `glory`?
> 
> WRT the former, if testharness.js is considered a blocker, related comments, flames, etc. should be sent to public-test-infra@w3.org (unless James suggests otherwise).

Is testharness.js considered hard? My personal experience is that that may be true but not its fault. If you've used another testing framework such as Jasmine or Mocha before, I reckon it ought to be reasonably easy to pick it up. The syntax differs but the concepts are all pretty much the same.

But for all the hype that testing gets from the development community loudmouths, I get the impression that the proportion of developers who actually use it is fantastically low. I recently gave a talk that introduced testharness.js and I had foolishly expected the attendees — Web developers — to have at least some passing familiarity with the common testing tools of their trade (which is to say, to at least have looked at the docs or at some tests even if they haven't actually used them for their projects). Noticing a lot of blank stares though, I asked who had played with those before — not a single hand went up.

There isn't much we can do about that — it's not as if there weren't already loads of testing advocacy going on. Perhaps the only thing that can make testharness.js easier on people would be better docs? Perhaps something along the lines of what the Jasmine folks recently did: http://pivotal.github.com/jasmine/. Not only is it a good tutorial, it's also runnable.

> WRT the later, Marcos did a good job of creating a cover page for each of the web application packing spec's Test Suites and he acknowledged the Test Contributors (e.g. [1]) so there is at least one way to provide some recognition (although the Contributors could be more prominent like Editors in specs).

I think that making the Test Tzar for a spec right up there with the Editor ought to be a requirement in all specs. "Test Contributors" sounds secondary though. If you can find me a good name for it, I'll add support for it to ReSpec post-haste. Conformance Lead? It should be something that can shine on a CV.

>> It would certainly help if some in the 200+ lurkers were to speak up! Perhaps it is that there's something missing in making it possible, easier, simpler, clearer, better incentivised to apply the former to the latter. 
> 
> I think someone recently mentioned (perhaps at last May's AC meeting) the idea of a "testing tax". For example, a mandatory requirement to attend CoreMob's upcoming f2f meeting could be submitting at least 25 test cases (using testharness.js) to one of the specs this group considers "core". [No pain, no gain and such ;-).]

I love the idea but the problem is always enforcement. If someone doesn't pay their share, you need to actually physically exclude them from the meeting — otherwise it's just wishful thinking. That will already prove hard. Then you're going to have to provide exemptions: someone new to the group, someone who has done a huge amount of work on something else, the invited expert on his own dime who's dedicating what time he can, someone whom you know has had recent trouble in their family… Before you know it exemptions are common enough that the group loses its sense of tax fairness and you're in trouble. I think we're better off trying to positively incentivise this rather than try to force it. Besides, I can easily write 25 "tests" over breakfast before the meeting — they just wouldn't be tests you'd want to actually use :)

-- 
Robin Berjon - http://berjon.com/ - @robinberjon

Received on Wednesday, 13 June 2012 09:36:12 UTC