W3C home > Mailing lists > Public > www-dom-ts@w3.org > April 2002

Re: Re: jsUnit 1.3.0 alpha

From: <edwardhieatt@earthlink.net>
Date: Mon, 15 Apr 2002 19:15:48 -0400 (EDT)
To: Dimitris Dimitriadis <dimitris@ontologicon.com>
Cc: Robert Clary <bclary@netscape.com>, Curt Arnold <carnold@houston.rr.com>, www-dom-ts@w3.org, Edward Hieatt <edward@jsunit.net>
Message-ID: <Springmail.0994.1018912517.0.85874200@webmail.pas.earthlink.net>
I suppose I should look at the DOM tests to really be able to advise you on how to structure things, but there are a couple of points that caught my attention which seem to violate the usual Unit Testing ideas:

- You should never have tests that rely on previous tests having been run.  Every test should be able to run independently of all the others.  If you need to create complicated fixture, so be it; perhaps the test that needs to be run first should be broken up so that the setUp of the later test can invoke the relevant parts.  The fact that JsUnit runs Test Functions in a determined order is not something that should be relied on.  JUnit, for example, does not guarantee the order of the running of Test Methods within a Test Class.

- In a typical development environment, speed of tests is crucial.  One wants to encourage people to run tests as often as they like whilst they are developing.  Perhaps this situation is different, because you are working in a very distributed manner geographically, however.  At my company we have over 2000 unit tests, all of which are run whenever anyone integrates code; we strive to keep the total time taken under 10 minutes.

For reference, you might want to look at Kent Beck's "Extreme Programming Explained".

By the way - I think I misunderstood Curt's original posting.  Sorry for the confusion.

- Edward

On Tue, 16 Apr 2002 00:49:18 +0200 Dimitris Dimitriadis <dimitris@ontologicon.com> wrote:

One issue which will certainly be raised (has already been raised in 
part in the DOM WG) is the complexity of tests; testing module 
interdependency will be a nightmare if we can't run tests that rest on 
results of running previous tests. We could either write long massive 
test descriptions for those cases, or make the framework smart enough to 
load several tests and pass results from one test to another test, then 
run the second test (this will be asked for with level 3, take L/S for 
example).

So I guess the answer to your question is, No, we don't need to be able 
to run more than one test (currently) but we may want that in any case.

Memory requirements are a limiting factor, but I think we cant engineer 
smoothly and avoid the worst case scenario. Speed is, at least to me, as 
you mention, not a limiting factor. We have had voices arguing for the 
opposite, but my personal feeling is that if the gain from running more 
tests is larger than the loss of waiting for a few more minutes it's 
worth it. Then again, it's frustrating to do one change in your code, 
compile, test, and have that procedure take you x minutes longer.

Ideas?

/Dimitris

On Tuesday, April 16, 2002, at 12:33 , Robert Clary wrote:

> [bc] inline
>
> Dimitris Dimitriadis wrote:
>> comment inlined
>> On Tuesday, April 16, 2002, at 12:11 , Robert Clary wrote:
>>> Curt Arnold wrote:
>>>
>>>> Another thing that would be useful is a clean-up function that runs 
>>>> after
>>>> all the tests have been completed.  The existing jsUnit based HTML 
>>>> tests
>>>> leave a large number of windows open since they keep unmodified 
>>>> documents
>>>> around for potential later use, however there is nothing to hook to 
>>>> close
>>>> them all down when the tests are over.  My recollection is 
>>>> tearDown() fires
>>>> after each test and would not be appropriate, but I could wrong on 
>>>> that.
>>>
>>>
>>> The caching of unmodified test documents seemed to me to introduce a 
>>> great deal of complexity without necessarily adding a corresponding 
>>> degree of benefit. However that is the subject of a different 
>>> discussion I think.
>>>
>>> It may be possible to do what you want by organizing the tests into 
>>> suite of suites and having the final suite perform the cleanup.
>>>
>>> This reminds me however that the Document Loader in jsUnit currently 
>>> is limited to a single document at a time. This may be a limitation 
>>> that needs to be addressed in the proposed jsUnit. How important is 
>>> it to load multiple test documents at the same time in general and in 
>>> the DOM TS in particular?
>>>
>> [dd] Am I off track presuming that this could be useful if we utilize 
>> the JsUnit TestSuite option?
>
> [bc] No, I don't think you are off track. The current design can 
> probably be modified to support loading documents into separate windows 
> although handling the onload notification in Mozilla may be problematic.
>
> The question I have is "do any of the tests actually require that two 
> or more documents be open at the same time?". If so, then this is a 
> must have feature for jsUnit and the DOM TS.
>
> However, if it is only a convenience originally introduced as a partial 
> work around for problems encountered earlier due to asynchronous 
> document loading, is the complexity worth it?
>
> I consider Memory performance to be an absolute requirement for 
> supporting as many testers as possible. We can not require each tester 
> to have 1G of ram to run the tests. However I do not consider speed as 
> important. If the tests take a few minutes longer without document 
> caching, no one will be excluded from running the tests.
>
> /bc
>
>
Received on Tuesday, 16 April 2002 14:16:29 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Monday, 6 April 2009 12:58:46 GMT