Re: Re: jsUnit 1.3.0 alpha

With respect to the loading of files, the tests themselves are independent
of
anything that has been previously run.  It was just convenient in java to
only reload a test file if the in-memory representation had been altered
in anyway.  I think this has carried over into the ECMAScript version,
although it is not necessary.  As long as the time to run the tests is
acceptable, I don't see a need to keep track of whether or not a test has
modified the in-memory representation of the data -- it's just as easy to
load the test file each time and go ahead and run the test.

--Mary

----- Original Message -----
From: <edwardhieatt@earthlink.net>
To: "Dimitris Dimitriadis" <dimitris@ontologicon.com>
Cc: "Robert Clary" <bclary@netscape.com>; "Curt Arnold"
<carnold@houston.rr.com>; <www-dom-ts@w3.org>; "Edward Hieatt"
<edward@jsunit.net>
Sent: Monday, April 15, 2002 7:15 PM
Subject: Re: Re: jsUnit 1.3.0 alpha


> I suppose I should look at the DOM tests to really be able to advise you
on how to structure things, but there are a couple of points that caught my
attention which seem to violate the usual Unit Testing ideas:
>
> - You should never have tests that rely on previous tests having been run.
Every test should be able to run independently of all the others.  If you
need to create complicated fixture, so be it; perhaps the test that needs to
be run first should be broken up so that the setUp of the later test can
invoke the relevant parts.  The fact that JsUnit runs Test Functions in a
determined order is not something that should be relied on.  JUnit, for
example, does not guarantee the order of the running of Test Methods within
a Test Class.
>
> - In a typical development environment, speed of tests is crucial.  One
wants to encourage people to run tests as often as they like whilst they are
developing.  Perhaps this situation is different, because you are working in
a very distributed manner geographically, however.  At my company we have
over 2000 unit tests, all of which are run whenever anyone integrates code;
we strive to keep the total time taken under 10 minutes.
>
> For reference, you might want to look at Kent Beck's "Extreme Programming
Explained".
>
> By the way - I think I misunderstood Curt's original posting.  Sorry for
the confusion.
>
> - Edward
>
> On Tue, 16 Apr 2002 00:49:18 +0200 Dimitris Dimitriadis
<dimitris@ontologicon.com> wrote:
>
> One issue which will certainly be raised (has already been raised in
> part in the DOM WG) is the complexity of tests; testing module
> interdependency will be a nightmare if we can't run tests that rest on
> results of running previous tests. We could either write long massive
> test descriptions for those cases, or make the framework smart enough to
> load several tests and pass results from one test to another test, then
> run the second test (this will be asked for with level 3, take L/S for
> example).
>
> So I guess the answer to your question is, No, we don't need to be able
> to run more than one test (currently) but we may want that in any case.
>
> Memory requirements are a limiting factor, but I think we cant engineer
> smoothly and avoid the worst case scenario. Speed is, at least to me, as
> you mention, not a limiting factor. We have had voices arguing for the
> opposite, but my personal feeling is that if the gain from running more
> tests is larger than the loss of waiting for a few more minutes it's
> worth it. Then again, it's frustrating to do one change in your code,
> compile, test, and have that procedure take you x minutes longer.
>
> Ideas?
>
> /Dimitris
>
> On Tuesday, April 16, 2002, at 12:33 , Robert Clary wrote:
>
> > [bc] inline
> >
> > Dimitris Dimitriadis wrote:
> >> comment inlined
> >> On Tuesday, April 16, 2002, at 12:11 , Robert Clary wrote:
> >>> Curt Arnold wrote:
> >>>
> >>>> Another thing that would be useful is a clean-up function that runs
> >>>> after
> >>>> all the tests have been completed.  The existing jsUnit based HTML
> >>>> tests
> >>>> leave a large number of windows open since they keep unmodified
> >>>> documents
> >>>> around for potential later use, however there is nothing to hook to
> >>>> close
> >>>> them all down when the tests are over.  My recollection is
> >>>> tearDown() fires
> >>>> after each test and would not be appropriate, but I could wrong on
> >>>> that.
> >>>
> >>>
> >>> The caching of unmodified test documents seemed to me to introduce a
> >>> great deal of complexity without necessarily adding a corresponding
> >>> degree of benefit. However that is the subject of a different
> >>> discussion I think.
> >>>
> >>> It may be possible to do what you want by organizing the tests into
> >>> suite of suites and having the final suite perform the cleanup.
> >>>
> >>> This reminds me however that the Document Loader in jsUnit currently
> >>> is limited to a single document at a time. This may be a limitation
> >>> that needs to be addressed in the proposed jsUnit. How important is
> >>> it to load multiple test documents at the same time in general and in
> >>> the DOM TS in particular?
> >>>
> >> [dd] Am I off track presuming that this could be useful if we utilize
> >> the JsUnit TestSuite option?
> >
> > [bc] No, I don't think you are off track. The current design can
> > probably be modified to support loading documents into separate windows
> > although handling the onload notification in Mozilla may be problematic.
> >
> > The question I have is "do any of the tests actually require that two
> > or more documents be open at the same time?". If so, then this is a
> > must have feature for jsUnit and the DOM TS.
> >
> > However, if it is only a convenience originally introduced as a partial
> > work around for problems encountered earlier due to asynchronous
> > document loading, is the complexity worth it?
> >
> > I consider Memory performance to be an absolute requirement for
> > supporting as many testers as possible. We can not require each tester
> > to have 1G of ram to run the tests. However I do not consider speed as
> > important. If the tests take a few minutes longer without document
> > caching, no one will be excluded from running the tests.
> >
> > /bc
> >
> >
>
>

Received on Tuesday, 16 April 2002 16:27:53 UTC