W3C home > Mailing lists > Public > www-dom-ts@w3.org > November 2001

Re: ECMA harness

From: Dimitris Dimitriadis <dimitris@ontologicon.com>
Date: Sat, 17 Nov 2001 19:08:58 +0100
Message-Id: <200111171810.fAHIAKU19329@mail.24-7webhosting.com>
Cc: "Mary Brady" <mbrady@nist.gov>, <www-dom-ts@w3.org>
To: "Jason Brittsan" <jasonbri@microsoft.com>
Hi Jason

Thanks for the clarification. I think the general outline of your 
submitted example is quite good, please help those producing the 
particular framework.

However, I do have one concern; the DOM TS aims at testing 
imnplementations' support for the DOm specification, thus also 
exceptions. I don't know if leaving them out from the TS is compatible 
with the aims of it, I will however investigate this with the DOM WG and 
get back as soon as possible.


On Friday, November 16, 2001, at 09:26  PM, Jason Brittsan wrote:

> Hi Dimitris,
> My comment could take on several meanings, depending on the design of
> the test harness.  Originally, I meant that there should be an option to
> only run the tests that match the capabilities of the client in
> reference to HTML-only implementations and implementations that do not
> support exceptions.  This is consistent with discussions that took place
> early on in the development of this test suite.  I'm not proposing any
> modularization other than that.
> Depending on the implementation of the harness, we could also provide
> ways of running only certain portions of the test suite, based on the
> needs of the user.  The current NIST harness provides some basic
> functionality by using a SELECT control to specify the "DOM Category"
> and another SELECT control to specify the "DOM Interface."  The test
> case selection process could be made better.  I've attached a *sample*
> of what this could look like.  (THIS IS ONLY A MOCK-UP!)  In addition,
> running the suite would be automated, or at least take less time for a
> person to run the conformance suite.
> Flexibility in test case selection would lead to more useful reporting.
> Users would only get back the information they desire instead of sorting
> through test case areas that don't concern them.  Also, the results
> would be posted on a single HTML page (or XML file?) instead of
> requiring the tester to visit each interface, record the results, and
> move on to the next interface.
> -Jason
> -----Original Message-----
> From: Dimitris Dimitriadis [mailto:dimitris@ontologicon.com]
> Sent: Friday, November 16, 2001 12:23 AM
> To: Jason Brittsan
> Cc: Mary Brady; www-dom-ts@w3.org
> Subject: Re: ECMA harness
> Hi Jason
> Please provide a more detailed account of what this would mean; running
> different parts of the test suite? Have different test suites built to
> begin with in accordance with existing browser capabilities? Currently
> we haven't limited the suite in any other way than defined by the DOM
> specification, except for entity exapansion and whitespace preservation
> in parsers.
> In order to come to an understanding about the harness thus: how should
> we act in this matter? Should we modularize the test suite in some way?
> We have discussed this in the past, and given the fact that we want to
> release the test suite as soon as possible it seems a good idea to make
> this explicit very soon.
> In our previous discussion though, we decided to go for using a
> modularization that stayed as close as possible to the DOM
> specification. On the other hand, we obviously want to be able to run
> the test suite in all major browsers. IE can be tested running the Ecma
> tests with the JSunit framework by running ant dom1-core-gen-jsunit to
> build the appropriate code.
> /Dimitris
> On Monday, November 12, 2001, at 02:32  PM, Jason Brittsan wrote:
>> Hi Mary... my apologies for not responding sooner.  Today is my first
>> day back from vacation.
>> Of course, I will be happy to provide any assistance I can with the
> test
>> harness.
>> The test harness should be able to run tests based on the capabilities
>> of the client.  Therefore we need to support this in the harness UI.
>> I believe that flexibility in reporting is our best strategy.
>> -----Original Message-----
>> From: Mary Brady [mailto:mbrady@nist.gov]
>> Sent: Tuesday, October 30, 2001 8:39 AM
>> To: www-dom-ts@w3.org
>> Subject: ECMA harness
>> In building the ECMA harness, I have started with the original harness
>> that was provided from the NIST web site:
>>     http://xw2k.sdct.itl.nist.gov/dom/index.html
>> This harness uses whatever DOM implementation is running on the
>> client side, attempts to run available tests, and reports the results.
>> Each of the tests expect to have access to common xml load routines
>> and common assertion routines.  I expect that we can use the same
>> code that is currently being used by the jsunit harness. The following
>> needs to be done:
>> 1) Integrate current load/assertion routines -- Mary
>> 2) Validate load routines
>>             -- IE (Jason)
>>             -- Mozilla (Do we have a Netscape volunteer?)
>> 3) Validate DOMException codes
>>             -- IE (Jason)
>>             -- Mozilla ?
>> 4) Determine high level interface -- all
>>             -- Do we want to run all tests, or be able to
>>                 discriminately pick appropriate tests?
>> 5) Determine reporting mechanism
>>             -- simply dump returns from tests?
>>             -- color-code results?
>>             -- display expected vs actual?
>>             -- possibly modify code to accomodate
>>                 what we want to display.
>> 6) Access to other testing resources ?
>>             -- test assertions, <subjects>
>>             -- view source code
>>             -- view portion of spec being tested.
>> Anything else?
>> --Mary
<Attachment missing>
Received on Saturday, 17 November 2001 13:12:10 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:34:03 UTC