RE: ECMA harness

I apologize for my lateness in replying.
 
Internet Explorer has an ECMAScript implementation that does not support
exceptions, per se.  We return error codes from the scripting engine.
 
-----Original Message-----
From: Arnold, Curt [mailto:Curt.Arnold@hyprotech.com] 
Sent: Monday, November 19, 2001 8:29 AM
To: 'www-dom-ts@w3.org'
Subject: RE: ECMA harness
 
I have a lot of not fully realized thoughts on this topic and it is more
complicated than it appears.  However, I can't go into it right now.  I
think that we can have this covered.
 
Jason can you outline the particular DOM and ECMAScript implementations
that you are concerned with supporting?  Particularly which of the
ECMAScript implementations do not support exceptions and what mechanism,
if any, is there to detect the error conditions raised from DOM
interactions.
 
 
-----Original Message-----
From: Mary Brady [mailto:mbrady@nist.gov] 
Sent: Monday, November 19, 2001 10:15 AM
To: Jason Brittsan; Dimitris Dimitriadis
Cc: www-dom-ts@w3.org
Subject: Re: ECMA harness
	We have not yet looked at allowing the new 
	jsunit-based harness to discriminately run 
	tests -- it currently runs all available tests at 
	once and provides a report.  Have you tried 
	to run this yet Jason?  I'd be interested in 
	your reaction.  We do, however, have the 
	ability to insert inside a particular test that it 
	will only run with a particular module -- we have 
	not yet done this -- but I would think that doing 
	so would cover the HTML-only tests.  Maybe 
	Curt can comment on how we should use 
	this feature, and what the transform does 
	with this information.  
	 
	--Mary
	 
		----- Original Message ----- 
		From: Jason Brittsan <mailto:jasonbri@microsoft.com>  
		To: Dimitris Dimitriadis
<mailto:dimitris@ontologicon.com>  
		Cc: Mary Brady <mailto:mbrady@nist.gov>  ;
www-dom-ts@w3.org 
		Sent: Friday, November 16, 2001 3:26 PM
		Subject: RE: ECMA harness
		 
		Hi Dimitris,
		
		My comment could take on several meanings, depending on
the design of
		the test harness.  Originally, I meant that there should
be an option to
		only run the tests that match the capabilities of the
client in
		reference to HTML-only implementations and
implementations that do not
		support exceptions.  This is consistent with discussions
that took place
		early on in the development of this test suite.  I'm not
proposing any
		modularization other than that.
		
		Depending on the implementation of the harness, we could
also provide
		ways of running only certain portions of the test suite,
based on the
		needs of the user.  The current NIST harness provides
some basic
		functionality by using a SELECT control to specify the
"DOM Category"
		and another SELECT control to specify the "DOM
Interface."  The test
		case selection process could be made better.  I've
attached a *sample*
		of what this could look like.  (THIS IS ONLY A MOCK-UP!)
In addition,
		running the suite would be automated, or at least take
less time for a
		person to run the conformance suite.
		
		Flexibility in test case selection would lead to more
useful reporting.
		Users would only get back the information they desire
instead of sorting
		through test case areas that don't concern them.  Also,
the results
		would be posted on a single HTML page (or XML file?)
instead of
		requiring the tester to visit each interface, record the
results, and
		move on to the next interface.
		
		-Jason
		
		-----Original Message-----
		From: Dimitris Dimitriadis
[mailto:dimitris@ontologicon.com] 
		Sent: Friday, November 16, 2001 12:23 AM
		To: Jason Brittsan
		Cc: Mary Brady; www-dom-ts@w3.org
		Subject: Re: ECMA harness
		
		Hi Jason
		
		Please provide a more detailed account of what this
would mean; running 
		different parts of the test suite? Have different test
suites built to 
		begin with in accordance with existing browser
capabilities? Currently 
		we haven't limited the suite in any other way than
defined by the DOM 
		specification, except for entity exapansion and
whitespace preservation 
		in parsers.
		
		In order to come to an understanding about the harness
thus: how should 
		we act in this matter? Should we modularize the test
suite in some way? 
		We have discussed this in the past, and given the fact
that we want to 
		release the test suite as soon as possible it seems a
good idea to make 
		this explicit very soon.
		
		In our previous discussion though, we decided to go for
using a 
		modularization that stayed as close as possible to the
DOM 
		specification. On the other hand, we obviously want to
be able to run 
		the test suite in all major browsers. IE can be tested
running the Ecma 
		tests with the JSunit framework by running ant
dom1-core-gen-jsunit to 
		build the appropriate code.
		
		/Dimitris
		
		
		On Monday, November 12, 2001, at 02:32  PM, Jason
Brittsan wrote:
		
		> Hi Mary... my apologies for not responding sooner.
Today is my first
		> day back from vacation.
		>
		> Of course, I will be happy to provide any assistance I
can with the
		test
		> harness.
		>
		> The test harness should be able to run tests based on
the capabilities
		> of the client.  Therefore we need to support this in
the harness UI.
		>
		> I believe that flexibility in reporting is our best
strategy.
		>
		> -----Original Message-----
		> From: Mary Brady [mailto:mbrady@nist.gov]
		> Sent: Tuesday, October 30, 2001 8:39 AM
		> To: www-dom-ts@w3.org
		> Subject: ECMA harness
		>
		> In building the ECMA harness, I have started with the
original harness
		> that was provided from the NIST web site:
		>
		>     http://xw2k.sdct.itl.nist.gov/dom/index.html
		> This harness uses whatever DOM implementation is
running on the
		> client side, attempts to run available tests, and
reports the results.
		> Each of the tests expect to have access to common xml
load routines
		> and common assertion routines.  I expect that we can
use the same
		> code that is currently being used by the jsunit
harness. The following
		> needs to be done:
		>
		> 1) Integrate current load/assertion routines -- Mary
		> 2) Validate load routines
		>             -- IE (Jason)
		>             -- Mozilla (Do we have a Netscape
volunteer?)
		> 3) Validate DOMException codes
		>             -- IE (Jason)
		>             -- Mozilla ?
		> 4) Determine high level interface -- all
		>             -- Do we want to run all tests, or be able
to
		>                 discriminately pick appropriate tests?
		> 5) Determine reporting mechanism
		>             -- simply dump returns from tests?
		>             -- color-code results?
		>             -- display expected vs actual?
		>             -- possibly modify code to accomodate
		>                 what we want to display.
		> 6) Access to other testing resources ?
		>             -- test assertions, <subjects>
		>             -- view source code
		>             -- view portion of spec being tested.
		>
		> Anything else?
		>
		> --Mary
		>