- From: Dimitris Dimitriadis <dimitris@ontologicon.com>
- Date: Sat, 17 Nov 2001 19:08:58 +0100
- To: "Jason Brittsan" <jasonbri@microsoft.com>
- Cc: "Mary Brady" <mbrady@nist.gov>, <www-dom-ts@w3.org>
- Message-Id: <200111171810.fAHIAKU19329@mail.24-7webhosting.com>
Hi Jason Thanks for the clarification. I think the general outline of your submitted example is quite good, please help those producing the particular framework. However, I do have one concern; the DOM TS aims at testing imnplementations' support for the DOm specification, thus also exceptions. I don't know if leaving them out from the TS is compatible with the aims of it, I will however investigate this with the DOM WG and get back as soon as possible. /Dimitris On Friday, November 16, 2001, at 09:26 PM, Jason Brittsan wrote: > Hi Dimitris, > > My comment could take on several meanings, depending on the design of > the test harness. Originally, I meant that there should be an option to > only run the tests that match the capabilities of the client in > reference to HTML-only implementations and implementations that do not > support exceptions. This is consistent with discussions that took place > early on in the development of this test suite. I'm not proposing any > modularization other than that. > > Depending on the implementation of the harness, we could also provide > ways of running only certain portions of the test suite, based on the > needs of the user. The current NIST harness provides some basic > functionality by using a SELECT control to specify the "DOM Category" > and another SELECT control to specify the "DOM Interface." The test > case selection process could be made better. I've attached a *sample* > of what this could look like. (THIS IS ONLY A MOCK-UP!) In addition, > running the suite would be automated, or at least take less time for a > person to run the conformance suite. > > Flexibility in test case selection would lead to more useful reporting. > Users would only get back the information they desire instead of sorting > through test case areas that don't concern them. Also, the results > would be posted on a single HTML page (or XML file?) instead of > requiring the tester to visit each interface, record the results, and > move on to the next interface. > > -Jason > > -----Original Message----- > From: Dimitris Dimitriadis [mailto:dimitris@ontologicon.com] > Sent: Friday, November 16, 2001 12:23 AM > To: Jason Brittsan > Cc: Mary Brady; www-dom-ts@w3.org > Subject: Re: ECMA harness > > Hi Jason > > Please provide a more detailed account of what this would mean; running > different parts of the test suite? Have different test suites built to > begin with in accordance with existing browser capabilities? Currently > we haven't limited the suite in any other way than defined by the DOM > specification, except for entity exapansion and whitespace preservation > in parsers. > > In order to come to an understanding about the harness thus: how should > we act in this matter? Should we modularize the test suite in some way? > We have discussed this in the past, and given the fact that we want to > release the test suite as soon as possible it seems a good idea to make > this explicit very soon. > > In our previous discussion though, we decided to go for using a > modularization that stayed as close as possible to the DOM > specification. On the other hand, we obviously want to be able to run > the test suite in all major browsers. IE can be tested running the Ecma > tests with the JSunit framework by running ant dom1-core-gen-jsunit to > build the appropriate code. > > /Dimitris > > > On Monday, November 12, 2001, at 02:32 PM, Jason Brittsan wrote: > >> Hi Mary... my apologies for not responding sooner. Today is my first >> day back from vacation. >> >> Of course, I will be happy to provide any assistance I can with the > test >> harness. >> >> The test harness should be able to run tests based on the capabilities >> of the client. Therefore we need to support this in the harness UI. >> >> I believe that flexibility in reporting is our best strategy. >> >> -----Original Message----- >> From: Mary Brady [mailto:mbrady@nist.gov] >> Sent: Tuesday, October 30, 2001 8:39 AM >> To: www-dom-ts@w3.org >> Subject: ECMA harness >> >> In building the ECMA harness, I have started with the original harness >> that was provided from the NIST web site: >> >> http://xw2k.sdct.itl.nist.gov/dom/index.html >> This harness uses whatever DOM implementation is running on the >> client side, attempts to run available tests, and reports the results. >> Each of the tests expect to have access to common xml load routines >> and common assertion routines. I expect that we can use the same >> code that is currently being used by the jsunit harness. The following >> needs to be done: >> >> 1) Integrate current load/assertion routines -- Mary >> 2) Validate load routines >> -- IE (Jason) >> -- Mozilla (Do we have a Netscape volunteer?) >> 3) Validate DOMException codes >> -- IE (Jason) >> -- Mozilla ? >> 4) Determine high level interface -- all >> -- Do we want to run all tests, or be able to >> discriminately pick appropriate tests? >> 5) Determine reporting mechanism >> -- simply dump returns from tests? >> -- color-code results? >> -- display expected vs actual? >> -- possibly modify code to accomodate >> what we want to display. >> 6) Access to other testing resources ? >> -- test assertions, <subjects> >> -- view source code >> -- view portion of spec being tested. >> >> Anything else? >> >> --Mary >> > <Attachment missing>
Hi Jason Thanks for the clarification. I think the general outline of your submitted example is quite good, please help those producing the particular framework. However, I do have one concern; the DOM TS aims at testing imnplementations' support for the DOm specification, thus also exceptions. I don't know if leaving them out from the TS is compatible with the aims of it, I will however investigate this with the DOM WG and get back as soon as possible. /Dimitris On Friday, November 16, 2001, at 09:26 PM, Jason Brittsan wrote: > Hi Dimitris, > > My comment could take on several meanings, depending on the design of > the test harness. Originally, I meant that there should be an option to > only run the tests that match the capabilities of the client in > reference to HTML-only implementations and implementations that do not > support exceptions. This is consistent with discussions that took place > early on in the development of this test suite. I'm not proposing any > modularization other than that. > > Depending on the implementation of the harness, we could also provide > ways of running only certain portions of the test suite, based on the > needs of the user. The current NIST harness provides some basic > functionality by using a SELECT control to specify the "DOM Category" > and another SELECT control to specify the "DOM Interface." The test > case selection process could be made better. I've attached a *sample* > of what this could look like. (THIS IS ONLY A MOCK-UP!) In addition, > running the suite would be automated, or at least take less time for a > person to run the conformance suite. > > Flexibility in test case selection would lead to more useful reporting. > Users would only get back the information they desire instead of sorting > through test case areas that don't concern them. Also, the results > would be posted on a single HTML page (or XML file?) instead of > requiring the tester to visit each interface, record the results, and > move on to the next interface. > > -Jason > > -----Original Message----- > From: Dimitris Dimitriadis [mailto:dimitris@ontologicon.com] > Sent: Friday, November 16, 2001 12:23 AM > To: Jason Brittsan > Cc: Mary Brady; www-dom-ts@w3.org > Subject: Re: ECMA harness > > Hi Jason > > Please provide a more detailed account of what this would mean; running > different parts of the test suite? Have different test suites built to > begin with in accordance with existing browser capabilities? Currently > we haven't limited the suite in any other way than defined by the DOM > specification, except for entity exapansion and whitespace preservation > in parsers. > > In order to come to an understanding about the harness thus: how should > we act in this matter? Should we modularize the test suite in some way? > We have discussed this in the past, and given the fact that we want to > release the test suite as soon as possible it seems a good idea to make > this explicit very soon. > > In our previous discussion though, we decided to go for using a > modularization that stayed as close as possible to the DOM > specification. On the other hand, we obviously want to be able to run > the test suite in all major browsers. IE can be tested running the Ecma > tests with the JSunit framework by running ant dom1-core-gen-jsunit to > build the appropriate code. > > /Dimitris > > > On Monday, November 12, 2001, at 02:32 PM, Jason Brittsan wrote: > >> Hi Mary... my apologies for not responding sooner. Today is my first >> day back from vacation. >> >> Of course, I will be happy to provide any assistance I can with the > test >> harness. >> >> The test harness should be able to run tests based on the capabilities >> of the client. Therefore we need to support this in the harness UI. >> >> I believe that flexibility in reporting is our best strategy. >> >> -----Original Message----- >> From: Mary Brady [mailto:mbrady@nist.gov] >> Sent: Tuesday, October 30, 2001 8:39 AM >> To: www-dom-ts@w3.org >> Subject: ECMA harness >> >> In building the ECMA harness, I have started with the original harness >> that was provided from the NIST web site: >> >> http://xw2k.sdct.itl.nist.gov/dom/index.html >> This harness uses whatever DOM implementation is running on the >> client side, attempts to run available tests, and reports the results. >> Each of the tests expect to have access to common xml load routines >> and common assertion routines. I expect that we can use the same >> code that is currently being used by the jsunit harness. The following >> needs to be done: >> >> 1) Integrate current load/assertion routines -- Mary >> 2) Validate load routines >> -- IE (Jason) >> -- Mozilla (Do we have a Netscape volunteer?) >> 3) Validate DOMException codes >> -- IE (Jason) >> -- Mozilla ? >> 4) Determine high level interface -- all >> -- Do we want to run all tests, or be able to >> discriminately pick appropriate tests? >> 5) Determine reporting mechanism >> -- simply dump returns from tests? >> -- color-code results? >> -- display expected vs actual? >> -- possibly modify code to accomodate >> what we want to display. >> 6) Access to other testing resources ? >> -- test assertions, <subjects> >> -- view source code >> -- view portion of spec being tested. >> >> Anything else? >> >> --Mary >> >
Attachments
- text/html attachment: testsuiteui.html
Received on Saturday, 17 November 2001 13:12:10 UTC