- From: Dimitris Dimitriadis <dimitris@ontologicon.com>
- Date: Fri, 22 Feb 2002 12:35:44 +0100
- To: bclary@netscape.com (Robert Clary)
- Cc: Mary Brady <mbrady@nist.gov>, www-dom-ts@w3.org
comments inlined On Friday, February 22, 2002, at 04:39 , Robert Clary wrote: > [bc] Comments in line. > > Mary Brady wrote: > >> Bob, >> >> Thanks for the input. Let me join Dimitris in welcoming you to this >> effort. >> We welcome >> any help, but in particular, are very happy to have someone who can >> help >> ensure that >> the tests interface well to Mozilla. >> >> I can see that some of your comments bring up the issue of >> "conformance". >> You are >> correct in pointing out that some folks think of the whole nine yards >> when >> the term >> "conformance" is used -- that is, a comprehensive test suite, >> certification, >> validation, etc. >> I think that you can see that this effort hopes to someday be >> comprehensive, >> but we are >> not yet there. >> >> Additional comments inlined. >> >> --Mary >> > [bc] > > Yes, people do have expectations when "conformance" is used to describe > a test. It is these expectations that I wish to make sure are > appropriate. > [dd] This we all share. I've had problems with the term 'conformance' but I think we've solved most issues with it so far. However, your views are important to us and will be discussed. > I was very disappointed in the low volume of posts when the DOM TS was > announced on Slashdot, but as you can see from the comments (the few > that there are) > > <http://slashdot.org/comments.pl?sid=27983&cid=0&pid=0&startat=&threshold= > 0&mode=nested&commentsort=0&op=Change> > > these tests are already being used to denigrate Mozilla's > implementation of the DOM Core. > [dd] I was very disappointed too by the numbers, but wouldn't be too alarmed by the comments themselves. Except for this version being our first release and thus subject to change (which is indicated in every second piece of information regarding the DOM TS), C. Arnold pointed out that comparisons and claims made simply cannot be made. I understand that it is alarming that people claim they have seen "how conformant Mozilla and IE really are" but we will have to wait to deal with this issue until the framework is properly set and we can have something more official to say (not saying it is ootright erroneous, but it needs polishing). > mozilla.org's contributers consider standard conformance to be one of > their highest goals and they work very hard to achieve it. In a fair > test, I believe Mozilla's implementation of the DOM Core is superior to > Internet Explorer/MSXML's. However I am afraid that people will never > know that unless improvements to the DOM TS are made. > [dd] My own job (I'm currently freelancing, so you can appreciate how much time this takes from my day to day job) is to strive for implementation of web and information standards. My job in this effort is not to give rates of conformance, but provide the tools necessary to do so. If the tools are flawed, I'll help in fixing them, which I understand your mission to be too. > As an evangelist, it makes my job very difficult when web developers > consider the standard to be whatever Internet Explorer does. For the > DOM TS to further that view is beyond understanding. > [dd] I wouldn't go so far as to say that the DOM TS (intentionally) furthers any view on conformance with the DOM specifications. As has been the case from the start, we formed this group a year or so ago to work with the then largest bulk of tests, the NIST load, and to try to find a framework to give each implementation a fair shot at running them (and other commited in the future). The issue with Mozilla's problems with asynchronicity has been known from the original days of the NIST DOM test suite (Mary, please provide a relevant date, I cannot remember right now), and we have continually tried to contact relevant persons to solve it. I'm hoping that this will start happening now and I feel confident it will, given the intention of each and everyone involved to strive for standard implementation. > And for some background, I did not get involved in this because of my > employment with Netscape. I became involved in testing DOM > implementations (perhaps a better phrase than testing conformance) > before I became an employee of Netscape. I became an employee of > Netscape because of my interest in promoting standards based web > development. In fact, if you read your email from the Summer of 2000, > you will find correspondence with me on this subject. [dd] I once joined the DOM WG for that particular reason, so I know very well what you mean. > > additional comments in line > > >> >>> [snip] >>> >>>> [dd] I agree with most of your points and think that they should be >>>> considered for implementation in the next versions of the DOM TS. >>>> However, it was anticipated quite early that we would not have a >>>> complete test suite, which is indicated in the DOM TS Process >>>> document >>>> that received acceptance from the DOM WG, IG and other related >>>> parties >>>> before being published: >>>> >>>> <quote> >>>> (point 2) The DOM test suites are intended to be used as a tool to >>>> aid >>>> implementors in developing DOM implementations. Validation and >>>> certification of these implementations are outside the scope of this >>>> work. The tests and test suites will be provided for information and >>>> help only. However, we intend to produce as comprehensive, functional >>>> and general tests as possible, and this should be the overall goal in >>>> designing and implementing the DOM TS. >>>> </quote> >>>> >>>> (http://www.w3.org/2002/01/DOMConformanceTS- >>>> Process-20020115#requirements) >>>> >>>> There is a clear intention to produce as comprehensive, functional >>>> and >>>> general tests as possible, but to claim is made that these will be >>>> complete. >>>> >>> [bc] Ok, I understand the intent however since these tests do not >>> compose a conformance test, that fact should be made very clear so >>> that >>> no confusion can arise from the inadvertant appearance of >>> "Conformance" >>> in the documentation. >>> >>> >> [mb] I'm not sure I follow you here. With respect to the term >> "Conformance", I >> think that the web pages for the test suite are clear in what is >> meant. I >> do understand >> that "conformance" can imply different things to different folks -- >> that's >> why so much >> effort was expended in defining what the test suite process is all >> about. I >> would hope >> that folks aren't reaching conclusions without first reading this >> information. As far as >> the tests themselves go, each test was created because there was a >> clause in >> the specification >> that defined what should happen. We tried to ensure that each test was >> well-grounded >> in the specification, and in the xml representation of the test, there >> are >> subject pointers to >> the area of the spec being tested. In addition, here at NIST, we have >> the >> actual clauses >> from the spec that were used to develop the tests. If you believe a >> test to >> be flawed, >> please identify a specific test, and discuss why you believe it to not >> be >> grounded in >> the specification. These tests represent a first pass -- through an >> open >> and iterative process, >> we hope to improve implementations, the specification, and of course >> the >> tests themselves. >> [dd] As far as the documentation goes, I dod hope that I had written it clearly enough. If there is any possibility of misunderstanding as far conformance claims go, I'll see to that they are eliminated in furuter revisions. > [bc] > > So long as their is no confusion in the web developer community I > would agree with you. However, if there is confusion as to the meaning > of the results of the DOM TS, then I think we need to address that > fact. > [dd] You're right. However, I'm afraid parts of the web developer community didn't take the time necessary to read the documentation, and there is only so much one can do about that. In any case, I'll make it as clear as possible. > Let me state once and for all. It is not my mission to fudge/nudge the > DOM TS so that Mozilla performs well on the DOM TS. It is my mission to > make the DOM TS as complete, accurate and fair as possible. Whether the > DOM TS that results from this effort shows Mozilla's DOM implementation > to be adequate or not is secondary. My goal is to provide the > developer of any browser that chooses to develop a DOM implementation > the means to test it as well as provide to any web developer a fair and > accurate way to test the implementations of browsers they wish to > support. > [dd] I don't think anyone misinterpreted your intentions, and I for one am happy that you're aboard. I'm sure you can contribute greatly to this effort and it seems that your goal is shared by all involved in this group. > Such a DOM TS would be a great boon to mozilla.org developers and would > help them pinpoint errors in their implementation. You can be assured > that bugs would be filed in our database immediately. In fact, I became > acutely aware of the DOM TS when mozilla.org contributers began filing > bugs against the browser based upon the DOM TS results. It was then I > discovered how inadequately Mozilla was being tested. > Mary, Thanks for replying. I look forward to working with all of you. > > Bob > > [dd] Given that your email triggered this thourough a discussion, I'm > sure we'll solve most problems and that the next release will be more > complete and fair for everyone. Again, welcome. /Dimitris
Received on Friday, 22 February 2002 06:34:46 UTC