Re: Notes on validome test suite / validators comparison

Hi Olivier,

Here is our statement:

> * http://www.validome.org/out/ena1004
> * http://www.validome.org/out/ena4021
> * http://www.validome.org/out/ena2002

Here Validome advices the user to use our XML-Validator, as a HTML-Validator is not the appropriated tool to check XML...;-)
 
> * http://www.validome.org/out/ena1005
> * http://www.validome.org/out/ena1007
> * http://www.validome.org/out/ena1011
> * http://www.validome.org/out/ena1012
> * http://www.validome.org/out/ena1014
> * http://www.validome.org/out/ena1015
> * http://www.validome.org/out/ena4019
> * http://www.validome.org/out/ena4020
> * http://www.validome.org/out/ena2
> * http://www.validome.org/out/ena13
> * http://www.validome.org/out/ena14
> * http://www.validome.org/out/ena5006
> * http://www.validome.org/out/ena5007
> * http://www.validome.org/out/ena5008
> * http://www.validome.org/out/ena5009
> * http://www.validome.org/out/ena5010
> * http://www.validome.org/out/ena5011
> * http://www.validome.org/out/ena5021
> * http://www.validome.org/out/ena5025
> * http://www.validome.org/out/ena5026
> * http://www.validome.org/out/ena5027
> * http://www.validome.org/out/ena5028
> * http://www.validome.org/out/ena6030
> * http://www.validome.org/out/ena5030

Here we corrected our claims, sorry for not keeping the comparison up to date. BUT: Until you renewed the W3C-Validator by implementing LibXML, the announcements were right and you knew this about an year... (http://lists.w3.org/Archives/Public/www-validator/2006Apr/0072.html)

> * http://www.validome.org/out/ena4011
> HTML 4.01 document with no system Id.
> Validome sends a warning... Not necessary per the spec.
> W3C Markup validator passes validation.
> Why is W3C validator marked as faulty here? References please?

http://www.w3.org/TR/1999/REC-html401-19991224/struct/global.html#h-7.2
Other way: Where is specified, that System-Id can be missed?
 
> * http://www.validome.org/out/ena4012
> XHTML doctype without system Id, but valid public id.
> Validation should report an error (both validators do), but why does  
> validome count this as a fatal error?

That one has coding reasons.
 
> * http://www.validome.org/out/ena4023
> Validome says valid. OpenSP and W3C Markup validator says not valid.
> I'd tend to trust opensp here. The comparison page's claim that  
 validome is the only validator doing the right thing is very dubious.
 
> * http://www.validome.org/out/ena4024
> Ditto above. The comparison page's claim that validome is the only  
> validator doing the right thing is very dubious.

What is here dubious? It's about SGML (not HTML) documents.
 
 
> * http://www.validome.org/out/ena8
> W3C markup validator uses algorithm for charset detection, finds  
> none, uses fallback
> Validome uses... exactly the same algorith (to the point of having  
> almost the same error message...), finds no charset, yields a fatal  
> error.
> I'm very curious to know why validome passes and w3c markup validator  
> fails here. I think the opposite: validome's taste for fatal error is  
> a grave failure in usability.

The "old" W3C-Validator made a fallback o US-ASCII, the "new" to UTF-8. Can you explain this, please?
We asked many times W3C-Germany and Bjoern Hoehrmann in regard to the *correct* behaviour of an validator in the case of a fallback, but we didn't get any *exact* answer. In this case, the specs are very unexact and ambiguous. Please give us a *mandatory" answer - with a link reference to appropriate specifications - upon this case. The only clear case till now is XHTML, there validators should make a fallback to UTF-8 (depending on MIME-Type), HTML is still ambiguous...
 

> * http://www.validome.org/out/ena2008
> * http://www.validome.org/out/ena2009
> * http://www.validome.org/out/ena2010
Old, deprecated examples because of unclear and ambiguous specifications.
 
> * http://www.validome.org/out/ena2041
> The comparison page is incorrect. The W3C Markup validator has the  
> proper behavior here.

The W3C-Validator doesn't detect the conflict.
 
> * http://www.validome.org/out/ena5020
> I strongly disagree that the W3C Markup's validator behavior is  
> incorrect, here.
> text/html is allowed for XHTML 1.0

We don't claim here, the behaviour of the W3C-Validator is wrong, we say that we miss the appropriate note.
In accordance to http://www.w3.org/TR/2002/REC-xhtml1-20020801/#media a XHTML document should be delivered with MIME-Type text/html when it meets the guidelines of HTML compatibility. That is what a validator shoul claim and Validome does it.
 
> * http://www.validome.org/out/ena7003
> I'd like to see a reference for this.

http://www.w3.org/TR/html401/struct/links.html#h-12.2.3
"...The id attribute, on the other hand, may not contain character references."
 

> * http://www.validome.org/out/ena7005 (and 7006)
> This has nothing to do with validation. If validome emulates some of  
> the features of a link checker, compare it to link checkers, not  
> validator. This test is moot.

http://www.w3.org/TR/html401/struct/links.html#h-12.2.4
"A reference to an unavailable or unidentifiable resource is an error"
...
"If a user agent cannot locate a linked resource, it should alert the user"

Where is here the "moot"? The W3C-Specification is very clear in this case...
 

> * http://www.validome.org/out/ena3002
> This test is bogus. Sorry. An XML declaration also happens to be a  
> proper SGML PI. Giving a warning asking the HTML4 author "are you  
> sure you want this here" may be a good idea. Making this a fatal  
> error is wrong, wrong, wrong.

If a XML-declaration is allowed in SGML, I'd like to see a reference for this.
*If* this should be right, ehat about the priority of the encoding attribute within declararion vs. META-element??????

> * http://www.validome.org/out/ena3006
> The comparison page is incorrect. Output of a warning for a shorttag  
> construct is a good thing (dev version of w3c validator actually does  
> it) but not required. The current W3C Validator's behavior is not wrong.
> 
> * http://www.validome.org/out/ena3007
> ditto. Learn about shorttags. Validome is actually wrong here, this  
> should not be reported as an error, at most a warning.

Oh, her we have hundred opinions of the case. Could you please show us a *exact* reference?

BTW:

At the moment, we are implementing the W3C Validator in a free out of the box software solution for Windows users, together with validome.
When trying to implement, there are some inconsitencies/bugs we found:
 
1. The W3C-SGML-Parser uses two catalog files: xml.soc and sgml.soc. Within xml.soc there are 21 points missing, all regarding SVG 1.1 Tiny and "SVG 1.1 Basic.
2. We missed 6 DTDs, necessary to get the download package running.
3. Your LibXML-Implementation was not correct - you just use the catalog files of your SGML-Parser instead of taking care of the the "official" catalog specification (http://www.xmlsoft.org/catalog.html#Simple).
Because of this, LibXML tries to get the external DTDs instead of the local ones.
 
You write within your CVS (http://dev.w3.org/cvsweb/validator/httpd/cgi-bin/check?rev=1.574&content-type=text/x-cvsweb-markup):
 
# [NOT] loading the XML catalog for entities resolution as it seems to cause a lot of unnecessary DTD/entities fetching (requires >= 1.53 if enabled)
#$xmlparser->load_catalog( File::Spec->catfile($CFG->{Paths}->{SGML}->{Library}, 'xml.soc') );
 
That is not right, as your implementation is not correct. BUT: Youcan download the fixes on http://www.validome.org/W3C_fix.rar, with the fixes it works (Problem 1+2+3 solved).
 
Best regards,
 
Alex