W3C home > Mailing lists > Public > public-qa-dev@w3.org > October 2002

[check][logval] automating validator's testing using logvalidator

From: Olivier Thereaux <ot@w3.org>
Date: Tue, 22 Oct 2002 12:08:33 +0900
To: public-qa-dev@w3.org
Message-Id: <8C6DA990-E56B-11D6-8898-000393BAB03A@w3.org>

A few days ago, I wrote :

> As a sidenote, during the QA Team face-to-face I received the AI to 
> work on completing and (semi)-automating the "test" suite for the 
> validator. (will talk about this in another mail, later)

So, here's the plan...

As far as I know, there are "test files" here and there, that are used 
to test the validator's behaviour, either when changing the prod 
instance, or just to test the dev one.

I know at least :
http://validator.w3.org/dev/tests/
http://www.w3.org/2002/09/xhtml-i18n-tests/
and I guess Masayasu Ishikawa (HML activity lead) has tons of others.

Some tests are here to check error messages, strange cases, etc, but 
most of those are either "should be valid" or "should be invalid". It 
should be straightforward to automate this using the logvalidator, and 
it should save a lot of time.

The logvalidator is usually fed logs (hence the name...) but it's easy 
to make it run on a specific list of URIs.

Simple (very basic) use of the logval to test the validator would be to 
run both ("should be valid" and "should be invalid") lists through it 
and check "by hand" that the results are correct. Refinements could 
include analysis of the results, diff-ing the results for tests run on 
the prod validator and on the tested instance, etc.

This, of course, relies on the HTTP headers feature of the validator, 
and if this one is broken or simply not working (e.g on the current 
:80), it won't work.

Thoughts?

P.S: logvalidator at http://www.w3.org/QA/Tools/LogValidator

-- 
Olivier
Received on Monday, 21 October 2002 23:08:36 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 19 August 2010 18:12:43 GMT