- From: Olivier Thereaux <ot@w3.org>
- Date: Tue, 22 Oct 2002 12:08:33 +0900
- To: public-qa-dev@w3.org
A few days ago, I wrote :
> As a sidenote, during the QA Team face-to-face I received the AI to
> work on completing and (semi)-automating the "test" suite for the
> validator. (will talk about this in another mail, later)
So, here's the plan...
As far as I know, there are "test files" here and there, that are used
to test the validator's behaviour, either when changing the prod
instance, or just to test the dev one.
I know at least :
http://validator.w3.org/dev/tests/
http://www.w3.org/2002/09/xhtml-i18n-tests/
and I guess Masayasu Ishikawa (HML activity lead) has tons of others.
Some tests are here to check error messages, strange cases, etc, but
most of those are either "should be valid" or "should be invalid". It
should be straightforward to automate this using the logvalidator, and
it should save a lot of time.
The logvalidator is usually fed logs (hence the name...) but it's easy
to make it run on a specific list of URIs.
Simple (very basic) use of the logval to test the validator would be to
run both ("should be valid" and "should be invalid") lists through it
and check "by hand" that the results are correct. Refinements could
include analysis of the results, diff-ing the results for tests run on
the prod validator and on the tested instance, etc.
This, of course, relies on the HTTP headers feature of the validator,
and if this one is broken or simply not working (e.g on the current
:80), it won't work.
Thoughts?
P.S: logvalidator at http://www.w3.org/QA/Tools/LogValidator
--
Olivier
Received on Monday, 21 October 2002 23:08:36 UTC