Re: EARL and locating a problem...

I'm sorry that I wasn't able to contribute more to the discussion last
Monday.

I see some of discussion points to be too specific at the moment
because we risk of loosing track of the larger picture. The problems
to be solved are difficult and still vague.

I would suggest to follow two steps:
1. specify a service that integrates test reports and allows queries
    on them
2. work on its implementation using earl (including definition of a
    location/normalization thta has all the important properties:
    tolerant wrt edits, tolerant wrt different tools, ...)

In this way we can face three different problems separately, namely
deciding what kind of interface evaluation tools should provide (to
such a service), which functionalities the service should offer to end
users, and how to use EARL (and therefore which features it
has to support) to implement it.

As for step 1 my ideas are:

* each tool (lift, accverify, bobby, ...) should be able to export
   information on the resources being tested, the checkpoint followed,
   the outcome of the test, when the test has been done, ...
   This info might be expressed as earl statements.

* the tool should also specify where in the resource the problem
   occurs (location of the problem). As there are many ways to do it
   (offset, index number, xpointer)  and some of them depend on the
   parser the tool is based on, then let's invent a syntax for
   expressing these different representations of locations.
   Example: (method = dom-index, parser = dreamweaverMX), or (method =
   char-offset, parser = explorer4), or (method=xpointer, parser=mozilla)

* the service should take as input this info and by using an internal
   normalization (eg. mozilla parser that adds unique ids to all the
   elements) should convert it so that the appropriate earl statements
   can be made. This will require tuning and tweaking to handle more
   than one type of location representations.

Then we can work on the second problem (which functionalities make
sense for the end user) and third one (how to use EARL to support
them).

We can also experiment with different internal normalizations
(pointers, object collections, ....) since
they are shielded from (a) the tools, (b) the end users.
Only when
- we understood which functionalities make sense for end users, and
   their interaction structure
- we understood a viable way to represent locations

can we decide on a specification for EARL that supports these two
things.

My 2 euro cents :-)
Giorgio

Charles McCathieNevile wrote:
> Hi folks,
> 
> during the meeting we discussed the question of whether earl should provide a
> pointer to where it finds a specific problem within a page, or whether it is
> better to write earl statements about objects within a page, and then
> describe the page as a collection of objects that do or don't meet a given
> set of requirements. I took an action item to carry the discussion on outside
> the meeting.
> 
> I used to believe the latter, so I will try to explain why I have moved to
> the former...
> 
> Part of this has been pragmatically motivated by talking to developers of
> repair tools. In order to do repairs it is important to know where the repair
> should be made, if possible. Both approaches work, up to here, but from this
> point it becomes tricky.
> 
> As I see it, the problem is not that we need to know what needs retesting
> afterwards. That is more or less the same problem in both methods. Details of
> whether changing a particular item, or repairing a fault, can be expected to
> have or not have an effect on other related requirements is probably beyond
> the scope of EARL and lives in the land of testing processes.
> 
> But details of what a page contains need to be provided,and need to be exact,
> if we are not going to have a location pointer for a particular result that
> is independent of the test object (which may or may not have a larger scope).
> This requires us to be able to specify exactly what a page consists of, and
> we may have to divide it along several different axes for different types of
> tests. That strikes me as a lot of work that only sometimes pays off.
> 
> Moreover, it isn't necessary to define that as a method of locating problems
> in order to use it. The place where it strikes me as a valuable technique is
> in assessing databases behind content management systems. But even there, it
> seems like a messy way to deal with problems in the template used to produce
> an instance of content for rendering to the user. It is something that can
> always be done, but there seem like good reasons for using what is
> essentially the shortcut of allowing a relevantLocation or something similar
> to be a property of a result.
> 
> An interesting paralell is Annotea. It is possible to annotate something
> directly even if it is only a part of a larger document, but the annotea
> design instead annotates a whole document and provides a location as the
> information about what exactly was located. It might be interesting to know
> if this was a deliberate design decision, and why.
> 
> cheers
> 
> Chaals
> 
>

Received on Friday, 28 June 2002 06:54:51 UTC