log of chat: EARL and TDPL

Sean and I spent some time chatting today about EARL and a test point 
description language.  Here's the log of the chat. I've edited out 
irrelevant bits.

21:14:37 <wendy> sbp: you suggested changing { :Page earl:passes :TestCase } .
21:14:39 <wendy> to
21:14:45 <wendy> [ rdf:subject :Page; rdf:predicate earl:passes; rdf:object 
:TestCase ]
21:14:52 <sbp> yep
21:15:15 <wendy> hmmm. could you explain why?
21:15:39 <sbp> contexts are kinda special to CWM, and I'd hoped they'd 
catch on by now. But people are still just discussing away, and reification 
is here to stay. Ooh, that rhymes
21:16:14 <sbp> actually, they just discussed it today on RDF IG
21:16:29 <sbp> Seth asked Tim if a context with one statment as a member is 
equivalent to a statement ID
21:16:47 <sbp> Tim said he stays out of those sort of arguements :-)
21:17:03 <sbp> so reifying might be better
21:17:16 <wendy> when you say "I'd hoped they'd catch on" who is they?
21:17:25 <wendy> people other than those creating CWM?
21:17:32 <wendy> you want others to use contexts?
21:18:16 <sbp> they: the world and it
21:18:19 <sbp> and it's dog
21:18:27 <wendy> :)
21:25:03 <wendy> so, when are you going to write an EARL validator? :)
21:25:13 <sbp> it only implemented "earl:pass" :-)
21:25:30 <sbp> EARL Validator: there are two different approaches I could take
21:25:53 <sbp> 1) Play about with CWM/Llyn
21:26:01 <sbp> 2) Beg Aaron to work on his RDF API
21:26:08 <AaronSw> Already on it!
21:26:15 <sbp> problem solved
21:26:17 <AaronSw> Running the unit test now...
21:27:00 <sbp> we're looking for a CGI form whereby one can paste in a URI, 
and have it "validate" the EARL
21:27:16 <sbp> in as much as one can validate an RDF Model. Really, you 
just check it for consistency
21:28:02 <sbp> which CWM is good at. Perhaps I'll try that approach too
21:28:19 <wendy> hmm. i thought it was the syntax we wanted to validate, 
not the model. am i confusing things?
21:28:41 <sbp> but I'm a bit saddened by the whole state of EARL at the 
moment: test case work messing things up, etc.
21:28:50 <sbp> syntax: just use the RDF Validator for that
21:29:44 <sbp> with a model validator, you want to check that people are 
not doing _:Checkpoint earl:passes _:Page . or anything silly like that

21:31:36 <wendy> hmm. the other day you said s/earl:suite/earl:id.  but 
this uses earl:suite.
21:31:46 <wendy> or am i switching them?
21:31:46 <sbp> oh, feck

21:33:15 <sbp> I'm not even sure what earl:suite does anymore
21:33:34 <sbp> I mean, it's meant to link to a set of conformance criteria
21:33:41 <sbp> e.g. all of WCAG 1.0
21:34:20 <sbp> but perhaps it would be better to link it to a URI/bNode, 
and then link all of the IDs off of that. Sort of like the machine readable 
version of WCAG that I drafted up
21:34:28 <wendy> yes, linking to all of wcag 1.0 could be called a suite, 
but it's an id. not sure we need to distinguish...
21:34:34 <wendy> although some of the recent thread gets into that.
21:34:54 <wendy> if you want to make a less granular claim you could...but 
i'm not sure how it would change how it gets processed.
21:35:08 <sbp> well, when processing, you need to be explicit
21:35:27 <sbp> it's not enough to be fuzzy to any degree, you have to state 
things in incredibly pedantic detail
21:35:34 <sbp> which is probably why I like it :-)
21:35:53 <wendy> yes, good to be pendantic.  but, i'm trying to think of a 
scenario of how this would be processed differently.
21:35:54 <sbp> so using WCAG as a kind of bag may seem good on the surface, 
but the processor won't know what's in the ba
21:36:08 <wendy> yes - exactly - unless that bag is defined somewhere...
21:36:14 <wendy> perhaps suite = WCAG 1.0
21:36:29 <wendy> WCAG 1.0 is a bag for {set of checkpoint URIs}
21:36:32 <sbp> and WCAG's annoying, because really it's a bag with three 
other bags in it, all of the different checkpoints, and you can have 
overlaps of the bags
21:36:40 <wendy> actually- 3 bags. one for each level of conformance of wcag.
21:36:41 <sbp> yep, that's how I wanted to model it
21:36:49 <wendy> :) exactly.
21:36:53 <sbp> :-)
21:37:29 <wendy> ok. so in this instance i've used an id, but suite should 
be possible, as long as we have a way to define what is in the suite.
21:37:43 <wendy> which, is basically just an rdf:bag.
21:38:09 <sbp> DAML lists are considered "cooler"
21:38:26 <wendy> I don't know anything about DAML, really...could you explain?
21:38:53 <sbp> er... it's an ontology. so, it's a language which allows you 
to make other languages with astonishing detail
21:38:59 <sbp> like a schema language
21:39:18 <sbp> but it has other little knick knacks in it, like daml:Seq, 
or whatever it's called
21:39:28 <sbp> we use DAML a little bit in the EARL schema
21:40:17 <wendy> yes - that's the only bit of DAML i know is in the EARL 
schema.
21:40:57 <sbp> the schema is at http://www.daml.org/2001/03/daml+oil#
21:41:18 <sbp> oh, and there's a really good walkthrough: 
http://www.daml.org/2001/03/daml+oil-walkthru
21:42:25 <sbp> Any idea what we're going to do about the TPDL? I kinda 
decided all of a sudden that EARL walks a weird tightrope act between 
linking to a test case, putting up metadata for a test case, and actually 
defining the test case
21:42:59 <sbp> Daniel's original properties had a list of test case stuff 
with a note to enumerate them out later, but we never did, and in the end 
we lost interest in that aspect and got rid of some of the properties
21:43:15 <wendy> yes, it is in an odd grey area.
21:43:19 <wendy> Not sure what we'll do.
21:43:22 <sbp> so we have a kind of semi-pointer/semi-metadata language for 
test cases in EARL
21:43:26 <wendy> I expect it will end up in QA.
21:43:40 <sbp> but the question is: what do we do for EARL?
21:43:40 <wendy> BUT, i would like to iron out as much of that as possible 
before handing off.
21:43:47 <wendy> right. yes.
21:44:09 <wendy> What are your thoughts? Add back the properties that were 
stripped out?
21:44:47 <wendy> incorporate parts of Chris/Josh's test case dtd?
21:45:00 <sbp> well, if a TPDL lanuage exists at all, it will be separate 
from EARL. Well, IMO it should be separate from EARL. Even Al said it would 
be good as a Semantic WEb project, but I'm damned if I'm going to be able 
to work on that as well, what with another degree coming up and so forth, 
so the course for EARL looks a bit shaky
21:45:32 <wendy> do you think more needs to be stripped from earl to remove 
the greyness?
21:45:40 <sbp> well, the question is whther or not to standardize a TPDL, 
and then if so, how do we incorporate that into EARL assuming that it 
doesn't turn out as an RDF language, and if not, how EARL copes with the 
mess that will be created
21:45:51 <sbp> I'm not sure if we need to remove stuff, or add it, or what
21:45:59 <sbp> well... let me slow down a bit...
21:46:06 <wendy> what was your reaction to Dimitris' post?
21:46:28 <wendy> it seemed to me that he was saying the DOM TS ML was 
specific to DOM and not something to base other test suites on.
21:47:22 <sbp> we have the pointer to the test case at the end of the 
statement. That's al;ways going to be there. The more sympathetic the test 
case is to EARL, the less mess we end up with in EARL itself. At a level of 
full compatability iwth EARL (i.e. if TPDL is an RDF language), we can just 
simply put the URI in the reified triple. THat would be the perfect, most 
wonderful solution that coyld happen, and we can all go for a drink and 
dance around the maypole
21:47:29 <sbp> so we're faced with the alternatives...
21:47:42 <sbp> yeah, I'm still digesting what Dimitris wrote
21:47:53 <wendy> then there is the  test case vs test distinction.
21:47:57 <wendy> maypole :)
21:47:59 <AaronSw> * AaronSw wonders what a TPDL is.
21:48:05 <sbp> he said that if TPDL was standardized, that he'd try to move 
over to it, which I thought was really cool
21:48:14 <sbp> TPDL - Test Point Definition Langauge
21:48:22 <wendy> yes, he seems very open to finding an elegant solution.
21:48:26 <AaronSw> Ah.
21:48:56 <sbp> but, that's a lot of work that somebody will have to do. The 
problem with the legant solution is that it takes work away from DOM TS and 
EARL (good), but heaps it all into a group that will have to be chartered 
to cope with it, surely
21:49:14 <sbp> and it's going to be months, if not years, until a good TPDL 
is created
21:49:23 <wendy> seems that it is NISTs job.
21:49:26 <wendy> they are the experts.
21:49:32 <sbp> and even then, it will probably end up being less than 
compatable with either EARL or DOM TS
21:49:43 <sbp> yeah, but are NIST keeping an eye on EARL and the DOM TS?
21:49:46 <wendy> * wendy wonders about large corps that do testing...they 
must have something...likely proprietary. :(
21:50:10 <wendy> actually, NIST has participated some in DOM TS, but I 
think are drooling over QA participation.
21:50:20 <wendy> thus, expect if it is to happen, it will be in QA.
21:50:21 <sbp> If QA sat down and did it, it would benefit us. if someone 
else does it, we'll have even more of a headache, waiting for them to 
finish, and so forth
21:50:30 <wendy> esp. w/DD as one of the leads.
21:50:45 <sbp> yeah, absolutely
21:50:53 <wendy> thus, you and i just need to join QA! help make it happen! :)
21:51:19 <sbp> heh, with all the bundles of free time that we have, eh? :-)
21:51:38 <wendy> free time. :)
21:51:52 <AaronSw> And the validator folks are joining QA too... it's 
getting quite popular it seems.
21:52:14 <sbp> the problem is, we've been working on EARL for ages and ages 
now, and with just one more turn of the wheel, it could be basically 
compltete. And now there's the problem of the TPDL, which could be a whole 
other process... aaargh! I just want to get EARL finished
21:53:34 <wendy> finishing EARL: will it every be? :)
21:53:55 <wendy> TSDL: it would be extremely cool, but it seems part of the 
second phase.
21:54:03 <wendy> there's still tons of work to do for earl.
21:54:17 <sbp> yeah, but TPDL should hav come first, and it's coming second!
21:54:18 <wendy> most elegant solution includes TSDL, but earl at least 
makes current situation better.
21:54:27 <wendy> hindsight is 20/20.
21:54:34 <sbp> yeah, true
21:54:47 <wendy> we should have thought about users as well as devs, but we 
focused on devs and now jim ley is upset.
21:54:52 <wendy> ya live. ya learn.
21:55:11 <sbp> and anyway, EARL 0.95 is unimplementable to devs right now
21:55:25 <wendy> EARL came first, TPDL will come second.  egg before 
chicken as it were, I guess.
21:55:43 <sbp> yeah
21:56:02 <wendy> yikes, unimplementable? missed ya there. what do you mean?
21:56:06 <sbp> O.K., so I ask you: what do you want me to do now?
21:56:12 <sbp> oh, O.K., hang on...
21:56:28 <sbp> unimplementable: well, we're going to change it 
substantially, I hope
21:57:18 <sbp> there are so many little bits which just bug the crap out of 
me that we could do with changing now. But then there is the resistance 
from Chaals, which is valuid, and goes something like "we've already 
learned a lot, we have a lot more to learn"
21:57:35 <sbp> I had hoped that using RDF for all of this would soften the 
blow. But.. um... it hasn't really
21:57:54 <sbp> well, it has. Just not as much as I would have hoped
21:58:15 <wendy> the blow of what? implementability?
21:58:20 <sbp> the 0.9 to 0.95 thing was really cool
21:58:27 <sbp> the blow of version changes
21:58:32 <wendy> ah. gotcha.
21:59:02 <sbp> now, if we go to 1.0, we are in effect saying "this is done, 
go implement it"
21:59:26 <wendy> what was nick's reaction?
21:59:32 <sbp> TPDL is threatening, because it means we cannot conceivably 
come up with the above until we work out how it fits in with EARL
21:59:38 <sbp> Nick's reaction to what?
21:59:53 <wendy> possibly moving to 1.0 in the midst of implementing 0.95.
22:00:04 <sbp> oh, I don't think I told him yet :-)
22:00:07 <wendy> lol
22:00:22 <sbp> and I should note that it's not that 0.95 is 
unimplementable... it's that 1.0 will be better
22:00:32 <wendy> yes. gotcha.
22:00:37 <wendy> ok, then. here's what i know:
22:00:45 <wendy> 1. we don't have a tpdl yet.
22:00:58 <wendy> 2. we want to make sure earl can jive w/tpdl once it exists.
22:01:06 <wendy> 3. but we need to go ahead w/earl.
22:01:15 <wendy> therefore, i suggest that we anticipate earl 2.0.
22:01:25 <wendy> earl 1.0 exists today w/out tpdl.
22:01:30 <wendy> once tpdl exists, we modify earl.
22:01:51 <wendy> only problem then is need to make sure all earl 1.0 data 
doesn't go stale
22:02:01 <wendy> and that there is some sort of easy migration like from 
0.9 to 0.95
22:02:01 <sbp> * sbp sobs
22:02:19 <sbp> 0.9 to 0.95 was easy 'cause we just bugfixed it
22:02:37 <sbp> 1.0 to 2.0 will be scary
22:02:55 <wendy> but not impossible?
22:03:14 <sbp> as long as no data is added, no. it's not impossible
22:03:40 <wendy> and, data is likely to be added in 2.0 to jive w/tpdl.
22:03:43 <wendy> right?
22:04:01 <sbp> I'm not sure what TPDL is (it doesn't exist yet), so I can't 
say for sure
22:04:02 <wendy> :)
22:04:07 <sbp> :-)
22:04:26 <wendy> how difficult would it be to come up with a skeleton of 
what tpdl might look like?
22:04:32 <wendy> earl took a while since we were all learning.
22:04:41 <wendy> tpdl might be easier since we've done some of this.
22:05:03 <wendy> but, i still think tpdl is best owned by qa.
22:05:16 <sbp> yes, I agree. You'd have to ask them, I guess
22:05:17 <wendy> therefore, perhaps we try to get this on the qa agenda for 
their upcoming f2f?
22:05:38 <sbp> oh, absolutely!
22:05:44 <wendy> hmmm.
22:06:00 <wendy> * wendy figures out how to fit in tpdl into 
w3.org/2001/10/22-earl-uaag/all.htm
22:06:19 <sbp> let QA do it :-)
22:06:21 <wendy> this was supposed to be my slideset for a presentation to 
the UA WG on monday, but i think it will be a more genearl resource.
22:06:35 <sbp> it's a really good little page
22:06:45 <wendy> well, but i think we can offer some wisdom to get them 
started.
22:06:51 <wendy> we've been thinking about this for quite a while! :)
22:06:55 <wendy> thx.
22:07:03 <sbp> so, WCAG would have to create test point stuff. How do you 
feel about that?
22:07:18 <wendy> we already are and we need to do it.
22:07:27 <wendy> WCAG needs to be less ambiguous.
22:07:35 <wendy> AERT was born b/c of the ambiguities.
22:07:45 <wendy> it should have come from WCAG in the first place.
22:07:48 <sbp> well, a TPDL would both help and hinder GL as much as ERT, 
right?
22:07:59 <wendy> yes.
22:08:05 <sbp> yep, I certainly agree with that
22:08:31 <wendy> also would help UA WG as working on test suite for UAAG's CR.
22:08:36 <wendy> and XAG's CR
22:08:44 <sbp> perhaps I'll look in my mailbox, and someone will have sent 
me a specification for a decent TPDL langauge in RDF?
22:08:48 <sbp> * sbp checks his mail
22:08:51 <wendy> :)
22:08:59 <sbp> nah, oh well
22:09:05 <sbp> sorry, you were saying? :-)
22:09:21 <sbp> XAG's CR... oh, man... dependencies!
22:09:32 <sbp> we should have done TPDL first!
22:09:41 <wendy> lol
22:09:49 <sbp> no, seriously! EARL would have been just chaff then

22:15:43 <wendy> well, i'll write up some stuff on the tpdl and include it 
in this thing i'm working on.
22:15:48 <sbp> actually, that's your decision
22:15:55 <sbp> what should I do next for ERT?
22:16:09 <wendy> well, hmmm.
22:16:24 <wendy> i'm primarily concerned about what we are going to tell 
WCAG about using EARL on sites.
22:16:45 <wendy> so, any help with that would be good.
22:17:05 <sbp> just make something up! that's what I do
22:17:05 <wendy> secondly, i'm still real interested in what bugs you about 
0.95 and how you would move to 1.0.
22:17:20 <wendy> that whole thing we discussed on monday.
22:17:32 <wendy> :)
22:17:36 <sbp> well, for a start: bug with 1.0: how do we make conformance 
claims about a whole site?
22:17:46 <wendy> yes. that needs attention.
22:17:49 <sbp> s/1.0/0.95
22:17:59 <wendy> yes, that's how i read it :)
22:18:18 <sbp> well, we could nick CWM's stuff for 0.95, or add something 
specialized to EARL 1.0
22:18:30 <sbp> s/or/and\/or/
22:18:46 <sbp> [[[
22:18:46 <sbp> It could be, for example:-
22:18:46 <sbp>    _:a earl:testSubject _:b .
22:18:46 <sbp>    _:b log:uri [ log:startsWith "http://example.org/" ] .
22:18:46 <sbp> It may turn out that we add some terms in the EARL 1.0 
vocabulary to offer
22:18:47 <sbp> a "standardized solution" for this.
22:18:49 <sbp> ]]]
22:20:22 <sbp> ah, we could even say
22:20:53 <sbp> _:x a earl:Site; earl:testSubject <http://example.org/> . # 
would that work?
22:21:03 <wendy> sbp: what terms do you think we would need to add in EARL 
1.0? "applies-across-domain" or "starts-at-base-uri" or stuff like that?
22:21:06 <sbp> Aaron: would it be valid to say that?
22:21:26 <AaronSw> It's well-formed RDF.
22:21:43 <sbp> yeah, we could add loads of stuff. I dunno why we were so 
conservative before: we should just add stuff 'till the schema explodes
22:21:51 <wendy> lol
22:22:09 <sbp> it doesn't matter if we add stuff and it's useless. It 
totally stuffs us if we don't add something and find we need it later on
22:22:49 <wendy> * wendy considers how this would tie in w/html...
22:22:53 <sbp> for example... if I wanted to use a single page as a test 
case, I'd do... oh actually no, I don't think it does make sense
22:23:13 <sbp> tie in with HTML?
22:23:22 <wendy> well...
22:23:35 <wendy> lots of straws here... HTML base element...
22:23:45 <sbp> unless we created a new disjoint class. Ooh, that might work
22:23:58 <sbp> HTML base element?
22:24:27 <wendy> hmm. no nothing there.
22:25:12 <wendy> there are two pieces that we need to look at here on out:
22:25:18 <wendy> 1. how the earl is structured
22:25:26 <wendy> 2. how the html references it
22:25:34 <wendy> keep in mind the following scenario:
22:26:02 <wendy> search engine ferrets out a set of pages that seem to fit 
the user's query then prioritizes them based on EARL.
22:26:14 <wendy> based on EARL compared to user preferences.
22:26:24 <wendy> hmmm. wonder if that means we would be keeping user 
preferences in EARL?
22:27:03 <wendy> or rather what is the earl model (rather than how structured)
22:27:24 <sbp> note to self: split :TestSubject up into smaller pairwise 
disjoint classes, restrict :testCase so that the domain is a union of those 
classes, spend ages telling people why I hacked it that way
22:28:02 <sbp> Wendy: example user's query?
22:29:28 <wendy> a user has uses a search engine (SE).  SE knows that the 
user needs text equivalents (WCAG 1.0 cp 1.1)
22:29:50 <wendy> SE searches database for user's query "rdf api tests" it 
finds several pages, but puts highest
22:30:02 <wendy> priority on this one that claims it conforms to WCAG 1.0 
cp 1.1
22:30:09 <wendy> not the most compelling example...
22:30:40 <wendy> the user could also say, "don't show me sites that don't 
make a claim, since they'll likely be useless to me."
22:30:44 <sbp> ah, I get it now. No, you're right
22:31:15 <wendy> about which part - user prefs in EARL?
22:31:53 <sbp> er... the preferences file would be something like a filter 
in whatever you're using to process the EARL, which in this case in the SE. 
The filters would be in whatever query/inference language the SE 
implements, which is annoying... standardized inference language, please!
22:32:08 <sbp> did I get that right?
22:33:45 <wendy> sounds reasonable, although I don't know that much about SEs.
22:33:46 <sbp> neither do I
22:33:59 <wendy> know anyone who does?
22:34:10 <sbp> Nick, perhaps?
22:34:24 <wendy> worth a shot.
22:34:30 <wendy> we ought to take this to the list for discussion.
--
wendy a chisholm
world wide web consortium
web accessibility initiative
seattle, wa usa
/--

Received on Friday, 19 October 2001 18:53:51 UTC