EARL - The Evaluation And Report Language

What Is EARL?

EARL is a machine processable model and vocabulary for reporting evaluations. EARL does not constrain you on the types of evaluations that you make, but its vocabulary is biased towards evaluating software and hardware online, for example rating pages for accessibility, validating code, or rating software.

The EARL Model

The general EARL model consists of a "context", which may include details about the person or machine running the test, and a series of assertions. These assertions consist of three parts - the thing being evaluated, the test that it's being evaluated against, and the validity (e.g., pass or fail). For example, a very simple EARL evaluation, in prose, might be something like "Sean says that page x passes checkpoint y". However, EARL is a machine processable langauge. This means that we have standard ways of saying things, so that machine can interpret these easily. This allows us to make highly-grained evaluations, and analyze or process the results in manners which are difficult for humans.

The EARL Syntax

EARL builds on top of the Resource Description Framework Model and Syntax specification, which has been a W3C recommendation since February 1999. RDF also forms the building blocks for the Semantic Web, so by building EARL on top of this, we open up the possibility for Semantic Web "compliance", and all the benefits that brings with it.

RDF is generally an excellent solution for languages such as EARL, because it is a metadata format. Metadata are simply data about data; i.e. details about something. RDF is based on things called "triples". This is a very simple concept, and has a close analogy in the English languagee in "noun verb noun". For example, a triple might be something like "Sean likes Chocolate".

As you can tell, there must be some twist to the language! There is in that fact that instead of using words in our triples, we use URIs. URIs are the strings starting with "http:", or "ftp:" etc. that you use in your browser and other Web applications. These strings are very useful to us in EARL because they are decentralized. Decentralization means that there is no one central authority controlling something, so in our case, there is no one central authority that controls the delegation of all URIs, although certain URIs do have registrars, such as ICANN. However, the two simple facts are that 1) anyone can get a URI, 2) anything that is on the World Wide Web has a URI. Therefore, if it is on the Web (has a URI), EARL can say something about it. If it's not on the Web, you can easily create a URI, or talk about it simply by using its properties.

An Example Of EARL

Time for some real code. We're going to be using a simple format called "N3" here, although you do not need to learn it for the purposes of this discussion, or even to write EARL, for EARL can be written in any RDF syntax.

In N3, we simply write out the "noun verb noun" triple (which technically is comprised of a "subject", a "predicate", and an "object"), much like in English. So for example, "Sean likes Chocolate" might be written as:-

:Sean :likes :Chocolate .

Note the full stop (period) at the end of the triple. The colons before the words simply indicate that these are actually URIs. We can declare elsewhere in the document what these colons stand for exactly, but this need not concern us here. All we have to note for now is that anything with a simple colon is something that that the user has made up, and anything that starts with "earl:" is something that is in the EARL vocabulary.

The simplest kind of EARL evaluation is:-

:Sean earl:asserts { :MyPage earl:passes :CP1 } .

Which is English might be something like "Sean asserts that 'MyPage' passes 'CP1'". Note the curly brackets in there: this allows us to use a triple where you might only have one term. It's like nesting a triple inside a triple. (In RDF, this is called "reification", although in Notation3, sets of reified statements are allowed, so we call the thing in the curly brackets a "context". It's a lot like quoting something).

The EARL Vocabulary

As we have already stated, EARL consists of a set of terms that people can use, i.e. a vocabulary. We have already used two of the EARL terms above in our example, "earl:asserts", and "earl:passes". These particular terms are (in RDF) called "properties". Not all things are properties, only the terms that connect other terms (i.e. the "verbs", although it is possible to use noun clauses). The thing in the middle of a triple is always a type of property.

As well as properties, RDF (and hence EARL) has a set of terms called classes. These are very much like the nouns of a sentence, except that classes are simply sets of objects. For example, "WebPage" might be a class. Some thing can "be" a WebPage, and there may be many WebPages. Other classes might include people, operating systems, or whatever. RDF then lets us say that something is a type of that class - it belongs to that class.

For example:-

:Sean a :Person .
:Fido a :Dog .
<http://www.w3.org/> a :WebPage.

The word "a" is used in the examples above as shorthand for "rdf:type". "rdf:type" is simply a special term that the RDF specification defines for people implementing RDF to use. As you can tell from the examples above, it's a very simple term - you can read it as "is a type of", "is a", "belongs to class", and so on.

This is a very useful thing for us to use in EARL, because it lets us talk carefully about the evaluation. Let's take our example from above again:-

:Sean earl:asserts { :MyPage earl:passes :CP1 } .

We could now say what type of thing each part of this evaluaton is:-

:Sean a earl:Person .
:MyPage a earl:WebContent .
:CP1 a earl:Checkpoint .

EARL has many such terms, for more details, try the EARL schema.

Note that classes can also have sub classes and super classes. For example, we can say that:-

earl:Person rdfs:subClassOf earl:Animal .

This simply says that person is a sub class of animal, in other words that all humans are animals, that some animals are humans, and that something which isn't an animal isn't a human. You can think of this in terms of a diagram:-

Human Animal

There can be many such sub classes and super classes, although usually these are of minimal importance.

Back to typing, we can say that:-

{ :Sean earl:asserts
     { { :MyPage earl:passes :CP1 } a earl:Assertion }
} a earl:Evaluation .

And that:-

:Sean a earl:assertor .

Which brings us to the whole stuff about domains, ranges, and all that stuff. Hmm... this got quite "technical" quite fast.

Sean B. Palmer