Editor: Christophe Strobbe (DocArch, Katholieke Universiteit Leuven).
Test Case Description Language (TCDL) is an XML vocabulary for describing test files and evaluation scenarios for such test files, intended to be included in test suites for user interface guidelines such as Web Content Accessibility Guidelines (WCAG) 2.0.
This is a draft for discussion by the TSD TF only.
@@todo
This document defines the features and syntax of Test Case Description Language (TCDL). The objective of this language is to define metadata for test cases. In addition to formal metadata and links to relevant sections in technical specifications, TCDL also supports the definition of test scenarios for the validation of test samples (for example by end users) against usability and accessibility guidelines. TCDL defines metadata for detailed descriptions of test cases, for tracking the status of test cases during the development process, for matching test cases with user profile metadata, etcetera.
TCDL can be used to support the development of suites of test samples for sets of accessibility guidelines such as the Web Content Accessibility Guidelines 2.0 (WCAG 2.0).
This specification reuses element and attribute types from other
specifications. These element and attribute types can be recognized by the
namespace prefixes that precede the names (for example, html:
,
dc:
, xlink:
). TCDL files need
not use the same namespace prefixes as this specification, except where the
namespace prefix is xml:
, which is reserved by the
XML specification (XML 1.0). References to data types
defined in XML Schema Part 2: Datatypes
consistently use the xs:
namespace prefix.
The content models for the element types state whether a child element type is optional or required; child element types are not repeatable unless this is explicitly stated.
This specifcation uses certain keywords as defined in RFC 2119 to indicate requirement levels; these keywords are emphasized like the following examples: must, must not, should.
The root element type of a TCDL file is
testCaseDescription
. This element type has the following
attributes and namespace declarations:
id
: an identifier for the test case that is unique that is
unambiguous and unique withing the context of the test case (the context
can be a test suite) (required);xml:lang
:
the default language of the TCDL file (optional);dir
: the default text direction of
the TCDL file (optional);xmlns="http://bentoweb.org/refs/TCDL2.0/"
: the default
namespace (required);xmlns:dc="http://purl.org/dc/elements/1.1/"
: namespace for
Dublin Core Element Set (required);xmlns:xlink="http://www.w3.org/1999/xlink"
: namespace for
W3C XLink (required);xmlns:html="http://www.w3.org/1999/xhtml"
: namespace for
XHTML 1.x (required);xmlns:xsd="http://www.w3.org/2001/XMLSchema"
;xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
;xsi:schemaLocation
: for linking namespace
URIs with the URIs of the
corresponding schema documents.Note: the namespace prefixes need not be the same as those in the above list.
Note: for extensibility, the
testCaseDescription
element type can have any other attribute
types in the target namespace (http://bentoweb.org/refs/TCDL2.0/); schema
processors will validate attributes for which they can find declarations and
raise errors if they are invalid (<xs:anyAttribute
namespace="##targetNamespace" processContents="lax"/>
).
TCDL metadata are organised in five major sections, which will be explained below:
formalMetadata
element type,
required),technologies
element type,
required),testCase
element type, required),rules
element type, required),namespaceMappings
element type,
optional),Extension
),In addition, there are also several extension elements and extension points. See “Versioning XML Vocabularies” (Orchard 2003) and “Extensibility, XML Vocabularies, and XML Schema” (Orchard 2004) for more background on the extensiblity of XML vocabularies.
Below is the skeleton of a TCDL file.
<?xml version="1.0" encoding="UTF-8"?>
<testCaseDescription id="wcag20060427_sc1.1.1_l1_001"
xml:lang="en"
xmlns="http://bentoweb.org/refs/TCDL2.0"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:html="http://www.w3.org/1999/xhtml"
xsi:schemaLocation="http://bentoweb.org/refs/TCDL1
http://bentoweb.org/refs/schemas/tcdl1.xsd
http://bentoweb.org/refs/TCDL2.0
http://bentoweb.org/refs/schemas/tcdl2.0.xsd
http://purl.org/dc/elements/1.1/
http://dublincore.org/schemas/xmls/simpledc20021212.xsd
http://www.w3.org/1999/xhtml
http://www.w3.org/2004/07/xhtml/xhtml1-strict.xsd
http://www.w3.org/1999/xlink
http://bentoweb.org/refs/schemas/xlink.xsd"
>
<formalMetadata>
<!-- child elements omitted -->
</formalMetadata>
<technologies>
<!-- child elements omitted -->
</technologies>
<testCase complexity="atomic">
<!-- child elements omitted -->
</testCase>
<rules>
<!-- child elements omitted -->
</rules>
<namespaceMappings>
<!-- child elements omitted -->
</namespaceMappings>
</testCaseDescription>
Note: There is no normative schema for XLink; see Appendix B of the XLink specification (XLink 1.0). In the BenToWeb project, the W3C XML Schema for XLink 1.0 (METS XLink Schema, v. 2, Nov. 15, 2004), from the Metadata Encoding and Transmission Standard (METS) Official Web Site (Library of Congress) was borrowed with permission.
formalMetadata
Element Type [stable]The formalMetadata
element type contains formal or
administative metadata.
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).The formalMetadata
element type also has a number of required
and optional child element types, which occur in the following order:
title
(required),dc:creator
(required),dc:language
(required),dc:rights
(required),date
(required),status
(required),version
(optional),source
(optional),dc:contributor
(optional),Extension
(optional extension element),title
Element TypeThe title
element type contains the title of the test case.
It does not need to be unique (for example, in the test suite containing the
test case) and its lenght can vary depending on the complexity of the test
materials. However, the title should be sufficiently
descriptive to allow a quick identification of the issue exhibited by the
test case.
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).This element type may not only contain character data but also the XHTML element types specified in XHTML-Style Inline Markup.
dc:creator
Element TypeThe dc:creator
element type is borrowed from the Dublin Core
Metadata Element Set (DCMES 1.1) and
identifies the person or organization primarily responsible for creating the
test case.
This element type has no attribute types.
The dc:creator
element type has the data type xs:string and
must therefore contain only character data.
dc:language
Element TypeThe dc:language
element type is borrowed from the Dublin Core
Metadata Element Set (DCMES 1.1) and
identifies the language of the intellectual content of the test case.
This element type has no attribute types.
The dc:language
element type has the data type xs:string and
must therefore contain only character data. However,
in TCDL the value of dc:language
must be one of the language tags defined by BCP 47
(the successor of RFC 3066). The value must not be empty or null.
dc:rights
Element TypeThe dc:rights
element type is borrowed from the Dublin Core
Metadata Element Set (DCMES 1.1) and
describes rights held in and over the test case.
This element type has no attribute types.
The dc:rights
element type has the data type xs:string and
must therefore contain only character data.
date
Element TypeThe date
element type identifies the date when the test case
was created.
This element type has no attribute types.
The date
element type has the data type xs:date
(which is inspired by ISO 8601). Example:
2004-10-01 means 1 October 2004.
status
Element TypeThe status
element type identifies the state of the test case
during the development process of the test suite, by means of one of an
enumerated list of values (for example, “draft”, “pending bugfix”,
“accepted”, “rejected”).
This element type has no attribute types.
The status
element type has the data type xs:string, but the
values should really be chose from a limited list. Defining the list of legal
values is outside the scope of this specification because these values depend
on the context of the test suite. For an example, see the Conformance Test Process For WCAG
2.0, which contains a description of the steps that must be performed so
that a test case may be added to a test suite for WCAG
2.0.
version
Element TypeThe version
element type contains an identifier that allows
one to distinguish between different revisions of a test case. Note that the
W3C QA Framework Primer defines a new
version of a specification as a significant functional change and
enhancement
. TCDL does not define what consitutes a
new version of a test case because this depends on the context in which the
language is used.
Note: the WG Note on Test Metadata contains the following note about “version”:
ISSUE: It is unclear whether it is necessary to apply version numbers to individual tests. More common practice is to version the entire test suite. Ideally tests will not need to be revised as the specification evolves. However, this is sometimes necessary, and at a minimum each Working Group should define SpecRef and possibly other metadata such as Grouping elements to indicate which versions of the specification an individual test supports.
This element type has no attribute types.
The version
element type has the data type xs:string.
source
Element TypeThe source
element type is used to document the source of a
borrowed test case and related information: the name and
URL of the original test suite, the URL
of the original test file, its creator, whether the file was copied literally
or with changes, and rights held in and over the original test case.
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).The source
element type can only contain other element types
in the following order:
testSuite
(required),sourceFile
(required and repeatable),dc:rights
(optional),comment
(optional).testSuite
Element TypeThe testSuite
element type is used to document the test suite
from which a test case was borrowed.
This element type has the following attribute:
xlink:href
(required) to specify the location of the test
suite.The testSuite
element type can only contain one element type
(and no character data):
name
(required) to specify the name of the test suite.This element type may not only contain character data but also the XHTML element types specified in XHTML-Style Inline Markup.
sourceFile
Element TypeThe sourceFile
element type is used to document a file that
was borrowed from another test suite.
This element type has the following attribute types:
changed
(required) to specify whether the file was
borrowed with changes (“true”) or without any changes
(“false”),xlink:href
(required) to specify the location of the
original file.The sourceFile
element type can only contain one element type
(and no character data):
dc:creator
(required).The dc:creator
element type identifies the person or
organization primarily responsible for creating the borrowed test file. For
attribute types and content model, see the dc:creator
element type above.
dc:rights
Element TypeThe dc:rights
element type describes rights held in and over
the borrowed test file. For attribute types and content model, see the dc:rights
element type above.
comment
Element
TypeThe comment
element type is a general-purpose container for
comments on the borrowed materials, such as changes that were performed,
errors that were found etcetera.
This element type can only contain XHTML-style paragraphs.
dc:contributor
Element TypeThe dc:contributor
element type identifies the person or
organization responsible for making contributions to the content of the test
case.
This element type has no attribute types.
The dc:contributor
element type has the data type
xs:string.
technologies
Element Type [stable]The technologies
element type contains the followiing
information about the technologies used in a test case:
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).The technologies
element type contains one or more instances
of the technicalSpec
element type.
technicalSpec
Element TypeThe technicalSpec
element type identifies a technology
(proprietary, W3C recommendation or other type of
technical specification) used in the test case. If a test case uses
server-side scripting to generate content on the fly,
technicalSpec
identifies the generated content type, not the
server-side scripting language. Languages and formats that are embedded or
inlined in another format, for example inline CSS or an embedded PNG file in an HTML file are also considered as
technologies in the context of this specification. There is one
technicalSpec
element for each file technology used by the test
case.
This element type has the following attribute types:
xlink:href
(required),baseline
(optional).The xlink:href
attribute type contains the
URL of the technical specification of the file format.
According to the XLink 1.0 specification,
[t]he value of the
href
attribute must be a URI reference as defined in RFC 2396, or must result in a URI reference (…).
The baseline
attribute type specifies whether a technology
(technicalSpec
) or a feature (testElement
) is
inside or outside the baseline assumed by the test case. The value
“included” means that something is inside the baseline; the value
“excluded” means outside the baseline. (For information about baselines
in WCAG 2.0,
see Technology
assumptions and the "baseline" in Web Content
Accessibility Guidelines 2.0 (W3C Working Draft 27 April
2006) and About
Baselines and WCAG 2.0.)
[@@References]
Note: at the time of writing [@@date], it is not entirely clear whether WCAG baselines will only contain complete technologies or also mention specific features. The issue was discussed in March, with some arguments in favour of very granular baseline information. WCAG conformance claims don't need to specify what technologies were used outside the baseline, but it seems desirable that test case metadata are more precise in this area.
Note: the requirement that there is a formal specification or
recommendation for each technology works for many technologies
(HTML, CSS,
ECMAScript, Flash, PDF, PNG,
JPEG) but may be more problematic for others
(vendor-specific HTML extensions, the currently
non-standerdized XMLHttpRequest
object, …).
The technicalSpec
element type can contain two other element
types:
specName
(required),testElements
(optional).specName
Element TypeThe specName
element type contains the name of the
technology, more specifically the title of the technical specification that
defines the technology. This element type may not only contain character data
but also the XHTML element types specified in XHTML-Style Inline
Markup.
testElements
Element TypeThe testElements
element type contains a description of the
elements or features used by the test case. This element can be omitted for
binary formats (PDF,
Flash, …). This element type has no attribute types. It contains at least
one instance of the testElement
element type.
testElement
Element TypeThe testElement
element type contains a description of a
specific element or feature. There is one testElement
for each
feature. For CSS, the name
of the rule, selector, property, property value, … can be provided. For
proprietary element and attribute types (Microsoft's bgsound
,
Netscape's embed
, etcetera), a link to the company's
documentation can be provided.
The testElement
element type has an optional baseline
attribute. It also contains
one instance of the elementName
element type, followed by one
instance of either the specReference
element type or the
specQuote
element type.
The elementName
element type
identifies the name of the element or feature. This element type has the
following attribute types:
xml:lang
(optional),dir
(optional),localname
(required),namespace
(optional).The localname
attribute type identifies the name of the feature. If the feature is an
element or attribute from an XML vocabulary,
localname
contains the unqualified name.
The namespace
attribute type identifies the namespace URI of the
feature, if relevant. This attribute type must not
be used if the technology defining the feature is not an
XML vocabulary.
elementName
is an empty element type; it does not contain
child elements or characer data.
The specReference
element type contains
the title of the relevant section in the specification. Use the title
provided by the specification wherever possible; otherwise provide a succinct
and meaningful title for the relevant section in the specification.
This element type has the following attribute types:
xml:lang
(optional),dir
(optional),xlink:href
(required).The xlink:href
attribute type contains the URL of the relevant
section in the technical specification.
The specReference
element type may not only contain character
data but also the XHTML element types specified in XHTML-Style Inline
Markup.
The specQuote
element type contains a quote from the relevant section of the
specification. specQuote
can be used if it is not possible to
provide an exact pointer to the relevant section in the specification (with
specReference
).
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).The specQuote
element type may not only contain character
data but also the XHTML element types specified in XHTML-Style Inline
Markup.
testCase
Element Type [stable]The testCase
element type describes what is necessary for the
evaluation of the test case.
This element type has only one attribute type, complexity
,
with the values “atomic” or “complex”. A test case is atomic if it
applies to only one “accessibility rule” (for example, a
WCAG 1.0 checkpoint, a WCAG 2.0 success
criterion or a Section 508 rule), or complex if it applies to multiple
“accessibility rules” from the same set (i.e. the same
guidelines document or the same set of legal requirements).
The testCase
element type can contain six other element
types:
description
(required),purpose
(required),expertGuidance
(optional),preconditions
(optional),requiredTests
(required),files
(required).description
Element TypeThe description
element type contains a summary of the test
materials and how they are to be used in the test case. The description
indicates to the accessibility expert/scenario author what to expect and what
will happen when a user interacts with the test materials. The description of
the behaviour is especially important for test materials with interactive
components such as forms.
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).The description
element type may not only contain character
data but also the XHTML element types specified in XHTML-Style Inline
Markup.
purpose
Element TypeThe purpose
element type contains a description of the
intention of the test materials. The purpose contains an explanation of the
expected evaluation result in regard to the relevant rules, checkpoints or
success criteria. It does not need to repeat the “rule” or success
criterion for which the test case is developed. (Strictly speaking, this goes
beyond the Purpose
in W3C Working Group Note on Test Metadata.)
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).This element type can only contain XHTML-style paragraphs.
expertGuidance
Element TypeThe expertGuidance
element type contains guidance to
reviewers (for example, accessibility experts if TCDL is used for a
WCAG test suite) for evaluating the test case.
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).This element type can only contain XHTML-style paragraphs.
preconditions
Element TypeThe preconditions
element type contains conditions that must
be met before this test is executed. See Preconditions
in W3C Working Group Note on Test Metadata.
This element type is provided for extensibility; it can contain any
elements in any namespace (##any
).
requiredTests
Element TypeThe requiredTests
element type describes
Note: in Bentoweb, each scenario has a mandatory identifier that is unique within the test case description. This identifier is necessary to map end user feedback to the scenarios.
The TCDL specification does not define any attribute
types for the requiredTests
element type but allows the use of
any attribute types for extensibility.
The requiredTests
element type can contain the following
element types in the following order:
testMode
(required),scenario
(optional and repeatable),Extension
(optional extension element),testMode
Element TypeThe testMode
element type specifies the type of evaluation
that is required to get the test case “accepted”.
This element type has no attribute types and allows only string values (xs:string).
Note: BenToWeb uses the following test modes:
scenario
Element TypeThe scenario
element type specifies a scenario that can
generate data for validating the test case. In TCDL, this
element type is geared towards end-user evaluation of test samples. For this
purpose, it is possible to define several types of questions (see below).
This element type has the following attribute:
id
(required) for an identifier that is unique within the
TCDL file.The scenario
element type can contain the following element
types in the following order:
dc:creator
(optional),date
(optional),status
(optional),scenarioNotes
(optional),userGuidance
(optional and repeatable, one per language),questions
(required),experience
(required),disabilities
(optional).For dc:creator
, date
and status
,
see above: the dc:creator
element
type, the date
element type, and
the status
element type.
scenarioNotes
Element TypeThe scenarioNotes
element type is a container for any kind of
notes on the scenario, for example, its purpose.
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).The scenarioNotes
element type can only contain XHTML-style paragraphs.
userGuidance
Element TypeThe userGuidance
element type contains additional guidance to
the end user who evaluates the test case, especially with regard to the
configuration of their user agent and/or assistive technology. The
xml:lang
attribute is required.
This element type has the following attribute types:
xml:lang
(required),dir
(optional).Note: the languages must
match those used for the questions defined in the questions
element. This means that for each language version of a question, there
must be a corresponding language version of
userGuidance
, and vice versa, unless the scenario does not need
user guidance (in which case there is no userGuidance
element
for any of the languages in which the questions are available).
This element type can only contain XHTML-style paragraphs.
questions
Element TypeThe questions
element type contains one or more question that
can be presented to a human evaluator (an end user) to evaluate a test case.
In this context, end-user evaluation is a method for getting feedback on a
technique or a specific aspect of a technique's implementation, for example,
the use of certain elements and/or attributes to meet a
WCAG 2.0 success criterion or a specific contrast ratio in
a web page.
If the evaluation panel (a group of end-users not involved in the development of the test cases) is multilingual, it may be necessary to provide the questions in different languages. In that case, there should be one question or set of questions per language that needs to be supported. The use of different language versions for questions, user guidance and labels must be consistent: when any of these items is translated, each of the other elements (assuming that they are used) must also be translated.
This element type has the following attribute types:
xml:lang
(required),dir
(optional).The questions
element type can contain only one of the
following five element types, which are mutually exclusive:
openQuestion
, oryesNoQuestion
, oryesNoOpenQuestion
, orlikertScale
, ormultipleChoice
.openQuestion
Element TypeThe openQuestion
element type contains an open question with
a textarea, which can be presented to a user.
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).The openQuestion
element type contains the following element
types, which occur in the following order:
questionText
(required and repeatable, one per language), which contains the actual
question, andspace
(required) which specifies the size of the text area
in which the user can enter a response.The questionText
element type has a
required xml:lang
attribute and an optional dir
attribute. There is one questionText
element per language in
which the scenario needs to be available. As noted, above, the language
versions of questionText
must match those of
userGuidance
, if any user guidance is provided. The
questionText
element type can only contain XHTML-style paragraphs.
The space
element
type has two required attribute types: rows
, which specifies
the number of visible text lines, and columns
, which specifies
the visible width in average character widths. The values of the
rows
and columns
attributes must be positive integers. The resulting text area
must allow the user to enter more lines than
specified by the rows
attribute and longer lines specified by
the columns
attribute.
yesNoQuestion
Element TypeThe yesNoQuestion
element type contains a yes-no-question,
with a third option (for options such as “Don't know”, “Not
applicable”, etcetera).
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).The yesNoQuestion
element type contains the following element
types, which occur in the following order:
questionText
(required and repeatable, one per language), which contains the actual
question, andoptionYes
(required),optionNo
(required),optionOther
(optional).The questionText
element
type is defined above.
The element types
optionYes
and optionNo
are empty element types
with the following attribute types:
value
(required) for the value that will be sent to the
server if the user selects this option,expected
(optional) to specify if this option is what the
question author would expect from the user (indicated with the values
“yes” or “no”),xml:lang
(optional),dir
(optional).The element types optionYes
and optionNo
are
empty because it is not necessary for TCDL to contain the
strings “yes” or “no” (or any translations of these):
the process that transforms the scenarios into a user interface can fill in
these strings.
The optionOther
element type specifies an
alternative for “yes” or “no”, for example “I don't know” or
“Other”. There must be one “other
option” for each language in which the questions are available. Unlike the
element types optionYes
and optionNo
,
optionOther
has child elements: it can only contain XHTML-style paragraphs.
The optionOther
element type has the following attribute
types:
expected
(optional) to specify if this option is what the
question author would expect from the user (indicated with the values
“yes” or “no”),xml:lang
(required),dir
(optional).yesNoOpenQuestion
Element
TypeThe yesNoOpenQuestion
element type contains a
yes-no-question, with a third option (for options such as “Don't know”,
“Not applicable”, etcetera) and an open question with a text area. It is
a combination of the yesNoQuestion
and openQuestion
element types defined above.
The yesNoOpenQuestion
element type contains the following
element types, which occur in the following order:
questionText
(required and repeatable, one per language), which contains the question
for the yes-no part and possibly also for the open part,optionYes
(required),optionNo
(required),optionOther
(optional),questionText
(optional and repeatable, one per language), which contains the question
for the open part if this is not already contained in the first
questionText
(s),space
(required), which specifies the size of the text
area in which the user can enter a response.The element types questionText
, optionYes
, optionNo
,
optionOther
and space
are defined above.
likertScale
Element TypeThe likertScale
element type specifies a Likert scale.
The Usability First™ Usability Glossary defines Likert scale as follows:
a type of survey question where respondents are asked to rate the level at which they agree or disagree with a given statement. For example:
I find this software easy to use.
strongly disagree 1 2 3 4 5 6 7 strongly agree
A Likert scale is used to measure attitudes, preferences, and subjective reactions. In software evaluation, we can often objectively measure efficiency and effectiveness with performance metrics such as time taken or errors made. Likert scales and other attitudinal scales help get at the emotional and preferential responses people have to the design. Is it attractive, fun, professional, easy?
This element type has the following attribute type:
display
(required), which defines how a list of choices is displayed: vertically,
horizontally or as a drop-down list (with the values
“vertical”, “horizontal” and
“dropdown”, respectively).The default for display
is vertical (for greater
accessibility to screen readers). Dropdowns are not recommended if more than
one choice needs to be selected.
The likertScale
element type contains the following element
types, which occur in the following order:
questionText
(required
and repeatable, one per language), which
contains the actual question, andlikertLevel
(required, exactly five times).The likertLevel
element type defines a level in the Likert
scale.
The element has the following attribute types:
number
(required) for the number of the level (1-5, consecutively),expected
(optional) to specify if this option is what the question author would
expect from the user (indicated with the values “yes” or
“no”).The likertLevel
element type contains the
following elements, which occur in the following order:
label
(required and repeatable, one per language), for the value displayed to
the user,value
(required), for the value to be submitted to the
server.The label
element
type specifies the label associated with the choice. There is one
label
element per language in which the scenario needs to be
available.
This element type has the following attribute types:
xml:lang
(required),dir
(optional).The label
element type may not only contain character data
but also the XHTML element types specified in XHTML-Style Inline
Markup.
The value
element
type specifies the value to be submitted by the user. There is only one
value
element in every likertLevel
, so that the
same value is sent to the server, regardless the number of languages the
label
is available.
This element type has no attribute types. Its data type is xs:string.
multipleChoice
Element TypeThe multipleChoice
element type contains a multiple choice
question. It is possible to limit the number of responses to one (for radio
buttons) or to allow multiple reponses (for check boxes). This element type
is strictly a generalisation of likertScale
, which can be seen
as a short-hand notation of a specific usage of
multipleChoice
.
The multipleChoice
element type has the following attribute
types:
select
(required), with the values “one” or
“multiple”,display
(required).The select
attribute type defines whether the user can select
one or multiple choices (i.e. one radio button versus
multiple check boxes): if a multipleChoice
element has a
select
attribute with the value “one”, the
HTML equivalent is rendered as a set of radio buttons; if
the select
attribute has the value “multiple”, the
HTML equivalent is rendered as a set of check boxes.
The multipleChoice
element type contains the following
element types, which occur in the following order:
questionText
(required
and repeatable, one per language), which
contains the actual question, andchoice
(required, at least two instances), for each of the
options presented to the user.The choice
element type has the following attribute types:
The choice
element type contains the following element types,
which occur in the following order:
label
(required and repeatable,
one per language), for the value displayed to
the user,value
(required), for the value
to be submitted to the server.experience
Element TypeThe experience
element type specifies a user's level of
experience with the specified types of software. There are three categories
of software: assistive technologies, user agents and devices. User agent also
covers browser plugins.
The experience
element type has no attribute types. It
contains the following element types, which occur in the following order:
AssistiveTechnology
(optional and repeatable),UserAgent
(required and repeatable),Device
(optional and repeatable).AssistiveTechnology
, UserAgent
and
Device
are empty element types. AssistiveTechnology
lists assistive technologies such as screen readers, screen magnifiers, voice
recognition software, alternative keyboards and alternative pointing devices.
UserAgent
here refers to any software that retrieves and renders
Web content for users. This may include Web browsers, media players,
plug-ins, and other programs — including assistive technologies — that
help in retrieving and rendering Web content. [@@check
WCAG glossary] Device
reters to the hardware
on which user agents and/or assistive technologies run: personal computers,
personal digital assistants, mobile phones, …
Note: in BenToWeb, assistive technologies are not user agents but are a category of their own.
The element types AssistiveTechnology
, UserAgent
and Device
have the following attribute types:
minimumLevel
(required),type
(required),product
(optional),version
(optional).The attribute type minimumLevel
specifies a user's required
level of experience with a certain software or device on a scale from 1
(low/novice) to 5 (high/expert). How these levels are established is outside
the scope of this specification but depends on the context where
TCDL is used. This attribute type contains integers from 1
up to and including 5.
The attribute type type
specifies the type of software or
device. This attribute type contains only character data (data type
xs:string).
Note: BenToWeb uses the following values for
type
:
AssistiveTechnology
: “screenreader”,
“screenreader with magnification”, “magnification software”,
“speech and hearing support software”, “Braille display”,
“speech recognition software”, “CCTV”,
“alternative input devices”;UserAgent
: “browser”, “talking browser”,
“plugin”;Device
: “PC”,
“PDA”, “MobilePhone”, “Television”.The attribute type product
specifies a product in the
category specfied in type
.
The attribute type version
specifies a version of the product
specified in product
. If the version
attribute type
is used, the product
attribute type must also be used. The data type of version
is xs:decimal.
experience
and User ProfilesThe experience information in a TCDL file can be used to filter specific scenarios for users with a specific profile, so that it is possible to gather information on how a test case is used by someone with a specific browser, a specific screen reader, etcetera. For this to work, the system that matches scenarios to user profiles would need a matching algorithm like the following:
AssistiveTechnology
,
UserAgent
and Device
) have an AND relationship:
a user's profile only matches if there is a match between each of the
classes in experience and the corresponding part of the user profile;type
value have an
AND relationship: a user's profile only matches if each of these types is
listed in his/her profile;type
value have an OR
relationship: a user's profile results in a match if any product of this
type is in his profile, unless experience
also lists
specific products;type
value have an AND
relationship: a user's profile only matches if each of these types is
listed in his/her profile;type
value have an OR
relationship: a user's profile results in a match if any product of this
type is in his profile, unless experience
also lists
specific products;type
value have an OR relationship:
a user's profile results in a match if any product of this type is in his
profile, unless experience
also lists specific products;AssistiveTechnology
, UserAgent
and
Device
), values of product
and
version
need an exact match in the user profile.experience
: ExamplesA few examples of how to match scenarios to user protiles with the above
algorithm are discussed below. For the sake of simplicity, we will disregard
to minimumLevel
attribute values.
<!-- example 1 -->
<AssistiveTechnology type="screenreader"
minimumLevel="3"/>
<UserAgent type="browser" minimumLevel="1"/>
This matches all user profiles with a certain screenreader AND a certain browser.
<!-- example 2 -->
<AssistiveTechnology type="screenreader"
minimumLevel="3"/>
<AssistiveTechnology type="alternative input
devices" minimumLevel="3"/>
<UserAgent type="browser" minimumLevel="1"/>
This matches all user profiles with a certain screenreader AND an alternative input device AND a certain browser.
<!-- example 3 -->
<AssistiveTechnology type="screenreader"
minimumLevel="3"/>
<UserAgent type="browser" product="Microsoft Internet Explorer" version="4" minimumLevel="1"/>
<UserAgent type="browser" product="Microsoft Internet Explorer" version="5" minimumLevel="1"/>
<UserAgent type="browser" product="Microsoft Internet Explorer" version="6" minimumLevel="1"/>
This matches all user profiles with a certain screenreader AND any of the specified versions of Microsoft Internet Explorer (4, 5 or 6, but not 5.5).
<!-- example 4 -->
<AssistiveTechnology type="screenreader"
minimumLevel="3"/>
<AssistiveTechnology type="magnification
software" minimumLevel="3"/>
<UserAgent type="browser" product="Microsoft Internet Explorer" version="4" minimumLevel="1"/>
<UserAgent type="browser" product="Microsoft Internet Explorer" version="5" minimumLevel="1"/>
This matches all user profiles with any screenreader AND any magnification software AND any of the specified versions of Microsoft Internet Explorer (4 or 5 but not 5.5).
<!-- example 5 -->
<AssistiveTechnology type="screenreader"
product="JAWS" minimumLevel="3"/>
<AssistiveTechnology type="screenreader"product="Window-Eyes" minimumLevel="3"/>
<UserAgent type="browser" product="Microsoft Internet Explorer" version="4" minimumLevel="1"/>
<UserAgent type="browser" product="Microsoft Internet Explorer" version="5" minimumLevel="1"/>
This matches all user profiles with any version of JAWS or Window-Eyes AND any of the specified versions of Microsoft Internet Explorer (4 or 5 but not 5.5).
<!-- example 6 -->
<AssistiveTechnology type="screenreader"
product="JAWS" minimumLevel="3"/>
<AssistiveTechnology type="Braille display"
minimumLevel="3"/>
<UserAgent type="browser" minimumLevel="1"/>
<Device type="PC" minimumLevel="2"/>
This matches all user profiles with any version of JAWS AND any Braille display AND any browser AND a PC.
disabilities
Element TypeThe disabilities
element type specifies one or more
disabilities to which the test case is relevant. This enables matching test
cases with user profiles and prevents that end users get test cases that are
irrelevant to them (for example, a test case on colour contrast should not be
matched with a blind user).
The disabilities
element type has no attribute types. It can
only contain one element type:
disability
(required and repeatable).The disability
element type can only contain character data
(data type: xs:string). Defining a list of disabilities is beyond the scope
of this specification.
Note: BenToWeb uses a list of disabilities that contains the following values (non-exhaustive):
The values above are not based on the WHO's International Classification of Functioning, Disability and Health (@@reference). One must also take into account that many of these terms may be found objectionable by persons they are supposed to describe. For some of these terms it is hard to find equivalents in other languages; literal translations may have a different meaning than in English.
files
Element TypeThe files
element type contains information about each of the
test files that make up the test case.In a XHTML test
suite for WCAG 1.0 or 2.0, for example, there is usually
only one XHTML test file per test case (not counting
supporting files such as external stylesheets , images, etcetera), but some
checkpoints or success criteria, for example, those on consistent navigation,
consistent style etcetera, require a set of files.
The files
element type has only one attribute type:
sequential
(optional), with the value “true” or
“false” (the default).The sequential
attribute type specifies whether the order of
the files (if there are several) matters for the evaluation of the test case:
“true” means that the files should be processed in the order in which
they occur in the test case description; “false” means that the files can
be processed in random order. The default value is “false”.
The files
element type can only contain one element type:
file
(required and repeatable).file
Element
TypeThe file
element type has the following attribute types:
xlink:href
(optional),hrefLang
(optional).The xlink:href
attribute type specifies the
URL of the test file or test sample. Supporting files such
as included images, external style sheets and dummy pages to which forms are
submitted (if only the form is relevant to the test) etcetera, need not be
documented. (Only what the Last Call Working Draft of WCAG
2.0 calls "Web units" needs to be listed.)
The hrefLang
attribute type specifies the language of the
file or sample identified by xlink:href
(if the attribute is
present) or httpRequest
(explained below).
The file
element type can only contain one element type:
httpRequest
(optional).httpRequest
Element TypeThe httpRequest
element type specifies data for creating a
HTTP request, such as media type, encoding, HTTP headers
and HTTP body.
The httpRequest
element type has the following attribute
types:
mediatype
(optional),enctype
(optional).The mediatype
attribute type specifies the value of the
Accept header of the HTTP request. @@Reference: list of
MIME types at IANA.
The enctype
attribute type specifies the
MIME type of the body of an HTTP POST
request. This can be used in form submission with an HTML
form (see HTML's enctype
attribute type) and
with XForms. [@@refer to 'media types': RFC 2045:
'Multipurpose Internet Mail Extensions (MIME) Part One:
Format of Internet Message Bodies']
The httpRequest
element type contains the following element
types, which occur in the following order:
httpHeaders
(optional),httpBody
(optional).The httpHeaders
element type specifies headers for an
HTTP request to a server. This element type has no
attribute types and can contain only one element type:
header
(required and repeatable).The httpBody
element type specifies content for the
“body” or “payload” of an HTTP POST request.This
element type has no attribute types and can contain only one element type:
variable
(optional and repeatable).The element types header
and variable
are empty
element types with the following attribute types:
name
(required),value
(required).The name
attribute type of the header
element
type only allows names of HTTP methods. The
name
attribute type of the header
element type and
the value
attribute type of both the header
and
body
element types allow any character data (data type:
xs:string).
rules
and
rule
[stable]The rules
element type contains the “rules” that are
tested by a test case. “Rule” is here used as a generic term for
accessibility requirements such as
The rules
element type has no attribute types. It can only
contain one element type:
rule
(required, repeatable).rule
Element
TypeThere is one rule
element per “rule” that is documented
in the test case.
The rule
element type has the following attribute types:
id
(required), for the URI of the
rule,primary
(required), to specify if the rule is
“primary” (value “yes”) or not (value “no”).The id
attribute type specifies the URI of
the rule (data type xs:anyURI). The id
attribute type does not provide a direct reference to the relevant online
source because it may not always be possible to reference “rules”
directly (for example, because they are not available in
HTML). The id
attribute type contains a URL
with a fragment identifier: the URL part locates a “Rulesets XML” file, the
fragment identifier refers to a rule ID which in turn references
the relevant rule in an online source.
The primary
attribute type specifies whether a rule is
“primary” or not: primary rules are those against which the test case is
supposed to be evaluated; other rules are informational.
The rule
element type contains the following element types,
which occur in the following order:
locations
(required),functionalOutcome
(optional),techComment
(optional).locations
Element TypeThe locations
element type specifies the location or
locations where relevant (especially faulty) code occurs, more specifically
code that exhibits an issue (especially an error) relevant to the specified
“rule”.
This element type has the following attribute type:
expectedResult
(required).The expectedResult
attribute type specifies the expected
result of an evaluation of the code against the specified “rule”. The
following values are allowed: “pass”, “fail”, “cannotTell” and
“notApplicable”. The value “cannotTell” is used if there is ambiguity
of unclarity (for example, dependence on the implementation or version of a
user agent or assistive technology, ambiguity in the accessibilty rules or in
the specification of the file format). The value “notApplicable” can be
used to point out explicitly that a specific location in the code is not
relevant.
The locations
element type can only contain one element type
(and no character data):
location
(optional and repeatable).The location
element type specifies a location in the code.
Sometimes a combination of features (markup, rules, etcetera) constitute an
error when taken together (even if these features are not errors when
considered separately). In that case, there should be one
location
element per feature/error. At least XPath or a
combination of line and column number must be
used.
The location
element type has the following attribute
types:
xlink:href
(optional),line
(optional),column
(optional),xpath
(optional),The xlink:href
attribute type specifies the file in which the
issue occurs. This attribute type must be used when
the files
element specifies more than one file; otherwise it can
be omitted: if the files
element contains only one
file
element, the file specified there is assumed to contain the
issue.
The line
and column
attribute type specify the
line and column, respectively, where the relevant code is located. These
attribute types must always be used together. The
value “1” denotes the first position.
The xpath
attribute type specifies the location of an issue
by means of an XPath expression as defined in XPath
1.0.
functionalOutcome
Element
TypeThe functionalOutcome
element type specifies the functional
outcome of the test: a description of why the test case passes or fails, in
terms that relate to a user's experience (as opposed to technical comments
about source code).
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).The functionalOutcome
element type can only contain XHTML-style paragraphs.
techComment
Element TypeThe techComment
element type specifies a technical comment on
the test: a technical description of why the test case passes or fails, and
other technical information (for example, issues in certain browsers or
assistive technologies).
This element type has the following attribute types:
xml:lang
(optional),dir
(optional).The techComment
element can only contain XHTML-style paragraphs.
namespaceMappings
and namespace
[stable]This optional section is important for test cases based on test files that
use XML-based technologies. The technology features (see
the section on Technologies above) used in a test case exist in an
XML namespace1 and should be referenceable with XPath. The
normal mechanism for mapping namespaces to URIs by
declaring them with xmlns:prefix= "namespaceURI"
cannot be used
for this, because it is necessary to be able to define empty prefixes and to
extract the namespace mappings with a common XML
API.
This section describes content models and data types that are used by several element types in TCDL.
This content model is a mixture of character data and the following XHTML elements:
em
(emphasis);strong
(strong emphasis);dfn
(defining instance of a term or phrase);code
(program code);q
(inlined quote);samp
(sample);kbd
(keyboard input, something a user would input);var
(variable);cite
(citation);abbr
(abbreviation);acronym
;sub
(subscript);sup
(superscript);br
(forced line break);span
(generic container);bdo
(I18N
bidirectionality override).The XHTML 1.0 specification allows that the above
element types contain other element types. However, in
TCDL the above element types must
not contain element types other than those in the above list and the
a
element type (for links). Note that the schema for TCDL does not enforce
this constraint.
This content model is a sequence of one or more p
elements.
The content model of these p
elements is the
XHTML-style inline markup described above.
xml:lang
and hrefLang
The xml:lang
attribute type specifies the base language of an element's attribute
values and text content. The default value of this attribute is unknown. This
attribute type is imported from the XML schema for
XML 1.0, which is available at http://www.w3.org/2001/xml.xsd. The
value of xml:lang
is inherited; the data type is xs:language.
The hrefLang
attribute type specifies the base language of the resource designated by
the xlink:href
attibute on the same element; its data type is
xs:language.
dir
Attribute TypeThe only values allowed for this attribute type are ltr
(for
left-to-right text) and rtl
(for right-to-left text). When the
attribute is not used, the text direction is left to right, unless another
text direction is inherited from an ancestor element.
@@discuss the Extension
element type and the etensibility
points.
@@todo describe rulesets, which are used in the rules
section
of TCDL.
@@todo add XML Schema.
The example below uses only required element and attribute types, except
the optional xlink:href
attribute on the file
element.
<?xml version="1.0"
encoding="UTF-8"?>
<testCaseDescription id="wcag20060427_sc3.1.1_l1_001"
xml:lang="en"
xmlns="http://bentoweb.org/refs/TCDL2.0"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:html="http://www.w3.org/1999/xhtml"
xsi:schemaLocation="http://bentoweb.org/refs/TCDL1
http://bentoweb.org/refs/schemas/tcdl1.xsd
http://bentoweb.org/refs/TCDL2.0
http://bentoweb.org/refs/schemas/tcdl2.0.xsd
http://purl.org/dc/elements/1.1/
http://dublincore.org/schemas/xmls/simpledc20021212.xsd
http://www.w3.org/1999/xhtml
http://www.w3.org/2004/07/xhtml/xhtml1-strict.xsd
http://www.w3.org/1999/xlink
http://bentoweb.org/refs/schemas/xlink.xsd"
>
<formalMetadata>
<title>German text with English language
identification</title>
<dc:creator>John.Doe@example.com</dc:creator>
<dc:language>en</dc:language>
<dc:rights>Copyright
Example.com</dc:rights>
<date>2006-08-31</date>
<status>draft</status>
</formalMetadata>
<technologies>
<technicalSpec><!-- xlink:href is optional -->
<specName>
<html:acronym>XHTML</html:acronym>™ 1.0: The Extensible HyperText Markup Language
(Second Edition)
</specName>
</technicalSpec>
</technologies>
<testCase complexity="atomic">
<description>Web page with an excerpt of
Peter Handke's play<html:span lang="de">Publikumsbeschimpfung</html:span>(in German); the<html:code>html</html:code>element has
a<html:code>lang</html:code>attribute with
the value "en" (for English) instead of "de" (for German).
</description>
<purpose><p>This test case is
intended to fail because the language of the page is not correctly
identified.</p>
</purpose>
<requiredTests>
<testModes>
<testMode>oneExpert</testMode>
</testModes>
</requiredTests>
<files>
<file xlink:href="wcag20060427_sc3.1.1_l1_001.html"/>
<!-- @@note xlink:href is actually optional -->
</files>
</testCase>
<rules>
<rule primary="yes"
id="http://bentoweb.org/refs/rulesets.xml#WCAG2_20060427_3.1_meaning-doc-lang-id">
<locations expectedResult="fail"/>
</rule>
</rules>
</testCaseDescription>
@@todo add XML Schema.
On 12 October 2003, the Quality Assurance Working Group (QA WG) published the submission Test Case Description Language 1.0. Many sections of this document are unfinished; according to a message of Lofton Henderson to the QA WG mailing list, this version was going to be superseded.
@@todo describe diferences between the two TCDLs.
On 14 September 2005, the Quality Assurance Working Group (QA WG) published a Working Group Note on Test Metadata. (See also Testcase Metadata, draft, 9 September 2005.)
The W3C
Working Group Note “Test Metadata” (14 September 2005) suggests
fourteen metadata elements for test suites. The table below shows how these
metadata elements map to TCDL elements. (In this table,
TCDL elements are identified by means of XPath expressions
without a namespace prefix and with the root element —
testCaseDescription
— omitted, except for the attributes of
that element.)
W3C Metadata | TCDL | Comment |
---|---|---|
Identifier | testCaseDescription/@id |
|
Title | formalMetadata/title |
In TCDL, titles are not necessarily unique; they don't duplicate the identifier. |
Purpose | testCase/purpose |
|
Description | testCase/description |
In TCDL, the description should help the reviewer understand how the test materials are constructed and the behaviour when a user interacts with it. |
Status | formalMetadata/status |
|
SpecRef | rules/rule and technology/technicalSpec |
In TCDL, two types of specifications need to be referenced: the accessibility guidelines (“rules”) and the specifications of the technologies in the test. |
Preconditions | testCase/preconditions |
Not used in BenToWeb, because test cases map directly to WCAG success criteria instead of tests. |
Inputs | testCase/files/file/httpRequest |
Only for the special case where it is necessary to (re)create a specific HTTP request. |
ExpectedResults | rules/rule/locations/@exptectedResult and rules/rule/functionalOutcome |
|
Version | formalMetadata/version |
It may be possible to rely on a version control system instead? |
Contributor | formalMetadata/dc:conributor |
|
Rights | formalMetadata/dc:rights and formalMetadata/source/dc:rights |
TCDL can document rights related to a test case and rights related to borrowed materials. |
Grouping | - | (Because TCDL describes only single test cases, grouping should happen at a higher level.) |
SeeAlso | technology/recommendation (and rules/rule/techComment ) |
This work has been undertaken in the framework of the project BenToWeb — IST-2-004275-STP — funded by the IST Programme of the European Commission.