Test Results Summary - Nov 26

To have a quick view of where we stand implementation wise, I did a 
fast summary of current test results as reported in Sandro's thing, 
counting only the approved tests (as of 11/26) - here's where we 
stood.  I did this by hand (needed something to keep me awake an 
extra hour :->) but it shows we've got some nice implementations out 
there doing well
  -JH
p.s. Note that some implmentations were only tested on part of the 
tests, and therefore reproting as a percentage is not indicative of 
how they did on the tests they were tested on

test results = Nov 26, 2003

NAMESPACE

1 - Use of OWL namespace  (9 Lite)
  5 implementations tested, all get 100%

ENTAILMENT

2 - Entailment tests (27 Lite)
10 implementations tested - results range from 3% - 100% 
  top results are Euler 100%,  Pellet 85%, Bossom 74%

3 - Entailment tests (26 DL)
10 implementations tested - results range from 3% - 100% 
  top results are Euler 100%,  Pellet 96%, Hoolet 69%

4 -Entailment tests (37 Full)
7 implementations tested (note 8 reported, but OWLP may be a mistake)
results ranged from 13-100%
  3 best Euler 100%, Pellet 86%, Bossam 78%

8 - Non-entailment tests (8 light)
  6 implementations tested, results from 0% - 100%
  3 best FOWL 100%, OWLP 87%, Pellet 62 % (Euler, Surnia report 0%)

9 - Non-entailment tests (5 DL)
  7 implementations tested, results from 0%-100%
  3 best FOWL 100%, Pellet 80%, OWLP 60 % (Euler, Surnia report 0%)

10 - Non-entailment tests (5 Full)
  4 implementation tested, results from 0% - 100%
Fowl 100%, Pellet 80% (Euler, Surnia report 0%)

OWL for OWL

13 - OWL for OWL (1 Lite)
4 reporting, 0%-100%
  OWLP, FOWL, Euler get 100%

14 - OWL for OWL (1 DL)
4 reporting, all get 100% (OWLP, FOWL, Euler, Jena2)

15 - OWL for OWL (4 Full)
4 reporting, 50%-100%
Cerebra, FOWL, Euler 100%

CONSISTENCY

16 - Consistency (25 Lite)
8 implementations reported, 0% - 100%
3 best Pellet 100%, Cerebra 84%, ConsVisor 80%

17 - Consistency (26 DL)
8 implementations reported 0%-100%
Cerebra 100%, Pellet 92%, Hoolet 84%

18 - Consistency (13 Full)
6 reported, 0% -100%
FOWL 100%, ConsVISor 100%, Pellet 92%

21 - Consistency (6 extra credit)
6 reported, 0%-100%
FOWL 100%, ConsVISor 100%, Pellet 83%

22 - Inconsistency (29 Lite)
10 reported 0%-100%
Pellet 100%, Euler 100%, Hoolet 93%, FaCT 82%

23 - Inconsistency (37 DL)
10 reported, 0%-100%
Pellet 100%, Hoolet 97%, Euler 97%, FaCT 89%

24 - Inconsistency (7 Full)
8 reported, 0%-100%
Euler 100%, FOWL 71%, Jena2 71%

IMPORT

29 - Import entailment (1 Lite)
5 reported, all at 100%

30 - Import Entailment (2 Full)
5 reported, all at 100%

31 - import Level (3 Lite)
4 reported, all at 100%

32 - import Level (2 DL)
4 reported, all at 100%

33 - import Level (2 Full)
4 reported, 3 at 100%

SYNTACTIC

35 - syntactic level (103 Lite)
4 reported, all at 100%

36 - Syntactic level (96 DL)
4  reported, all at 100%

37 - Syntactic Level (70 Full)
4 reported, 98-100%
Jena2, Pellet, WonderWeb 100%

41 - Syntactic Level (1 Extra credit DL)
4 reported, all 100%

42 Syntactic Level (6 Extra Credit Full)
4 reporting, all 100%

-- 
Professor James Hendler			  http://www.cs.umd.edu/users/hendler 
Director, Semantic Web and Agent Technologies	  301-405-2696
Maryland Information and Network Dynamics Lab.	  301-405-6707 (Fax)
Univ of Maryland, College Park, MD 20742	  240-277-3388 (Cell)