Fwd: Re: XDMDL feedback

Please see my conversation below with Jason regarding the XDMDL straw-man, bindings, topologies and the definition of xAPI verbs. This evening, I will circulate an agenda for tomorrow's call. Do reply if you there is anything you would like to include.

Best, Crispin.

---------- Forwarded message ----------
From: Crispin Weston <crispin.weston@saltis.org>
Date: Thu Apr 09 2015 11:13:16 GMT+0100 (GMT Summer Time)
Subject: Re: XDMDL feedback
To: Haag, Jason <jason.haag.ctr@adlnet.gov>
Hi Jason,
Many thanks for the feedback. My comments are inline.




On 8 April 2015, Haag, Jason <jason.haag.ctr@adlnet.gov> wrote:
> Hi Crispin,
> 
> Thanks again for sharing the paper. I had planned to get this feedback to you sooner, but didn't have as much time last week. Here are some thoughts:
> 
> General Thoughts:
> * I know I have preached this already, but since this working group activity is being driven by a W3C requirement perhaps we should align as much as possible with existing W3C standards based on things like RDF(Linked Data) to express/model/represent the relationships between the XDMDL components. I know there is a learning curve here, but I wonder if there is a way to model everything by first using your UML approach or simply using entities, classes and properties without focusing on a specific markup or schema (e.g. XML, RDFS).
I take your point on this and do not want to prejudge the best output format - the main reason that I use XML is because that is what I am familiar with. It might be best to adopt a W3C approach, all other things being equal, though I would not want to adopt a binding that was less suitable, just because it was W3C. That suggests that we need to play around with different bindings and maybe discuss some of the abstract characteristics of the solution by testing them out in different ways.

I would prefer to complete my first draft using XML, with which I am familiar, then consider alternative bindings, which will help us (a) consider which binding is best (both technical and political reasons count) and (b) help elucidate & refine the underlying, abstract principles and requirements all the better, for expressing them in different ways. If you are able to create alternative bindings, great, if not, I will just have to get going up the learning curve.

> * Your desire to use xml is not unusual. See this post (<http://www.mikepadgett.com/technology/technical/cmaptools-for-concept-mapping-and-owl-authoring/>). What might be helpful is if we use your UML approach and CMapTools with someway of easily translating that into OWL. IF not, I'm willing to try and do it manually, but it will be a learning curve. I'm still learning about how to do this myself.
I am not particularly attached to UML, which I have used in somewhat unusual ways to express concept maps rather than class diagrams. I also see the advantage of using a tool which will enable diagrams to be modified and extended (kind of the point of the whole exercise...) so I will be very happy to play around with your CMAP tools - but it would be good if we can play in parallel and compare notes.
> * In case we define more than just two types of digital resources, I found this description/usage to be helpful too for describing information vs. non-information resources: <https://code.google.com/p/tdwg-rdf/wiki/Beginners1URIs#1.1__What_is_a_resource?>
Thanks for this link, which is an interesting comparison to my diagram on page 5 - the RDF terms "information" and "non-information" resources correspond closely to my "digital" and "physical". But their "non-information" is broader than my "physical" because it encompasses people and concepts (which I term "entities") as well as "relationships" which does not appear on my diagram.

I am not sure that this is necessarily a question of right and wrong, but rather helpful and unhelpful in our context. In fact, it rather illustrates my point about the meaning of terms ("resource") being dependent on their ontological context. In the case of RDF, the definition of a resource as "anything that can be defined and which can be assigned an identifier" might well be useful - but in the context of ed-tech, I would argue that it isn't. The generally accepted meaning of the term is defined as "A stock or supply of money, materials, staff, and other assets that can be drawn on by a person or organization in order to function effectively" - although I suggest that teachers are not so comfortable with the "human resource". I doubt thatmost people are comfortable with using the term to refer to concepts and relationships. I think that this matters because I think we need to create a terminology with which our target audience is comfortable.

A second reason why I don't think that the RDF definition is helpful for our purposes is that a word that refers to everything is of little use. We want to be able to make distinctions. It seems to me to be useful to distinguish between things, actors/agents, metadata, and relationships. Maybe you still need a high level term to refer to "stuff" in general - I propose "[ecosystem] content", on the assumption that this is not a term that will be much used.

I would also acknowledge that my diagram contains a potential flaw in that it is useful to have a term that encompasses software services and their users (actors/agents?), while in my diagram, software services are described as a sort of resource, which is distinct from "actor".

These are just my thoughts. I think the discussion of these rather high level conceptual maps are important and perhaps we need to dedicate a weekly call to the subject.
> * The regular use of "LMS" might be confusing to some here. Is our work predicated on the existence of an ecosystem or even an LMS? Although it states, "(LMS—or other appropriate software service)" I fear that those reading this proposal are going to instinctually ignore this unless they have an LMS. Is this XDMDL proposal specifically targeting LMS applications? It seems like could be perceived that way although I "think" that it isn't the intent. Aren't you targeting more than just LMS and SCORM-like use cases for this concept of a data model declaration language? Is that the use case we are starting with?
Not the intent - and very happy to change. The ecosystem diagram really shows a particular use case, which assumes some sort of gateway service that controls launch, so a different diagram might be appropriate for the case in which launch is initiated independently, with outcome data being reported to an LRS, for example. The intention is that one "item" type in the XDMDL library would be a "topology", which would define the different software services and relationships between them - so the whole modelling language becomes agnostic with respect to what is talking to what. What I call a "profile" item then binds together a data model, topology service roles, bindings, transports etc. Maybe "profile" is insufficient. The intention is that any ecosystem can be modelled/defined.
> * In the proposal, it states "If the software (or its human user) decides that the extent of interoperability is sufficient, the LMS can launch the activity and interact with the AP, providing the contextual information that is required to make sense of the activity and harvesting outcome and monitoring data." What does this mean exactly? So the LMS/software or the end user can decide if interoperability is sufficient? It seems too subject or abstract for my understanding. What if they determine the interoperability is not sufficient? What are the criteria for doing so?
I see this as an important question. What I mean is that services should be able to discover each other on the fly. They might not recognise the profile, but they might recognise all the components, including all the mandatory data declarations. There might be a middle usage category between "mandatory" and "optional" which declares different levels of conformance (as the old SCORM test harness provided with their RTE1, 2, 3 definitions). This level of compatitibility could then be declared to the user at registration time, not at launch time, when in formal education it is normally too late to discover that software does not integrate as you had hoped.

This might be over-ambitious and my desire to enable a more flexible sort of interoperability may need to find a middle way, making it easier, at least as a first step, for groups of companies to declare the semantics of the interoperability that they have achieved in consistent and easily adopted ways - but not necessarily on the fly. Perhaps I need to sketch out some thoughts/use cases in response to your challenge and we need to think this through with different use cases. It might be something on which we could have a pre-discussion this Friday, and a more considered discussion in a couple of weeks time, after you have had time to introduce the OWL concept mapping.

> Feedback on Blogpost (<http://wiki.saltis.org/display/XDMDL/2015/03/29/The+meaning+of+terms>)
> 
> * Totally agree with you on the perspective of the verb having more meaning through context. However, with context (which can vary significantly as you pointed out in your post) there might be some inference that can be made by humans. But for machine-readability the approach to using linked data and referencing existing RDF representations such as wordnet provides us with powerful semantic machine-readability without the need to define the additional learning activity context. This machine URI dereferencing of linked data is what is adopted by the W3C and the semantic community at large for a vocabulary data set. So it isn't something we just came up with. I believe we should try to adopt this approach for xAPI verb data sets. Others believe that we eventually need to adopt a more formal ontology approach for the entire xAPI, but that is a much larger effort. Our current situation is that we aren't following a true linked data approach so there is no automatic semantic meaning and readability made by machines. This impacts analytics and reporting interoperability of LRS applications. But even before that, it impacts discoverability, which in turn impacts the duplication of terms.
I do not doubt that you did not make it up. I just have some niggling suspicions about the whole approach to ontologies, including whether it has really proved itself useful in this sort of situation. My niggle is the assumption that I have found, eg in the work on InLOC work and Claude Ostyn's work on competencies, that terms can be given absolute meanings independent of the ontology in which they are embedded.

People are sloppy. So they think that "finished" in one ontology means the same as "finished" in another ontology - so the term can be "reused". But the fact that it is the same word, or even has the same definition, does not mean that it will be used the same way - even by cold logical machines. And we don't want to lock everyone down by presenting them with one, single ontology - we want to allow people to experiment and innovate by creating different systems to do different things. I think a recurring problem in our world is that we see things through different perspectives yet use similar words, and so end up misunderstanding each other. That is why I suspect the dictionary definition is often a false friend.

The other sense in which humans are sloppy is that they use bad grammar. So I was making the point that the meaning of the verb is often dependent on the object. But the natural objects of many of the xAPI verbs are very different - what you master, what you void, what you answer are all different sorts of thing. So one needs to know how those different things relate to each other. But people, not being precise in the way that they express those relationships, are likely to confuse activities with learning objectives - I think this is an error that the current xAPI documentation falls into sometimes, suggesting that you might "master" an "activity" when the proper object for the verb "master" is an objective. My solution to this problem is to aim for rule-based systems in which the verb is always associated with a particular object. i.e. if you have finished something, it is always the attempt; if you have mastered something, it is always a learning objective; if you have answered something, it is always an interaction, if you have completed something, it is always the task - and all of these objects (objective, interaction, attempt and task, all have well defined relationships). This reduces the scope for misunderstanding that exists with a machine-readable syntax which tries (but probably fails) to mimic the flexibility of natural language.
> * Of the examples of completed, the one we have associated with for the ADL vocabulary is the first one actually. <http://xapi.adlnet.mobi/verbs/#completed>

I am not very keen on your definition of "complete" for the reasons I give above. It allows you to use the verb in relation to may different types of object. You can master a learning objective (e.g. "pass") without completing the task: if the pass mark is 6/10, it can be achieved by answering only 6 questions. You can complete the task without completing the experience (see my previous example of a maze which does require you to walk every piece of it to complete it). If people can "complete" a learning objective, can you be sure that they will reference the objective in the [object] element? Or will they reference the activity? And how do you deal with the fact that when they report the completion of a learning objective, they will require completely different [result] and [context] records (e.g. to handle evidence) to when they complete a task? I think that all this boils down a a design problem that follows from trying to pack too much meaning into the verb. Better, in my view, to express all this inter-related and inter-dependent information in a single [result] record and fall back on a simple vocabulary of verbs: [actor] "did" [object] [with result: <put everything here in a structured form>]

> * We feel this one is more closely related to the definition of completed based on SCORM. In fact, the ADL vocabulary primarily consists of SCORM-like verbs. <http://xapi.adlnet.mobi/verbs/> If the XDMDL work is going to be heavily focused on SCORM as it seems,

See comment above. This is not the intention. I need to work up more use-cases to show that XDMDL is agnostic in relation to the topology/use case. Though it would be good to talk to Jono and Tom about their SCORM work once I have made that clear. But I do think that different world views (of which SCORM is one) need to be encoded in different profiles that are intenally consistent. Alternative models might focus on multi-player activities, the return of creative product, or (pace xAPI) student-initiated learning, for example.
> * then perhaps we should ask Jono and Tom to come to our next meeting and provide a deep dive on the SCORM profile of xAPI. I'm thinking that some of your ideas in the proposal for SCORM-like concepts (SCORM Score, Interactions) might need to be explained to them for consideration in the current SCORM profile effort as well. Or perhaps it would be good to see how they are currently handling it.
> * 
> * The main reason for suggesting some form of vocabulary control in the xAPI specification is to achieve consistency in the description, storage, and retrieval of xAPI data. However, as you have pointed out here are two fundamental challenges with natural language: !) Two or more terms can be used to represent a single concept and 2) Two or more words that have the same spelling can represent different concepts. Depending upon the use case, many different forms of "completed" could be used. But which one you use depends upon how it is represented via linked data. In this way, linked data is also metadata.
> * I can't help but think that what you are proposing (a high level data model declaration language/schema) is needed, but it seems like this might be the same thing as or require the similar type of work as defining a core ontology for xAPI.
I would be great to explore this. I suggest that the best way is to create models and test them. Last time, Frank and Avron suggested that, if my proposal was to be used for modelling purposes, I really needed to create my tool. I am setting about this - though it will take a little time. You may be able to model different approaches using tools that are already available (I am talking about the DMD piece, not the conceptual mapping). It would be great if you wanted to make the running for a couple of week while I tried to make some progress with the tool. Otherwise, we will necessarily need to continue with a more abstract discussion.
> Will provide more feedback as time permits. Overall, I think the doc is great, but I think eventually it might help if we combine it with the details you provided in your blog post for better understanding. I found myself flipping back and forth between the doc and the blog post. Talk to you soon!
Absolutely. Another take-away from the call was the need for an accessible prospectus/marketing document. Perhaps this is something that could be done while waiting for tools.

Many thanks again. Look forward to the continuing discussions - and see you on Friday week I hope.

Best wishes, Crispin.
> 
> 
> 
> Kind Regards,
> 
> 
> J Haag
> -------------------------------------------------------
> Advanced Distributed Learning Initiative
> +1.850.266.7100(office)
> +1.850.471.1300 (mobile)
> jhaag75 (skype)
> <http://motifproject.org> (MoTIF Project)
> <http://ml.adlnet.gov> (Web)
> <http://twitter.com/mobilejson> (Twitter)
> <http://linkedin.com/in/jasonhaag> (LinkedIn)
>