Re: ISSUE-648: Can schema be made a bit more jaxb friendly?

Hi Stephan,

I tried the modified xsd with <xs:complexType name="Others"> through JAXB and got the same Document.java as you. I noticed that both Document.java and BundleConstructor.java still exhibit the JAXBElement artifacts that are not as user-friendly to use.

public class Document {
    @XmlElementRefs({
        @XmlElementRef(name = "wasInformedBy", namespace = "http://www.w3.org/ns/prov#", type = JAXBElement.class, required = false),
        @XmlElementRef(name = "wasAttributedTo", namespace = "http://www.w3.org/ns/prov#", type = JAXBElement.class, required = false),
        @XmlElementRef(name = "hadMember", namespace = "http://www.w3.org/ns/prov#", type = JAXBElement.class, required = false),
        @XmlElementRef(name = "wasInvalidatedBy", namespace = "http://www.w3.org/ns/prov#", type = JAXBElement.class, required = false),
        …

This then may rule out proposed approach (b) of using new element:
  <prov:others>
          <!-- here, xsd:any non-prov element -->
  </prov:others>

---

I also tried another approach where we keep the schema as is, but leverage customizations in JAXB bindings mapping file to tweak the behavior to avoid JAXBElements. I'm attaching two variants of customized JAXB mapping file (bindings1.xjb and bindings2.xjb) to set generateElementProperty="false", which according to [1] removes the generated JAXBElement<T>. However, I have not successfully gotten this customization to fully remove the JAXBElement<T>. Even if it did, it should be noted that the presence of JAXBElement<T> is required by JAXB to enable full round-trip marshaling/unmarhsalling of elements under certain conditions. Removing JAXBElement<T> makes the round-tripping "lossy".

So it begs the question of wanting prov-xml schemas such that JAXB does not need to generate JAXBElement<T> in the first place without forcing generateElementProperty="false". The approach we did of removing the modularity of <xs:group name="documentElements"> did achieve more user-friendly JAXBElement-less bindings, however at the cost of forcing prov element ordering.. I am also attaching the modified version of prov-core.xsd that has xs:group removed, requires prov element ordering, but no longer naturally needs JAXBElements.  <xs:element name="internalElement" abstract="true" /> is kept in this schema.

Note that this approach results in the lost of "scruffiness" in prov-xml—one of the noted desirements.

---

This may leave us with the remaining viable option of keeping the schema as is, moving on with the review, and providing a JAXB-friendly variant schema in the FAQ.

I agree that since the jaxb-friendly version of prov.xsd includes all dependencies {prov-core.xsd + prov-dictionary.xsd + prov-links.xsd}, it makes more sense to make a merged prov.xsd the only one required for jaxb-friendly use.

--Hook

[1] http://docs.oracle.com/cd/E19879-01/820-1072/ahiid/index.html


From: Stephan Zednik <zednis2@rpi.edu<mailto:zednis2@rpi.edu>>
Date: Wed, 3 Apr 2013 15:44:54 -0700
To: "Hua, Hook (398C)" <hook.hua@jpl.nasa.gov<mailto:hook.hua@jpl.nasa.gov>>
Cc: "public-prov-wg@w3.org<mailto:public-prov-wg@w3.org>" <public-prov-wg@w3.org<mailto:public-prov-wg@w3.org>>
Subject: Re: ISSUE-648: Can schema be made a bit more jaxb friendly?


On Apr 3, 2013, at 11:50 AM, "Hua, Hook (398C)" <hook.hua@jpl.nasa.gov<mailto:hook.hua@jpl.nasa.gov>> wrote:

Hi Stephan,

We use substitutionGroup="prov:internalElement" in both prov-dictionary and prov-links. So this approach would imply removing them there as well?

The FAQ entry would propose a solution whereby the abstractElement + substitutionGroup pattern was no longer used.  Perhaps by providing a processing script that can be used to generate new schemas which combines definitions from prov-core and extensions, perhaps by simply linking to simpler schema we manually generate.

Effectively, if we want nice schema for jaxb we may have to generate new schemas such as

jaxb-friendly: prov.xsd = {prov-core.xsd + prov-dictionary.xsd + prov-links.xsd}
jaxb-friendly: prov-dictionary.xsd = {prov-core.xsd + prov-dictionary.xsd}
jaxb-friendly: prov-links.xsd = { prov-core.xsd + prov-linnks.xsd}

I am not sure how useful the extension-specific jaxb-friendly schemas are since the jaxb-friendly prov.xsd will be core + both extensions.

We may also find a jaxb bindings solution that allows us to keep the abstract element... but after a fair bit of searching I have not found an obvious solution following this path.

We could also decide to make the jaxb-friendly prov.xsd schema the 'official' schema described by the note, but we would have to decide on that TODAY.  This is option 1) from the current discussion.

Could you send either the updated xsds or generated JAXB *.java bindings?

attached.  I took out the groups because each group was only being referenced once so they didn't really provide much simplification or organization.  I am amendable to adding them back.



Thanks,
Hook


From: Stephan Zednik <zednis2@rpi.edu<mailto:zednis2@rpi.edu>>
Date: Wed, 3 Apr 2013 11:25:52 -0600
To: "Hua, Hook (398C)" <hook.hua@jpl.nasa.gov<mailto:hook.hua@jpl.nasa.gov>>
Cc: "public-prov-wg@w3.org<mailto:public-prov-wg@w3.org>" <public-prov-wg@w3.org<mailto:public-prov-wg@w3.org>>
Subject: Re: ISSUE-648: Can schema be made a bit more jaxb friendly?


On Apr 3, 2013, at 11:16 AM, "Hua, Hook (398C)" <hook.hua@jpl.nasa.gov<mailto:hook.hua@jpl.nasa.gov>> wrote:

Hi Stephan,

Proposed solution (b) of <xs:complexType name="Others"> is an interesting one. Though it validates, do you still see JAXBElements in the generated JAXB bindings?

Did you try it yourself?

With prov:internalElement still in the document/bundle sequences you still get the JAXBElements in the generated JAXB bindings.  If you remove prov:internalElement, as we would suggest in the FAQ-entry, then you no longer get JAXBElements in the generated bindings.

--Stephan



--Hook


From: Stephan Zednik <zednis2@rpi.edu<mailto:zednis2@rpi.edu>>
Date: Tue, 2 Apr 2013 20:07:04 -0600
To: Luc Moreau <L.Moreau@ecs.soton.ac.uk<mailto:L.Moreau@ecs.soton.ac.uk>>
Cc: <public-prov-wg@w3.org<mailto:public-prov-wg@w3.org>>
Subject: Re: ISSUE-648: Can schema be made a bit more jaxb friendly?
Resent-From: <public-prov-wg@w3.org<mailto:public-prov-wg@w3.org>>
Resent-Date: Wed, 3 Apr 2013 02:07:34 +0000


On Apr 2, 2013, at 2:25 PM, Luc Moreau <L.Moreau@ecs.soton.ac.uk<mailto:L.Moreau@ecs.soton.ac.uk>> wrote:

Hi Stephan,
Thanks for still looking into a solution for this issue, and summarizing the options.

May I suggest option 1.5.

a) keep use of abstract element and substitution,
    but in FAQ, add an xml processing step, that inserts all included files in a single one,
    and replaces the abstract element with the concrete ones..

    With this, you keep the modularity, but offer a JAXB way of handling it..

b) introduce a mechanism to support any non prov element.
    I am not sure what the right option is.

I think this option is not as radical as your option 1.


An option for xsd:any not discussed yet I think  is to introduce a new element:
    <prov:others>
          here, xsd:any non-prov element
    </prov:others>

Thoughts?

I made this update locally and all our example xml files still validate so we weren't using any non-prov elements at the document or bundle level (just as attributes) in our examples.  I will do a quick check to ensure that all examples in the Note are still valid but they were based on the xml examples in the repository so I expect that to be the case.

I propose we hold a straw poll  by email regarding adopting Luc's suggestions above so I can implement tonight and get the note staged for review (may go to stage early tomorrow morning).

PROPOSED:

a) keep use of abstract element and substitution,
    but in FAQ, add an xml processing step, that inserts all included files in a single one,
    and replaces the abstract element with the concrete ones.

b) introduce a new element:

<prov:others>
          <!-- here, xsd:any non-prov element -->
  </prov:others>

to be implemented as:

<xs:element name="others" type="prov:Others"/>

  <xs:complexType name="Others">
<xs:sequence>
<xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
  </xs:complexType>

and the prov:others element to be referenced in the document and bundle sequences.

--Stephan

Luc

On 02/04/13 20:56, Stephan Zednik wrote:
Hi Luc,

I've looked into ways to update the schema while retaining our current encapsulation strategy of having extension schemas uses substitution group elements on the prov:internalElement element and I have yet to figure out a way to do it.  I also have yet to figure out if we can provide a solution to improve JAXB binding without forcing all non-prov elements to be after the prov elements in document and bundle containers.

We are getting to the end of our timeframe for addressing this so we should make a decision ASAP.

Possible solutions:

1) implement the changes mentioned by Luc, which would be:

a) remove use of abstract element and substitution groups in prov-core and extension schemas
b) introduce ordering that requires xsd:any after prov elements in document and element containers

these changes would force us to also do the following:

c) remove include statements from prov.xsd and extension schemas, all schemas contain local declarations of prov core types
d) update the Note document to reflect changes

PROS:
- Removes the JAXBElement.class type from the @XmlElementRef annotations in generated classes

CONS:
- Removes our strategy for modularity and encapsulation in the schemas
- Restricts non-prov elements to only be valid after prov-elements in bundle and document containers

2) leave the schema and Note as they are; look into developing a separate JAXB-friendly schema which can be introduced in a FAQ entry.  Write FAQ entry about our experiences with JAXB binding irregardless of if a separate JAXB-friendly schema can be successfully developed.

PROS:
- Increases our window to get the work done, we may not have determined the best solution yet
- Does not require a change to the existing Note or schema
- Allows us to introduce jaxb annotations in the FAQ schema that may help with the schema binding
- Preserves exiting modularity and encapsulation in the schemas

CONS:
- The JAX-friendly schema will likely need to be a more restrictive schema than the official schema (such as perhaps requiring xs:any elements outside of choices at the end of sequences), this may lead to problems in trying to unmarshall valid PROV-XML that does not conform to the JAXB-friendly schema.

3) Do nothing

I prefer 2) or 1) over not addressing this request.

--Stephan

Hi

I have ported the ProvToolbox and the ProvValidator to the new XML schema.
I just wanted to report on my experience with the schema and JAXB.
Obviously, others may have better experience with JAXB and may be able
to help on some of the issues I encountered.

Everything worked fine, except:
- <xs:element ref="prov:internalElement abstract=true/>
- extensibility <xs:any namespace="##other"/> in Document and Bundle


These two constructs, while processable by JAXB, are not JAXB-friendly.

Indeed, JAXB compiles the schema in a list containing all possible statements.

protected List<Object> entityAndActivityAndWasGeneratedBy;

However, the presence on an abstract element and an <any/> element result in the
content of that list to be of type:


@XmlElementRefs({
@XmlElementRef(name = "used", namespace = "http://www.w3.org/ns/prov#", type = JAXBElement.class),
@XmlElementRef(name = "wasAssociatedWith", namespace = "http://www.w3.org/ns/prov#", type = JAXBElement.class),
@XmlElementRef(name = "person", namespace = "http://www.w3.org/ns/prov#", type = JAXBElement.class),
@XmlElementRef(name = "entity", namespace = "http://www.w3.org/ns/prov#", type = JAXBElement.class),
@XmlElementRef(name = "wasInfluencedBy", namespace = "http://www.w3.org/ns/prov#"
.....
})

@XmlAnyElement(lax = true)
protected List<Object> entityAndActivityAndWasGeneratedBy;

where all data structures are wrapped up in this unpleasant JAXBElement.

Without these features, we get a much more natural mapping:
@XmlElements({
@XmlElement(name = "entity", namespace = "http://www.w3.org/ns/prov#", type = Entity.class),
@XmlElement(name = "activity", namespace = "http://www.w3.org/ns/prov#", type = Activity.class),
@XmlElement(name = "wasGeneratedBy", namespace = "http://www.w3.org/ns/prov#", type = WasGeneratedBy.class),
@XmlElement(name = "used", namespace = "http://www.w3.org/ns/prov#", type = Used.class),
@XmlElement(name = "wasInformedBy", namespace = "http://www.w3.org/ns/prov#", type = WasInformedBy.class),
....
})

So, how I did I solve the problem? I inserted the extension schemas into the schema file, and hence got rid of the abstract element. I am ok with this.. We could possible provide the utility to that transformation.

For the extensibility, I used a different definition. It happens to
parse prov-xml compliant xml. When serializing, it puts all
extensibility elements at the end. This is not a satisfactory
solution, and is likely to be dependent of the jaxb implementation (though I am not entirely sure).


<xs:complexType name="Document">
<xs:sequence>
<xs:choice maxOccurs="unbounded">
<xs:group ref="prov:documentElements"/>
<xs:element name="bundleContent" type="prov:NamedBundle"/>
</xs:choice>
<xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
</xs:complexType>

Can something be done to make the XML schema a bit more jaxb friendly,
while still keeping the same flexibility? Thoughts welcome.

Cheers,
Luc


--
Professor Luc Moreau
Electronics and Computer Science   tel:   +44 23 8059 4487
University of Southampton          fax:   +44 23 8059 2865
Southampton SO17 1BJ               email: l.moreau@ecs.soton.ac.uk<mailto:l..moreau@ecs.soton.ac.uk>
United Kingdom                     http://www.ecs.soton.ac.uk/~lavm