Re: EXI for JSON structure

Thanks for continued scrutiny of these possibilities.

There was some excellent dialog captured in meeting minutes from last week.

	https://www.w3.org/2016/02/02-exi-minutes.html

Incidentally it would be helpful for the entire list if minutes might be posted after each meeting, and perhaps a link provided in agendas.

Given that prior dialog, getting back on topic:

Regarding the current JSON encoding, it has a major virtue of being able to handle any JSON document correctly.

In some sense, if you solely use JSON itself, there is no further meta-information to take advantage of when trying to make an EXI JSON encoding as efficient as possible.

This is similar to using EXI with XML documents that do not have an XML schema.

So a key insight is that use of a JSON schema is necessary, and might allow a more "tuned" application of EXI algorithms.

That would then lead us to consider a pair of approaches to JSON compression: document based (no schema) or informed by JSON schema.

A potential path forward thus presents itself: comparison of expressive power between XML Schema and JSON Schema, anticipating possibly improved EXI encodings for JSON.

This approach might also raise suggested commonality that best supports the design of XML and JSON data structures with equivalent information models.

If we get that far, we will have gone a long way towards understanding whether bidirectional mappings between XML, EXI and JSON are feasible.

It will also illustrate the differences and tradeoffs between a strict object model (e.g.) and a less-strict document model (e.g. XML PSVI).

Onward we go...   8)


On 2/8/2016 6:34 PM, Takuki Kamiya wrote:
> Hi Daniel,
> 
> I agree that we ultimately should define JSON schema to EXI grammar
> mappings.
> 
> If JSON schema to EXI grammar mapping is defined, I think one important
> aspect to consider is how to make it feasible to reuse as much existing implementation
> infrastructure as possible.
> 
> In defining the mapping, it would be beneficial to existing implementations
> if the mapping can be implemented by simply converting JSON schema to
> similarly structured XML schema (or to in-memory schema model). However,
> not all implementations do not need to follow this path.
> 
> Say we have the following JSON structure where both name and title are
> optional.
> 
> {
> 　“name”: “Taro”,
> 　“title”: “poet”
> }
> 
> The corresponding pseudo-XML schema (XML schema-like with some
> violation of XML schema rules) is:
> 
> <xs:element name="map">
>    <xs:complexType>
>      <xs:element name="string" minOccurs="0">
>        <xs:complexType>
>          <xs:simpleContent>
>            <xs:extension base="xs:string">
>              <xs:attribute name="key">
>                <xs:simpleType>
>                  <xs:restriction base="xs:string">
>                    <xs:enumeration value="name"/>
>                  </xs:restriction>
>                </xs:simpleType>
>              </xs:attribute>
>            </xs:extension>
>          </xs:simpleContent>
>        </xs:complexType>
>      </xs:element>
>      <xs:element name="string" minOccurs="0">
>        <xs:complexType>
>          <xs:simpleContent>
>            <xs:extension base="xs:string">
>              <xs:attribute name="key">
>                <xs:simpleType>
>                  <xs:restriction base="xs:string">
>                    <xs:enumeration value="title"/>
>                  </xs:restriction>
>                </xs:simpleType>
>              </xs:attribute>
>            </xs:extension>
>          </xs:simpleContent>
>        </xs:complexType>
>      </xs:element>
>    </xs:complexType>
> </xs:element>
> 
> Even if a processor successfully read this pseudo schema, the two
> AT("string") have different types, so the proto-grammar cannot be
> processed by the rules defined in EXI 1.0.
> 
> Thank you,
> 
> Takuki Kamiya
> Fujitsu Laboratories of America
> 
> 
> 
> -----Original Message-----
> From: Peintner, Daniel (ext) [mailto:daniel.peintner.ext@siemens.com]
> Sent: Thursday, February 04, 2016 4:42 AM
> To: Takuki Kamiya; public-exi@w3.org
> Subject: AW: EXI for JSON structure
> 
> Hi Taki,
> 
> thank you for sharing your experiments with us.
> 
> EXI for JSON uses a very generic XML schema that allows representing any JSON document. That said I do see your usecase of applying a more detailed schema though!
> 
> The JSON snippet you referred to was
> 
>           "link" : {
>               "manager" : "Boss"
>               "subordinates" : "worker"
>            }
> 
> by saying the "link" property in the above is a map that consists of either or both of "manager" and "subordinates".
> 
> The JSON snippet would be converted to XML as follows:
> 
>      <j:map key="link">
>          <j:string key="manager">Boss</j:string>
>          <j:string key="subordinates">worker</j:string>
>      </j:map>
> 
> However, building an XML schema that deals with the properties you shared does not seem to work.
> The approach you proposed provides some benefit but does not handle all issues:
> 
> * It is not schema-valid and causes "cos-nonambig" issues.
>    (We do not know how XML schema processors deal with it)
> 
> * It assumes that keys are valid tag-names.
>    e.g., a key "fo ba" with spaces might cause issues as attribute name,
>    at least when serialized to XML
> 
> * It still does not express properties of JSON such as a map is unordered, meaning
>    that manager and subordinates can appear in any order
>    (I think this is much more complicated if you deal with more possible entries)
> 
> * It tries to solve an issue in XML schema that might be solved in JSON schema.
>    The use-case is JSON and at best people might have an JSON schema document which
>    should be used to generate EXI grammars
> 
> I think in a schema-informed case what we really want to see is something like this (expressed in EXI grammars):
> 
> link0:
>    SE("manager") link1
>    SE("subordinates") link2
>    EE
> 
> link1:
>    SE("subordinates") link2
>    EE
> 
> link2:
>    EE
> 
> manager0:
>    CH manager1
> 
> manager1:
>    EE
> 
> subordinates0:
>    CH subordinates1
> 
> subordinates1:
>    EE
> 
> This is very different from the generic approach.
> That said, I do have a hard time to see how we can achieve what we are looking for.
> 
> In my experiments I failed to create an XML instance that could be used with a generic schema and which is also feasible for a more appropriate schema (which is not in conflict with XML schema rules such as "cos-nonambig" or "cos-element-consistent") :-(
> 
> I am glad if we can find a solution though!
> 
> Thanks,
> 
> -- Daniel
> 
> ________________________________
> Von: Takuki Kamiya [tkamiya@us.fujitsu.com]
> Gesendet: Dienstag, 2. Februar 2016 02:17
> An: public-exi@w3.org
> Betreff: EXI for JSON structure
> 
> Hi,
> 
> I have looked at an example to exercise how EXI for JSON [1] works with schemas.
> 
> Below is an example snippet of a JSON consisting of one record "person".
> 
>        {
>          "person" : {
>            "id" : "Boss",
>            "name" : {
>              "family" : "Smith",
>              "given" : "Bill"
>            },
>            "email" : "smith@foo.com",
>            "YearsOfService" : 20,
>            "weight" : 175.4,
>            "birthday" : 1955-03-24,
>            "link" : {
>              "manager" : "Boss"
>              "subordinates" : "worker"
>            }
>          }
>        }
> 
> "link" property in the above is a map that consists of either or both of "manager"
> and "subordinates".
> 
> Currently, EXI for JSON carries names as the value of "key" attribute.
> This makes it difficult to generate EXI grammar, because in both "manager"
> and "subordinates" cases, the name is contained in the value of "key"
> attribute. EXI spec expects the distinction be explicitly made in the terminal
> symbols. (See section 8.5.4.2.2 [2] in EXI spec.)
> 
> I therefore would like to suggest the following structure in XML.
> 
>          <map link="">
>            <string manager="">Boss</string>
>            <string subordinates="">worker</string>
>          </map>
> 
> A structure like the one above would permit the rule in EXI spec
> section 8.5.4.2.2 to work.
> 
> The corresponding schema would be as follows:
> 
> <xs:element name="map"><!-- link -->
>    <xs:complexType>
>      <xs:sequence>
>        <xs:element name="string" minOccurs="0"><!-- subordinates -->
>          <xs:complexType>
>            <xs:simpleContent>
>              <xs:extension base="xs:string">
>                <xs:attribute name="subordinates" use="required">
>                  <xs:simpleType>
>                    <xs:restriction base="xs:string">
>                      <xs:enumeration value=""/>
>                    </xs:restriction>
>                  </xs:simpleType>
>                </xs:attribute>
>              </xs:extension>
>            </xs:simpleContent>
>          </xs:complexType>
>        </xs:element>
>        <xs:element name="string" minOccurs="0"><!-- manager -->
>          <xs:complexType>
>            <xs:simpleContent>
>              <xs:extension base="xs:string">
>                <xs:attribute name="manager" use="required">
>                  <xs:simpleType>
>                    <xs:restriction base="xs:string">
>                      <xs:enumeration value=""/>
>                    </xs:restriction>
>                  </xs:simpleType>
>                </xs:attribute>
>              </xs:extension>
>            </xs:simpleContent>
>          </xs:complexType>
>        </xs:element>
>      </xs:sequence>
>      <xs:attribute name="link" use="required">
>        <xs:simpleType>
>          <xs:restriction base="xs:string">
>            <xs:enumeration value=""/>
>          </xs:restriction>
>        </xs:simpleType>
>      </xs:attribute>
>    </xs:complexType>
> </xs:element>
> 
> 
> Thank you,
> 
> [1] https://www.w3.org/TR/2016/WD-exi-for-json-20160128/<&smime=14.3.123.2https://www.w3.org/TR/2016/WD-exi-for-json-20160128/>
> [2] https://www.w3.org/TR/2014/REC-exi-20140211/#eliminatingSymbols<&smime=14.3.123.2https://www.w3.org/TR/2014/REC-exi-20140211/#eliminatingSymbols>
> 
> Takuki Kamiya
> Fujitsu Laboratories of America
> 
> 
> 


all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman@nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman

Received on Tuesday, 9 February 2016 15:56:42 UTC