Forms F2F Cambridge Day 3 Morning March 25, 2010

* Present

Charlie Wiecha, IBM
Erik Bruchez, Orbeon
John Boyer, IBM
Nick van den Bleeken, Inventive Designers
Steven Pemberton, CWI/W3C
Leigh Klotz, Xerox (minutes)
Uli Lissé, DreamLab

* Agenda


Charlie Wiecha: Demos
Charlie Wiecha: Here is a mortgage demo of XForms: required with red asterisk, constraints violations showing up as messages, and dependency graphs. I'm also showing Dojo widgets with SNBs. The principal field shows with a Dojo NumberTextBox widget. MIP events aren't yet implemented, but value-change is using the existing Dojo notification.
Steven Pemberton: So the advantage is that it formats the numbers?
Charlie Wiecha: It's coded in the widget for Dojo.
John Boyer: There's a lot of Dojo machinery for that sort of thing.

Charlie Wiecha: Back to JSON. JSON in the wild can be pretty wild. In Ubiquity I've implemented one policy for using JSON, without extensibility. JSON gives you structures and arrays. Atomic values are numbers, strings, and booleans. Composites are arrays and structs.

<script  id="loan_init" type="application/json">
<instance id="loan" src="#loan_init" />

Nick van: How do you go back?
Charlie Wiecha: This example doesn't; it gets JSON in, you can update it, and you get it back. You lose document order, mixed content. So is it sufficiently interesting? JSON is a data-centric format. Even so there are variants not handled by this approach, as in the example from XSLTForms and Wikipedia we saw yesterday, which was anonymous.
Nick van: How about the transforming of currency elements? In JSON it's clear that the first is USD, Canadian, Euro. If you have a second, with British Pounds they would be merged in your XML as siblings and you don't know.
John Boyer: It would be good to look at the corresponding XML. The currency element is not in the XML. All you have is the currency element below principal.
Charlie Wiecha: This isn't the same data as the canonical loan form.
John Boyer: How do you indicate a node is an enumeration or an attribute.
Charlie Wiecha: There's no attribute support whatsoever. It's element only.
John Boyer: What's the philosophical model? From beginning to end this is pure JSON data and won't ever be XML?
Steven Pemberton: It is XML because it's in an XPath model but do you mean it won't be serialized?
Charlie Wiecha: I mean, is there a set of data-centric (not document-centric) services that don't care about order and don't have mixed content.
Steven Pemberton: Even if we did want to start from XML and produce JSON we can say that you do it this way.
Charlie Wiecha: We don't have enough machinery to do that; we need places for transformers that preserves validity checking. You get the serialize event but not the whole submission pipeline.
Nick van: Is it possible to paste that in the IRC?
Charlie Wiecha: Can you run the JSON through the PicoForms converter?
Steven Pemberton: It's easier for us to read JSON than for JSON to read XML.
Charlie Wiecha: It doesn't imply that we can have a fixed policy.
Steven Pemberton: There seem to be two principal uses: non-anonymous thing with name, and anonymous with raw structure.
Charlie Wiecha: The indexing is 1- vs 0-origin but it's similar. I've seen JSON examples that aren't possible to parse: pure numeric property names, or non-XML characters. We can't control the JSON in the wild and we can't control it. But in general we need to plug in transformer code. If you have to write an application-specific transform and have to understand the XPath, what's the usability?
Charlie Wiecha: I believe we can design the places to plug in, but
Erik Bruchez: I think it's an impedance mismatch with XPath.
Charlie Wiecha: You said yesterday that every data form comes with a query language.
Erik Bruchez: ...
John Boyer: So take a step back for the reason we're doing this.
Steven Pemberton: Because the world is full of JSON data. Kenneth did it for customers.
Erik Bruchez: An implementation experiment is OK. But something for the standard must look reasonable enough.
Charlie Wiecha: The standardization question is the extension mechanism for arbitrary transforms.
John Boyer: We're doing this to connect to JSON services that are formats we have no control over, right? As opposed to convenient use of newly-designed JSON services.
Steven Pemberton: Right.
Charlie Wiecha: It would be a simpler goal and possibly a JSON-to-JSON layer to map.
John Boyer: JSLT?
Charlie Wiecha: JSON-Path
Nick van: Picoforms adjusted their XPath engine so that it recognizes the dots for paths but they don't think it should be standardized.
Charlie Wiecha: So JSONPath or JQUery, but build dependency graph. I don't know if there's enough function machinery in those libraries to do the kind of constraints and MIP expressions. It's not intended.
Leigh Klotz: It's not an issue about the JSON syntax on the XPath data model, but JSON syntax used on a JSON data model and it's not clear how you would do rebuild and match structural changes.
Nick van: I pasted in the output of PICOForms converter.
Charlie Wiecha: It's the same as I have.
Erik Bruchez: What about un-named items?
Nick van: It's called JSON in theirs.
Erik Bruchez: That's ambiguous.
Leigh Klotz: We may have to get over those concerns.

John Boyer: What about the currency element?
Charlie Wiecha: Just currency/currency.
John Boyer: So iterating across an array is the same as duplicate tag values?
Charlie Wiecha: Yes, the same as Picoforms.
Leigh Klotz: So the PicoForms parser doesn't deal with backslash or backslash-u.
Charlie Wiecha: No, actually, you can't have multiple ones of the same name. It's a field name in a structure.
Steven Pemberton: You'd spot it and generate an array.
Leigh Klotz: It has to be homogeneous? currency,currency,pizza,currency?
Charlie Wiecha: It would have two structure entries.
Leigh Klotz: So you un-interleave them.

Charlie Wiecha: So the application-specific transforms seems to be the problem.
Steven Pemberton: Don't we cover most of the cases?
Charlie Wiecha: Maybe

Leigh Klotz: This is fitting the XPath model to JSON data. It's not clear that the game is over for that.

Nick van: Two anonymous structures in a row of the same structure crashes the PicoForms converter.

Steven Pemberton: You can do it like this using types.
Nick van: It's a lot of predicates in XPath.
Erik Bruchez: That could be the canonical representation you could then transform.
Nick van: We can have multiple profile transforms with qnames: one that supports everything, and others for dialects.
Erik Bruchez: Are we defining those?
Charlie Wiecha: The library Leigh pointed out for fish.
Erik Bruchez: Those go the other way.
Charlie Wiecha: We can give them a handful of xpath functions. The anonymous case, for example.
Steven Pemberton: We can have a defined mapping for what to do with non-name characters.
Leigh Klotz: Right, that's what I mean by "get over it." You just might not be able to distinguish a control-character key from an key named "u0017."
Erik Bruchez: We could have a full canonical representation that's not very XPath friendly and then use transformations from that XML to friendlier XML.
Nick van: We could support a number of standard transforms.
Erik Bruchez: That puts us in the business of JSON-to-XML format.
Leigh Klotz: does what you asked; it does the full Badgerfish conversion and then the mapped conversion.
Erik Bruchez: But the badgerfish page is gone.

Charlie Wiecha: So do we want to provide serialization steps for conversion for interacting with existing JSON services?
Charlie Wiecha: So do we want to take our XML convert to full JSON?
Steven Pemberton: I don't think so.
John Boyer: You can jury-rig a script tag to implement submission?
Steven Pemberton: jparse will replace eval.
Erik Bruchez: The benefit is not speed; it's because it's more for Javascript syntax.
John Boyer: It's not JSON, but JavaScript with parens.
Charlie Wiecha: It's data-centric, not document-centric.
Erik Bruchez: The only benefit is consuming and maybe producing JSON.
John Boyer: There exist services you can call cross-domain by generating a script tag dynamically.
Charlie Wiecha: Mark has a Ubiquity library for doing that.
John Boyer: You get the JavaScript inline and callbacks.
Erik Bruchez: That's JSONP.
Leigh Klotz: Raman showed the TAG this WikiPedia page:
Erik Bruchez: Producing and consuming JSON is a benefit. It is rising as a data format. Sometimes services provide only XML.
John Boyer: If our goal is to connect to those services we need to look at what those services do. If they expect a function name then that's what we have to be responsive to.
Erik Bruchez: We don't have to support JSONP; that's a second step.
John Boyer: Then what's the second step?
Charlie Wiecha: A straight REST call is my understanding of the primary use case.
Nick van: Wikipedia, Google.
Charlie Wiecha: OpenSocial
John Boyer: OK
Charlie Wiecha: The transform is the key issue.
Erik Bruchez: The Twitter API will give you JSON or JSONP. I doubt there are services that force JSONP at this time.
John Boyer: Can a form talk to the wikipedia service without cross-site problems.
Leigh Klotz: Yes, does it.
Nick van: It's not cross-domain scripting; it's parsed.
John Boyer: It's because it's a GET.
Nick van: Yes.

Charlie Wiecha: So do we agree?

Resolution 2010-03-25.1: The primary use case for JSON with XForms 1.2 is to support interaction with existing data-centric JSON services, not for full-fidelity document-centric XML import/export.

Nick van: How about posting JSON?
Charlie Wiecha: Yes, posting JSON is covered.

Leigh Klotz: In your example you initialized your XPath data instance from a #id reference to a script tag containing JSON syntax.
Charlie Wiecha: Yes. And for submission I used serialization="application/json". It works because the geo service works with the named one.
Leigh Klotz: The Google example I showed used (as Nick found) repeated anonymous structures.
Steven Pemberton: We should test against the Wikipedia and Google services. Does the number as name one appear?
Charlie Wiecha: Yes, there ones in the wild with numbers.
Steven Pemberton: So there's three cases?
Leigh Klotz: And the mixed one. They could fix this with a made-up name for the unnamed structure.

Charlie Wiecha: OK, so there's also a query-language vs data model mismatch. Do we do that too?
Leigh Klotz: I think it's valuable and people do it: some people parse XML into JSON, use dot-notation, and then re-serialize as XML. So it's interesting, but it's not this task.
Charlie Wiecha: So we don't do that as part of this task.
Leigh Klotz: Right. It's interesting but hard and isn't this task.

Erik Bruchez: I'm still uncomfortable about us defining the conversions.
Charlie Wiecha: We don't. We might define a common starter set but we need to allow extensibility.
Nick van: We can have standard mappings and the one that supports everything.

Erik Bruchez: It seems like there is some research work needed.

Charlie Wiecha: We need to define success cases.
Nick van: Google, Twitter, Wikipedia.
Charlie Wiecha: Without having to write transformers.

Erik Bruchez: The Badgerfish document is gone.
Leigh Klotz: I'll write the author and ask.

Action 2010-03-25.1: Charlie Wiecha to research common use cases to support with named JSON conventions and a straw-man for dealing with transforms.