Re: Form Serialization

Form Serialization
By Garrett Smith, August 25, 2007 (with much gratuitous help from Jonas Sicking)

1. Problem:
No standard, reliable way to associate a form with an Ajax request.
Scenario:
A web site has a form containing multiple form fields.
This data needs to be sent to the server to be verified.
However the author does not want to affect history state
After submitting the form, a message appears on the page displaying
customized details from the server.
>From the user's perspective, he is still on the same page.

JavaScript can be used to control the UI by providing status and
feedback in real time to the user about her actions. This enables a
smooth, continuous user experience and usability (fun, responsive,
easy to use applications).

The need has manifested itself strongly in all of today's popular Ajax
libraries, however:
i.	nearly all of them get it wrong.
ii.	none of them serialize a file

(Ironically, Internet Explorer 4-7 gets the BUTTON wrong (uses
innerHTML instead of value, sends BUTTON when not clicked)).

The need for File serialization is formally recognized, is being
standardized in the File and FileList specifications(1), and is
implemented in Mozilla Firefox (2).  File serialization now makes it
possible to send a file using XMLHttpRequest, although the
functionality has not gotten mainstream acceptance (too early).

The workaround that most libraries take is to fall back to creating a
form to a hidden IFRAME to which the FORM is submitted.
The current (partial) technological solutions are:
1)	serialize the form using a JavaScript library, and use XMLHttpRequest
2)	target an iframe
3)	use standard form submission (not Ajax)
Other possible proposals (new):
1)	Support Form Serialization
2)	a new Solution, XMLHttpRequest method setForm( )
3)	add new features to IFRAME
4)	ignore this case
Proposal:
Add to HTMLFormElement interface, the following methods:
    toJSONString
    getDataSetString

Both methods will return a serialized representation of the forms
successful controls.
Background:
Many Ajax apps use forms to provide a UI to model data. There are some
benefits to this approach:
pro: More explicit "stub" on the client. The URL and params are in the
content layer (in the FORM).
  	pro: can be used in a degradable approach for non-Ajax clients.
 	pro: Entities can modeled on the client using standard UI controls
Approach #1: Serialize the form using JavaScript and use an XmlHttpRequest
pro: does not affect history state
pro: existing standard (9)
pro: Standard support HTTP methods GET, POST, HEAD, PUT, DELETE, and OPTIONS
pro: Standard access to response headers
pro: the ability to abort a call and track a calls state
pro: XMLHttpRequest is reported to be faster than an IFRAME (5)
pro: XMLHttpRequest supports responseXML in an abstract view, as model
data on the client

XMLHttpRequest supports other HTTP methods. These methods help to
differentiate different types of actions. For example, DELETE
(idempotent) can be used to remove an item from a user's shopping
cart.
Problems with Current approach of using an XMLHttpRequest for FORM
data submission:
con: If an XHR-associated FORM that does not use a file input has been
changed to have a file input, then the entire serialization to
generate a string to send with the XHR will require a change to an
IFRAME for browser that don't support File serialization.
con: the FORM must be serialized by hand.  Most Ajax apps already use
a library; however Ajax libraries offer some considerable problems:
con: Most of them are inconsistent with each other in terms of form
serialization(6)
con: complex, pushes too much work to the browser-side javascript
con: Adding a file input to an Ajax-associated form submission makes
the form ineligible for XHR
con: the library is unnecessarily complicated by the file input and by
the form serialization

Conclusion: XHR is a good solution, however, FORM Serialization is
somewhat complicated and authors often make honest mistakes in
attempts to serialize the HTML FORM's successful controls.
The current (partial) technological solution, pros and cons:
Approach #2: Target a hidden IFRAME
pro: simple
pro: works in all modern browsers
Problems associated with using an IFRAME:
con: Changes history state in most browsers, but behavior is
inconsistent and nonstandard (3)
con: No support for methods other than GET or POST
con: Can't access response headers
con: It's a hack. IFRAME is not directly intended to be hidden and
used for the targeting of requests, but is instead intended to "allow
authors to present documents in multiple views" (4)
con: does not support ability to abort a call
con: can not track a calls state
con: Browsers may offer side effects to accommodate the intended use
of IFRAME (inconsistencies such as the "click and throb" in IE)

Conclusion: Using an IFRAME to accommodate this use case will work,
but has certain implications that make it less desirable.
Approach #3: Use standard FORM submission
Advantages to Using Standard FORM Submission
pro: commonly accepted, valid development methodology
pro: simple
pro: standards-based: relies on the standard html FORM
pro: no change required
Disadvantages to Using Standard FORM Submission
con: has page refresh
con: cannot be used for widgets
con: one-way communication in a page
con: has page refresh

Conclusion: Does not correctly satisfy use case.
Other possible proposals (new), pros and cons
Proposal #1: Support FORM Serialization:
     toJSONString
     getDataSetString

File serialization has been recently recognized, is being standardized
(1), and is implemented in Mozilla Firefox (2)
pro: can be used to fulfill use case
pro: decrease hacks and increase developer productivity
pro: reduce bandwidth (less bloated library code)
pro: will make form serialization easier (no borked-up, bloated,
hand-rolled library code)
pro: browser already support this natively, behavior is well-defined
by the w3c FORM specification:
    http://www.w3.org/TR/html401/interact/forms.html#h-17.13.1
pro: Allows for two-way communication within a page
pro: more efficient on server resources
pro: standard XMLHttpRequest that are well-defined by the w3c and
well-supported in modern browsers
con: will require permanent modification to the HTMLFormElement interface
con: more complicated
con: requires JavaScript enabled
con: content is rendered in the IFRAME's document as it is parsed
(this makes no sense for model XML or JSON).

	Pros and cons of each sub-feature:
a. getDataSetString
pro: most common use case to submit a serialized form
pro: Data Set strings are standardized (8) and supported in all server
side APIs
con: adds a new method to API, permanently changing the API
con: URI encoding greatly increases file size.

b. toJSONString
pro: wide support in a all server-side languages for unmarshalling JSON objects.
pro: useful for storage in a browser-side cache – a common metaphor
pro: could also be useful for offline storage
pro: toJSONString is already going to be part of Object.prototype in
ES4. This is planned to be supported in Firefox 3 and IE8, via a
plugin distributed with the browser (10)
		pro: easier to read and parse than URIs
		pro: does not need URI Encoding/Decoding (unless sent as GET)
con: no way to set a FORM's data with the JSON object; incomplete, one
way API (we really need a method form.setData( jsonObj ) )

Note: There is a speculation in the JSONRequest whitepaper that JSON
will be used to transfer data between sites and will be more widely
used in the future (11)

Conclusion: Can be used to satisfy use case.
Proposal #2:  a new solution, XMLHttpRequest method setForm( form )
pro: very simple to use.
pro: decrease hacks and increase developer productivity
pro: reduce bandwidth (less bloated library code)
con: does not support JSON marshalling (although this could be added
to the form's enctype)
con: will require permanent modification to the XMLHttpRequest's
public interface
con: does not cover PUT, DELETE, only GET and POST. (although this
could also be changed)

Proposal #3:  a new solution, add new features to IFRAME.
       Add the ability to get progress events and cancel loads on the
window object.
       Modify the HTML specification to state that IFRAME can also be
used as a transport mechanism.

HTMLIframeElement {
attribute useHistory boolean
attribute render boolean

useHistory – boolean, default value is true
when false, the history will not be included in IFRAME loads.
render – boolean, default value is true
	when false, the IFRAME does not render its contentDocument

pro: IFRAME is already in use
pro: load events follow Progress Events 1.0 working draft
pro: Standard access to response headers
pro: The ability to abort a call and track a calls state and
pro: responseXML in an abstract view, as model data on the client.
con: adds a new use case to the existing IFRAME element which requires
changes to the existing IFRAME element. IFRAME is not intended to be
hidden and used for the targeting of requests, but is instead intended
to "allow authors to present documents in multiple views" (4)
con: does not cover other HTTP methods (PUT, DELETE, only GET and
POST. (could also be changed)
con: content is rendered in the IFRAME's document as it is parsed
(this makes no sense for model XML or JSON).
con: contentDocument (an alternative to responseXML) is rendered by
the IFRAME, which is not desirable in the use case.
	con: conflates two use-cases into one element
1) IFRAME is designed to display content in an inline frame.
2) New: IFRAME can also be used as a transport mechanism by adding new
attributes:
useHistory
render
	con: requires additional mechanism to cancel an IFRAME load
	con: current browsers don't provide standard access to response headers
	con: requires a new attribute to have the IFRAME not change history
	con: If the contentDocument contains scripts, they will be
interpreted when the IFRAME is loaded.
(This may cause unwanted side effects)
con: XMLHttpRequest supports other HTTP methods, such as DELETE (an
idempotent method that is useful for deleting an entity on the
server). These methods help to differentiate different types of
actions.

Conclusion: Can be used to satisfy use case.
Proposal #4:  Ignore this case
This is not really an option, as current web applications make heavy
use of this with existing technological methodologies mentioned
earlier.
References:
(1) W3C: File Upload. http://www.w3.org/TR/file-upload/
(2) Need an API to store <input type=file> data offline and access
when online https://bugzilla.mozilla.org/show_bug.cgi?id=371432
(3) HTML Techniques for Web Content Accessibility Guidelines 1.0:
Frames http://www.w3.org/TR/WCAG10-HTML-TECHS/#frames
(4) W3C: 16.1 Introduction to frames
http://www.w3.org/TR/html401/present/frames.html#h-16.1
(5) XML HTTP Performance and Caching, Emil A Eklund,
http://me.eae.net/archive/2005/04/02/xml-http-performance-and-caching/
(6) JavaScript Form Serialization Comparison:   jQuery • dojo •
Prototype • YUI • MochiKit http://www.malsup.com/jquery/form/comp/
(6) Comment #6: Use Cases for File Serialization by Vladimir Vukicevic
(:vlad) https://bugzilla.mozilla.org/show_bug.cgi?id=371432#c6
(8) Forms in HTML Documents: Form Submission
http://www.w3.org/TR/html401/interact/forms.html#h-17.13.1
(9) W3C: The XMLHttpRequest Object
http://www.w3.org/TR/2007/WD-XMLHttpRequest-20070618/
(10) Bugzilla: Implement JSON extensions to the JS language.
https://bugzilla.mozilla.org/show_bug.cgi?id=340987
(11) JSONRequest http://www.json.org/JSONRequest.html
(12) Progress events 1.0
http://dev.w3.org/cvsweb/~checkout~/2006/webapi/progress/Progress.html?rev=1.16&content-type=text/html;%20charset=iso-8859-1#XHR

Received on Saturday, 25 August 2007 19:54:11 UTC