RE: [clipboard events] seeking implementor feedback on using CID: URI scheme for pasting embedded binary data

Hi Hallvord!

The IE11 API you mentioned is msConvertURL [1] (also on the IE blog [2]), and it was designed as a simple way for sites to choose DataURI or Blob for otherwise inaccessible images. We default to DataURI to be interoperable with Firefox’s current design of always doing DataURI for local images on the clipboard. With a little bit of code, blobs can be used instead. Our clipboardData.files implementation contains only images at the moment because we have only partial support of the current version of the Clipboard API spec. This means that with a quick ‘for’ loop over those images, javascript can choose to create blobs, manage the blob’s memory, and upload the blobs to a server. As you mention, it is not possible to tell which file/image corresponds to which <img> because it’s really designed as a simple approach for cases where a site wants to always use blob or dataURI for images that they couldn’t otherwise access.

We are considering doing the CID approach as well in the future. It is nice to have the additional control of seeing which <img> src you are changing, and it will likely work better for copy, not just paste like convertURL. We believe that convertURL does not block using the CIDs you have in the current spec. To better understand your approach and allow us to help move it forward, can you give us sample javascript that a site would use to set the DataTransferItems for HTML and the related images during copy? Second, can you provide the javascript for how a site would put them into the pasted markup during paste?

Regarding a couple other questions you ask:
* As far as I’ve seen, WebKit/Blink do not yet support images with local-system sources. They do support binary images on the clipboard using ClipboardData.items like in the example I found online [3].
* Some sites prefer DataURI to Blob because it’s all inline and doesn’t require sending separate objects or managing memory, so I don’t think DataURI is something we should discount.

Looking forward to seeing your sample code!
Ben Peters

[1] http://msdn.microsoft.com/en-us/library/ie/dn254951(v=vs.85).aspx 
[2] http://blogs.msdn.com/b/ie/archive/2013/10/24/enhanced-rich-editing-experiences-in-ie11.aspx

[3] http://strd6.com/2011/09/html5-javascript-pasting-image-data-in-chrome/


-----Original Message-----
From: Hallvord R. M. Steen [mailto:hsteen@mozilla.com] 
Sent: Thursday, January 23, 2014 5:30 PM
To: public-webapps
Subject: [clipboard events] seeking implementor feedback on using CID: URI scheme for pasting embedded binary data

> Hi,
> pasting HTML that contains embeds (images, video) into a rich text editor is a use case we should cover. It's currently handled in different ways - 

> * IE11 supports pasting images as either data: URLs or blobs [1] (and has a non-standard method to fill in a gap in the blob approach). I don't understand from this blog post how/if it supports referencing the binary parts from the HTML. If for example you paste a snippet from a Word page that contains two images, the DataTransferItemList is presumably populated with two image files, which can be processed/uploaded using the blob method - but how is the script processing the data supposed to know what IMG tag in the pasted HTML each image file belongs to?

> * Pasting stuff as data: URLs seems like a hack, wasting memory and requiring quite some extra processing if there is a lot of data.

> * Firefox apparently happily passes on file:/// URLs with local paths and all [2], this is of course a bug.

> * Right now I'm not sure what WebKit/Blink - based implementations do. Test results welcome!

> As the editor of the Clipboard Events spec, I'm proposing a somewhat different take on this: "cid:"-URIs for embeds. See http://dev.w3.org/2006/webapi/clipops/clipops.html (search for "cid:").

> The idea is that rather than embedding potentially very huge data: URLs or reference local files in the embedded markup, we add a reference to the DataTransferItemList, and use the index of this reference to construct a cid: URI in the markup that clipboardEvent.getData('text/html') will see. The script processing this data can then pull out the cid: URIs, do drag-and-drop style file uploads for referenced clipboard parts, and update the data to refer to the locations on the server eventually (maybe first using an intermediate placeholder image or something like that.)

> AFAIK, outside of the used in HTML intended for E-mail, this would be the first usage of CID: URIs in web platform specs. I'm looking for feedback regarding whether this is implementable and a good solution. I haven't had much (if any) feedback from implementors on this issue yet, so thank you all in advance for your ideas and input.
-Hallvord

> [1] http://blogs.msdn.com/b/ie/archive/2013/10/24/enhanced-rich-editing-experiences-in-ie11.aspx

> [2] https://bugzilla.mozilla.org/show_bug.cgi?id=665341

Received on Wednesday, 12 February 2014 09:36:06 UTC