Re: [clipboard events] seeking implementor feedback on using CID: URI scheme for pasting embedded binary data

On Wed, Feb 12, 2014 at 1:25 AM, Hallvord R. M. Steen <hsteen@mozilla.com>wrote:

> > Hi Hallvord!
>
> Hi Ben! Thanks for responding to my request for feedback - especially
> since the IE team has done some interesting work in this area and is
> arguably ahead of the rest! :-)
>
> > The IE11 API you mentioned is msConvertURL [1] (also on the IE blog
> [2]), and it was designed as a
> > simple way for sites to choose DataURI or Blob for otherwise
> inaccessible images.
> <X>
> > As you mention, it is not possible to tell which file/image corresponds
> to which <img> because it’s
> > really designed as a simple approach for cases where a site wants to
> always use blob or dataURI for
> > images that they couldn’t otherwise access.
>
> That's more or less what I thought, based on the blog post, so thanks for
> describing it in detail.
>
> > We are considering doing the CID approach as well in the future. It is
> nice to have the additional
> > control of seeing which <img> src you are changing, and it will likely
> work better for copy, not just
> > paste like convertURL.
>
> Actually, I haven't truly considered the copy case here yet. I've sort of
> assumed that given that you can put multiple bits of data on a clipboard,
> the various clipboard implementations should already have a way one piece
> of data can reference one specific other piece of data - I haven't really
> found the technical details here.
>

​I don't actually know of any native clipboard formats that allow data
associated with one format to reference other data in the same operation,
so browsers would have to roll their own implementation/figure out how to
get it to work well with native apps.


>
> Of course it would be nice to support a script that wants to generate
> random HTML with embedded files to place on the clipboard (although I think
> most of those use cases can already be met by using URLs and assuming that
> any software reading HTML from the clipboard can understand URLs..).
> However, one can imagine a use case for example with a CANVAS app where the
> script wants to copy the state of the CANVAS as an image inside HTML it
> places on the clipboard - having the script create src="cid:n" type markup,
> append files, and make the UA translate that to the platform's native
> clipboard implementation's way of referencing one part on the clipboard
> from another part..
>
> > We believe that convertURL does not block using the CIDs you have in the
> current spec.
>
> I suppose not, but perhaps the more relevant question is: should we
> standardise convertURL? Would it still have a use case if we take the cid:
> route? (And I guess a related question is: given that we've done data: URLs
> for a while, how much content will we break if, say, Firefox moves from
> data: to cid:? Do we need to make cid: opt-in somehow, like you're doing
> with convertURL?)
>

I'm trying to make sure I correctly understand how the IE11 version of this
works. From the sample (
http://msdn.microsoft.com/en-us/library/ie/dn254935(v=vs.85).aspx), it
looks like if a user pastes in some HTML that references local images, IE11
automatically captures the referenced files into the clipboard. Then the
page uses msConvertUrl() rewrites references to the src attributes in the
text/html DataTransferItem to reference the blob URLs, right?

Given that the drag data store is in "read only" mode at this point, it
seems weird to allow mutations at this point.

In addition, from a security perspective, what stops a malicious website
from embedding something like <img src="file:///etc/passwd"
style="display:none"></img> in the markup?


>
> > To better understand your approach and allow us to help move it forward,
> can you give us sample
> > javascript that a site would use to set the DataTransferItems for HTML
> and the related images during copy?
>
> As I said, I have not really considered this use case - so the spec
> doesn't actually cover this. If we want to make this work, I suppose the JS
> would look somewhat like this?
>
> document.addEventListener('copy', function(e){
>         // So, you want to copy the current status of your game? No
> problem.
>         // Let's say this game generates a number of PNG graphics from
> CANVAS tags
>         // and keeps them in memory in ArrayBuffers or something
>
>         var html = '<div><b>player</b>\'s medals: <img src="cid:1"><img
> src="cid:2"></div>';
>         e.clipboardData.items.add(html, 'text/html');
>         e.clipboardData.items.add(new File(medals[0], 'medal.png',
> {type:'image/png'}));
>         e.clipboardData.items.add(new File(medals[1], 'medal.png',
> {type:'image/png'}));
>         e.preventDefault();
>
> }, false);
>
> > Second, can you provide the javascript for how a site would put them
> into the pasted markup during paste?
>
> The way I thought this would work, would be that the site starts XHR
> uploads from the paste processing, and shows some intermediate 'loading'
> animation or something before it gets the final URLs back from the server.
> A bit like this (although some things could be more elegant, like the
> insertion of the data which needs to take cursors and selections into
> account):
>
> http://jsfiddle.net/2Qqrp/
>
> Thinking about it, it may be considered somewhat wasteful (or exceedingly
> slow - if for example the embedded data is a video) to do it this way - but
> it quickly gets complex and/or confusing if we have a some "show this local
> file until it's uploaded, then reference the online file instead" magic..?
>
> -Hallvord
>
>

Received on Monday, 10 March 2014 22:44:32 UTC