[w3c/editing] Should we modify the clipboard API spec to match Safari's behavior for HTML-referenced media? (#285)

## Background
If a user copies rich text with images from a native app such as Microsoft Word, and, using Chromium, attempts to paste into a web app such as Outlook or GMail, the images (and other media referenced by the HTML) will not be pasted. The images in the HTML on the clipboard have src attributes with file URIs and those will not be loaded when pasted into a document served from the web.

### Current Clipboard API Specification
The [current clipboard API](https://www.w3.org/TR/clipboard-apis) spec [has a mechanism](https://www.w3.org/TR/clipboard-apis/#to-process-html-paste-event) that would allow authors to access HTML referenced media when pasted, but no browser implements this approach.  

According to the spec, user agents should replace file URIs with cid:\<index>, where \<index> refers to the position of a corresponding file in the items collection of the ClipboardEvent’s DataTransfer object.

Authors could then handle the event as follows:

```javascript
addEventListener("paste", e => {
    // get the HTML from the clipboard
    const html = e.clipboardData.getData("text/html")
    
    // parse the HTML into a temporary document
    const clipboardDocument = document.implementation.createHTMLDocument()
    clipboardDocument.write(html)
    
    // find all elements having a src with a cid reference
    const elements = clipboardDocument.querySelectorAll("[src^='cid:']")
    for (const element of elements) {
        const index = parseInt(element.getAttribute("src").substring(4), 10)
        if (index) {
            if (e.clipboardData.items.length > index) {
                // Get the file associated with this element
                const file = e.clipboardData.items[index]
                // Create a blob URL to load the content
                element.setAttribute("src", URL.createObjectURL(file))
                // Perform other operations as appropriate, 
                // e.g. saving the image
            }
        }
    }
 
    // Sanitize the clipboard document (omitted)
    sanitizeClipboardHTML(clipboardDocument)
 
    // Paste the contents of the document somewhere
    pasteClipboardHTML(clipboardDocument)
 
    // Override default behavior
    e.preventDefault()
})
```

### Safari Implementation (Unspecified)
Safari, has crafted its own (more direct) approach by turning the referenced media into blobs and replacing the file URIs with their corresponding blob URIs before the markup is ever provided to the authors.  Additional details regarding Safari’s implementation are available in this [blog post](https://webkit.org/blog/8170/clipboard-api-improvements/) and this [change list](https://trac.webkit.org/changeset/223440/webkit).

The code from above changes to:

```javascript
addEventListener("paste", e => {
    // get the HTML from the clipboard
    const html = e.clipboardData.getData("text/html")
    
    // parse the HTML into a temporary document
    const clipboardDocument = document.implementation.createHTMLDocument()
    clipboardDocument.write(html)    
 
    // find all elements having a blob URL to process them
    const elements = clipboardDocument.querySelectorAll("[src^='blob:']")
    for (const element of elements) {
        // Get the blob associated with this element
        fetch(element.getAttribute("src"))
            .then(r => r.blob())
            .then(b => {
                // Process the blob in some site-specific way
            })        
    }
 
    // Sanitize the clipboard document (omitted)
    sanitizeClipboardHTML(clipboardDocument)
 
    // Paste the contents of the document somewhere (omitted)
    pasteClipboardHTML(clipboardDocument)
 
    // Override default behavior
    e.preventDefault()
})
```

## Proposal
This issue proposes that we update the current spec to match Safari’s implementation, by referencing blobs in HTML instead of items in the DataTransferItemList with cid references.

### Security Considerations
In order to prevent authors from round-tripping file URIs through the clipboard to read arbitrary content from a user’s file system, user agents need a reliable way to detect when the contents of the clipboard were copied from a trusted application that the user has installed.  This mechanism may be platform-specific and would likely not be specified as part of a standard.  

One approach would be for user agents to examine the formats present on the clipboard at time of paste and determine if it’s possible for those formats to have been put there by an author (clipboard formats that can be written by the author are currently restricted by browsers). 

Another approach would be to check the contents of the HTML format for something that an author could not have put there.  The header written for the HTML format on Windows, for example, is not controllable by the author today and some native applications indicate the source of the content in a way that would distinguish it from clipboard content created by an author.

If the restrictions placed on what can be written to the clipboard are lifted in the future, the lifting of those restrictions must be accompanied by a required [mark of the web](https://github.com/dway123/clip-motw/blob/master/explainer.md), which would indicate that the data is not to be trusted for the purposes of making referenced media bits available to authors on paste.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/w3c/editing/issues/285