Re: Stickers in contenteditable?

Just to clarify, we are not trying to ship this now .


I am also totally fine with not having this shipped in this version of the spec.


My goal here was to gauge if there is any interest in this from Web Authors and to also brainstorm on how would we go about exposing it.


Hence, any feedback is greatly appreciated.


- Grisha


Sent from Outlook<http://aka.ms/weboutlook>

________________________________
From: johanneswilm@gmail.com <johanneswilm@gmail.com> on behalf of Johannes Wilm <mail@johanneswilm.org>
Sent: Thursday, August 10, 2017 12:41:54 AM
To: Gary Kačmarčík (Кошмарчик)
Cc: Grisha Lyukshin; public-editing-tf@w3.org
Subject: Re: Stickers in contenteditable?

On Thu, Aug 10, 2017 at 12:48 AM, Gary Kačmarčík (Кошмарчик) <garykac@google.com<mailto:garykac@google.com>> wrote:
I can pass along questions you might have and/or get you in touch with some people if needed.

However, I don't see how "stickers" by themselves warrant any special support -- they're basically just inline images with no real consistency between vendors or even between sticker packs from the same vendor.

Having stickers provided by the OS/UA is a non-starter because there is no unified registry (nor do we want to maintain one). Even if there was a registry (like Unicode is for emoji), there are still issues where users on one platform get annoyed unless their stickers/emoji show up *exactly* as they appeared when the message was composed. Many users specifically misuse emoji based on the appearance (cf. eggplant emoji) and don't want the recipient to get a different image.

I think that is part of the point: The user agent provides images that are sent as images. There is no common registry. We cannot dictate where the user agent gets those images from, but one could for example imagine an Android soft keyboard bundling some images and also allowing users to add their own images to the collection.

The difference from other types of images would be the metadata provided with them.


So to support stickers properly, we'd really just need to support generic inline images.

As for inline images - the main difficulty has been getting the caret to move around them correctly. I believe that was actually solved a while ago in all the browsers. There were more "exotic" elements (such as SVGs, or canvas elements) that have been more problematic. So unless there are other issues I am unaware of, I think images are OK as they are right now.


Anything beyond that feels out of scope for the editing api.


At the beginning, when I first heard about it, my first thought was also that this was just another example of browser-overreach, where the browser tries to do add more features to the text editor, and in the end it just never really works because it ends up being too complex, buggy and doesn't work across browsers or cannot be used in combination with JavaScript operations that are needed to keep the entire thing going. Also, the type of editor I have been working on is more in the academic corner, so this didn't really seem to apply.

But then I saw some points to it:
* If Microsoft is going to add sticker support to some input controls and we don't have an inputtype for it, they will probably just use paste. Thereby we lose semantic information about what is coming in. Also, once we add the ability to turn off specific input types, editors that don't want stickers won't have any way to explain to the user agents that they are not interested in stickers (which would likely cause the user agents to disable the input controls related to stickers).
* Maintaining a sticker collection and use them across web apps is difficult. This would make it somewhat more easily accessible.
* Stickers should have more than just the image data that we could also get in a paste. There could for example be different size versions of the image so that the JS edito can decide which one makes more sense. CKEditor mentioned that one would need a type to understand if the sticker is meant as an inline emoji-replacement or an block-level sticker.


But I worry that spending time adding support for inline images/rich content (while quite useful) would be a distraction from nailing down the remaining issues.

I am also skeptical. The remaining issues are definitely more important to resolve. "Rich content" is more problematic. Remember that everything that goes into the editing element has to go through the JS editor's way of understanding the world. If we added support for say 3 or 4 different pieces of richtext, then JS editors can probably figure out to deal with that. But fi we just add an infinite amount of arbitrary richtext, then the JS editor will likely deal with that like it deals with paste: it will filter through it and keep those parts it understands. That may in many cases mean that the richtext content will end up garbled in the output.



I think that this might be worth doing, but I'm not convinced that it's worth doing right now.

I would also preferably want to finish those things we set out to do first. But if Microsoft is going to launch this sticker bar in a few weeks, like Apple launched the touchbar last year (?), then I would like to get what is needed in place so that JS editors can dela with the new type of input. It's not optimal that the users get more new ways of inputting data, and once they go to the web they discovered that using these controls breaks all their favorite websites.



On Wed, Aug 9, 2017 at 1:23 PM, Grisha Lyukshin <glyuk@microsoft.com<mailto:glyuk@microsoft.com>> wrote:

In addition, to stickers, there could be any rich content, really. We could have something like "richContentInput" with some metadata, containing what the content is, or something similar.


@Chrome folks: do you happen to have contacts in Gmail, Google+ or any other Google editors that have some kind of messaging capability that we could inquire about this and other editing related questions?


- Grisha


Sent from Outlook<http://aka.ms/weboutlook>

________________________________
From: johanneswilm@gmail.com<mailto:johanneswilm@gmail.com> <johanneswilm@gmail.com<mailto:johanneswilm@gmail.com>> on behalf of Johannes Wilm <mail@johanneswilm.org<mailto:mail@johanneswilm.org>>
Sent: Tuesday, August 8, 2017 10:57:07 AM
To: public-editing-tf@w3.org<mailto:public-editing-tf@w3.org>
Subject: Stickers in contenteditable?

Hey,
today we spoke about potentially adding a way to add stickers within contenteditable as a new input Type. Basically, the OS/User agent would provide an image which then could be used across apps. The OS would decide whether it will allow users to add their own stickers and how that process would be, or whether it only would allow standard OS-provided stickers.

One of the advantage of providing stickers from the user agent rather than having stickers implemented in each web app would be that it would allow users to use the same sticker across web apps.

Grisha explained his idea about it, and I must say it appears to be useful for at least some use cases. It may be obvious that stickers are not suitable for all types of formal writing, but for a lot of what people write in a more social setting, they may be useful. For example, last year I was contacted by a group of young media people in one Central American country. They wondered if there was a way that they could spread an emoji in celebration of a national day in that country which they hoped to hoped to spread through the national media of for people to use. Unfortunately I had to tell them that emojis were simply unicode characters, and that none of the provided alternatives seemed suitable for their purpose. Had one had this sticker system in place, that may have been an alternative.

Similar images may be useful for this purpose in other geographic areas (check for example how people in the UK try to use the remembrance day poppy in social media posts [1]) or related to specific subcultures that won't all be coverable with emojis simply because there are so many of them. It may be down to a family or single person who decides to spread the same image across several social media networks or forum posts.

But the question remains: Is this something other browser vendors and the developers of web apps making use of contenteditable would find useful? I am trying to reach out to various JS editor developers and it would be good if people here could give feedback as well and if possible contact the JS projects that could potentially use this within their organizations.

As I see it, this would require adding an input type to beforeinput/input that specifies that the user would like to insert a sticker image together with the image data. "Stickers" would then also be among the features that could be specified to be turned off for a specific contenteditable element.

[1] http://www.mirror.co.uk/tech/add-remembrance-poppy-2016-facebook-6751550


--
Johannes Wilm
http://www.johanneswilm.org

tel: +1 (520) 399 8880<tel:(520)%20399-8880>




--
Johannes Wilm
http://www.johanneswilm.org

tel: +1 (520) 399 8880