minutes 2022-04-13

[17:03] <johanneswilm> https://github.com/w3c/editing/issues/424
[17:04] <GameMaker> snianu this is about the effects of delayed clipboard
rendering
[17:04] <GameMaker> snianu if a site registered for this format and the
user closed the browers
[17:05] <GameMaker> snianu what effect does closing the tab on the formats
[17:05] <GameMaker> snianu what effect does this have on the existing
features of the browsers
[17:05] <GameMaker> snianu going through the options
[17:05] <GameMaker> snianu first options, before unload handler, currently,
the dialog text is very generic
[17:06] <GameMaker> snianu if we showed this dialog to the user it would be
confusing to the user
[17:06] <GameMaker> snianu and the site wouldn't be able to use pf cache
[17:06] <GameMaker> snianu if there is a site that is abusing this, then
that can effect performance
[17:06] <GameMaker> snianu have a timeout, say 10 seconds, and if they
authors are not able to produce the format in that time
[17:07] <GameMaker> johanneswilm this would be something that is added from
the browser side, would you need to display the generic message?
[17:08] <GameMaker> johanneswilm could you not have one that was related to
clipboard, about items being lost because the tab is being closed
[17:08] <GameMaker> snianu are you talking about the one before unload, or
a new dialog
[17:09] <-- estade (~estade@a11e3e9c.public.cloak) has left this server
(Ping timeout: 180 seconds).
[17:09] <GameMaker> johanneswilm I understand the problem about the user
closing a tab, and in that tab is a clipboard item being rendered, and that
they current warning is very generic, because you do not allow the user to
specify the message, but why can't the browser make a message that explains
what is going on
[17:10] <GameMaker> snianu native excel has this notification, we could
show a seperate confermation dialog, the reason that we don't allow author
to specify what the message is is for security and to not let malicious
websites trick the users.
[17:11] <GameMaker> snianu it's also confusing, if you have more than one
format, will you show them all, it's confusing to the user, maybe you could
make it work with timeouts.
[17:12] --> whsieh (~whsieh@a11e3e9c.public.cloak) has joined this channel.
[17:12] <GameMaker> jsbell clarifying that snianu was walking through
option 1,a nd johanneswilm was jumping to options 5, I do not have any
additional options at this time
[17:12] --> estade (~estade@a11e3e9c.public.cloak) has joined this channel.
[17:13] <GameMaker> smaug this approach might work if you just have some
static thing, you can render the whole clipboard content at onces.
[17:13] <GameMaker> smaug there was also a discussion about streaming, and
no browers would want to delay to wait for streaming, if the user is
closing the tab, we are not going to wait too long
[17:14] <GameMaker> snianu I think the undelay options could also be used
maliciously, we could mitigate this by adding a timeout
[17:15] <GameMaker> snianu option 2-  we could forcefully trigger all the
callbacks until we fired the unload event, we also run into the issue of it
taking to long, but we could add a timeout. this won't affect the pfcash
directly, but it directly affect the navigation
[17:16] <GameMaker> snianu this is one approach that might be viable. there
is some concern about slowing down the brower tab, but this is better than
the loader event
[17:17] <GameMaker> estade that seems like it still has the same problem
where we are delaying the shutdown. the general idea is chromium is that
every user visible action should complete quickly, and this sounds like it
will have the same issue, and so it will take more than that amount of
time, it will effect how snappy it feels to close the tab
[17:18] <GameMaker> snianu it will effect the browser shutnown behhavior,
but you could inform the user and if they don't want it, they can ignore it
and shutdonw
[17:19] <GameMaker> johanneswilm having a timeout, wouldn't that also
affect the workflow of copying something, closng the tab and pasting the
content, and that would be confusing to the user, it may even be the same
[17:19] <GameMaker> johanneswilm workflow, but return different results
[17:20] <GameMaker> snianu excel writes lowfidelity formats onto the
clipboard and we could do that before closing
[17:20] --> sanketj (~uid392014@a11e3e9c.public.cloak) has joined this
channel.
[17:20] --> alexk (~alexk@a11e3e9c.public.cloak) has joined this channel.
[17:20] <GameMaker> sanketj asking a clarifying question - are you saying
that the brower - is your concern showing a dialog and that slows things dow
[17:22] <GameMaker> estade I was talking about the later, I'm saying that
in general, for any kind of user action, we expect it to complete as
quickly as possible, in this case, we have an expensive operation, we are
trying to no perform the expernsive operation at all, but we really don't,
if there is an expensive operation, we do it in the backround, our solution
can't be to do it and delay the user
[17:23] <GameMaker> estade what we currently have with the promises is that
it keep things smooth for the user, and we want to keep that smooth timing,
with few exceptions, user interactions are supposed to be very snappy
[17:23] <GameMaker> sanketj I think that's fair, but a user could write an
upload handers
[17:24] <GameMaker> jsbell with more formal descriptions of the pay
lifecycle, navigation isn't just about an unload hander, because a tab
might be unloaded when it's not in front because of memory pressure, you
cna't just add a before unload handler, because it might never show to the
user
[17:25] <GameMaker> estade yes, sites can already delay shutdown, with
before unload, you can't trust every site, they will try to prevent you
from leaving, so this is just one more thing that they would add on top of
that it's not good to add more ways to be annoying
[17:26] <GameMaker> sanketj I wanted to take a step back and maybe talk
about what do we wnat to happen in this case, here the user has performed a
copy, some of the formats have been created, if the tab is being closed, I
think there's two behaviours, sometime the site will just drop all the
formats, or the source may keep some of the cheaper, most commonly used
formates.
[17:27] <GameMaker> sanketj what are folks ideas on how to achieve one or
both of those, that the siite might want to keep some of the formats
[17:27] <GameMaker> sanketj what other options do people have in mind
[17:27] <GameMaker> smaug I would expect that only those non-delayed
sources to be used, that's a simple approach, and it's hard to think of
what else we could do, espeically if we want to support streaming
[17:28] <GameMaker> johanneswilm this is already something that happens
with some programs on some OSes
[17:28] <GameMaker> sanketj I have only checked this on windows, not on
mac, but we can follow up on that, to all these points, I am leaning toward
that, as maybe a simple thing that we can do
[17:29] <GameMaker> sanketj Excell native generates 22 formats, a majority
of them need to be delayed rendered, are all of those format really that
expensive to generate?
[17:29] <GameMaker> sanketj we need to make sure that that behaviour still
continues to wrok for the apps that have expressed interest in this
[17:30] <GameMaker> estade if you're asking what a high level step back is,
I'd say we need a UX expert or researcher to see what the expected user
expiernece is
[17:31] <GameMaker> estade we present copy as if this is immediate, but
we're running into a case where it's not cheap, so we're decisving the
user. if I'm using gimp or photoshop, if I choose 'export as' then that is
not fast, and I expect that to be slow, but copy is supposed to do all of
these slow conversions, that will not be fast
[17:31] <GameMaker> estade if the user understands that the copy can be
slow, then it will change their expectations
[17:32] <GameMaker> estade if I worked on an app, I would consider having
something like "copy as..." so if I'm copying out of photoshop into figma,
[17:32] <GameMaker> estade it's an almost impossible problem, and it
shouldn't be presented to the user as something that is fast when it's not
[17:33] <GameMaker> jsbell two questions, building on evan, for the
partners we've talked to about this, is there ever not an immediatly
rendered format? the app might populate the clipboard with many types, but
is there ever one that is not immediatly renders
[17:35] <GameMaker> jsbell bulding on that, for native OSes, is there a
notion of an application owning a clipboard, and if that could be lost, if
there's 10 formats, and only one is immeditatly rendered, the application
will have to know that those delayed rendering types are gone, but the
browers will need to say that there's 10 options, but 9 are now gone, does
anyone know how that works
[17:36] <GameMaker> sanketj I don't know the answer to the last one, I
wanted to respond to the UX point, delayed clipboard rendering has been
around for a long time, whatever UX patterns we are hoping to gain insights
from, they already exist, and there's other apps other than excel and
photoshop
[17:37] <GameMaker> sanketj to the questions about if there were any
formats that would not delay render - excel wants to sill delay render text
and html, but that is a desire, the could potentially be some way to fall
back, right now they have some threathold for some limits for copies that
are too big. At the minimum I wwant to write this to the clipboard
[17:38] <GameMaker> smaug About the UX side, webbrowers are quite different
than native apps, so getting some feedback from UX folk would be helpful
[17:38] <GameMaker> sanketj in excell there are sheets which are like tabs,
you could overwrite data in a collaborative app. It's not exactly like it
is in the browser.
[17:39] <GameMaker> johanneswilm can you talk to the UX people at your
companies for some feedback on what to do for this issue?
[17:40] <GameMaker> jsbell I think we need more clarity before engaging
with UX, and it's too general, so we need to narrow it down yet
[17:41] <GameMaker> estade I think that at some point we need to engage
with UX, we're at the point right now that we're trying to get signals from
other vendors if they want to support this, and if it's even possible for
us to implement this in a way that works with our architecture
[17:42] <GameMaker> estade presicent is really important, but there's
different kind of presendent, sometimes it was from the 90s when it was
really about a differnt type of computing, but there was also a time where
browers didn't have tabs, we can try and set better standards now
[17:43] <GameMaker> johanneswilm we had a discussion about what formats
this would be for? did anyone find example of such formats?
[17:43] <GameMaker> sanketj we shared them in the original thread, from
Excel and Adobe
[17:44] <GameMaker> snianu We mentioned them before
[17:44] <GameMaker> johanneswilm are these use cases not something that
could be used with a special professional format?
[17:45] <GameMaker> sanketj for adobe, it's when they copy some specific
special types of layers, they want a fallback images if it's not being
pasted into another adobe program
[17:45] <GameMaker> sanketj when we're pasting into Excel, we want to use
our special web formate, but if we're pasting into something else, we want
the more generic version
[17:45] <GameMaker> sanketj excel makes 22 formats, but we can't do that
from the web, because they are not often used
[17:46] <GameMaker> johanneswilm that was an answer, it's still not clear
to me if some of those formats couldn't be a special menu options, say you
wanted to copy it into a special format, could you not say "copy as image"?
[17:47] <GameMaker> sanketj the site could choose to do that, but when when
a user copies something from photoshop, they have to do something
expelicite, having 5 or 20 'copy as' menu items is a worse user experirnece
[17:48] <GameMaker> sanketj the paste target will trying to use the best
format that is available
[17:48] <GameMaker> jsbell this is only of the ways where things just work
for the user in a really great way.
[17:49] <GameMaker> jsbell what we've heard from different partnets, memory
usage is also an issue, we're literally dealing with gigabye formats,
having multiple version of that can be a really expensive
[17:49] <GameMaker> jsbell do we need to consider delayed rendering for
those formats as well? Do we need to consider populated the drag and drop
data with multiple formats.
[17:50] <GameMaker> estade There was the example of photoshop saving as a
photoshop thing and an image, so it seems like we should, we should be
doing this because if we don't, there's a whole can of worms.
[17:51] <GameMaker> estade I don't think that if anyone knew we could paste
into paint from excel, but I do think that is something that people do want
[17:51] <GameMaker> estade the user might want to be able to select what
thing to copy something as
[17:52] <GameMaker> sanketj excel adds so many formats, the point that I'm
making more generally, it can be super expensive to general so many of
these formats, we're asking these well established apps, you have to change
this well established behaviour.
[17:53] <GameMaker> sanketj to me, that feels like a non-starter, I don't
want to be dismissive, but putting the onus on apps to make 'copy as.."
options, but I'm open if people disagree
[17:54] <GameMaker> whsieh that was just in response to the drag and drop,
at least in WebKit, it already works for lazy delievered types, it really
exploits the assumptions that the source application is alive, there's no
user expectation that if the source application crashes that the drop will
still work.
[17:55] <GameMaker> johanneswilm If I want to make an image, I want to be
able to make that expeiciatly. I do consider the difference between with
and without cvs
[17:55] <GameMaker> sanketj I wanted to say that excel generates 4 bitmap
formats, so giving the user the option there is too much.
[17:56] <GameMaker> sanketj should we talk this back to UX folks, but we
need to have a better view of what our options are
[17:57] <GameMaker> sanketj if you close the browers, all the delayed
formats go away, or you could do a dialog, but the page author could do
that with 'before unload', we could also wait for all those formate could
be writte, and also if there's a timeout.
[17:57] <GameMaker> estade So, I would only suggest one of those things,
which is everything just gets dropped.
[17:58] <GameMaker> estade I think I would not give them any options then
you haven't really solved something
[17:58] <GameMaker> johanneswilm I think that also one of the options could
be, but for certian formats, could we just have extra formats for some
power users.
[17:59] <GameMaker> johanneswilm could we just have this specific menu item
[17:59] <GameMaker> johanneswilm then the ux people can still says this is
a terrible idea
[17:59] <GameMaker> sanketj we're happy to follow up with microsoft UX
people. but we need other companies.
[18:00] <johanneswilm> johanneswilm: continue next time?
[18:01] <johanneswilm> jsbell: we can discuss async via the issue on what
are proposals for ux ppl to look at
[18:01] <johanneswilm> sanketj: sounds good

-- 
---
Johannes Wilm, PhD
tel +46766922220
https://www.johanneswilm.org