Re: Hello & a proposal

I arrived at this group from https://www.w3.org/community/wicg/ where there
is no mention of discourse! It looks more active than this mailing list so
I guess I'll make another post about this there.

Previous discussion on public-webapps starts here:
https://lists.w3.org/Archives/Public/public-webapps/2015AprJun/0908.html

My key takeaways from the feedback were:
- the API should be designed to avoid intermediate copies of the data,
which increase the memory overhead (this sounds like it rules out using a
worker)
- ImageBitmap is probably not a good conversion source, since it may be a
premultiplied resource on the GPU
I incorporated this feedback to the latest draft.

As a web developer working on my own projects, I don't really feel like I'm
in a position to speak for the projects of other web developers. I guess I
read similar blogs and the odd Hacker News post, but I don't feel like I
have an answer to "why do other sites need this?" - am I supposed to be a
spokesperson for all developers? I think this feature is useful in a
general-purpose way to any web app that does any kind of image processing,
but I'm speculating.

Here's a realistic example of our own use case: the user drag and drops a
zip file in to the page. The zip contains 1000 PNGs (this is a realistic
workload in our case, not a number I have just fabricated). I want to
process these images based on their ImageData. To get the ImageDatas,
currently the only solution is loading the blobs by some means, then
synchronous use of drawing then getImageData on a canvas. If you have 1000
images, doing it all synchronously will take so long that either the
browser or OS will assume the page has frozen. It can be sliced up over
separate frames with timers so the browser keeps responding, but you are
still limited by the speed of a single core when the problem is perfectly
parallelisable, and the deliberate idle time to let the browser respond
slows down the overall processing speed. So overall you get to work at
about 75% of the speed of one core, when you could be using all cores. The
async methods I proposed allow the browser to schedule this over all cores,
significantly increasing performance, and saves me having to worry about
how to schedule a long queue of synchronous calls while still allowing the
browser to respond. (requestIdleCallback may help solve this, but still
will use less than one full core.)

All I need for this use case is an async ImageData.create function, but I
expanded the spec to cover more general purpose use cases to do with image
processing. Many of the use cases for getImageData/putImageData can
probably use these methods instead, and avoid synchronous jank. I guess I
should cover this in the spec draft?

Ashley



On 23 July 2015 at 10:08, Yoav Weiss <yoav@yoav.ws> wrote:

>
> On Wed, Jul 22, 2015 at 9:22 PM, Marcos Caceres <marcos@marcosc.com>
> wrote:
>
>> Hi Ashley,
>>
>> On July 22, 2015 at 12:47:49 PM, Ashley Gullen (ashley@scirra.com) wrote:
>> > Hello WICG. This sounds like an interesting idea. I'm new to writing
>> specs
>> > and this sounds like a useful place to participate.
>> >
>> > I wanted to bring up a small spec I drafted up recently:
>> > https://www.scirra.com/labs/specs/imagedata-conversion-extensions.html
>>
>> This already looks like a great start. Nice work.
>>
>
> +1 :)
>
>
>>
>> > I posted it to public-webapps, there was some discussion and feedback, I
>> > incorporated the feedback in to the spec, and then eventually discussion
>> > died off.
>>
>> Ok, let's gather those links so we can see what feedback you got. Please
>> post links to emails here so we can review them together.
>>
>> > I really need the features proposed in that spec, but I'm not really
>> > familiar with the spec process and I don't really know what to do next.
>>
>> Let's start with the above. We need you to articulate why you need it,
>> why other sites need it (maybe even which ones), and if possible we need to
>> show what various sites are already doing.
>>
>
> One thing that need emphasis is when do you need the image conversion to
> run. Is it a low priority task? Do you want to perform it in a worker?
> Should the browser treat it as something that mustn't occupy the UI thread?
>
>
>>
>> Here is a bit more detail about the process:
>> https://github.com/WICG/admin#contributing-new-proposals
>>
>> > Should I hassle implementors? Should I wait and see?
>>
>> Implementers are all here - no need to hassle anyone (hi from Mozilla!) :)
>>
>
> While that's true as far as representation is concerned, once this is
> ready, we'd probably want to grab the right people by the ear and get them
> to review it :)
>
>
>>
>> > Has everyone just
>> > forgotten about it? What's next? Any advice or guidance on where to go
>> next
>> > with my spec draft would be appreciated.
>>
>> We will help you pitch it. The document is missing the use cases, etc.
>> Let's start by seeing what feedback you got already from public-webapps,
>> then we can start bouncing it around a bit more.
>>
>
> I'd say that it'd also be good to move this work to a public repo. It can
> be under your org, or under the wicg's, but it'd make it easier for more
> people to comment, open issues, etc.
>
>