Re: JPEG 2000 (was: Multiple image files?)

I dug this up a while ago but didn't get too far into really understanding it:

http://people.cs.missouri.edu/~zengw/acmmultimedia01_httpstream_jpeg2k_final_preprint.pdf

Just like with Yoav's Responsive Images Container proposal (http://blog.yoav.ws/2013/09/Responsive-Image-Container), you need two HTTP requests -- in the proposal above, the first is for a separate index file which the client uses to determine what byte ranges to request from the .jp2, and in which order.

Unlike Yoav's proposal, implementing something like this would require clients to have a full understanding of JPEG2000's data structures & multi-resolution capabilities -- a much bigger burden than Yoav's proposal imposes ("simple" layer-compositing, which would work with any underlying format). The win is that you can make requests for arbitrary crops at arbitrary resolutions.

From a pure theoretical-purity-for-a-web-dev standpoint, I think something like this would be ideal:

HTML (pure content!):
<img src="enormous-image.jp2" id="itsBig" alt="Let's say its straight from your 24MP camera or something" />

CSS (pure layout!):
#itsBig {
width: 100%;
overflow: hidden;
}

/* art-direction cropping goes here */
@media (max-width:  40em) {
#itsBig {
margin: -25%;
}
}

With every client requesting just the bytes it needs from the full-res source file.

Allowing arbitrary crops via CSS would require the browser to wait for layout before requesting anything beyond the index… a non-starter. Yoav's fixed-progression-of-layers (which bakes art-direction cropping into the file) ensures that the browser can preload the first layer, knowing that no matter what the layout looks like, it's going to need at least those bytes.

Apparently HTTP2 affords some more efficiency with these sorts of multi-request loads, in ways that I also don't yet understand...


—eric



On Friday, October 18, 2013 at 9:06 AM, Frédéric Kayser wrote:

> Well, Mac users have enjoyed pretty good JPEG 2000 support for years since it is handled by CoreGraphics and used for the icons in OS X and therefore Safari (Mac, not sure about iOS) can display JPEG 2000 images, Adobe Photoshop can produce JPEG 2000 files, there are well-known libs like JasPer (used by ImageMagick, XnView) or OpenJPEG. JPIP (JPEG 2000 Interactive Protocol) image servers are available both commercially and as open source projects (CADI, ESA JPIP server) I don't know if using byte-range requests on a standard http server could do the trick to get some responsive features or if a whole SPIP server is needed. The best is probably to ask the OpenJPEG http://www.openjpeg.org/ and JasPer http://www.ece.uvic.ca/~frodo/jasper/ folks.
>  
> Regards
>  
> Aaron Gustafson wrote:
> > On Friday, October 18, 2013 at 8:55 AM, Frédéric Kayser wrote:
> > > Hello,
> > > I just wanted to share this is small excerpt from the introduction to JPEG 2000 in “Data Compression, The complete Reference” by David Salomon.
> > >  
> > > “One of the new, important approaches to compression introduced by JPEG 2000 is the “compress once, decompress many ways” paradigm. The JPEG 2000 encoder selects a maximum image quality Q and maximum resolution R, and it compresses an image using these parameters. The decoder can decompress the image at any image quality up to and including Q and at any resolution less than or equal R.
> > > In general, the decoder can decompress the entire image in lower quality and/or lower resolution. It can also decompress parts of the image (regions of interest) at either maximum or lower quality and resolution. Even more, the decoder or an image server can extract parts of the compressed stream and assemble them to create a new compressed stream without having to do any decompression. Thus a lower-resolution and/or lower-quality image can be created without the decoder having to decompress anything. The advantages of this approach are (1) it saves time and space (2) it prevents the build of image noise, common in cases where an image is lossily compressed and decompressed several times.
> > > JPEG 2000 also makes it possible to crop and transform the image. When an image is originally compressed, several regions of interest may be specified. The decoder or an image server can access the compressed data of any region and write it as a new compressed stream...”.
> > >  
> > > I'm not saying JPEG 2000 is the answer to all responsive images problems (it has its own drawbacks and it would require real image servers), but when you think about the possibilities and flexibility it offers…
> > > JPEG 2000 has been adopted by the DICOM (Digital Imaging and Communication in Medicine) standard, the more I read about it, the more I think that the web has overlooked JPEG 2000 and it should perhaps get a second chance to correct this “historical error”. The JPEG committee was aware of the design flaws and weaknesses of the original JPEG, that's why they continued their work and proposed new standards: the lossless mode of the original JPEG has evolved in JPEG-LS, and then came JPEG 2000.
> > >  
> > > Why, oh why, are we stuck in the early 90's when it comes to image file formats on the web?
> > >  
> > >  
> > >  
> >  
> >   
> > Agreed. An image file format has always been my preference. I was going down the route of progressive JPEG with a stop loading indicator, but JPEG 2000 sounds pretty awesome. The question is: what is support like?
> >  
> > --  
> > Aaron Gustafson
> > @AaronGustafson
> > aaron-gustafson.com (http://aaron-gustafson.com)