Re: Retargeted images redux: how it might actually be used on the web

On Mar 27, 2008, at 5:32 PM, Ben Darlow wrote:

>
> Previously (http://lists.w3.org/Archives/Public/www-style/2008Mar/0324.html 
> ) I discussed the subject of retargeted images as a CSS property,  
> which could be a useful way of flexibly incorporating photographic  
> or complex images into layouts where images can't always be  
> constrained to a given size. Considering my poor timing to suggest  
> this at the same time as David Hyatt posted his long list of very  
> interesting feature proposals based on work with WebKit, it's not  
> too surprising that my suggestion didn't really register on the  
> radar. So, rather than let it just slide I thought I'd rephrase my  
> suggestion, along with an example of how such a feature might be  
> useful, and a possible syntax for usage (feel free to shoot me down  
> on the latter; this is new territory for me).
>
> To illustrate the possible usage benefits, consider this crude  
> thumbnail representing a typical website layout: http://www.kapowaz.net/images/retargeted-320.png 
> . In this layout, a photo (!) frames the content at the top, below  
> which is a navigation bar, and two columns of content; one a fluid- 
> width text column, and another a fixed-width column with a logo and  
> other related content. If the photo itself has a fixed width, this  
> layout scheme is problematic; you can choose to either use a  
> background-image, whereupon detail at either side must be sacrificed  
> once the window is shrunk, or you must suffer horizontal scrollbars  
> (or worse, if the window is made larger than the width of the image,  
> unsightly blank space), or simply rescale the image down to a  
> suitable size.
>
> What image retargeting (or 'seam carving': http://www.seamcarving.com/) 
>  does is remove or add parts of the image in locations of lower  
> significance in order to allow the image to be retargeted to  
> required dimensions. How this might work in the context of the above  
> thumbnail is shown in http://www.kapowaz.net/images/ 
> retargeted-250.png. Here, you may notice that the aspect ratio of  
> individual elements of the image of significance — the mountain  
> range, birds and sun in the sky — has been maintained, but the scene  
> itself has been made to fit the available page width.
>
> The way I envisage that such a technique could be implemented would  
> require a new CSS attribute for image elements, for instance:
>
> #heading img {
> 	retarget-direction: horizontal;
> 	retarget-max-width:1000px;
> 	retarget-min-width:400px;
> }
>
> The retargeting can be applied in both axes, so the value of  
> retarget-direction could be horizontal, vertical or both. The  
> attributes for maximum and minimum width would be used to prevent  
> artefacts which tend to occur when an image is retargeted to extreme  
> sizes (both small and large). An image element which is being  
> retargeted would have a block display mode, and would automatically  
> expand to the full width of its containing element.
>
> Attempting to implement this as a patch for any of the open source  
> browsers out there is really a task beyond me, but if anybody is  
> interested at least one individual has already implemented the  
> technique using the SDL library (http://kometbomb.net/2007/09/04/image-retargeting/ 
> ). I would imagine that a native implementation in one of the modern  
> browsers would be able to perform such transformations vastly more  
> quickly than the alternative (Flash, e.g. rsizr.com).
>
> All comments and suggestions welcomed!

I think this is a neat idea.  It does seem to me like only one CSS  
property would be necessary though.  Really isn't this just about  
picking a scaling algorithm for an image, e.g.,

image-scale-algorithm: seam-carving;

?

WebKit actually already supports two scaling modes (although the  
ability to use the low quality interpolation mode is not exposed to  
CSS but is instead WebKit API).

Limiting the scope of the seam carving could just involve actually  
clamping the size of the object where the image is used, e.g., with  
normal max-width/min-width properties.

One issue with adding properties that affect images is that you run  
into the problem of wanting to do the same thing with images specified  
in CSS, such as background images, list images and border images.

We've run into the same issue with the desire to add support for  
compositing modes to images.   Right now we have a composite attribute  
extension on our <img> element in WebKit, but we ended up having to  
add a -webkit-background-composite property just for background images  
to accomplish the same thing.  I hate this, which is one reason you  
haven't heard me propose it to CSS yet.

There are other "controls" one could envision wanting to add per  
image, such as more control over how images animate.

My idea for how to do this would involve pointing to a new kind of rule:

background-image: image(foo);

with a block like:

@image foo { image-scale-algorithm: seam-carving; }

Or perhaps a more generic name like "media-resource" would be  
appropriate for video and audio as well.

Is this algorithm unencumbered and freely implementable?

dave
(hyatt@apple.com)

Received on Friday, 28 March 2008 19:09:41 UTC