Re: [w3c/manifest] feat: add support for adaptive icons (#563)

So there's quite an iceberg of complexity here, and unfortunately this reply is very long.

## High-level semantics

So before we even get into the Manifest syntax, we need to talk about the high-level semantics of adaptive icons.

### Separation of concerns

I think conceptually that we're dealing with two mostly unrelated issues here:
* Separating out the foreground/background layers of an icon, for the purpose of parallax or "pop" animations.
* Masking an icon, where the UA or OS crops out part of a full-bleed image to create an icon of uniform shape.

Either of these makes sense without the other, as shown in this table:

\- | Non-masking (developer-controlled shape) | Masking (OS-controlled shape)
--- | --- | ---
**Non-parallax (single image)** | Traditional icon | iOS
**Parallax (foreground + background)** | ? | Android O

Admittedly, the "non-masking + parallax" option makes less sense than the others, and we don't know of any platform that does parallax without masking, but it is still reasonable. I think it would be helpful if we think of these as two separate (but somewhat related) features.

Another thought I had is that while masking is clearly an important thing on both Android and iOS going forward, the parallax effect seems very much like a gimmick specific to Android that, while fun, is hardly noticeable (the animation almost always takes place *under your finger*) and may not be worth developing web tech for. So let's keep it in mind, but separate out these two concerns.

### Foreground without background

_"A foreground image, which can be considered the main icon, and a background image, which may and may not be used given the platform."_

So there are three possible ways for UAs to display an icon:

* a) just `src`,
* b) just `foreground`,
* c) `foreground` composited on `background`.

Basically, this (and the new examples) suggests that "`src` is the old deprecated way; a new site puts their full logo in `foreground`, with some other background in `background`, keeping `src` merely as a fallback." There are several problems with this:

* Now `foreground` has to be designed to look good by itself or on `background`.
* You can't make an icon where part of your logo is in `foreground`, and part in `background`.

Have a play with [these](https://adapticon.tooo.io/). Look how many of them play with slicing part of the logo in `foreground` and part in `background`, and consider how they would look if you just showed the foreground. Also look at [these examples](https://medium.com/google-design/designing-adaptive-icons-515af294c783).

* Google Maps would be just half the logo in `foreground`.
* Google Hangouts would just be the white quotation marks without any green.
* The Play Store logo in the latter article would just be a white square with a transparent triangle cut out of it.
* I made my own *totally not official* [adaptive Chrome logo](https://adapticon.tooo.io/#/bg=https://i.imgur.com/WbkYq4b.png/fg=https://i.imgur.com/dM1Jmzo.png) (based on [this SVG](https://vignette.wikia.nocookie.net/logopedia/images/c/ca/Google_Chrome_for_Android_Icon_2016.svg/revision/latest?cb=20160305053723)). The `foreground` is just the blue circle, which doesn't form a logo on its own.

Basically, we can't ask developers to design a `foreground` + `background` logo for masking, and then allow UAs to optionally strip out the background. We should require that UAs either use `foreground` and `background` together, or use `src` and ignore the adaptive icon.

### Masking and proportions

This is a huge source of complexity (and confusion), and I'll note up front that we could reduce a lot of the complexity here if we had masking, but not parallax.

_"The inner 2/3 of the images appears within the mask. Thus, that part is what makes up the final image, i.e. by default there will be a margin of 1/6 of the size. For special animations, each layer may move up to 1/3 of the icon size in each of the directions."_

This isn't nearly detailed enough, and also it's straight out of the Android Oreo spec. I'm wary of committing to the Android proportions, because they may not match how other platforms treat icons, and also Android may change in the future.

For some reason the official Android docs don't tell you this, but [this Medium article](https://medium.com/google-design/designing-adaptive-icons-515af294c783) does: the icon space has two key zones: the "dead zone" and the "safe zone".

* The "dead zone" is a square outer border of 1/6 of the icon size on each side. All pixels in this zone are guaranteed to *never* be seen in any mask, except when animating/parallaxing. The inverse of the dead zone (the pixels that *may* be seen) is a square of 2/3 of the width and height of the icon.
* The "safe zone" is a circle of 11/36 of the icon size in radius. All pixels in this zone are guaranteed to be seen in all masks.

![Safe zone within a rounded square mask](https://cdn-images-1.medium.com/max/800/1*d3ieGM2uVdKjrlF9y-G-0w.png)

We need to spec something similar, specifying both the dead and safe areas. This allows developers to design the full-square version of their logo within the dead zone (making sure any important parts are also within the safe zone), and then extending the edges of the image outwards to cover the dead zone. That's what I did with my Chrome logo experiment above:

Background:
![Adaptive Chrome logo, background](https://i.imgur.com/WbkYq4b.png)
Foreground:
![Adaptive Chrome logo, foreground](https://i.imgur.com/dM1Jmzo.png)

We would also need to spec the "default" representation of an icon (i.e., what the icon looks like without parallax) as an algorithm, something like this:

> 1. Start with the background image.
> 2. Composite the foreground image onto the background.
> 3. Crop out 1/6 of the image size on each side (the dead zone).
> 4. You MAY apply a mask of any size, making any pixels that are more than 11/36ths of the image size away from the center (the safe zone) transparent. You MUST NOT make any pixel within the safe zone transparent. It is RECOMMENDED that you make all such pixels transparent (i.e., a circle).

The big unknown for me is, how do we specify the dead zone in a way that isn't locked to Android Oreo's specifications? What if a future platform comes along where the dead zone is 1/2 the width, instead of 1/3? We have to decide on a particular number for the dead zone. We can accommodate any platform with a smaller dead zone than what we choose, by cropping. We can't accommodate a platform with a bigger dead zone. Oreo has a fairly big dead zone, so we could just go with 1/3, but it is arbitrary and the only reason we chose it is because of Oreo.

If we were *only* doing maskable icons (not parallax), then not only would we not need separate background/foreground images, but we also wouldn't need to spec dead zone, only safe zone. This would remove steps 2 and 3 above, and avoid the arbitrary decision about how big to make the dead zone. Then the Chrome logo would just be this:

![Maskable Chrome logo](https://i.imgur.com/3aYQPyA.png)

Which frankly is much easier for developers to understand, and we wouldn't need to have 55% of the image cropped out in almost all cases.

## Manifest syntax

OK onto the actual details of the Manifest.

Dom and I discussed this and we're not really fans of just shoving the new adaptive icons alongside the existing `src`. For one thing, it means each adaptive ImageResource needs a corresponding `src`. What if you supply an adaptive ImageResource at a different size to the rest of your traditional images? What if you want to supply an SVG ImageResource (with "sizes: any") while your traditional images are PNGs (with multiple resources of different, fixed size?)

It's weird for there to be a "type" member for ImageResource, when an ImageResource can now contain three different images, potentially with different types.

Further, I can see what you're trying to accomplish with "steps for inferring background", but it seems quite arbitrary to always bundle the foreground and background in the same ImageResource, *except* in this special case, where you provide a separate scalable background, with a magic algorithm that associated a scalable background with all of the other images that have no background. Does that special IR with the scalable background require a scalable "src"? The algorithm totally ignores "type", "purpose" and "platform" of the scalable background when it copies it into the fixed-size ImageResources.

Basically, I think one ImageResource should represent one image file. The little-used "purpose" member should be used to indicate adaptive icons.

We propose that instead of adding "foreground" and "background" members, we just add new "purposes" of "foreground" and "background", using the "src" member. This means if you want an adaptive icon, you specify three ImageResources (x the number of sizes you need):

```json
{
  "icons": [
    {
      "foreground": "icon/fg.png",
      "background": "icon/bg.png",
      "src": "icon/fallback.png",
      "sizes": "432x432",
      "type": "image/png"
    }
  ]
}
```
becomes:
```json
{
  "icons": [
    {
      "src": "icon/fallback.png",
      "sizes": "432x432",
      "type": "image/png"
    },
    {
      "purpose": "foreground",
      "src": "icon/fg.png",
      "sizes": "432x432",
      "type": "image/png"
    },
    {
      "purpose": "background",
      "src": "icon/bg.png",
      "sizes": "432x432",
      "type": "image/png"
    }
  ]
}
```

You would still be required to supply a non-purposed icon as the "main" icon, and then optionally specify an alternate "adaptive" icon in the form of a foreground and background purpose. Both of these are optional, but if one is missing, it would just be equivalent to fully transparent. (So, you can make a non-parallaxing but still full-bleed-and-masked icon by supplying just a background.)

This also means you can mix and match sizes arbitrarily. You could supply just a high-res background and have it scaled by the UA, while the foreground is specified in 10 different sizes. Or you could supply an SVG foreground with many fixed-size backgrounds. So you get your "steps for inferring background" for free.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/w3c/manifest/pull/563#issuecomment-351290827