[webvtt] VTT Type Proposal for time-coded general-flash metadata (#512)

cookiecrook has just created a new issue for https://github.com/w3c/webvtt:

== VTT Type Proposal for time-coded general-flash metadata  ==
Earlier today, Apple released a [new strobing mitigation media accessibility accommodation in tvOS 16.4, iOS 16.4, and macOS 13.3, called "Dim Flashing Lights."](https://developer.apple.com/accessibility/#whats-new) (Additional links below)

Prior to this, the current typical behavior with flashing media is that a viewer is warned *once* before before a video plays. Even in a feature length film, there is no indication of when the flashing may occur or information about the intensity and type of strobing.

On Apple devices, we can now adjust for some strobing/flashing in real time with onboard hardware and software, but as Apple's TV+ content is viewable on many others platforms, we'd also like to be able to do this on non-Apple hardware. 

We think proposing this as an accessibility standard makes sense, as other platforms may wish to adopt their own form of strobing mitigation or warnings.

So we'd like to propose using a WebVTT metadata track to allow streaming services and content owners to denote areas of flashing in video time codes similar to a caption track.

### Minimum proposal
```webvtt
WEBVTT

1
00:00:10.123 --> 00:00:15.432
{
  type: "video.strobing.general-flash",
  level: "75",
}

2
00:00:47.462 --> 00:01:04.028
{
  type: "video.strobing.general-flash",
  level: "100",
}

```

We're not particular to the proposed key names or value formats, but the type value above aligns with an expected update to HLS, and accounts for future expansion into other common strobing patterns such as `red-flash` and `spatial-pattern`.

### Variant proposal that accounts for algorithm and version

Ideally, in order to ensure all content publishers, software developers, and users have the same understanding and representation of the level, the VTT cue format could account for different, documented algorithms and versions for each of those. 

[Apple published a new general flash algorithm (PDF)](https://developer.apple.com/accessibility/downloads/Video-Flashing-Reduction-12.pdf) for use alongside [the related native platform API](https://developer.apple.com/documentation/mediaaccessibility/flashing_lights/responding_to_changes_in_the_flashing_lights_setting) or for [use in other contexts](https://github.com/apple/VideoFlashingReduction/). 

In order to differentiate from other flash pattern algorithms in WebVTT (Harding, for example), the metadata cue could contain a published version and name or URI, such as the following:

```webvtt
WEBVTT

1
00:00:10.123 --> 00:00:15.432
{
  type: "video.strobing.general-flash",
  level: "75",
  test-uri: "https://developer.apple.com/accessibilty/...",
  test-version: "1.0",
}

2
00:00:47.462 --> 00:01:04.028
{
  type: "video.strobing.general-flash",
  level: "100",
  test-uri: "https://developer.apple.com/accessibilty/...",
  test-version: "1.0",
}

```

We're not particular to the proposed key names or values, and look forward to discussing further with the Working Group. 

### Prerequisite:
I have filed another issue we believe is important to consider alongside this one.

- #511


### Other documentation on the recently released feature

- [Dim Flashing Lights Overview](https://developer.apple.com/accessibility/#whats-new)
- [Explainer and Algorithms (PDF)](https://developer.apple.com/accessibility/downloads/Video-Flashing-Reduction-12.pdf)
- [Sample Code: Responding to changes in the flashing lights setting](https://developer.apple.com/documentation/mediaaccessibility/flashing_lights/responding_to_changes_in_the_flashing_lights_setting)
- [GitHub Sample Code: Detection of flashing lights in video content](https://github.com/apple/VideoFlashingReduction/)


Please view or discuss this issue at https://github.com/w3c/webvtt/issues/512 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 28 March 2023 04:50:24 UTC