W3C home > Mailing lists > Public > public-webrtc-logs@w3.org > July 2019

[mediacapture-main] Remove "The relative order of the tracks in the set is User Agent defined and the API will never put any requirements on the order.", et al. (#611)

From: guest271314 via GitHub <sysbot+gh@w3.org>
Date: Mon, 22 Jul 2019 14:54:56 +0000
To: public-webrtc-logs@w3.org
Message-ID: <issues.opened-471143001-1563807289-sysbot+gh@w3.org>
guest271314 has just created a new issue for https://github.com/w3c/mediacapture-main:

== Remove "The relative order of the tracks in the set is User Agent defined and the API will never put any requirements on the order.", et al. ==
Specify that when there is both an audio and a video track in a `MediaStream` the order of the `id` must be audio `0`, video `1`. 

Relevant specification language

>  The relative order of the tracks in the set is User Agent defined and the API will never put any requirements on the order.

> The conversion from the track set to the sequence is User Agent defined and the order does not have to be stable between calls.

> Since the order in the MediaStream 's track set is undefined, no requirements are put on how the AudioTrackList and VideoTrackList is ordered

If a `MediaStream` contains only a single track of kind audio or video nothing can be done, besides creating a complementary track for each missing track in the next file.

If a `MediaStream` contains both audio and video then the order described at the first sentence should be applied.

The reason for the change is for post-record editing, which becomes tedious when the order of the tracks is arbitrary.

Consider two `webm` files output by `MediaRecorder` at Chromium

```
$ ./mkvmerge -J 1.webm
{
  "attachments": [],
  "chapters": [],
  "container": {
    "properties": {
      "container_type": 17,
      "is_providing_timestamps": true,
      "muxing_application": "Chrome",
      "writing_application": "Chrome"
    },
    "recognized": true,
    "supported": true,
    "type": "Matroska"
  },
  "errors": [],
  "file_name": "1.webm",
  "global_tags": [],
  "identification_format_version": 12,
  "track_tags": [],
  "tracks": [
    {
      "codec": "Opus",
      "id": 0,
      "properties": {
        "audio_bits_per_sample": 32,
        "audio_channels": 1,
        "audio_sampling_frequency": 48000,
        "codec_id": "A_OPUS",
        "codec_private_data": "4f707573486561640101000080bb0000000000",
        "codec_private_length": 19,
        "default_track": true,
        "enabled_track": true,
        "forced_track": false,
        "language": "eng",
        "minimum_timestamp": 119000000,
        "number": 1,
        "uid": 40889044785409303
      },
      "type": "audio"
    },
    {
      "codec": "VP8",
      "id": 1,
      "properties": {
        "codec_id": "V_VP8",
        "codec_private_length": 0,
        "default_track": true,
        "display_dimensions": "1046x588",
        "display_unit": 0,
        "enabled_track": true,
        "forced_track": false,
        "language": "eng",
        "minimum_timestamp": 0,
        "number": 2,
        "pixel_dimensions": "1046x588",
        "uid": 17593832491204014
      },
      "type": "video"
    }
  ],
  "warnings": []
}

$ /mkvmerge -J 2.webm
{
  "attachments": [],
  "chapters": [],
  "container": {
    "properties": {
      "container_type": 17,
      "is_providing_timestamps": true,
      "muxing_application": "Chrome",
      "writing_application": "Chrome"
    },
    "recognized": true,
    "supported": true,
    "type": "Matroska"
  },
  "errors": [],
  "file_name": "2.webm",
  "global_tags": [],
  "identification_format_version": 12,
  "track_tags": [],
  "tracks": [
    {
      "codec": "VP8",
      "id": 0,
      "properties": {
        "codec_id": "V_VP8",
        "codec_private_length": 0,
        "default_track": true,
        "display_dimensions": "320x240",
        "display_unit": 0,
        "enabled_track": true,
        "forced_track": false,
        "language": "eng",
        "minimum_timestamp": 0,
        "number": 1,
        "pixel_dimensions": "320x240",
        "uid": 33743719209670163
      },
      "type": "video"
    },
    {
      "codec": "Opus",
      "id": 1,
      "properties": {
        "audio_bits_per_sample": 32,
        "audio_channels": 2,
        "audio_sampling_frequency": 48000,
        "codec_id": "A_OPUS",
        "codec_private_data": "4f707573486561640102000080bb0000000000",
        "codec_private_length": 19,
        "default_track": true,
        "enabled_track": true,
        "forced_track": false,
        "language": "eng",
        "minimum_timestamp": 8000000,
        "number": 2,
        "uid": 56209065175931709
      },
      "type": "audio"
    }
  ],
  "warnings": []
}
```

Not only does the number of channels need to be adjusted, though also, to merge the two files the developer needs to map which next track to append to the current track. For only two tracks that can be simple, for multiple tracks in arbitrary order, the task can become meticulous. 

This change will also provide uniformity for all implementations, for example, the ability to merge a `webm` file output by Chromium/Chrome and a `webm` file output by Firefox, setting aside for the moment that Chrome sets `audio_sampling_frequency` to `48000`, while Firefox sets the property to `44100`.

Please view or discuss this issue at https://github.com/w3c/mediacapture-main/issues/611 using your GitHub account
Received on Monday, 22 July 2019 14:54:58 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:22:25 UTC