RE: IMSC-HRM test suite

Hi Nigel,

Yes, the HRM tool has been useful in managing the complexity of transcoded content providing concrete evidence that transcodings were “too inefficient”.

I did not experiment with handcrafting documents that are just on one side or the other of conformance.  But from a gross view, the HRM worked as expected and there was no evidence that there is any problem.

It is a valuable tool to explain to transcoder manufacturers that the IMSC1 documents are “too complex” without having to hand-wave or point to commercial decoders that cannot properly render them.

Regards,
              Mike

From: Nigel Megitt <nigel.megitt@bbc.co.uk>
Sent: Tuesday, October 31, 2023 10:19 AM
To: Michael Dolan <mike@dolan.tv>; TTWG <public-tt@w3.org>
Subject: Re: IMSC-HRM test suite

Mike,

Thank you for that. It sounds to me like the HRM analysis was useful in this case.

Would you agree that this does not suggest a problem with the HRM algorithm or implementation?

Kind regards,

Nigel


From: Michael Dolan <mike@dolan.tv<mailto:mike@dolan.tv>>
Date: Tuesday, 31 October 2023 at 17:10
To: TTWG <public-tt@w3.org<mailto:public-tt@w3.org>>
Subject: RE: IMSC-HRM test suite
Resent from: <public-tt@w3.org<mailto:public-tt@w3.org>>
Resent date: Tuesday, 31 October 2023 at 17:10


External: Think before clicking
TTWG,

I recently exercised the HRM tool with ~3-minute clips of 2-second-duration MP4 segment files that were transcoded from a comprehensive CEA 608 MP2TS test file.  Two commercial transcoders were used to convert the file with the following results using https://hrm.sandflow.com/



1. A “normal” transcoder created IMSC1 segments with 2-sec duration with a “paint-on” of additional text over the 2 seconds, but no screen clearing. No HRM errors were reported.



2. An “unusual” transcoder created IMSC1 segments with all text re-drawn throughout the segment every 30-60ms. The HRM reported that every document was non-conformant.

FYI,
              Mike


From: Nigel Megitt <nigel.megitt@bbc.co.uk<mailto:nigel.megitt@bbc.co.uk>>
Sent: Friday, July 14, 2023 8:56 AM
To: TTWG <public-tt@w3.org<mailto:public-tt@w3.org>>
Subject: IMSC-HRM test suite

TTWG,

you may have noticed that Pierre opened a pull request https://github.com/w3c/imsc-hrm-tests/pull/1 to add IMSC HRM Text tests to the IMSC-HRM test suite.

These don't form a normative part of the specification, and we can see them as a starting point for the test suite, to give us a basis on which to iterate if we find any improvements that can be made.

In order to move IMSC-HRM on, and meet the CR exit criteria, we need implementations. To that end, Pierre has drafted an invitation calling for implementations and/or content to use as evidence of implementations, which can be found at https://docs.google.com/document/d/1J0kPIKXyS7RT_nTA4bYueC0e1AoatyLeHkWtr_5dZzc


That call for implementations includes a reference to the test suite: it would be helpful for that test suite not to be empty when we share it!

Our normal working mode is to treat the opening of a PR as a call for consensus, and begin the 10 working day countdown for the Decision policy at that time.

Given how near we are to the holiday season, and the nature of this pull request, I propose that we aim to merge the pull request a week earlier than that, exceptionally, so that we can also begin the process of sharing the call for implementations with our contacts.

If you would like more time to review the details of the pull request, please let me know as soon as possible, otherwise I hope we can agree during our next call on 2023-07-20 to merge it early. If you're happy to merge it sooner, please add an "approve" review to the pull request.

If you have any comments or suggestions for change on the call for implementation text, please use the Google doc facility to do that; I will also include it on the agenda for next week's call.

Nigel