RE: Open issues in the WOFF 2 draft spec

Hi Raph,

Thank you very much for such a rapid response to the issues raised in yesterday’s call, really appreciate it. And huge thanks to Zoltan for his  optimizations and the implementation of the proposed changes.

I’ve given another thought to the question about having table directory as part of the compressed stream vs. file header, and I think there may be a benefit in enabling WOFF2 client to parse the table directory content without decompressing the whole font, and considering that the byte count savings if we were to compress table directory would be insignificant, I am going to withdraw my suggestion to move it to the compressed font data and agree with you that we should keep it in the file header. This makes the question about known table tags quite relevant and the ability to extend the list of known table tags in the future is a nice feature. I’d recommend that we should go ahead and add all known table tags leaving using one of the free flag bits and keeping the last one reserved – I expect that the newly added tables will be officially ‘blessed’ at the ISO meeting next week.

Thank you for agreeing to making ‘glyf’ table transform mandatory, as this will make the compression more efficient (and eliminate ‘loca’ table from the compressed stream altogether). I don’t mind keeping the explicit transform bit flag – it doesn’t really affect anything and may turn out to be a useful feature if we ever come up with a way to transform other tables and squeeze few more bytes out of font data.

Regarding the UIntBase128 data type – with the header changes we already adopted (moving compressed length field to the WOFF2 header) there seem to be only one “original length” field left [plus the transformed length for ‘glyf’ table only] where this datatype is used. According to your own evaluation, using this datatype saves us only about 40+ bytes for a font – this is as much as can be saved by eliminating bounding box info if we apply glyf transform on a font subset containing only 5-6 glyphs (and much more if we consider other savings such as loca table elimination). I am not arguing against using UIntBase128, I am just looking for a proper justification, as I am sure these questions  will be raised in the future during the FPWD and Last Call reviews.

Thank you,
Vlad


From: Raph Levien [mailto:raph@google.com]
Sent: Thursday, March 27, 2014 2:38 PM
To: public-webfonts-wg@w3.org
Subject: Open issues in the WOFF 2 draft spec

Hi all,

   Thanks again for the great discussion in yesterday's conference call. It felt like we got to some really meaty issues, as well as how to word the spec properly and clearly. It also kicked off a flurry of activity on the compression and Chrome teams. We have a window where making relatively simple changes is not particularly painful for us. Now that we have a concrete spec proposal and an engaged discussion, I think we can focus on making the best spec possible, and hopefully still arrive at a consensus reasonably quickly.

   Zoltan agreed with Jonathan Kew's suggestion to move the compressed stream size out of the table directory and into the WOFF 2 header. As Jonathan said, this makes the format cleaner, has fewer combinations of flags that need to be checked, and frees up at least a bit or two to expand the known tags table. Zoltan suggests the following concrete proposal:

"I think the proposal that moves the compressed size to the header would be a huge win in the clarity of the spec. To future readers who are not familiar with the evolution of the format, the current state is embarrassingly complicated.

While I was at it, I also moved the transform bit from bit 5 to bit 7 and made bits 5 and 6 reserved for future use, now set to 0. This way, if there is an agreement to extend the list of known tables, it will be very easy to do in a backward compatible way."

   Not only that, but he went ahead and implemented it! The patch is here:

https://code.google.com/p/font-compression-reference/source/detail?r=22e2c328288610e63c03e8de0b9c18b1b926cff6


   This seems like a good solution to me, and, I believe it addresses Jonathan's concerns. I'd love to get feedback on this.

   The next open issue was expanding the known tags table. I thought I heard consensus for including including all three current color font proposals. I believe the relevant tags are:

Google proposal (https://color-emoji.googlecode.com/git/specification/v1.html)
CBDT
CBLC

SVG in OpenType (http://dev.w3.org/SVG/modules/fonts/SVG-OpenType.html)
SVG

Microsoft proposal (can't find authoritative reference, but http://blog.fontlab.com/font-tech/color-fonts/color-font-format-proposals/ has the info)'
COLR
CPAL

   What other tags make sense to include?

   The next issue was making the transform mandatory for the glyf and loca tables. I'm fine with this. I agree with the principle of making as few things optional as possible, unless there really is some compelling reason, and don't think non-transformed glyf rises to this level. As Vlad pointed out, for use cases that would benefit from lower cost processing than WOFF 2 encoding or decoding at the expense of file size, WOFF 1 still suffices well.

   I suggest that we keep the explicit transform bit in the flags word, as I think it makes the logic for parsing the table directory simpler, but I'm open to argument. Much of the language in the "constraints" section will become simpler, as it doesn't need to describe case analysis on whether the glyf table is transformed.

   The next question was whether to move the table directory inside the compression envelope. I'm going to argue against it, but again am very open to discussion.

   Based on simple analysis, I think the difference in file size will be negligible at best. Looking at one font (Open Sans encoded with the most recent font-compression-reference source), the table directory is 53 bytes. There is almost no redundancy in there, as it's already highly compressed. I think there is value in being able to extract certain information out of the font without having to start up a decompression stream. That information includes how much memory needs to be allocated for decoding, and also, for example, whether the font is TrueType, OpenType, or one of the color formats. As we discussed in the call, there is already a well-justified decision to have other information (the metadata) out of the compression envelope, so I don't see it making a difference one way or other for the cleanliness of the spec.

   Finally, I want to address a question that came up that I don't think are tied to potentially incompatible changes. Vlad asked for a justification of the UIntBase128 encoding for lengths. Removing this and using 32-bit ULONGS would roughly double the table directory size, bringing it from 53 to 98 bytes for the font I measured. If necessary, we could run experiments over a larger corpus, but I expect extremely little variation. incidentally, removing the "known tags" optimization would add 57 bytes (under the conservative assumption that we could get rid of the flags byte altogether now).

   How close are we? I think and hope pretty close. I'd really appreciate feedback on these open questions, and of course comments on the draft spec itself. I'm also happy to answer questions if there's anything I've missed.

Thanks,

Raph