Re: [EXTERNAL] [jfraichot@learningmachine.com] Re: VC API: handling large documents client to server

+1 Orie - there are definitely ways we can tackle this in the new data model

Mike Prorock
mesur.io

On Thu, Feb 10, 2022, 11:37 Orie Steele <orie@transmute.industries> wrote:

> base64url encoding is a major problem for signatures over complex data...
> when there is no applied compression.
>
> This needs to be fixed in the next version of the VC Data Model, if VC-JWT
> is to be useful for large credentials.
>
> Luckily there is
>  https://www.iana.org/assignments/jose/jose.xhtml#web-encryption-compression-algorithms
> <https://www.iana.org/assignments/jose/jose.xhtml#web-encryption-compression-algorithms>
>
> Regards,
>
> OS
> ᐧ
>
> On Thu, Feb 10, 2022 at 10:00 AM Julien Fraichot <
> Julien.Fraichot@hyland.com> wrote:
>
>> Thanks Manu for the great write up,
>>
>>
>>
>> It’s true that Blockcerts historically has made heavy use of base64
>> content as part of the data, and it was a partial problem since we didn’t
>> have much transport. Now that we are considering the problem it’s great to
>> hear what could be the industry standard, so I will try out the different
>> things you suggest.
>>
>>
>>
>> *From: *Mike Prorock <mprorock@mesur.io>
>> *Date: *Thursday, 10 February 2022 at 15:49
>> *To: *Manu Sporny <msporny@digitalbazaar.com>
>> *Cc: *W3C Credentials CG <public-credentials@w3.org>
>> *Subject: *[EXTERNAL] [jfraichot@learningmachine.com] Re: VC API:
>> handling large documents client to server
>>
>> *CAUTION: *This email originated from outside of Hyland. Do not click
>> links or open attachments unless you recognize the sender and know the
>> content is safe.
>>
>>
>>
>> +1 manu - great notes - I will note from our side that we utilize
>> hashlink (and related approaches) when dealing with larger binary data that
>> needs to be referenced in a credential - e.g. image of a product, etc.
>>
>>
>> Mike Prorock
>>
>> CTO, Founder
>>
>> https://mesur.io/
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Feb 10, 2022 at 9:39 AM Manu Sporny <msporny@digitalbazaar.com>
>> wrote:
>>
>> On 2/10/22 3:01 AM, Julien Fraichot wrote:
>> > Basically when calling a verification API from a (browser) client, there
>> > might be times where the documents could be quite large (few MBs). I am
>> > wondering if there are some strategies to reduce the payload that would
>> > also be standard when dealing with a VC API complying service?
>>
>> If you have a VC that is several MB in size, I would expect it to
>> struggle in
>> the ecosystem. Yes, they are legal, in the same way that attaching a 250MB
>> slide deck to an email is legal -- while the SMTP protocol allows for it,
>> most
>> mail servers will reject the message as too large.
>>
>> Typically, these large VCs happen because people are embedding
>> base-encoded
>> images directly into a VC. Instead, VC creators should consider modelling
>> their data differently -- e.g., use a hashlink, or some other way of
>> creating
>> a cryptographic hyperlink:
>>
>> https://datatracker.ietf.org/doc/html/draft-sporny-hashlink
>>
>> > I am researching gzipping on the client and tried more exotic
>> approaches to
>> > no avail, so I’d be willing to hear what the people have thought on the
>> > matter.
>>
>> You get gzip compression during transmission (more or less) for free these
>> days, but that's not really going to save you given that you're probably
>> base-encoding the raw binary data. Quite counter-intuitively, doing that
>> makes
>> using gzip expand the file size instead of reducing it:
>>
>>
>> https://stackoverflow.com/questions/38124361/why-does-base64-encoded-data-compress-so-poorly
>>
>> This is another reason the JOSE/JWT stack, when used with VCs, harm
>> wire-level
>> protocols -- everything is base64 encoded, and thus it effectively
>> destroys
>> any ability to compress data on the wire.
>>
>> Typical solutions to this problem require that you put the binary data
>> outside
>> of the VC, if at all possible. This works well for common static images
>> such
>> as logos. It is also possible to split the VC into two VCs... one with the
>> machine-readable data from the issuer (with a digital signature) and one
>> with
>> the image data from any source (without a digital signature, since, if
>> hashlinked, the signature will verify the validity of the image data).
>> That
>> latter approach can be more privacy preserving AND more complex than many
>> might feel is necessary.
>>
>> Selective disclosure schemes (such as BBS+) are another way to deliver a
>> subset of the information to a verifier without having to send the image
>> payload data.
>>
>> I expect this to be an active area of innovation for the next few years
>> with a
>> few proposals on standard design patterns that all industries could use.
>> This
>> problem appears most often with identification cards that have biometric
>> images embedded in them.
>>
>> -- manu
>>
>> --
>> Manu Sporny - https://www.linkedin.com/in/manusporny/
>> Founder/CEO - Digital Bazaar, Inc.
>> News: Digital Bazaar Announces New Case Studies (2021)
>> https://www.digitalbazaar.com/
>>
>> ----------------------------------------- Please consider the environment
>> before printing this e-mail -----------------------------------------
>>
>> CONFIDENTIALITY NOTICE: This message and any attached documents may
>> contain confidential information from Hyland Software, Inc. The information
>> is intended only for the use of the individual or entity named above. If
>> the reader of this message is not the intended recipient, or an employee or
>> agent responsible for the delivery of this message to the intended
>> recipient, the reader is hereby notified that any dissemination,
>> distribution or copying of this message or of any attached documents, or
>> the taking of any action or omission to take any action in reliance on the
>> contents of this message or of any attached documents, is strictly
>> prohibited. If you have received this communication in error, please notify
>> the sender immediately by e-mail or telephone, at +1 (440) 788-5000, and
>> delete the original message immediately. Thank you.
>>
>
>
> --
> *ORIE STEELE*
> Chief Technical Officer
> www.transmute.industries
>
> <https://www.transmute.industries>
>