Re: [w3c/FileAPI] Proposal: `Blob.from()` for creating virtual Blobs with custom backing storage (Issue #209)

jimmywarting left a comment (w3c/FileAPI#209)

> I do think further discussion of this proposal needs an explanation of how Blobs that are postMessaged are dealt with.

Yes, i know - i figured that when i wrote this. Sending a `postMessage` with a custom backing store would not be easily serializable, perhaps maybe there could be an internal flag saying that they are not structured cloneable. perhaps maybe the clone could become more like a shortcut / symbolic link so that if you try to read blob with custom storage on another thread, then it would ask the owners thread to supply data back.
One solution to it could be to send a new `MessageChannel` or a transferable `ReadableStream` of some sort when cloning it.
The other thread would just ask the owner: "Please give me the data from A to B and give me back a ReadableStream"

The idea where not just only meant for downloading things, i have had this idea for a long time and also being able to create own blob's in Deno to create Blob/Files backed up by the filesystem that lacks something similar to Node's `fs.openAsBlob()`

i mean, i tried doing something like `Deno.openAsBlob()` 2 days ago: https://github.com/denoland/deno/issues/27664#issuecomment-2888726897
it feels rather hacky - and this solution won't work in browser, cuz they read data more privately. backend servers rely on public properties / methods that can be patched.

--- 

i could also imagine creating a objectURL of an blob with custom backing store where you assign it to a `<video>` seeking in it would be easy to do using 206 partial responses. 

--- 

I once created a [zip reader](https://github.com/jimmywarting/zip-go/tree/main?tab=readme-ov-file#reading-a-zip) that reads the central directory at the very end of the file to get all the entries and return them as File-Like objects

if a file is uncompressed it would be as easy to just [slice a zip](https://github.com/jimmywarting/zip-go/blob/76736bc9acfecf4181cbcf172752e5cd2f44b5be/lib/read.js#L33) file of where it start / ends, but if it is compressed then it would all be async.

i first created this Http-File-Like: https://github.com/transcend-io/conflux/issues/31 a very long time ago and others have done similar [things](https://github.com/ThaUnknown/pwa-haven/blob/main/shared/URLFile.js)

https://www.npmjs.com/package/vinyl - a very large, old popular concept of a very simple metadata object that describes a file, it's essentially just the same thing as a `Blob` with an own custom backing store. I just think it would be cool to standardize them to a actual Blob object such that they can also work together with

- URL.createObjectURL()
- FileReader
- Response()
- Request()
- FormData.append()
- fetch(blob:url),
- Blob() 
- File()
- createImageBitmap
- etc

---

Speaking of IndexedDB and transaction. 
I have seen folks using IndexeDB to store large AI model in IndexedDB. 
if you think about how then need to go about storing it they would 1) first have to download everything into memory 2) and only then can they store that blob.

if you could create a very simple blob with own custom backing store then you would never have to fill up your memory. the file can just be piped directly to the disc

it's a win win. 
sure the transaction would stall for a bit, but i think this solution is actually a good way to pipe data directly into IndexedDB without filling the memory.

but ofc there is now better ways to store it using `Cache` storage or opfs. But the idea of piping data into indexeddb is still a good argument rather than creating a blob in memory first and then save it to indexeddb.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/w3c/FileAPI/issues/209#issuecomment-2892501992
You are receiving this because you are subscribed to this thread.

Message ID: <w3c/FileAPI/issues/209/2892501992@github.com>