Blob and Files

[jimmywarting]

Hi, I'm coming from the NodeJS world and help maintain node-fetch and fetch-blob
I have also helped and reviewed NodeJS buffer.Blob implementations so it follow the spec more correctly.
NodeJS are now looking into adding something like our blobFrom(path) implementation from our fetch-blob/from.js and create something like fs.blobFrom(path)

I hope to migrate to Deno sometime and I would like to see your blob implementation being improved upon. I also hope to see something like the native file system spec being implemented so it would be possible to retrieve a File from the FileSystem and have something like Blob and files backed up by the FileSystem that actually extends the Blob class. (your blob impl can't really do that atm)

There is a few things about your blob implementation that i have some thoughts on...

• The first and very simple easy thing that are missing is symbol.toStringTag on both Blob and the File Class
• The way you handle blobs underlying byteSequence isn't really how the spec explains it - You handle it as single Uint8Array Buffer container internally that holds all of the data. I know the spec says something in terms of "append the bytes it represents to the byteSequence" - it can be a bit fuzzy, but what it actually means it that a blob part can come from any arbitrary place, it could come from the filesystem, memory (like string, arrayBuffer, arrayBufferViews) and also from other slices of another blob or file.

When browser creates Blob/Files from the filesystem then it just merely stat the file for the size and last modified date and creates a dummy blob (called BlobDataItem in chromium) that don't have any underlying data held in the memory
When you slice this file then you are not really copying/reading the data
The only thing you should actually do is: Create a new Blob with a references to the old Blob and change the size and offset from where it should start reading from and where it should end.

We also treated our blob data as a single buffer but it all changed when we instead operated on a blob parts sequences and had to rewrite the hole slice method b/c of parts that now came from the file system and not only from the memory.
The benefit of this lead to a memory reduction and slicing a blob from memory would not clone the data - it only created a new references point from where it should start/stop reading another blob from. This is more or less how the browser dose it behind the scenes

[lucacasonato]

Hey @jimmywarting

The first and very simple easy thing that are missing is symbol.toStringTag on both Blob and the File Class

Surprised there is no web platform test for this. Anyway, should be super simple to fix,

The way you handle blobs underlying byteSequence isn't really how the spec explains it - You handle it as single Uint8Array Buffer container internally that holds all of the data. I know the spec says something in terms of "append the bytes it represents to the byteSequence" - it can be a bit fuzzy, but what it actually means it that a blob part can come from any arbitrary place, it could come from the filesystem, memory (like string, arrayBuffer, arrayBufferViews) and also from other slices of another blob or file.

This is on our radar. Currently blobs are always fully stored in isolate memory. Our implementation is technically spec compliant (we pass the web platform tests), but it is not ideal and not what the spec intends. In the future we want to transparently store blobs on disk after a certain size threshold of in memory storage has been reached (similar to how it is done in Chromium: https://chromium.googlesource.com/chromium/src/+/master/storage/browser/blob/README.md). I am planning to tackle this either this, or next quarter. Once we have the ability to asynchronously stream large blobs to/from disk (for example new Response().blob()), the infrastructure will be in place to have blobs backed by other asynchronous data sources.