GrB_Vector_(de)serialize

[victorstewart]

any fundamental reason why these functions don't exist, or just simply "on the docket"?

[DrTimothyAldenDavis]

They should exist, in my opinion. See GraphBLAS/graphblas-api-c#36 . I only follow the spec, and I've put in an issue to the GraphBLAS C API committee to consider adding them. I've added my own as GxB_Vector_serialize and deserialize.

[DrTimothyAldenDavis]

I closed this by mistake ... should be left open until the C API spec is updated.

[victorstewart]

sounds good.

i prepend the size of the serialized buffer first, then the buffer, then align the tail to 2MB as i'm using O_DIRECT. so when i control the memory it's as simple as a single meta write. but when i don't control the memory i have to do it in up to 3 stages (head + body + tail). so just makes the code more unnecessarily complex though still less than 40 lines. and 10 lines the other way.

[victorstewart]

oh and thanks for these functions! allowed me to delete the multi hundred line serialization and deserialization functions i'd written myself. less code is always more.

[DrTimothyAldenDavis]

Glad to help! Yes, the serialize/deserialize methods are very important. I put a lot of work into them to make the serialized blobs as compact as possible, now with ZSTD compression by default.

I hadn't considered the idea of rounding up the size to a multiple of 2MB, but then that would only make sense if all you had was the blob ... but you want to (naturally) add the blobsize at the front.

Actually, the first 8 bytes of the serialized blob contains the size of the blob itself. But the blob is supposed to be opaque, and that feature of my blob is not supposed to be visible to the user application. So you can't rely on that fact, unfortunately.

See this line for the first thing in my blob header:

GraphBLAS/Source/GB_serialize.c

Line 301 in b5ae1ed

GB_BLOB_WRITE (blob_size_required, size_t) ;

... shhhh ... you didn't see that line of code :-)

[victorstewart]

i just ran into a need for this. when forking, to then serialize the copy-on-write copies of the matrix and vector objects into IPC shared memory segments between the parent and child process, there's of course no way with the current API to instruct GrB_Vector_serialize to write into the shared memory region.