bytes type

[cameron1024]

I'm not sure if this has been discussed before, apologies if this is a duplicate.

My team is evaluating surrealdb for our application, which involves storing quite a lot of raw byte arrays (i.e. Vec<u8>). Currently, it seems like the best way to do this in surrealdb is to base64 encode the bytes and store them in a string. This works, but isn't ideal for us, since we're storing quite a lot of data, and this is a bit inefficient (plus we now need a little bit of logic to handle the base64 decoding/encoding).

Firstly, is there a better way to do this? From what I can tell, the documents are serialized to json before sending to the database, so a simple array of integers isn't going to be space-efficient (since it presumably needs space for tags). Is this assumption correct?

Secondly, if I am correct that base64 encoded strings are the best way to store binary data, is there scope for a new type for raw byte arrays (perhaps bytes). If so, I'd be happy to work on such a feature, since it's pretty important for my company's use case.

[tobiemh]

Hi @cameron1024, we'd definitely like to add a raw binary field type. Do you have any ideas how you would like this to work?

[cameron1024]

I guess something similar to the BYTEA type in postgres makes sense - i.e. a string without any requirements placed on its encoding.

Without having dived too deeply into the code, it looks like simply adding a variant to Value like Bytes(Vec<u8>) would probably slot in pretty nicely.

[realonbebeto]

I share in this concern. I don't know if this has been handled explicitly or implicitly because Vec (bytes) in Rust is not being serialised/deserialised as needed to/from bytes in surrealdb.

Any updates?

[tobiemh]

Hi @cameron1024 and @realonbebeto there ia already a bytes type in SurrealDB. It's been there since SurrealDB 2.0 I think, and is optimised in SurrealDB 3.0!

[tobiemh]

@realonbebeto the new SurrealDB 3.0 Rust SDK handles byte types correctly, so byte types should be handled correctly now.

[realonbebeto]

Thank you @tobiemh

[Ekleog]

Seems like there were no news on this topic since march, so I'm going to up it :)

I'm coming with the same idea: is there a way for me to store images inside surrealdb? I'd rather avoid base64 encoding; and don't really care about any search semantics: I'd just need it to behave like a key-value store, basically.