Design choices and new features for big upcoming update to bnum

[isaacholt100]

Hello everyone! I'm opening this discussion as I have been working on a new version of bnum which will include the biggest changes to the crate so far, and I would love to hear everyone's input on what various design choices which I'm thinking of adding/changing. I'll list the things I'm thinking of changing/adding below: comments on any/all of them would be much appreciated.

Space efficient types that are at least as fast as BUint and BInt

The biggest change will be that there will only be one unsigned integer type and one signed integer type. These will effectively be the same as BUintD8 and BIntD8, with the same API and layout (i.e. an array of u8 digits). The important difference will be that, where relevant, methods will iterate over batches of 16 u8 digits at a time, rather than 1 digit at a time. This effectively means the integer will in fact operate on u128 digits (16 * 8 = 128) and will use methods on u128 rather than u8 ints. To give an example, the BitAnd implementation is a simple digit-wise logical AND. Rather than computing AND for each pair of u8 digits, we compute AND for each pair of "effective" u128 digits (formed from 16 consecutive u8 digits). It doesn't matter if the total number of bits is not a multiple of 128, as the last effective u128 digit will be leading zero-padded.

• The main benefit is that this new type will be as fast as or faster than even a BUint (u64 digits) of the same size (since we are iterating over 2 times fewer effective digits), while retaining maximum control over the size of the integer. Other benefits are reduced crate size and faster compile times (as there are 4 times fewer integer types), and being able to make the {from,to}_{b,l,n}_e_bytes available on stable.
• The only downside I can think of is that computations with these would require slightly more space (e.g. if you want a U24, it would compute using one u128 rather than 3 u8s), but the integer itself would be stored with the desired number of bytes.
• Something to decide on is what to call these new integer types: BUintD8 and BIntD8 seem unnecessary, as now these are the only two integer types. But BUint and BInt might cause confusion as these currently use 64 bit digit, not 8. So perhaps just Uint and Int.

Allowing arbitrary bit widths

Currently, the bit width has to be a multiple of 8. If being able to specify an arbitrary bit width as another const-generic parameter is of interest, I would consider adding this. If this was implemented, there would be some decisions to make about the API:

• Until the Rust feature generic_const_exprs is stabilised, a const-generic parameter specifying the number of digits would be required as well.
• Currently, I'm thinking the best option is for the type to be Uint<Digits, Bits>, where Bits is by default 0, which would signify that the integer uses Digits * 8 bits (we can't have the default be Digits * 8 as this requires generic_const_exprs. The only thing is having 0 to mean Digits * 8 bits looks a bit strange (although a 0 bit width integer is not needed so maybe not a big issue).
• There would be compile time assertions that Bits lies in the range (Digits * 8, Digits * 8].

Use the 2024 edition of Rust (requires MSRV 1.85.0)

• Benefits are that some of the stabilised methods from const_option can be used, which will mean that the unchecked_... methods can become const, and const_mut_refs can be used.
• Relatedly, the parse_str_radix is not necessary now, given it can be achieved with U::from_str_radix(s, r).unwrap(). So I'm thinking of removing it.

Use the error traits from core::num

Currently, bnum defines its own error types ParseIntError and TryFromIntError, which are used for the same purpose as the error types with the same names from core::num. This is because those errors from core::num can't be initialised via something like a new method and their fields are private. However, it is possible to create them by deliberately invoking an error, e.g.

const E: ParseIntError = u8::from_str_radix("a", 10).unwrap_err();

So a possible design change is to use these error types from core::num rather than bnum's own ones. But maybe that way of creating them seems a bit hacky/bad practice.

Remove implementations of e.g. From<u128> for now

Since it is currently not possible (on stable) to write trait implementations only for certain range of values of a const-generic parameter, the existence of implementations of From<n> for primitive integers n is not quite correct, since the number of bits of the bnum could be smaller than the number of bits of n. So a possible change is to remove these implementations and only implement TryFrom<n> instead (and hopefully conditional trait impls based on const-generics are added to stable Rust relatively soon). Conversions that don't return a Result would still be possible via the CastFrom and As traits.

Put some functionality behind crate features

To reduce compile times, the functionality for signed integers, as well as functionality that requires the alloc crate, could be put behind crate features (called e.g. signed-ints and alloc) which would be enabled by default. In particular, if alloc functionality was optional, then bnum could be used in environments with no global allocator.

Remove the Add<Digit> impl

It doesn't really make sense to have this and not Add<u> for other primitive integers u, and I think having all of those would be messy.

Arbitrary fixed-width floats

This would be a huge new feature, but I've left it to the end as it's obviously separate from all the integer-related changes. I've been working on the functionality for floats on and off for some time, and basically all of the methods from f32 and f64 are there apart from the mathematical functions (e.g. exp, log, sin) and the mathematical constants (e.g. PI, E). I'm not sure how much this would be used, as the software implementations would obviously be quite a bit slower than f32 and f64, but it would be interesting to see how many people would like to have this added.

Other things

Any other comments or suggestions are welcome! Thank you for reading and looking forward to hearing your thoughts!

[krakow10]

My use case is implementing fixed-point numbers for a custom physics system in a hobby project that will hopefully end up as a completed indie game. The interesting part is that multiplying two numbers gives a widened result type (A*B = C where size_of::<C>() == size_of::<A>() + size_of::<B>()), so I can do a complete physics calculation without any rounding errors. I can share the code if you like, but it's somewhere other than github and not pretty to look at.

Space efficient types that are at least as fast as BUint and BInt

I'm all for it if you think it's an improvement. This could potentially enable fixed point types smaller than 64 bits which may be useful for on-disk formats and preventing the calculations from growing too large. My only concern would be how u128 math performs on webassembly, but my game's performance is already abysmal on wasm32 and web won't be relevant to the project for years.

Currently, the bit width has to be a multiple of 8

This is not a limitation for my use case 👍

Use the 2024 edition of Rust (requires MSRV 1.85.0)

I am already using it 😄

Remove implementations of e.g. From

This affects my code, but I am willing to change to update it since it's just implemented as a convenience (i.e. MyNumber*2) and my code could use an update for better code quality.

Put some functionality behind crate features

👍

[isaacholt100]

Thanks @krakow10! I'm unfamiliar with how u128 math works on web assembly - does it use u64 operations under the hood?

[krakow10]

Looks like wasm32 only has an i64 type - but I'm not an expert and only briefly tested it so far.

[al8n]

Hi, I am expecting the arbitrary bit widths implementation and arbitrary width floats. This is like Zig's uN, iN, and fN, and those types are quite useful in network programming and embedded programming.

[al8n]

Hi, one more thing, will pyo3 and wasm-bindgen be considered optional features? These two features can bridge Rust and the two most popular programming languages.

[isaacholt100]

Yeah I'm happy to add functionality for those - I don't have much experience with these crates, which traits from them would you want implemented?

[isaacholt100]

Hi everyone, I do apologise for the delay in releasing this version. I was quite ambitious about the number of new features I wanted to add, and so wasn't realistic with how long it would take. To give an update: the migration to the new integer types is almost complete, and I'm pleased to say that the new integer types have performance that matches or in several cases beats the performance of BUint and BInt from previous versions. This improved performance also comes with improved versatility, as now there are only two integer types, and each can have any bit width that is a multiple of 8 (before, the performance of a U264 would have been much slower than a U256 due to need to use BUintD8 rather than BUint; now, the performance will be almost identical).

Another significant change that I'm implementing is unifying the unsigned type Uint and the signed type Int into one single generic integer type, Integer. Integer has a const generic parameter N: usize for the number of bytes it contains, and a const generic parameter S: bool for whether it represents a signed or unsigned integer (so Uint<N> is a type alias for Integer<false, N> and Int<N> is a type alias for Integer<true, N>. This should lead to even greater reductions in the crate size, since a lot of logic is shared between these two types. Refactoring the code to unify these two types is not technically difficult but is fairly tedious and slow.

Another thing I've been doing is writing documentation for each method, rather than just linking to the Rust primitive equivalent, which will make the docs much clearer and easier to read.

I hope that the new version will be ready by the end of the year, but I am about to start my PhD, so am not sure how much time I will be able to spend on it for the next few months. If you have any questions or suggestions, please do let me know.