Reentrant calls in Lucet

[bubagl]

I noticed that Lucet explicitly disallows making "reentrant" calls, i.e., when a host callback calls to webassembly module. Has this been done on purpose? Is there a way to make such calls possible?

[acfoltzer]

Hmm, can you point out where you saw that reentrant calls are not allowed?

It's true that the interface for making them isn't great, but you can look up callbacks by Wasm function index and call them: https://docs.rs/lucet-runtime/0.6.1/lucet_runtime/vmctx/struct.Vmctx.html#method.get_func_from_idx

Since this returns a raw FunctionHandle, you have to be very careful about making sure you cast the function to the right type before invoking it. For that reason, I wouldn't recommend using this capability with untrusted Wasm at the moment.

[bubagl]

It seems we are talking about different things. For instance, let's say there is a wasm module:

void main() {
    call_host();
}

void foo() {}

and the host:

#[lucet_hostcall]
#[no_mangle]
pub fn call_host(ctx: &Vmctx) {
    let instance = unsafe { ctx.instance_mut() };
    instance.run("foo", &[]).unwrap();
}

The call from call_host back to wasm instance fails.

[pchickey]

The instance_mut method on Vmctx is extremely unsafe - it is not intended to be exposed to users of Lucet. The instance run methods need the instance's stack to be empty, but once inside a hostcall, the code is actually using that stack. Only the methods available on Vmctx through the lucet_runtime crate are safe to use in a hostcall.

If you need to call a WebAssembly function from inside a hostcall, the function Adam linked to is the only way to do it without corrupting the current stack. It's still much less safe than ideal, but we haven't done much to build out that aspect of Lucet because we don't use it in practice.

This corner of Lucet has some sharp edges. We recommend that many users are better suited to use Wasmtime, our sister Bytecode Alliance project, unless your application demands the particular features of Lucet (AOT, high concurrency) that aren't available in Wasmtime yet.

[bubagl]

use Wasmtime, our sister Bytecode Alliance project

do you mean that in general case it is recommended to switch to Wasmtime from Lucet?

[pchickey]

Yes. In the future, we will merge them into a single project, so that AOT and high concurrency are available for wasmtime. In the meantime, it is a better choice unless you have those particular needs. https://www.fastly.com/blog/how-lucet-wasmtime-make-stronger-compiler-together

[bubagl]

so that AOT and high concurrency are available for wasmtime

Is it possible to call the same "wasm vm" (not sure about their actual terminology) from multiple threads in wasmtime?

Also I like the fact that wasm instances in Lucet are very compact. It is possible to run thousands of instances in the same physical process. Is wasmtime comparable?

[pchickey]

WebAssembly instances are specified to be single-threaded, so you can't make simultaneous calls to an instance regardless of VM. You'll have to use a mutex or equivelant to ensure that only one thread has access to the instance at a time.

We haven't done any benchmarking of how many wasmtime instances you can make in a process. I don't expect it to be significantly more memory usage per instance, per se, but Lucet arranges allocations to amortize the number of memory management syscalls made, and allows alternative mechanisms like userfaultfd that can reduce kernel resource contention. We are in the planning stages of porting the way Lucet manages memory allocations to Wasmtime, so we expect that it will eventually be about the same as Lucet. That work isn't our top priority right now so let us know if it is important for your business case and we can figure out how to collaborate.

[bubagl]

You'll have to use a mutex or equivelant to ensure that only one thread has access to the instance at a time.

So it's possible to create an instance on one thread and to call it on a different one. Correct?

[pchickey]

Yes

[bubagl]

To me the ability of Lucet to produce native libraries looks like a killer feature. But it seems no other runtime adopted it. Are there some non-obvious problems in this approach?

[alterstep]

wasm2c -> no runtime! Direct function calls.

wasm2c: https://github.com/WebAssembly/wabt/tree/master/wasm2c

[bubagl]

From my experience wasm2c is pretty primitive comparing to Lucet. The resulting binary is slow and not all wasm features are supported.

[alterstep]

fast and all wasm features supported -> innative can create library, try it
but wasm2c code is easy to understand and performance is not bad. compile with optimizations!

innative: https://github.com/innative-sdk/innative

[iximeow]

Are there some non-obvious problems in this approach?

I think you mean to ask about other runtimes producing native libraries, correct? Wasmtime is building towards the ability to use native libraries as part of caching jit results, so I'd suggest following the development of that over the next few months.

As for potential issues, it really depends on the runtime and what guarantees they'd like to make - if a function isn't called, it doesn't need to be JIT'ed and then wouldn't take up space in memory, so by default emitting a native object could result in more memory use in some circumstances. It also complicates tiered compilation; I'm not sure if any WebAssembly runtimes do that at the moment, but adjusting a native library to match reoptimizations of a function would certainly be a complication.

In terms of emitting native libraries, Lucet still requires the host to embed lucet-runtime in some way or another, like other WebAssembly implementations involving a runtime. Lucet is closer to, but not currently able to, produce shared libraries that are statically linked with lucet-runtime. So to that end, I'd try to think of native libraries produced by lucetc as WebAssembly modules by another name, rather than native libraries you could use independently - they're just as dependent on the loader knowing how to run them as a runtime that loads .wasm instead of .so.

Edit: in contrast, it looks like innative that @alterstep linked above does produce native objects suitable for loading and executing without the host knowing about any WebAssembly runtime? I only learned about it nine minutes ago so I can't speak any further to it, but that's pretty neat :)

[bubagl]

performance is not bad. compile with optimizations!

Actually I was wrong. Tried several simple benchmarks and performance is more than reasonable. The runtime is limited and there is no Wasi support but still interesting ...