wasmtime / issue #3111 C/C++ API - Add "out of fuel" call... · git-wasmtime

Stream: git-wasmtime

Topic: wasmtime / issue #3111 C/C++ API - Add "out of fuel" call...

Wasmtime GitHub notifications bot (Jul 22 2021 at 17:23):

dshiell15 opened issue #3111:

Feature

Currently, the C/C++ API allows instantiating a Config with fuel enabled and an initial amount of fuel, but it would be nice to be able to define a callback or allow for some hook to catch out of fuel conditions such that the host application could decide whether to add more fuel and continue processing or terminate the wasm function.

Benefit

The Rust API already allows for indefinite async execution by configuring Store::out_of_fuel_async_yield() so this would help match functionality in the C/C++ APIs. It would also allow fair processing in application (server) embeddings which may be attempting to execute many wasm functions and would not want to block too long on any function.

Implementation

I'm not sure what is involved in exposing this functionality to the C/C++ API.

Alternatives

The main goal is to provide a way to. allow the host to make a decision about whether to refuel or terminate when the function's fuel is exhausted.

Thank you!

Wasmtime GitHub notifications bot (Jul 27 2021 at 21:02):

alexcrichton commented on issue #3111:

Could you describe a bit more the use case you have in mind for this? This functionality isn't even exposed in Rust right now, although I don't think there's necesarily anything fundamental about why it couldn't be.

I think though that an out-of-fuel callback could only decide to add fuel or trap, I don't think there's anything else we'd enable at this time (e.g. stack-switching or things like that), since that's within the realm of async bindings. Is your use case looking to dynamically determine whether more fuel should be added?

Wasmtime GitHub notifications bot (Jul 28 2021 at 00:14):

dshiell15 commented on issue #3111:

Hmm, yes, I'm thinking about allowing the host to dynamically determine whether more fuel should be added when an 'out of fuel' trap occurs. I saw the Store::out_of_fuel_async_yield() provides a way to yield and re-add a certain amount of fuel a certain amount of times and was hoping to be able to have some similar logic in the C-API.

More specifically, my use case is in a server context where I am using fuel to limit cpu consumption of a wasm function so that it can't block the server request processing for too long. The server already has a stack switching mechanism so really it just needs to be able to invoke a callback with the Store/Caller context so that it can make the decision about whether to add more fuel or trap.

Does that make sense?

Wasmtime GitHub notifications bot (Jul 29 2021 at 18:22):

alexcrichton commented on issue #3111:

Makes sense indeed!

FWIW I'd recommend being cautious about stack switching. Specifically if wasm is suspended while it's running Wasmtime has to reset its one TLS variable when wasm resumes on another thread. Wasmtime does this with async in Rust, but such functionality is not exposed through the C API at this time.

Wasmtime GitHub notifications bot (Jul 29 2021 at 18:43):

dshiell15 commented on issue #3111:

Thanks so much for the tip about the TLS resetting, I'll be sure to look out for that. In my case, the server is running on a single thread so hopefully it will not be an issue.

Wasmtime GitHub notifications bot (Apr 04 2023 at 06:42):

LLFourn commented on issue #3111:

I think this would be useful for me. When the fuel limit is reached I'd like to suspend the program until some condition is reached. The user who has requested the program be run may then satisfy the condition so it can resume.

Wasmtime GitHub notifications bot (May 09 2023 at 00:28):

martindevans commented on issue #3111:

An out-of-fuel callback would be very useful for me as well.

I'm using WASM in a game so the timing is pretty tight. It's pretty hard to know how much fuel to give the WASM up front - too little and no result is produced and the time spent is wasted, too much and the frame may be delayed. It would be much easier to give a very limited amount of fuel to it and then to drip feed more on request, checking if there's time remaining every time an out-of-fuel request is made.

Wasmtime GitHub notifications bot (May 09 2023 at 14:54):

alexcrichton commented on issue #3111:

@martindevans you might be more interested in epochs rather than fuel for your use case since a timer can be used to update the epoch. That would be more efficient for wasm itself but still achieve the desired effect of preventing wasm from executing for too long (and without the periodic checkins)

Wasmtime GitHub notifications bot (May 09 2023 at 15:13):

martindevans commented on issue #3111:

I am actually using epochs at the moment, to avoid the difficult of estimating the correct fuel values. But I think adding an out-of-fuel callback would be an improvement even over that.

Wasmtime GitHub notifications bot (Sep 19 2023 at 14:13):

rockwotj commented on issue #3111:

I have a proof of concept exposing the async functionality in the C API: https://github.com/bytecodealliance/wasmtime/compare/main...rockwotj:wasmtime:c-ma-its-async

Some of the qualifications:

You cannot pass arguments to call_async - my Rust was not good enough to figure out how to teach the compiler what I'm trying to do. Getting the lifetime of the translated vector parameters and the store context is quite difficult. However, for my usecase I only need to call WASI _start so it's fine to not pass/recv args.

Some of the rust may not be high quality - I'm really a C++ dev who dabbles in Rust :crab:

There still may be bugs.

I'd love to upstream this, but would probably need to figure out how to get call async with params/results first - help is welcome, or folks are welcome to steal that code and make it better if they are going to upstream it :)

Wasmtime GitHub notifications bot (Sep 19 2023 at 17:26):

alexcrichton commented on issue #3111:

Nice!

One question I'd have about that is what's the expected use case for the done boolean for host functions? Is the intention to enable epochs/fuel exclusively or to additionally enable host functions to be async themselves? I ask because only doing the former, enabling epochs/fuel, I think is the simpler way to go. That way all host functions are still synchronous but we could expose an interface for executing a wasm function which returns to the host on fuel running out or an epoch transition (instead of aborting)

Wasmtime GitHub notifications bot (Sep 19 2023 at 18:55):

rockwotj commented on issue #3111:

to additionally enable host functions to be async themselves?

This one :) I have an example of this working in our runtime and it's pretty awesome.

Wasmtime GitHub notifications bot (Sep 19 2023 at 18:57):

rockwotj commented on issue #3111:

Certainly there could be some API design I'm happy to iterate on.

I also have a need for "out of fuel" in addition to supporting async host functions. I think you have to do the "out of fuel" work of allowing the initial call into the VM to yield either way, but once you have that it's not a huge lift to add async functions (and sync functions still work).

Wasmtime GitHub notifications bot (Sep 19 2023 at 20:17):

rockwotj edited a comment on issue #3111:

to additionally enable host functions to be async themselves?

This one :) I have an example of this working in our application and it's pretty awesome.

Wasmtime GitHub notifications bot (Sep 19 2023 at 20:22):

alexcrichton commented on issue #3111:

Oh neat! Is the code public such that I'd be able to poke at it? I'm mostly curious how it all works out. For example how is what the host function is blocking on communicated to what's making the original async invocation of wasm?

Wasmtime GitHub notifications bot (Sep 19 2023 at 20:30):

rockwotj commented on issue #3111:

It exists in a very early form here: https://github.com/rockwotj/redpanda/blob/e5ff7ebf8bada9dbb77bce68be0364dcdc0e2f81/src/v/wasm/wasmtime.cc

It's written in c++ and using seastar.io which has a similar future abstraction to Rust's futures, but here is the high level approach:

Invoke the host function that returns a future:
https://github.com/rockwotj/redpanda/blob/e5ff7ebf8bada9dbb77bce68be0364dcdc0e2f81/src/v/wasm/wasmtime.cc#L302-L319

Record that pending future with the "engine" (our abstraction over an Instance):

https://github.com/rockwotj/redpanda/blob/e5ff7ebf8bada9dbb77bce68be0364dcdc0e2f81/src/v/wasm/wasmtime.cc#L325-L334

making sure to tail on a callback to update the context that the hostcall is finished.

The future in Rust polls that state with this callback: https://github.com/rockwotj/redpanda/blob/e5ff7ebf8bada9dbb77bce68be0364dcdc0e2f81/src/v/wasm/wasmtime.cc#L265-L270

The main driver of the host function when a poll call returns it looks to see if there is a future registered, then awaits it before calling poll again, otherwise asks the scheduler to do other work before resuming execution of polling the future:

https://github.com/rockwotj/redpanda/blob/e5ff7ebf8bada9dbb77bce68be0364dcdc0e2f81/src/v/wasm/wasmtime.cc#L652-L660

Wasmtime GitHub notifications bot (Sep 19 2023 at 20:34):

rockwotj edited a comment on issue #3111:

It exists in a very early form here: https://github.com/rockwotj/redpanda/blob/e5ff7ebf8bada9dbb77bce68be0364dcdc0e2f81/src/v/wasm/wasmtime.cc

It's written in c++ and using seastar.io which has a similar future abstraction to Rust's futures, but here is the high level approach:

Invoke the host function that returns a future:
https://github.com/rockwotj/redpanda/blob/e5ff7ebf8bada9dbb77bce68be0364dcdc0e2f81/src/v/wasm/wasmtime.cc#L302-L319

Record that pending future with the "engine" (our abstraction over an Instance(note that TODO is out of date):

https://github.com/rockwotj/redpanda/blob/e5ff7ebf8bada9dbb77bce68be0364dcdc0e2f81/src/v/wasm/wasmtime.cc#L325-L334

making sure to tail on a callback to update the context that the hostcall is finished.

The future in Rust polls that state with this callback: https://github.com/rockwotj/redpanda/blob/e5ff7ebf8bada9dbb77bce68be0364dcdc0e2f81/src/v/wasm/wasmtime.cc#L265-L270

The main driver of the host function when a poll call returns it looks to see if there is a future registered, then awaits it before calling poll again, otherwise asks the scheduler to do other work before resuming execution of polling the future:

https://github.com/rockwotj/redpanda/blob/e5ff7ebf8bada9dbb77bce68be0364dcdc0e2f81/src/v/wasm/wasmtime.cc#L652-L660

Wasmtime GitHub notifications bot (Sep 19 2023 at 22:37):

alexcrichton commented on issue #3111:

Thanks for the info! That all seems pretty reasonable to me. If you're able to allocate some time I think this'd be great to have upstream. I don't mind reviewing and can help out with any Rust fiddly-bits too.

Wasmtime GitHub notifications bot (Sep 20 2023 at 01:32):

rockwotj commented on issue #3111:

If you're able to allocate some time I think this'd be great to have upstream

I'm about to be OOO, but I think I can justify cleaning this up and sending patches out when I'm back. :+1:

can help out with any Rust fiddly-bits too

I'd appreciate this! The main thing that I was having problems with was teaching rust that wasmtime_func_call_async is with wasm_val_storage being kept alive (and the pointer stable) via the returned wasmtime_call_future_t owning the memory. Further complications are with CStoreContextMut being the reference that holds that initial memory and that func.call_async takes the store as a mutable reference.

I'll get something in draft mode when I'm back and maybe you can tell me how I'm holding the compiler wrong.

Wasmtime GitHub notifications bot (Sep 20 2023 at 14:37):

alexcrichton commented on issue #3111:

Ok sounds good, and yeah IMO the main thing to get designed is the C API and the documentation around its restrictions, and from that the Rust implementation should fall out without too too much effort (in theory), which I'm happy to help out wiht.

Wasmtime GitHub notifications bot (Oct 03 2023 at 15:00):

alexcrichton closed issue #3111:

Feature

Currently, the C/C++ API allows instantiating a Config with fuel enabled and an initial amount of fuel, but it would be nice to be able to define a callback or allow for some hook to catch out of fuel conditions such that the host application could decide whether to add more fuel and continue processing or terminate the wasm function.

Benefit

The Rust API already allows for indefinite async execution by configuring Store::out_of_fuel_async_yield() so this would help match functionality in the C/C++ APIs. It would also allow fair processing in application (server) embeddings which may be attempting to execute many wasm functions and would not want to block too long on any function.

Implementation

I'm not sure what is involved in exposing this functionality to the C/C++ API.

Alternatives

The main goal is to provide a way to. allow the host to make a decision about whether to refuel or terminate when the function's fuel is exhausted.

Thank you!

Last updated: Oct 23 2024 at 20:03 UTC