Stream: wasi

Topic: Resolving subtasks after parent task return

Victor Adossi (Nov 19 2025 at 01:33):

Hey @Joel Dice / @Alex Crichton I'm a bit stuck and have a question about the async_sleep_post_return_caller test -- I'm a bit stuck because I don't see how we keep polling the spawned subtask (i.e. calling the callback) after the first time.

Pending event is set when the subtask first starts (on_start() which is set up during canon_lower), but the first time you check it, it's supposed to removed (set to None in the spec docs -- Waitable#get_pending_event() via WaitableSet#get_pending_event() via the Subtask itself)...

So how could the subtask keep getting wait_until'd in the main loop? I think I'm missing something here.

It's a bit hard to explain but let me try again:

1. The initial callback call returns the code 2 (i.e. wait) and the waitable rep to watch is 1, the new Subtask itself
2. wait_until spits out { code: State.Subtask, index: 1, result: 1 }which makes sense because we have to wait for that spawned subtask, and the task is STARTED (result) -- the pending event fn was cleared for this
3. The callback is run a second time, and returns the unpacked code 2 (wait) w/ waitable 1 (the subtask) -- this all makes sense
4. We get into the second wait_until(), and the task will never resolve because we never re-set the pending event:

  # Task#wait_until -> Task#suspend_until -> Thread#suspend_until
  def wait_until(self, ready_func, thread, wset, cancellable) -> EventTuple:
    assert(thread in self.threads and thread.task is self)
    wset.num_waiting += 1
    def ready_and_has_event():
      return ready_func() and wset.has_pending_event() # <-- second condition will never be the case
    if not self.suspend_until(ready_and_has_event, thread, cancellable):
      event = (EventCode.TASK_CANCELLED, 0, 0)
    else:
      event = wset.get_pending_event()
    wset.num_waiting -= 1
    return event

Basically, I can't see/am missing the mechanism by which pending_event would ever get set again -- obviously on_progress() would be the thing to do this, BUT it wouldn't get called from on_start() established in canon_lower (we've already started), and not on_resolve() either because we're not done yet (the caller in the subtask has not task.returned)

Cheating and taking a look at the Rust impl, it looks like wasmtime basically just do another turn at the loop by creating another subtask under the current task (so 1 + n subtasks for every poll while the spawned tasks have not completed) -- this makes intuitive sense but I cannot find rationale in the spec for this.

If I'm reading the spec right, Subtasks only get created during the initial canon_lower call (i.e. on the code side this is responding to Trampoline::StartAsyncCall IIUC), so I can't figure out which mechanism is responsible for allowing the new subtask creation.

Also @Luke Wagner in case I'm misreading the spec here

Joel Dice (Nov 19 2025 at 14:56):

Victor Adossi said:

it looks like wasmtime basically just do another turn at the loop by creating another subtask under the current task (so 1 + n subtasks for every poll while the spawned tasks have not completed)

I'm not sure what you mean by this. Wasmtime isn't creating new subtasks on each poll or wait. All it's doing is being lazy about adding it to the caller's handle table. I.e. if the callee returns a result immediately, then there's no need to add it to the caller's handle table because, as far as the caller is concerned, the subtask is already finished.

Of course in this case the subtask will still do more stuff after it has returned, but that's invisible to the caller. And once the caller exits and the subtask still isn't done, the caller's caller (the host, in this case) will inherit the subtask. The host then allows that subtask and any transitive subtasks to complete by continuing to run the event loop until there's nothing left to do: https://github.com/bytecodealliance/wasmtime/blob/7fb8b55a8d3003a926753f4c3fcd676c813ebf98/crates/misc/component-async-tests/tests/scenario/post_return.rs#L90-L92

Joel Dice (Nov 19 2025 at 14:57):

I'm not sure how that all matches up to the spec code, but perhaps @Luke Wagner can help with that.

Victor Adossi (Nov 19 2025 at 15:19):

Ah, so I get that the host gets to run the subtasks until there's nothing left to do, via the tick() function -- my problem is with the check in the threads that looks like this:

# Task#wait_until -> Task#suspend_until -> Thread#suspend_until
  def wait_until(self, ready_func, thread, wset, cancellable) -> EventTuple:
    assert(thread in self.threads and thread.task is self)
    wset.num_waiting += 1
    def ready_and_has_event():
      return ready_func() and wset.has_pending_event() # <-- second condition will never be the case
    if not self.suspend_until(ready_and_has_event, thread, cancellable):
      event = (EventCode.TASK_CANCELLED, 0, 0)
    else:
      event = wset.get_pending_event()
    wset.num_waiting -= 1
    return event

My main question was how the subtask would get it's pending event set again after that first time.

The host IS currently trying to drive the subtask to completion but I'm stuck (well, the host is stuck) because it's running wait_until on a subtask that will never be ready according to that ready_and_has_event logic.

Victor Adossi (Nov 19 2025 at 15:26):

So it makes sense to me to keep running it, but maybe what you're saying is that once we're out of that loop of checking for event codes we don't enter it again? i.e. we break out of this loop:

    [packed] = call_and_trap_on_throw(callee, thread, flat_args)
    code,si = unpack_callback_result(packed)
    while code != CallbackCode.EXIT:
      thread.in_event_loop = True
      inst.exclusive = False
      match code:
        case CallbackCode.YIELD:
          event = task.yield_until(lambda: not inst.exclusive, thread, cancellable = True)
        case CallbackCode.WAIT:
          wset = inst.table.get(si)
          trap_if(not isinstance(wset, WaitableSet))
          event = task.wait_until(lambda: not inst.exclusive, thread, wset, cancellable = True)
        case CallbackCode.POLL:
          wset = inst.table.get(si)
          trap_if(not isinstance(wset, WaitableSet))
          event = task.poll_until(lambda: not inst.exclusive, thread, wset, cancellable = True)
      thread.in_event_loop = False
      inst.exclusive = True
      event_code, p1, p2 = event
      [packed] = call_and_trap_on_throw(opts.callback, thread, [event_code, p1, p2])
      code,si = unpack_callback_result(packed)
    task.exit()
    return

And just start running the callback in a tight loop? somewhere else?

Unless I'm reading it wrong (very possible!) progressing that subtask is going through that loop until we can get to CallbackCode.EXIT, and right now I'm stuck at WAIT because of the conditions of wait_until that will only be fulfilled once, essentially.

Joel Dice (Nov 19 2025 at 15:33):

At a high level, the host is looping until all tasks exit. In the case of this particular test, that will only happen once the host tasks representing calls to local:local/sleep#sleep-millis complete, at which point the caller of each host task will receive a returned event and exit.

Joel Dice (Nov 19 2025 at 15:34):

Again, though, any questions about the details of the Python code will have to be addressed by Luke. I haven't studied it in detail.

Victor Adossi (Nov 19 2025 at 15:47):

So in the Rust code, from what I can see is that it:

• breaks out of the loop of reading and interpreting results
• joins the guest waitable to the old set
• resets to the calling task

This is fine, but if I'm correct, this means that the subtask that was reparented is now being called in Store::poll_until via do_run_concurrent() -- is that right?

Assuming what I've said above is right (minus the creating new subtasks part early on!) the thing I was getting at is that there's nowhere in the spec that it notes to break out of that callback result interpretation loop bit and "just" run the subtask to completion.

Luke Wagner (Nov 19 2025 at 15:54):

@Victor Adossi spec-wise, the way that subtasks "keep running" even after having returned their value to the caller is that that callback loop runs in a new Thread created at the end of canon_lift and the Store has a list of all threads that are waiting to run and it runs them (via Store.tick) until there are no more left (which, it sounds like, is what Wasmtime is doing too)

Joel Dice (Nov 19 2025 at 15:54):

Victor Adossi said:

This is fine, but if I'm correct, this means that the subtask that was reparented is now being called in Store::poll_until via do_run_concurrent() -- is that right?

Yeah, once the subtask is inherited by the host, it and its subtasks become (or more accurately, remain) part of the general set of work-to-do stored in ConcurrentState and iterated on in poll_until

Victor Adossi (Nov 19 2025 at 15:55):

The way I could see this making sense in the current spec is if this was Thread#resume but the problem still stands that I don't see how you get out of that loop

Victor Adossi (Nov 19 2025 at 15:56):

Yup so @Luke Wagner this is what I'm doing, but my problem is how do you get out of that callback execution loop that I posted earlier -- it depends on having a pending event set on the subtask to keep executing it

Luke Wagner (Nov 19 2025 at 15:57):

The callback execution loop of a Task doesn't have any knowledge of a caller's Subtask (in fact, in the host-calls-component scenario, there is no caller Subtask)

Victor Adossi (Nov 19 2025 at 15:57):

My problem is that the spec makes it seem like the thread's work is that callback execution and result processing loop (until you get an EventCode.EXIT).

Victor Adossi (Nov 19 2025 at 15:57):

And yeah we're talking about a guest-> guest call here

Victor Adossi (Nov 19 2025 at 15:58):

the host-call to the component exits immediately basically, and a guest->guest component call lingers in the background being executed

Luke Wagner (Nov 19 2025 at 15:58):

Yes, well, the thread's work is the callback execution. The caller's Subtask isn't important; the callback execution loop just calls the on_start and on_resolve callbacks whenever it wants to and it's someone else's business of how to materialize the args and handle the results

Victor Adossi (Nov 19 2025 at 15:58):

The canon_lift for that guest->guest call dictates AFAICT that we call the callback until we get an exit

Luke Wagner (Nov 19 2025 at 15:59):

Correct

Luke Wagner (Nov 19 2025 at 15:59):

And in host->guest

Victor Adossi (Nov 19 2025 at 15:59):

Yup! So my problem is that I'm currently stuck in the wait_until branch

Victor Adossi (Nov 19 2025 at 15:59):

Because the only time we set a pending event is on-start() (the first time we start executing, that's good for ONE pass through the

Victor Adossi (Nov 19 2025 at 16:00):

And we'll never call on_resolve() because we never get to run the callback again and find out it's completed

Victor Adossi (Nov 19 2025 at 16:00):

Basically, there needs to be another call to on_progress() somewhere

Victor Adossi (Nov 19 2025 at 16:01):

as far as I can tell anyway.

Once you look at the pending event (via function) once, it's wiped -- it gets reset to None

Luke Wagner (Nov 19 2025 at 16:01):

Why isn't a Waitable in the WaitableSet passed to wait_until being set with a pending event?

Luke Wagner (Nov 19 2025 at 16:02):

That should happen independently of the callback loop that is waiting on the WaitableSet

Victor Adossi (Nov 19 2025 at 16:02):

So the pending event is set once, due to on_start() being called (and resultingly on_progress()

Victor Adossi (Nov 19 2025 at 16:02):

Ah sorry meeting! brb

Luke Wagner (Nov 19 2025 at 16:04):

It sounds like maybe you're conflating the on_progress events of a task A (which would be delivered to the supertask of A) with the on_progress events of a subtask of A. The callback loop of A can never be waiting on A's own events.

Victor Adossi (Nov 19 2025 at 16:09):

OK, meeting finished early -- I guess the first clarifying question I'd ask is what is the loop that the subtask is supposed to be in?

canon_lower sets the Subtask's callee w/ on_progress

Victor Adossi (Nov 19 2025 at 16:10):

I should say, on_progress via on_start and on_resolve

Victor Adossi (Nov 19 2025 at 16:10):

This line:

   subtask.callee = callee(thread.task, on_start, on_resolve)

Victor Adossi (Nov 19 2025 at 16:14):

Does subtask.callee() just get called in a loop somewhere? The Subtask() class does not seem to drive itself AFAICT -- so what I was understanding was that the task's callback loop is what drives that to completion (I'll call this the "callback & check" loop).

The thread that is running all this is running that callback & check loop (AFAICT)

Victor Adossi (Nov 19 2025 at 16:16):

So with what I have written now, here's what happens:

0. Host->Guest call runs, task.returns quickly, host gets a response.

   - The Thread's run loop is still going (we haven't gotten a CallbackCode.EXIT yet)
   - AFTER the task.return, the AsyncStartCall for the guest->guest call starts

1. The initial callback call returns the code 2 (i.e. wait) and the waitable rep to watch is 1, the new Subtask itself

2. wait_until spits out { code: State.Subtask, index: 1, result: 1 }which makes sense because we have to wait for that spawned subtask, and the task is STARTED (result) -- the pending event fn was cleared for this
3. The callback is run a second time, and returns the unpacked code 2 (wait) w/ waitable 1 (the subtask) -- this all makes sense
4. We get into the second wait_until(), and the task will never resolve because we never re-set the pending event:

Luke Wagner (Nov 19 2025 at 16:17):

The line subtask.callee = callee(thread.task, on_start, on_resolve) runs callee in a separate Thread so it can run its own event loop (or not... it can run synchronously or, in the future, as a stackful coroutine) and call on_start/on_resolve either synchronously or at some point in the future

Luke Wagner (Nov 19 2025 at 16:20):

So in that sequence, after 4, what I'd expect is that the subtask (later, concurrently, from its own Thread) calls on_resolve which calls on_progress which re-sets the pending event on the Subtask and thus turns the waiting event loop

Victor Adossi (Nov 19 2025 at 16:21):

OK, so that looks to be the bit I'm missing -- maybe that callee(...) like should be Store#invoke ?

Victor Adossi (Nov 19 2025 at 16:22):

Also, it's not clear that spawns a new thread that just... calls the callee forever/what the loop governing that behavior should be

Victor Adossi (Nov 19 2025 at 16:27):

AH OK, I got it:

Critically, calling a FuncInst never "blocks" (i.e., waits on I/O); if the callee would block, the FuncInst immediately returns a Call object representing the ongoing asynchronous and internally creates a Thread that can make progress via Store.tick.

Victor Adossi (Nov 19 2025 at 16:34):

Thanks, I think this is what I was missing -- the Subtask needs to start a completely separate Thread that calls the callee repeatedly in a loop and calls the on_start() and on_resolve() hooks.

So this line:

subtask.callee = callee(thread.task, on_start, on_resolve)

Is more like:

1. Start new Thread (set super task to prevent re-entrance)
2. In the body of the thread:

   - Run on_start()
   - Run callee in a tight loop... suspending after every one iteration?
   - once callee finished, run on_resolve()

Luke Wagner (Nov 19 2025 at 16:35):

Almost: on_resolve is called somewhere during the loop, and the loop keeps running until EXIT is returned

Victor Adossi (Nov 19 2025 at 16:35):

yes -- so the on_resolve would be called after task.return got called from the guest -> guest call

Victor Adossi (Nov 19 2025 at 16:35):

I'm calling that "once callee finished"

Luke Wagner (Nov 19 2025 at 16:37):

I'd say that the task.return built-in synchronously calls on_resolve which sets a pending event on any guest caller's Waitable and also does all the lowering shenanigans (calling realloc, copying return values, etc)

Victor Adossi (Nov 19 2025 at 16:38):

OK yup, that's definitely how I understand it

Victor Adossi (Nov 19 2025 at 16:38):

One clarification as well though -- that "tight loop" is supposed to be interpreting the callee results as well, no?

Luke Wagner (Nov 19 2025 at 16:40):

How do you mean "interpreting the callee results"?

Victor Adossi (Nov 19 2025 at 16:42):

I meant similarly to the other callback and check loop -- so when in this other thread I call the callee (which is an async callback) I was thinking I should be unpacking that result

Victor Adossi (Nov 19 2025 at 16:43):

not the task.return result obviously the direct async status/code result

Luke Wagner (Nov 19 2025 at 16:44):

Ah, yes, the loop in that thread unpacks the i32 return value of the lifted callee core function and the callback function (in a loop)

Victor Adossi (Nov 19 2025 at 16:45):

Yeah, so I basically need a second version of that loop in the new Thread

Luke Wagner (Nov 19 2025 at 16:45):

Yeah, there's one logical event loop per async callback-lifted function invocation

Victor Adossi (Nov 19 2025 at 16:46):

I had something similar before, but again this gets me to the problem of what wait_until

Victor Adossi (Nov 19 2025 at 16:48):

When that new Thread gets called the first time, we call on_start() and set the pending event.

IF inside the thread we're doing the callback & check loop, we're going to get stuck, because only the first pending event will allow us to progress (the second time we enter the wait_until branch, there's no pending event).

BUT if inside that loop we're NOT doing callback and check, and we're just running the callback, then in this case enough ticks will get it to eventually call on_resolve() (the component is just waiting a set amount of time)

Luke Wagner (Nov 19 2025 at 16:52):

When that new Thread gets called the first time, we call on_start() and set the pending event.

Technically, the Task.enter hits backpressure, it can suspend before calling on_start, such that an async caller will receive a subtask in the initial STARTING state.

Victor Adossi (Nov 19 2025 at 16:52):

So I guess what I'm getting at is -- I think the Thread that FuncInst is supposed to create anew might need it's semantics spelled out -- it doesn't seem like the semantics should be the exact same as the Task

Luke Wagner (Nov 19 2025 at 16:52):

I don't follow this "getting stuck" case...

Victor Adossi (Nov 19 2025 at 16:52):

Luke Wagner said:

When that new Thread gets called the first time, we call on_start() and set the pending event.

Technically, the Task.enter hits backpressure, it can suspend before calling on_start, such that an async caller will receive a subtask in the initial STARTING state.

Ah I will keep this in mind

Victor Adossi (Nov 19 2025 at 16:53):

Ah, so OK, it's this flow:

0. The initial callback call returns the code `2` (i.e. wait) and the waitable rep to watch is `1`, the new `Subtask` itself

1. `wait_until` spits out `{ code: State.Subtask, index: 1, result: 1 }`which makes sense because we have to wait for that spawned subtask, and the task is `STARTED` (`result`) -- the pending event fn was cleared for this
2. The callback is run a second time, and returns the unpacked code `2` (wait) w/ waitable `1` (the subtask) -- this all makes sense
3. We get into the *second* `wait_until()`, and the task will never resolve because we never re-set the pending event:

Luke Wagner (Nov 19 2025 at 16:53):

The semantics is spelled out by canon_lift: in guest-to-guest calls, callee is a curried canon_lift closure

Luke Wagner (Nov 19 2025 at 16:55):

I don't see why you're saying in step 3 that the pending event is never re-set... it gets re-set when on_resolve gets called by the subtask (at some point in time in the future)

Victor Adossi (Nov 19 2025 at 16:56):

Yup, that would make sense, and that is what makes me think that we need a similar "callback & check" loop.

The problem is, when I do the callback and check loop, I fall into the situation above.

It seemed to me like wasmtime was just running the callback (I assume I am wrong here) until it completed, but I haven't seen exactly what they do for this case other than just calling the callback repeatedly/polling the future until it's done.

Victor Adossi (Nov 19 2025 at 16:56):

Luke Wagner said:

I don't see why you're saying in step 3 that the pending event is never re-set... it gets re-set when on_resolve gets called by the subtask (at some point in time in the future)

Hmmn, do you have time to get on a quick call? that might be higher bandwidth!

Luke Wagner (Nov 19 2025 at 16:57):

(FWIW, I assume we're talking in the spec/Python terms, not how you're implementing this in JS)

Luke Wagner (Nov 19 2025 at 16:57):

Yeah, you bet

Luke Wagner (Nov 19 2025 at 16:57):

one sec...

Last updated: Dec 06 2025 at 06:05 UTC