Stream: wasmtime

Topic: reading an exported string

Nathaniel McCallum (Nov 17 2022 at 18:37):

I'm instantiating a module. I need it to export a string and read that string from wasmtime. This is a testing scenario, so I don't want to resort to a full witx/wit approach. What is the simplest way to do this?

Lann Martin (Nov 17 2022 at 18:39):

Compile time fixed or dynamic string?

Nathaniel McCallum (Nov 17 2022 at 18:39):

Compile time.

Lann Martin (Nov 17 2022 at 18:40):

You should be able to export pointer (offset) and length ints and read it from linear memory in wasmtime

Nathaniel McCallum (Nov 17 2022 at 18:40):

Something like:

#[no_mangle]
pub static STRING: &'static str = "foo";

Nathaniel McCallum (Nov 17 2022 at 18:41):

Yeah, that's what I expected too. But I haven't been able to figure out the exact invocation for that.

Lann Martin (Nov 17 2022 at 18:41):

You could make that work, though afaik the layout of str isn't stable

Nathaniel McCallum (Nov 17 2022 at 18:41):

Yup. I know.

Nathaniel McCallum (Nov 17 2022 at 18:42):

The moment I try to do that, module instantiation fails with:

command export 'STRING' is not a function

Lann Martin (Nov 17 2022 at 18:43):

I think you'd need to export STRING.as_ptr() (as usize?) and STRING.len()

Lann Martin (Nov 17 2022 at 18:43):

both are const fn

Nathaniel McCallum (Nov 17 2022 at 18:43):

Same error with an i32.

Nathaniel McCallum (Nov 17 2022 at 18:43):

It seems like wasmtime really doesn't like anything compiled for wasm32-wasi that exports anything that isn't a function.

Nathaniel McCallum (Nov 17 2022 at 18:45):

My next attempt was:

#[no_mangle]
pub fn string() -> &'static str {
    "foo"
}

... which linked and returned a function of:

FuncType { sig: WasmFuncType { params: [I32], externref_params_count: 0, returns: [], externref_returns_count: 0 } }

Nathaniel McCallum (Nov 17 2022 at 18:46):

So it is obviously wanting me to provide a pointer to memory as input to store the fat pointer in.

Nathaniel McCallum (Nov 17 2022 at 18:46):

But I'm not sure how it is possible to expose host memory directly to the guest like that.

Nathaniel McCallum (Nov 17 2022 at 18:47):

Attempting to return (*const u8, usize) produces the same signature.

Nathaniel McCallum (Nov 17 2022 at 18:48):

I think it wants me to pass in a pointer to 16 bytes of memory. But I don't know how.

Nathaniel McCallum (Nov 17 2022 at 18:49):

Val::null() produces an externref rather than an i32.

Nathaniel McCallum (Nov 17 2022 at 18:49):

So the function signatures don't match.

Jamey Sharp (Nov 17 2022 at 18:50):

In general, that requires exporting a malloc-like function from your wasm, so the host can allocate _guest_ memory. But I think there should be another way to do what you want... I'm not sure exactly what it is though.

Lann Martin (Nov 17 2022 at 18:52):

Can you pub static STRING_PTR: *const u8 = STRING.as_ptr();?

Lann Martin (Nov 17 2022 at 18:53):

(nope)

Nathaniel McCallum (Nov 17 2022 at 18:53):

Yeah, nope.

Nathaniel McCallum (Nov 17 2022 at 18:53):

Statics require Sync.

Lann Martin (Nov 17 2022 at 18:54):

I guess it makes sense - static location isn't known until link time

Lann Martin (Nov 17 2022 at 18:54):

You could write the pointer to a mutable static in an init function

Nathaniel McCallum (Nov 17 2022 at 18:54):

That sounds horrible. :)

Nathaniel McCallum (Nov 17 2022 at 18:55):

const STRING: &'static str = "Hello, world!";

#[no_mangle]
pub fn string() -> *const u8 {
    STRING.as_ptr()
}

#[no_mangle]
pub fn length() -> usize {
    STRING.len()
}

Lann Martin (Nov 17 2022 at 18:55):

Or use two separate pub functions for ptr and len; I don't know if multi-value return is implemented in rust (yet?)

Nathaniel McCallum (Nov 17 2022 at 18:56):

The above produces two functions with:

FuncType { sig: WasmFuncType { params: [], externref_params_count: 0, returns: [I32], externref_returns_count: 0 } }

Which means I can at least get the value out now.

Lann Martin (Nov 17 2022 at 18:56):

https://github.com/rust-lang/rust/issues/73755#issuecomment-650252985

Nathaniel McCallum (Nov 17 2022 at 18:57):

But I still don't know what to do with that integer.

Nathaniel McCallum (Nov 17 2022 at 18:58):

Since I'm not sure which APIs to call to get raw access to guest memory.

Lann Martin (Nov 17 2022 at 18:59):

https://docs.rs/wasmtime/2.0.2/wasmtime/struct.Instance.html#method.get_memory

Lann Martin (Nov 17 2022 at 18:59):

I forget what the default memory name is...

Lann Martin (Nov 17 2022 at 19:02):

"memory", typically

Nathaniel McCallum (Nov 17 2022 at 19:06):

Linker::get("memory") returns None.

Nathaniel McCallum (Nov 17 2022 at 19:07):

But the wat says it is exported.

Nathaniel McCallum (Nov 17 2022 at 19:08):

@Alex Crichton @Dan Gohman ^

Dan Gohman (Nov 17 2022 at 19:09):

If it's exported in wat, I'm not sure offhand why it wouldn't show up in Linker::get("memory")

Alex Crichton (Nov 17 2022 at 19:10):

you probably need to use Instance::get instead of Linker::get

Alex Crichton (Nov 17 2022 at 19:10):

Linker::get is only used for items defined into the linker, and if you're just instantiating through the linker that doesn't instantiate more things into the linker

Nathaniel McCallum (Nov 17 2022 at 19:12):

@Alex Crichton What is the right way to manually instantiate and call the start function?

Lann Martin (Nov 17 2022 at 19:13):

Instance construction runs the start function

Alex Crichton (Nov 17 2022 at 19:14):

yes presence of an Instance means the wasm start function was run

Lann Martin (Nov 17 2022 at 19:16):

Unless you mean the _start function, which - confusingly - is unrelated

Lann Martin (Nov 17 2022 at 19:17):

(i.e. an "old"-style WASI command)

Nathaniel McCallum (Nov 17 2022 at 19:19):

Wohoo! I managed to get an instance of a global!

Nathaniel McCallum (Nov 17 2022 at 19:19):

That feels like real progress.

Nathaniel McCallum (Nov 17 2022 at 19:36):

@Alex Crichton How do I translate the raw i32 result (which is obviously a pointer of some kind) into an offset into the exported memory?

Nathaniel McCallum (Nov 17 2022 at 19:37):

The result of the exported global is 1048616 and the size of the exported memory is 1114112.

Nathaniel McCallum (Nov 17 2022 at 19:37):

So that seems sane. But the value isn't at that offset.

Lann Martin (Nov 17 2022 at 19:38):

It should be - wasm guest pointers are offsets into their linear memory

Alex Crichton (Nov 17 2022 at 19:43):

yes it should be an offset into linear memory, if it isn't I don't know what it is

Nathaniel McCallum (Nov 17 2022 at 19:43):

I think it is a rust &str layout issue. The exported value points to just past the end of the string.

Nathaniel McCallum (Nov 17 2022 at 19:45):

#[no_mangle]
pub static CONFIG: &'static str = "Hello, world!\0";

The value of CONFIG as seen by wasmtime is 1048616, but the string actually starts at 1048600.

Alex Crichton (Nov 17 2022 at 19:46):

That's because CONFIG, as a pointer is &&'static str, you'd need to instead to static CONFIG: [u8; N] = *b"..." if you want the raw addrss

Nathaniel McCallum (Nov 17 2022 at 19:47):

Ah, right.

Jamey Sharp (Nov 17 2022 at 20:11):

or alternatively you should be able to read the I32 that represents the &&str, and then find the string at that address, right?

Nathaniel McCallum (Nov 18 2022 at 15:38):

@Jamey Sharp That's the approach I took.

Nathaniel McCallum (Nov 18 2022 at 15:41):

@Alex Crichton Since exploring this yesterday, I began thinking that a custom section might be more idiomatic. What is the status of custom section support? https://github.com/rust-lang/rust/pull/48883 was merged but I'm currently seeing cannot find attribute on nightly.

Alex Crichton (Nov 18 2022 at 15:45):

It's implemented in rustc and hasn't changed since, not sure why you'd get a cannot find attribute error

Nathaniel McCallum (Nov 18 2022 at 15:52):

@Alex Crichton Ah, vscode/analyzer was trying to build it for the host arch.

Last updated: Nov 22 2024 at 17:03 UTC