Stream: wasmtime

Topic: reading an exported string


view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:37):

I'm instantiating a module. I need it to export a string and read that string from wasmtime. This is a testing scenario, so I don't want to resort to a full witx/wit approach. What is the simplest way to do this?

view this post on Zulip Lann Martin (Nov 17 2022 at 18:39):

Compile time fixed or dynamic string?

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:39):

Compile time.

view this post on Zulip Lann Martin (Nov 17 2022 at 18:40):

You should be able to export pointer (offset) and length ints and read it from linear memory in wasmtime

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:40):

Something like:

#[no_mangle]
pub static STRING: &'static str = "foo";

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:41):

Yeah, that's what I expected too. But I haven't been able to figure out the exact invocation for that.

view this post on Zulip Lann Martin (Nov 17 2022 at 18:41):

You could make that work, though afaik the layout of str isn't stable

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:41):

Yup. I know.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:42):

The moment I try to do that, module instantiation fails with:

command export 'STRING' is not a function

view this post on Zulip Lann Martin (Nov 17 2022 at 18:43):

I think you'd need to export STRING.as_ptr() (as usize?) and STRING.len()

view this post on Zulip Lann Martin (Nov 17 2022 at 18:43):

both are const fn

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:43):

Same error with an i32.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:43):

It seems like wasmtime really doesn't like anything compiled for wasm32-wasi that exports anything that isn't a function.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:45):

My next attempt was:

#[no_mangle]
pub fn string() -> &'static str {
    "foo"
}

... which linked and returned a function of:

FuncType { sig: WasmFuncType { params: [I32], externref_params_count: 0, returns: [], externref_returns_count: 0 } }

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:46):

So it is obviously wanting me to provide a pointer to memory as input to store the fat pointer in.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:46):

But I'm not sure how it is possible to expose host memory directly to the guest like that.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:47):

Attempting to return (*const u8, usize) produces the same signature.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:48):

I think it wants me to pass in a pointer to 16 bytes of memory. But I don't know how.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:49):

Val::null() produces an externref rather than an i32.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:49):

So the function signatures don't match.

view this post on Zulip Jamey Sharp (Nov 17 2022 at 18:50):

In general, that requires exporting a malloc-like function from your wasm, so the host can allocate _guest_ memory. But I think there should be another way to do what you want... I'm not sure exactly what it is though.

view this post on Zulip Lann Martin (Nov 17 2022 at 18:52):

Can you pub static STRING_PTR: *const u8 = STRING.as_ptr();?

view this post on Zulip Lann Martin (Nov 17 2022 at 18:53):

(nope)

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:53):

Yeah, nope.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:53):

Statics require Sync.

view this post on Zulip Lann Martin (Nov 17 2022 at 18:54):

I guess it makes sense - static location isn't known until link time

view this post on Zulip Lann Martin (Nov 17 2022 at 18:54):

You could write the pointer to a mutable static in an init function

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:54):

That sounds horrible. :)

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:55):

const STRING: &'static str = "Hello, world!";

#[no_mangle]
pub fn string() -> *const u8 {
    STRING.as_ptr()
}

#[no_mangle]
pub fn length() -> usize {
    STRING.len()
}

view this post on Zulip Lann Martin (Nov 17 2022 at 18:55):

Or use two separate pub functions for ptr and len; I don't know if multi-value return is implemented in rust (yet?)

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:56):

The above produces two functions with:

FuncType { sig: WasmFuncType { params: [], externref_params_count: 0, returns: [I32], externref_returns_count: 0 } }

Which means I can at least get the value out now.

view this post on Zulip Lann Martin (Nov 17 2022 at 18:56):

https://github.com/rust-lang/rust/issues/73755#issuecomment-650252985

I wanted to use tuple in my code for wasm application. When i tried to compile my code to wasm using wasm32-unknown-unknown code is compiled but its not returning any return. But I see multi-value ...

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:57):

But I still don't know what to do with that integer.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 18:58):

Since I'm not sure which APIs to call to get raw access to guest memory.

view this post on Zulip Lann Martin (Nov 17 2022 at 18:59):

https://docs.rs/wasmtime/2.0.2/wasmtime/struct.Instance.html#method.get_memory

view this post on Zulip Lann Martin (Nov 17 2022 at 18:59):

I forget what the default memory name is...

view this post on Zulip Lann Martin (Nov 17 2022 at 19:02):

"memory", typically

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 19:06):

Linker::get("memory") returns None.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 19:07):

But the wat says it is exported.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 19:08):

@Alex Crichton @Dan Gohman ^

view this post on Zulip Dan Gohman (Nov 17 2022 at 19:09):

If it's exported in wat, I'm not sure offhand why it wouldn't show up in Linker::get("memory")

view this post on Zulip Alex Crichton (Nov 17 2022 at 19:10):

you probably need to use Instance::get instead of Linker::get

view this post on Zulip Alex Crichton (Nov 17 2022 at 19:10):

Linker::get is only used for items defined into the linker, and if you're just instantiating through the linker that doesn't instantiate more things into the linker

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 19:12):

@Alex Crichton What is the right way to manually instantiate and call the start function?

view this post on Zulip Lann Martin (Nov 17 2022 at 19:13):

Instance construction runs the start function

view this post on Zulip Alex Crichton (Nov 17 2022 at 19:14):

yes presence of an Instance means the wasm start function was run

view this post on Zulip Lann Martin (Nov 17 2022 at 19:16):

Unless you mean the _start function, which - confusingly - is unrelated

view this post on Zulip Lann Martin (Nov 17 2022 at 19:17):

(i.e. an "old"-style WASI command)

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 19:19):

Wohoo! I managed to get an instance of a global!

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 19:19):

That feels like real progress.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 19:36):

@Alex Crichton How do I translate the raw i32 result (which is obviously a pointer of some kind) into an offset into the exported memory?

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 19:37):

The result of the exported global is 1048616 and the size of the exported memory is 1114112.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 19:37):

So that seems sane. But the value isn't at that offset.

view this post on Zulip Lann Martin (Nov 17 2022 at 19:38):

It should be - wasm guest pointers are offsets into their linear memory

view this post on Zulip Alex Crichton (Nov 17 2022 at 19:43):

yes it should be an offset into linear memory, if it isn't I don't know what it is

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 19:43):

I think it is a rust &str layout issue. The exported value points to just past the end of the string.

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 19:45):

#[no_mangle]
pub static CONFIG: &'static str = "Hello, world!\0";

The value of CONFIG as seen by wasmtime is 1048616, but the string actually starts at 1048600.

view this post on Zulip Alex Crichton (Nov 17 2022 at 19:46):

That's because CONFIG, as a pointer is &&'static str, you'd need to instead to static CONFIG: [u8; N] = *b"..." if you want the raw addrss

view this post on Zulip Nathaniel McCallum (Nov 17 2022 at 19:47):

Ah, right.

view this post on Zulip Jamey Sharp (Nov 17 2022 at 20:11):

or alternatively you should be able to read the I32 that represents the &&str, and then find the string at that address, right?

view this post on Zulip Nathaniel McCallum (Nov 18 2022 at 15:38):

@Jamey Sharp That's the approach I took.

view this post on Zulip Nathaniel McCallum (Nov 18 2022 at 15:41):

@Alex Crichton Since exploring this yesterday, I began thinking that a custom section might be more idiomatic. What is the status of custom section support? https://github.com/rust-lang/rust/pull/48883 was merged but I'm currently seeing cannot find attribute on nightly.

This commit is an implementation of adding custom sections to wasm artifacts in rustc. The intention here is to expose the ability of the wasm binary format to contain custom sections with arbitrar...

view this post on Zulip Alex Crichton (Nov 18 2022 at 15:45):

It's implemented in rustc and hasn't changed since, not sure why you'd get a cannot find attribute error

view this post on Zulip Nathaniel McCallum (Nov 18 2022 at 15:52):

@Alex Crichton Ah, vscode/analyzer was trying to build it for the host arch.


Last updated: Dec 23 2024 at 13:07 UTC