alexcrichton opened issue #6741:
Currently the set of CLI flags added to Wasmtime has been ad-hoc and generally "throw another option in there" style historically, which is great for easily adding new options (which happens a fair bit), but I personally feel like there have been some growing pains that may be worthwhile to address sooner rather than later.
Some concerns I have with the current CLI are:
- Discoverability is hard right now as
--help
or-h
prints out quite a lot of information.- Not a lot of single-letter flags are in use so CLI commands tend to be quite long
- It's tough to review "at a glance" what Wasmtime's capabilities are on the CLI through the help page since there's so much to sift through
- Some options have little papercuts like trying to remember
--dir
vs--mapdir
, there's no way to inherit the caller's environment or a single env var from the environment easily, etc.- Options can be randomly
--disable-foo
or--enable-foo
and it's not clear why which is which when you're approaching the CLI for the first time.To be clear none of these are dealbreakers by any means. While I think the Wasmtime CLI is important the embedding API are where things "get serious" so it's ok for the CLI to not be perfect. That being said folks often interact with Wasmtime through the CLI to start off with and it's also mega-useful during development and debugging and such.
So given all the above I'd like to propose a (hopefully) brief bikeshed about the Wasmtime CLI. I'd like to ideally address the above concerns and provide guidelines of how to add new options into the future. This will hopefully clean up our CLI, make it a bit more ergonomic to use, all while not hindering our ability to easily add various knobs here and there as they're implemented.
At a high level my proposal is "let's do what
rustc
does". I'm very biased in that regard as I helped design rustc's CLI as well. Some things I like about rustc though are:
- The output of
rustc -h
fits more-or-less on one terminal. This serves as a good index from which you can learn more.- There are "option groups" that are behind other flags, such as
rustc -C foo
orrustc -Z foo
. These single-letter capital flags are easy to type and easy to remember and serve as a good place to house sub-options.- More information can be read through
rustc -C help
which provides an window into all the codegen capabilities of rustc- Instead of
--enable-foo
or--disable-foo
most rustc options are-C foo
for--enable-foo
or-C foo=no
for--disable-foo
. This makes it easy for callers to forcibly enable/disable something and you don't have to worry about what the defaults are since as a user you typically want to turn something on or off and you don't care too much about whatever the default happens to be.My proposal here for Wasmtime's option groups are:
Codegen options (
-C
or--codegen
)
- All
--disable-foo
flags become-C foo=no
- All
--enable-foo
flags become-C foo
-C foo
is shorthand for-C foo=yes
*=yes
can supporty
,yes
,true
, etc*=no
can supportn
,no
,false
, etcNon-boolean flags support
-C name=val
whereval
is parsed as whatever
name
wants.Can support comma-separation as well such as
-C foo,bar
or-C foo=n,bar
old cli flag new cli flag --compiler <COMPILER>
-C compiler=..
-O, --optimize
-C opt-level=N
/-O
--opt-level <LEVEL>
-C opt-level=N
--config <CONFIG_PATH>
-C cache-config=..
--disable-address-map
-C address-map=no
--disable-cache
-C cache=no
--disable-parallel-compilation
-C parallel-compilation=no
--epoch-interruption
-C epoch-interruption
-g
-C debuginfo
/-g
--dynamic-memory-guard-size <SIZE>
-C dynamic-memory-guard-size=...
--static-memory-forced
-C static-memory-forced
--static-memory-guard-size <SIZE>
-C static-memory-guard-size=N
--static-memory-maximum-size <MAXIMUM>
-C static-memory-maximum-size=N
--relaxed-simd-deterministic
-C relaxed-simd-deterministic
--cranelift-enable <SETTING>
-C cranelift-NAME
--enable-cranelift-debug-verifier
-C cranelift-debug-verifier
--enable-cranelift-nan-canonicalization
-C cranelift-nan-canonicalization
--cranelift-set <NAME=VALUE>
-C cranelift-NAME=VALUE
Runtime options (
-R
or--runtime
)
- Various
*-unknown-*
options are all-R unknown-*
now.- Same system for boolean flags as
-C
options
old cli flag new cli flag --allow-unknown-exports
-R unknown-exports-allow
--trap-unknown-imports
-R unknown-imports-trap
--default-values-unknown-imports
-R unknown-imports-default
--coredump-on-trap <PATH>
-R coredump=..
--disable-logging
-R logging=no
--disable-memory-init-cow
-R memory-init-cow=no
--dynamic-memory-reserved-for-growth <SIZE>
-R dynamic-memory-reserved-for-growth=...
--fuel <N>
-R fuel=N
--log-to-files
-R log-to-files
--max-instances <MAX_INSTANCES>
-R max-instances=N
--max-memories <MAX_MEMORIES>
-R max-memories=N
--max-memory-size <BYTES>
-R max-memory-size=N
--max-table-elements <MAX_TABLE_ELEMENTS>
-R max-table-elements=N
--max-tables <MAX_TABLES>
-R max-tables=N
--max-wasm-stack <MAX_WASM_STACK>
-R max-wasm-stack=N
--pooling-allocator
-R pooling-allocator
--trap-on-grow-failure
-R trap-on-grow-failure
--wasm-timeout <TIME>
-R timeout=N
WASI options (
-W
or--wasi
)
Separate from runtime options as this'll have WASI-proposal-specific
configuration options.Leave
--env
as its own standalone flag- Add support for
--env FOO
which inherits the env varFOO
from the outer
process.
old cli flag new cli flag --listenfd
-W listenfd
--tcplisten <SOCKET ADDRESS>
-W tcplisten=...
--wasi-modules <MODULE,MODULE,...>
-W common
--wasi-modules <MODULE,MODULE,...>
-W experimental-threads
--dir <DIRECTORY>
-W dir=..
--mapdir <GUEST_DIR::HOST_DIR>
-W dir=a::b
--env <NAME=VAL>
--env ...
Other flags
These are the other flags supported by Wasmtime which don't necessarily fit into the groups above. They're either important enough I think they should stick around at the top level or they have enough of their own "sub-syntax" that I'm not sure they fit well in the groups above.
--wasm-features <FEATURE,FEATURE,...>
- perhaps add something like a-F
shortcut, I find it a bit annoying to always type this out. Support--wasm-features help
for the extended help section on this.
--preload <NAME=MODULE_PATH>
- allow omittingNAME
and infer it by default
fromMODULE_PATH
.
--profile <STRATEGY>
- leave as-is--invoke <FUNCTION>
- leave as-is--allow-precompiled
- leave as-is--target <TARGET>
- leave as-is--output <OUTPUT>
- leave as-is--emit-clif <PATH>
- leave as-is
I'm curious what others think about this! If it's broadly amenable and folks are ok with the breakage I'm happy to implement this myself, I don't think it'll take all that long (perhaps just a day or so)
sunfishcode commented on issue #6741:
Overall I like the ideas here; my main concern is that an end user may not always know whether something is a "codegen" option or a "runtime" option, for example epoch interruption or memory configuration. I don't have a fully formed idea yet, but I wonder if it would make sense to instead group the options by something like:
- Performance tuning: optimization level, caching, parallel compilation, memory init cow, etc.
- Wasm semantics: NaN canonicalization, deterministic relaxed simd, max memory size etc., trap on grow failure, etc.
- Debugging: -g, logging, profiling, etc.
- WASI flags as you say
What do you think?
alexcrichton commented on issue #6741:
Nah yeah I agree. My best strawman for an ordering like you're suggesting is something along the lines of:
-P
- performance tuning-C
- codegen-D
- debugging-W
- wasm semantics-S
- WASI & system config<details>
old cli flag new cli flag -O, --optimize
-P opt-level=N
/-O
--opt-level <LEVEL>
-P opt-level=N
--dynamic-memory-guard-size <SIZE>
-P dynamic-memory-guard-size=...
--static-memory-forced
-P static-memory-forced
--static-memory-guard-size <SIZE>
-P static-memory-guard-size=N
--static-memory-maximum-size <MAXIMUM>
-P static-memory-maximum-size=N
--dynamic-memory-reserved-for-growth <SIZE>
-P dynamic-memory-reserved-for-growth=...
--pooling-allocator
-P pooling-allocator
--disable-memory-init-cow
-P memory-init-cow=no
--compiler <COMPILER>
-C compiler=..
--enable-cranelift-debug-verifier
-C cranelift-debug-verifier
--cranelift-enable <SETTING>
-C cranelift-NAME
--cranelift-set <NAME=VALUE>
-C cranelift-NAME=VALUE
--config <CONFIG_PATH>
-C cache-config=..
--disable-cache
-C cache=no
--disable-parallel-compilation
-C parallel-compilation=no
-g
-D debuginfo
/-g
--disable-address-map
-D address-map=no
--disable-logging
-D logging=no
--log-to-files
-D log-to-files
--coredump-on-trap <PATH>
-D coredump=..
--relaxed-simd-deterministic
-W relaxed-simd-deterministic
--enable-cranelift-nan-canonicalization
-W cranelift-nan-canonicalization
--fuel <N>
-W fuel=N
--epoch-interruption
-W epoch-interruption
--allow-unknown-exports
-W unknown-exports-allow
--trap-unknown-imports
-W unknown-imports-trap
--default-values-unknown-imports
-W unknown-imports-default
--max-instances <MAX_INSTANCES>
-W max-instances=N
--max-memories <MAX_MEMORIES>
-W max-memories=N
--max-memory-size <BYTES>
-W max-memory-size=N
--max-table-elements <MAX_TABLE_ELEMENTS>
-W max-table-elements=N
--max-tables <MAX_TABLES>
-W max-tables=N
--max-wasm-stack <MAX_WASM_STACK>
-W max-wasm-stack=N
--trap-on-grow-failure
-W trap-on-grow-failure
--wasm-timeout <TIME>
-W timeout=N
--listenfd
-S listenfd
--tcplisten <SOCKET ADDRESS>
-S tcplisten=...
--wasi-modules <MODULE,MODULE,...>
-S common
--wasi-modules <MODULE,MODULE,...>
-S experimental-threads
--dir <DIRECTORY>
-S dir=..
--mapdir <GUEST_DIR::HOST_DIR>
-S dir=a::b
</details>
One thing that feels a bit odd though is that I was hoping for a clean split where
wasmtime compile
would be a strict subset ofwasmtime run
options or something like that. For example there's no need forwasmtime compile
to support-S
or-D
options in theory. That's what I was roughly hoping for above wherewasmtime compile
would drop all the-R
options but keep all the-C
options.Here it's not so straightforward though because fuel is both a wasm semantics option and a compilation option. Other wasm semantic options like
max-memory-size
don't affect the compilation though. This may not be a great thing to optimize for though and the more I think about it the more it feels sort of arbitrary.In any case I like the idea of the categories you're thinking of too, would you move any of the above options around though?
sunfishcode commented on issue #6741:
Some bikeshed suggestions:
- Name the performance tuning flag
-O
, as in-O pooling-allocator
and so on, because that's familiar to me from common compiler flags, though I don't know if that will have the same association for others.- Rename
cranelift-nan-canonicalization
to justnan-canonlicalization
and maybe have winch fail if that flag is given and it isn't supported yet. Because from an end-user perspective, this is a semantics thing, not a cranelift thing.The
compile
vs.run
split is a good point. My best idea is to havecompile
accept all the same options asrun
, and record those options along with the output so thatrun
can default to them.run
could allow some of those flags to be overridden, but not all, because some will be baked into the compiled code. But it's awkward for users to figure out which flags need to be specified atcompile
and which can be overridden atrun
.But maybe that's ok? Maybe the priority should be to make the simple
run
user experience the simplest, and organizing the options in terms of tuning/semantics/debugging/etc. seems best for that?
fitzgen commented on issue #6741:
Don't feel like joining this particular bike shed, but one TODO item to record for posterity if we change the flags to enable different Wasm proposals is we will want to send a PR to update https://webassembly.org/roadmap/ as well.
alexcrichton commented on issue #6741:
I like the idea of
-O
for tuning, yeah, and renaming tonan-canonicalization
sounds good to me. I also agree it's probably best to optimize forrun
andcompile
can do what's necessary to support the flags, so given that I think the proposal would be:
old cli flag new cli flag -O, --optimize
-O opt-level=N
--opt-level <LEVEL>
-O opt-level=N
--dynamic-memory-guard-size <SIZE>
-O dynamic-memory-guard-size=...
--static-memory-forced
-O static-memory-forced
--static-memory-guard-size <SIZE>
-O static-memory-guard-size=N
--static-memory-maximum-size <MAXIMUM>
-O static-memory-maximum-size=N
--dynamic-memory-reserved-for-growth <SIZE>
-O dynamic-memory-reserved-for-growth=...
--pooling-allocator
-O pooling-allocator
--disable-memory-init-cow
-O memory-init-cow=no
--compiler <COMPILER>
-C compiler=..
--enable-cranelift-debug-verifier
-C cranelift-debug-verifier
--cranelift-enable <SETTING>
-C cranelift-NAME
--cranelift-set <NAME=VALUE>
-C cranelift-NAME=VALUE
--config <CONFIG_PATH>
-C cache-config=..
--disable-cache
-C cache=no
--disable-parallel-compilation
-C parallel-compilation=no
-g
-D debuginfo
/-g
--disable-address-map
-D address-map=no
--disable-logging
-D logging=no
--log-to-files
-D log-to-files
--coredump-on-trap <PATH>
-D coredump=..
--relaxed-simd-deterministic
-W relaxed-simd-deterministic
--enable-cranelift-nan-canonicalization
-W nan-canonicalization
--fuel <N>
-W fuel=N
--epoch-interruption
-W epoch-interruption
--allow-unknown-exports
-W unknown-exports-allow
--trap-unknown-imports
-W unknown-imports-trap
--default-values-unknown-imports
-W unknown-imports-default
--max-instances <MAX_INSTANCES>
-W max-instances=N
--max-memories <MAX_MEMORIES>
-W max-memories=N
--max-memory-size <BYTES>
-W max-memory-size=N
--max-table-elements <MAX_TABLE_ELEMENTS>
-W max-table-elements=N
--max-tables <MAX_TABLES>
-W max-tables=N
--max-wasm-stack <MAX_WASM_STACK>
-W max-wasm-stack=N
--trap-on-grow-failure
-W trap-on-grow-failure
--wasm-timeout <TIME>
-W timeout=N
--listenfd
-S listenfd
--tcplisten <SOCKET ADDRESS>
-S tcplisten=...
--wasi-modules <MODULE,MODULE,...>
-S common
--wasi-modules <MODULE,MODULE,...>
-S experimental-threads
--dir <DIRECTORY>
-S dir=..
--mapdir <GUEST_DIR::HOST_DIR>
-S dir=a::b
tschneidereit commented on issue #6741:
Also don't feel like I need to share my favorite shed coloring scheme, but a question: is it possible to have
help
output for specific flags? I.e., can I dowasmtime help run -O
(orwasmtime help -O
) and get output just for all-O
flags?
tschneidereit commented on issue #6741:
And another question: can we in redesigning this look into making flags available as env vars as well where reasonable? Ideally in a way that works for libwasmtime embeddings, not just the CLI? Obviously that doesn't make sense for most options, but it'd be excellent for debug/profiling things, for example.
alexcrichton commented on issue #6741:
I'm not sure of a way to configure clap to do that, but I've also found that if I'm looking for a specific option then
wasmtime --help | less
and searching forfoo
is good enough for exploring a specific option. Or alternatively with the groups above it'd bewasmtime -W help
or similar.And another question: can we in redesigning this look into making flags available as env vars as well where reasonable? Ideally in a way that works for libwasmtime embeddings, not just the CLI?
On one hand this is easy because I think clap exposes the ability to do this. On the other hand this isn't easy because embeddings aren't using clap they're using
Config
. I do agree though that for profiling and debugging options env vars are mega-useful since you don't have to thread them around everywhere.
alexcrichton commented on issue #6741:
I've made a PR for this at https://github.com/bytecodealliance/wasmtime/pull/6925
tschneidereit commented on issue #6741:
I guess we can close this?
alexcrichton commented on issue #6741:
Ah yes, thanks!
alexcrichton closed issue #6741:
Currently the set of CLI flags added to Wasmtime has been ad-hoc and generally "throw another option in there" style historically, which is great for easily adding new options (which happens a fair bit), but I personally feel like there have been some growing pains that may be worthwhile to address sooner rather than later.
Some concerns I have with the current CLI are:
- Discoverability is hard right now as
--help
or-h
prints out quite a lot of information.- Not a lot of single-letter flags are in use so CLI commands tend to be quite long
- It's tough to review "at a glance" what Wasmtime's capabilities are on the CLI through the help page since there's so much to sift through
- Some options have little papercuts like trying to remember
--dir
vs--mapdir
, there's no way to inherit the caller's environment or a single env var from the environment easily, etc.- Options can be randomly
--disable-foo
or--enable-foo
and it's not clear why which is which when you're approaching the CLI for the first time.To be clear none of these are dealbreakers by any means. While I think the Wasmtime CLI is important the embedding API are where things "get serious" so it's ok for the CLI to not be perfect. That being said folks often interact with Wasmtime through the CLI to start off with and it's also mega-useful during development and debugging and such.
So given all the above I'd like to propose a (hopefully) brief bikeshed about the Wasmtime CLI. I'd like to ideally address the above concerns and provide guidelines of how to add new options into the future. This will hopefully clean up our CLI, make it a bit more ergonomic to use, all while not hindering our ability to easily add various knobs here and there as they're implemented.
At a high level my proposal is "let's do what
rustc
does". I'm very biased in that regard as I helped design rustc's CLI as well. Some things I like about rustc though are:
- The output of
rustc -h
fits more-or-less on one terminal. This serves as a good index from which you can learn more.- There are "option groups" that are behind other flags, such as
rustc -C foo
orrustc -Z foo
. These single-letter capital flags are easy to type and easy to remember and serve as a good place to house sub-options.- More information can be read through
rustc -C help
which provides an window into all the codegen capabilities of rustc- Instead of
--enable-foo
or--disable-foo
most rustc options are-C foo
for--enable-foo
or-C foo=no
for--disable-foo
. This makes it easy for callers to forcibly enable/disable something and you don't have to worry about what the defaults are since as a user you typically want to turn something on or off and you don't care too much about whatever the default happens to be.My proposal here for Wasmtime's option groups are:
Codegen options (
-C
or--codegen
)
- All
--disable-foo
flags become-C foo=no
- All
--enable-foo
flags become-C foo
-C foo
is shorthand for-C foo=yes
*=yes
can supporty
,yes
,true
, etc*=no
can supportn
,no
,false
, etcNon-boolean flags support
-C name=val
whereval
is parsed as whatever
name
wants.Can support comma-separation as well such as
-C foo,bar
or-C foo=n,bar
old cli flag new cli flag --compiler <COMPILER>
-C compiler=..
-O, --optimize
-C opt-level=N
/-O
--opt-level <LEVEL>
-C opt-level=N
--config <CONFIG_PATH>
-C cache-config=..
--disable-address-map
-C address-map=no
--disable-cache
-C cache=no
--disable-parallel-compilation
-C parallel-compilation=no
--epoch-interruption
-C epoch-interruption
-g
-C debuginfo
/-g
--dynamic-memory-guard-size <SIZE>
-C dynamic-memory-guard-size=...
--static-memory-forced
-C static-memory-forced
--static-memory-guard-size <SIZE>
-C static-memory-guard-size=N
--static-memory-maximum-size <MAXIMUM>
-C static-memory-maximum-size=N
--relaxed-simd-deterministic
-C relaxed-simd-deterministic
--cranelift-enable <SETTING>
-C cranelift-NAME
--enable-cranelift-debug-verifier
-C cranelift-debug-verifier
--enable-cranelift-nan-canonicalization
-C cranelift-nan-canonicalization
--cranelift-set <NAME=VALUE>
-C cranelift-NAME=VALUE
Runtime options (
-R
or--runtime
)
- Various
*-unknown-*
options are all-R unknown-*
now.- Same system for boolean flags as
-C
options
old cli flag new cli flag --allow-unknown-exports
-R unknown-exports-allow
--trap-unknown-imports
-R unknown-imports-trap
--default-values-unknown-imports
-R unknown-imports-default
--coredump-on-trap <PATH>
-R coredump=..
--disable-logging
-R logging=no
--disable-memory-init-cow
-R memory-init-cow=no
--dynamic-memory-reserved-for-growth <SIZE>
-R dynamic-memory-reserved-for-growth=...
--fuel <N>
-R fuel=N
--log-to-files
-R log-to-files
--max-instances <MAX_INSTANCES>
-R max-instances=N
--max-memories <MAX_MEMORIES>
-R max-memories=N
--max-memory-size <BYTES>
-R max-memory-size=N
--max-table-elements <MAX_TABLE_ELEMENTS>
-R max-table-elements=N
--max-tables <MAX_TABLES>
-R max-tables=N
--max-wasm-stack <MAX_WASM_STACK>
-R max-wasm-stack=N
--pooling-allocator
-R pooling-allocator
--trap-on-grow-failure
-R trap-on-grow-failure
--wasm-timeout <TIME>
-R timeout=N
WASI options (
-W
or--wasi
)
Separate from runtime options as this'll have WASI-proposal-specific
configuration options.Leave
--env
as its own standalone flag- Add support for
--env FOO
which inherits the env varFOO
from the outer
process.
old cli flag new cli flag --listenfd
-W listenfd
--tcplisten <SOCKET ADDRESS>
-W tcplisten=...
--wasi-modules <MODULE,MODULE,...>
-W common
--wasi-modules <MODULE,MODULE,...>
-W experimental-threads
--dir <DIRECTORY>
-W dir=..
--mapdir <GUEST_DIR::HOST_DIR>
-W dir=a::b
--env <NAME=VAL>
--env ...
Other flags
These are the other flags supported by Wasmtime which don't necessarily fit into the groups above. They're either important enough I think they should stick around at the top level or they have enough of their own "sub-syntax" that I'm not sure they fit well in the groups above.
--wasm-features <FEATURE,FEATURE,...>
- perhaps add something like a-F
shortcut, I find it a bit annoying to always type this out. Support--wasm-features help
for the extended help section on this.
--preload <NAME=MODULE_PATH>
- allow omittingNAME
and infer it by default
fromMODULE_PATH
.
--profile <STRATEGY>
- leave as-is--invoke <FUNCTION>
- leave as-is--allow-precompiled
- leave as-is--target <TARGET>
- leave as-is--output <OUTPUT>
- leave as-is--emit-clif <PATH>
- leave as-is
I'm curious what others think about this! If it's broadly amenable and folks are ok with the breakage I'm happy to implement this myself, I don't think it'll take all that long (perhaps just a day or so)
Last updated: Dec 23 2024 at 12:05 UTC