Stream: warg

Topic: wasm-pkg registry config file format


view this post on Zulip Lann Martin (May 28 2024 at 13:14):

I have a draft PR up reworking the ad-hoc config file introduced by wasm-pkg-loader into a new wasm-pkg-common crate (along with some common packaging-related types).

Contribute to bytecodealliance/wasm-pkg-tools development by creating an account on GitHub.

view this post on Zulip Lann Martin (May 28 2024 at 13:20):

tldr:

# Override default registry
default_registry = "bytecodealliance.org" # this is the hard-coded default

[package_registries]
# Namespace prefix mapping is made more explicit with wildcard syntax
"wasi:*" = "wasi.dev"
# Allow mapping individual packages too
"example:foo" = "example.com"

[registry."bytecodealliance.org"]
# Optional backend type will override automatic detection (`.well-known/wasm-pkg/registry.json`)
type = "warg"

# Backend-specific config is opaque to the wasm-pkg-common crate; individual "sources" will parse these
[registry."wasi.dev".oci]
url = "https://private-registry.example.com"
auth = { username = "open", password = "sesame" }

# Alternative syntax with inline backend config table
[registry."wa.dev"]
# type = "warg" <- implicit due to well-known config
warg = { auth_token = "top_secret" }

view this post on Zulip Robin Brown (May 28 2024 at 13:50):

I don't think allowing or encouraging inline passwords/secrets is a good idea

view this post on Zulip Lann Martin (May 28 2024 at 13:55):

Eh yeah that isn't really part of the change just the first thing I came up with for the example

view this post on Zulip Lann Martin (May 28 2024 at 13:59):

Backend-specific config is opaque to the wasm-pkg-common crate; individual "sources" will parse these

I haven't actually integrated this part yet, which would produce more realistic examples

view this post on Zulip Lann Martin (May 28 2024 at 14:06):

Mostly interested in initial feedback on the top-level structure: [package_registries] and [registries.<name>.<backend>]. We'll probably be stuck with at least backward-compat code for whatever we land on for a while.

view this post on Zulip Robin Brown (May 28 2024 at 14:10):

Ya, I'm still thinking about the [package_registries] bit.

view this post on Zulip Robin Brown (May 28 2024 at 14:11):

I think it's good to offer people the tools to have local registry mappings, but the more we encourage people to do so, the more likely it is that when they go to push something its dependencies aren't compatible with the registry it's being published to in a way that breaks resolving

view this post on Zulip Lann Martin (May 28 2024 at 14:15):

I agree that it doesn't make a lot of sense at the global level, but I think its probably a requirement at the workspace level.

view this post on Zulip Lann Martin (May 28 2024 at 14:16):

Implicit here is the idea that this config file format would also apply to workspaces (e.g. ./.wasm-pkg/config.toml)

view this post on Zulip Lann Martin (May 28 2024 at 14:18):

https://github.com/bytecodealliance/wasm-pkg-tools/pull/25/files#diff-6c0858bfb1633845930b97569b9f9dd27862be071a56ea31cc00025a55094fe6R54-R62

I want to solidify a config format as at least a medium-term compatibility guarantee; we can always introduce a config file versioning scheme for revisions later if needed, but its nice to avoid to...

view this post on Zulip Robin Brown (May 28 2024 at 15:23):

Sure, the risk of making packages that aren't valid in your home registry is still present though.

view this post on Zulip Robin Brown (May 28 2024 at 15:24):

I'm not sure about the granularity here, does it make sense to make all mappings package-level and offer wildcards? I'd be tempted to let people only configure whole namespaces initially.

view this post on Zulip Robin Brown (May 28 2024 at 15:25):

In that same vein, what do you think about making this initial version rather minimal and expand as we have use cases / requests for things?

view this post on Zulip Robin Brown (May 28 2024 at 15:25):

Also, as an aside: I'd be interested in a workspace-level config that says "don't use any defaults from anywhere else everything must be specified here" that projects interested in high amounts of reproducibility can turn on.

view this post on Zulip Lann Martin (May 28 2024 at 15:26):

Admittedly the feature is currently driven by the very simple need to be able to override the registry for the root component, but I do think that the ability to patch single dependencies is too useful to ignore.

view this post on Zulip Lann Martin (May 28 2024 at 15:28):

If you patch in an incompatible component then composition will (hopefully/usually) fail, right?

view this post on Zulip Lann Martin (May 28 2024 at 15:30):

I guess an open question is whether a single overridden component's own dependencies would then be taken from that registry? :thinking:

view this post on Zulip Lann Martin (May 28 2024 at 15:30):

actually that question applies equally to namespace overrides

view this post on Zulip Lann Martin (May 29 2024 at 20:15):

Updated PR based on feedback:

# Override default registry
default_registry = "bytecodealliance.org" # this is the hard-coded default

[namespace_registries]
wasi = "wasi.dev"

[package_registry_overrides]
"example:foo" = "example.com"

[registry."bytecodealliance.org"]
# Optional backend type will override automatic detection (`.well-known/wasm-pkg/registry.json`)
type = "warg"
warg = { url = "https://warg.example.com" }

# Backend-specific config is opaque to the wasm-pkg-common crate; individual "sources" will parse these
[registry."wasi.dev".oci]
registry = "private-registry.example.com"

Last updated: Nov 22 2024 at 17:03 UTC