Stream: git-wasmtime

Topic: wasmtime / issue #4570 [Downstream Usability] Autobuild R...

Wasmtime GitHub notifications bot (Aug 01 2022 at 13:49):

lsm5 opened issue #4570:

This is a not a request for improvement in wasmtime itself, but rather in downstream ease-of-use which will need some upstream changes. I will submit a PR for this if upstream is cool with accepting it :crossed_fingers:

Feature

Enable automatic rpm package builds on every upstream PR merge.

Benefit

1. Users can conveniently use the latest features available upstream
2. No need to wait for upstream to cut a new release
3. Near-immediate availability to distro users
4. Delivered way quicker compared to official distro packages

Implementation

This will involve:

1. adding a .copr/Makefile and a .copr/wasmtime.spec such that:
   make -f .copr/Makefile srpm generates an srpm file which the copr infrastructure can use to build rpms on their build infra.

An example can be found for crun along with a successful rpm build.

We also build our other packages including Podman, Buildah and Skopeo.

2. adding a webhook to the wasmtime repo to trigger builds on the copr after every PR merge.

Alternatives

1. Get users to build from source
2. Wait for distro package to ship required features
3. I could do copr integration from a fork which would have to be perpetually rebased (i really hope not :sweat: :smile: )

/cc @giuseppe @font

Wasmtime GitHub notifications bot (Aug 01 2022 at 13:50):

lsm5 edited issue #4570:

This is a not a request for improvement in wasmtime itself, but rather in downstream ease-of-use which will need some upstream changes. I will submit a PR for this if upstream is cool with accepting it :crossed_fingers:

Feature

Enable automatic rpm package builds on every upstream PR merge.

Benefit

1. Users can conveniently use the latest features available upstream
2. No need to wait for upstream to cut a new release
3. Near-immediate availability to distro users
4. Delivered way quicker compared to official distro packages

Implementation

This will involve:

1. adding a .copr/Makefile and a .copr/wasmtime.spec such that:
   make -f .copr/Makefile srpm generates an srpm file which the copr infrastructure can use to build rpms on their build infra.

An example can be found for crun along with a successful rpm build.

We also build our other packages including Podman, Buildah and Skopeo similarly.

2. adding a webhook to the wasmtime repo to trigger builds on the copr after every PR merge.

Alternatives

1. Get users to build from source
2. Wait for distro package to ship required features
3. I could do copr integration from a fork which would have to be perpetually rebased (i really hope not :sweat: :smile: )

/cc @giuseppe @font

Wasmtime GitHub notifications bot (Aug 01 2022 at 13:54):

lsm5 edited issue #4570:

This is a not a request for improvement in wasmtime itself, but rather in downstream ease-of-use which will need some upstream changes. I will submit a PR for this if upstream is cool with accepting it :crossed_fingers:

Feature

Enable automatic rpm package builds on every upstream PR merge.

Benefit

1. Users can conveniently use the latest features available upstream
2. No need to wait for upstream to cut a new release
3. Near-immediate availability to distro users
4. Delivered way quicker compared to official distro packages

Implementation

This will involve:

1. adding a .copr/Makefile and a .copr/wasmtime.spec such that:
   make -f .copr/Makefile srpm generates an srpm file which the copr infrastructure can use to build rpms on their build infra.

An example can be found for crun along with a successful rpm build.

We also build our other packages including Podman, Buildah and Skopeo similarly.

2. adding a webhook to the wasmtime repo to trigger builds on the copr after every PR merge.

Alternatives

1. Get users to build from source
2. Wait for distro package to ship required features
3. I could do copr integration from a fork which would have to be perpetually rebased (I really hope not :sweat: :smile: )

/cc @giuseppe @font

Wasmtime GitHub notifications bot (Aug 01 2022 at 13:57):

lsm5 edited issue #4570:

This is a not a request for improvement in wasmtime itself, but rather in downstream ease-of-use which will need some upstream changes. I will submit a PR for this if upstream is cool with accepting it :crossed_fingers:

Feature

Enable automatic rpm package builds on every upstream PR merge.

Benefit

1. Users can conveniently use the latest features available upstream
2. No need to wait for upstream to cut a new release
3. Near-immediate availability to distro users
4. Delivered way quicker compared to official distro packages

Implementation

This will involve:

1. adding a .copr/Makefile and a .copr/wasmtime.spec such that:
   make -f .copr/Makefile srpm generates an srpm file which the copr infrastructure can use to build rpms on their build infra.

An example can be found for crun along with a successful rpm build.

We also build our other packages including Podman, Buildah and Skopeo similarly.

2. adding a webhook to the wasmtime repo to trigger builds on the copr after every PR merge.

Alternatives

1. Get users to build from source
2. Wait for distro package to ship required features
3. I could do copr integration from a fork which would have to be perpetually rebased (I really hope not :sweat: :smile: )

/cc @giuseppe @font @flouthoc

Wasmtime GitHub notifications bot (Aug 01 2022 at 14:31):

alexcrichton commented on issue #4570:

Personally I think this would be reasonable to add, although I have no experience myself building rpms in CI. I'd be happy to help review a PR to do so though! I suspect this wouldn't add too much time to CI and it should be fine to throw in the set of other release artifacts generated.

Wasmtime GitHub notifications bot (Aug 01 2022 at 14:58):

lsm5 commented on issue #4570:

Personally I think this would be reasonable to add, although I have no experience myself building rpms in CI. I'd be happy to help review a PR to do so though! I suspect this wouldn't add too much time to CI and it should be fine to throw in the set of other release artifacts generated.

@alexcrichton thanks a lot! RE: PR reviews, as long as we have successful copr builds with usable packages, we should be good to go. I guess @flouthoc and @font can help with testing those as well.

RE: CI time, this by itself will not add any CI time because all the Copr magic will occur post-merge, unless we add a test to generate the SRPM package (would be nice to have, but no biggie)

RE: artifacts, those would be generated on the Copr repo, for example: https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/04692536-crun/ lists the packages and logs for crun.

Wasmtime GitHub notifications bot (Aug 01 2022 at 16:00):

alexcrichton commented on issue #4570:

Hm sorry I may not understand then. I don't know what Copr is which also may not help. Currently we produce binary artifacts in CI for all runs (it's the dev tag in this repository) and I naively assumed that *.rpm packages would be built on CI in the same manner as the release binaries.

From what you mention though it may be the case that Copr is an external system which our CI/release process interacts with, but I'm not surew hat that would look like.

Wasmtime GitHub notifications bot (Aug 01 2022 at 16:57):

cfallin commented on issue #4570:

Copr is a build service that the Fedora project runs for individually-maintained add-on package repos (sort of like PPAs for Ubuntu). My understanding of @lsm5 's proposal above is that we would:

1. Accept some metadata (.copr subdirectory) into our repo;
2. Set a webhook notification so that updates to main in our repo trigger the Copr system.

@lsm5 item 1 above sounds totally reasonable to me. Re: item 2, it's technically possible, though I'm not sure how we feel about a precedent of lots of "push notifications" to downstream consumers (others have thoughts here?). Is there a way to set up your Copr repository so it polls our main branch at some interval?

Wasmtime GitHub notifications bot (Aug 01 2022 at 17:41):

lsm5 commented on issue #4570:

Copr is a build service that the Fedora project runs for individually-maintained add-on package repos (sort of like PPAs for Ubuntu).

Yes, exactly. People can host rpm packages without waiting for official Fedora reviews and it also allows for a lot of build customization not available on official Fedora environments. It's a lot quicker to ship things via Copr than via official Fedora channels.

Once the copr for wasmtime is in place, an interested end user would have to:

$ sudo dnf copr enable $COPR_ACCOUNT_NAME/wasmtime
$ sudo dnf install wasmtime

@lsm5 item 1 above sounds totally reasonable to me. Re: item 2, it's technically possible, though I'm not sure how we feel about a precedent of lots of "push notifications" to downstream consumers (others have thoughts here?). Is there a way to set up your Copr repository so it polls our main branch at some interval?

The webhook notifications will not bother any human users. It will only trigger new builds on the Copr system. And another thing I'd like to clarify, the rpm that will be available on the Copr will not be the official Fedora / CentOS package. Users will have to explicitly enable it on their environments. And any users that enable it will not be bombarded with any notifications. Just that the likelihood of them finding newer wasmtime packages is high with this Copr repo enabled compared to just the default distro package repos.

For the crun package, if you see the Default Build Source section in https://copr.fedorainfracloud.org/coprs/rhcontainerbot/podman-next/package/crun/ it has Committish: main, so it only triggers builds after each PR merge on the main branch.

So, long story short, nobody (upstream / downstream regardless) notices any difference unless they actually bother with enabling the Copr repo.

Wasmtime GitHub notifications bot (Aug 01 2022 at 17:49):

lsm5 commented on issue #4570:

btw, here's a relevant blog post which can give you a better idea with screenshots: https://hobo.house/2017/09/03/automate-rpm-builds-from-git-sources-using-copr/

Wasmtime GitHub notifications bot (Aug 01 2022 at 17:52):

lsm5 commented on issue #4570:

1. Accept some metadata (.copr subdirectory) into our repo

@cfallin yes, I'll PR for that soon-ish.

Wasmtime GitHub notifications bot (Aug 11 2022 at 15:15):

lsm5 commented on issue #4570:

@alexcrichton @cfallin Couple of things:

1. The webhook that would need to be added to wasmtime's settings will look like:


Of course, the exact URL would change based on the copr project which brings me to 2...

2. I'm hoping to make a docs PR to update installation instructions once #4688 is merged. So, I'd like to check with you what copr repo account_name/project_name you would prefer. The current repo I used for doing test builds is my personal lsm5/wasmtime repo and most of the packages I maintain related to podman go to rhcontainerbot/podman-next. Would you prefer something like a bytecodealliance/wasmtime or are you ok with rhcontainerbot/podman-next itself?
   

Wasmtime GitHub notifications bot (Aug 19 2022 at 19:30):

lsm5 commented on issue #4570:

@alexcrichton @cfallin @jameysharp gentle ping for thoughts on the previous comment + review on #4688 . Thanks!

Wasmtime GitHub notifications bot (Aug 22 2022 at 17:28):

jameysharp commented on issue #4570:

I suspect the silence here means that nobody on the core Wasmtime team wants to take responsibility for RPM builds. We try to make sure that if we merge something, we have a plan for how to maintain it. So while I think we're all interested in helping you make Wasmtime available to more people, I share some concerns that others have brought up. Let's see if we can talk through those.

The first question to address is who manages the Fedora account that publishes these packages. Since I don't think any of us feel prepared to do that, it shouldn't use the Bytecode Alliance name. I guess rhcontainerbot/podman-next is fine. But because we aren't familiar with Copr and won't have access to the Fedora account, we can't address any packaging-related bug reports, so I think it's important that you direct users of these packages somewhere other than this repo's issue tracker.

Then there's the question of the webhook configuration. I think it's a little weird if we have to manage a webhook that's sending notifications to an account that we don't control. Is it possible for you to manage the webhook without our involvement? I see GitHub supports PubSubHubbub which I think means you could create a webhook for any repo without needing admin access to that repo, but I haven't tried it.

The easy question is what to do with #4688. Since that just adds files in an out-of-the-way directory, I don't think anybody has any objections to merging it. As long as it's not likely to change often, it's fine. And when it does need to change, it's not a big deal for us to just approve and merge PRs.

In summary, if we can minimize how much the Wasmtime team has to be responsible for going forward, then we're happy to help you provide bleeding-edge packages for RPM-based distros. Thank you for your interest!

Wasmtime GitHub notifications bot (Aug 22 2022 at 19:16):

lsm5 commented on issue #4570:

I suspect the silence here means that nobody on the core Wasmtime team wants to take responsibility for RPM builds. We try to make sure that if we merge something, we have a plan for how to maintain it. So while I think we're all interested in helping you make Wasmtime available to more people, I share some concerns that others have brought up. Let's see if we can talk through those.

The first question to address is who manages the Fedora account that publishes these packages. Since I don't think any of us feel prepared to do that, it shouldn't use the Bytecode Alliance name. I guess rhcontainerbot/podman-next is fine. But because we aren't familiar with Copr and won't have access to the Fedora account, we can't address any packaging-related bug reports, so I think it's important that you direct users of these packages somewhere other than this repo's issue tracker.

Then there's the question of the webhook configuration. I think it's a little weird if we have to manage a webhook that's sending notifications to an account that we don't control. Is it possible for you to manage the webhook without our involvement? I see GitHub supports PubSubHubbub which I think means you could create a webhook for any repo without needing admin access to that repo, but I haven't tried it.

The easy question is what to do with #4688. Since that just adds files in an out-of-the-way directory, I don't think anybody has any objections to merging it. As long as it's not likely to change often, it's fine. And when it does need to change, it's not a big deal for us to just approve and merge PRs.

In summary, if we can minimize how much the Wasmtime team has to be responsible for going forward, then we're happy to help you provide bleeding-edge packages for RPM-based distros. Thank you for your interest!

Ack, thanks @jameysharp . Let me get back to you on this after checking with my colleagues. Might be a week or more.

Last updated: Nov 22 2024 at 16:03 UTC