Re: Heads Up: cirrus-ci is shutting down June 1st

On Mon, Apr 13, 2026 at 11:53 PM David Steele <david(at)pgbackrest(dot)org> wrote:
> On 4/10/26 06:29, Thomas Munro wrote:
> > Nested virtualisation to the rescue?
> >
> > https://github.com/cross-platform-actions/action
>
> I used this to migrate our FreeBSD tests [1] and it worked out OK. The
> only downside is it doesn't look like you can split out steps so all the
> commands end up logged together.

- name: Run Test
uses: cross-platform-actions/action(at)v1(dot)0(dot)0
with:
operating_system: freebsd
version: ${{matrix.os.version}}
run: |
uname -a
sudo pkg update && sudo pkg upgrade -y libiconv && sudo
pkg install -y bash git postgresql-libpqxx pkgconf libxml2 gmake perl5
libyaml p5-YAML-LibYAML rsync meson
cd .. && perl ${GITHUB_WORKSPACE?}/test/test.pl --vm-max=2
--no-coverage --no-valgrind --module=command --test=backup
--test=info --test=archive-push

Nice!

I guess the problems with this are:

1. It has to install the packages every time because it's not yet
using a pre-prepared image.
2. It has no ccache memory.
3. It has lost all that user-friendly stuff like artefact
archival/browsing, core file debugging etc.
4. IIRC the log URLs are not "public", you have to have to be logged
into an account to view them.

(That 4th point was one of Cirrus's unique advantages at the time we
selected it. We wanted to be able to share URLs for discussion on the
mailing list without requiring everyone to be a GitHub user.)

Perhaps for point 1, we could publish fast-start qemu images for
Debian, FreeBSD, NetBSD, OpenBSD*. The pg-vm-images repo that Andres
and Bilal maintain currently uploads images to Google Cloud's image
repository where Cirrus VMs can boot from them, but it could instead
publish qemu images to our own public URLs. I'm picturing a bunch of
images available as
https://ci.postgresql.org/images/qemu/arm64/{freebsd-15,debian-13,...}-with-postgresql-dependencies.img.
The point of per-arch variants would be to match common hosts for fast
kernel hypervisor support, eg on a Mac you want arm64, though you
could still run amd64 slowly if you need to. Could even do ppc and
riscv with emulation.

Qemu images should hopefully be usable in many different environments:

1. We could run them locally with some one-button command, and also
have images you can log into and hack on if you want.
2. We could run all of them or just the license-encumbered ones on
public clouds (not through a CI service) with some one-button command,
if you have an account.
3. We could use them in people's private GitHub/GitLab/... accounts
as you showed, just add
image_url=https://ci.postgresql.org/images/qemu/....
4. Cfbot could do any of those things, not sure what would be best.

For the license-encumbered OSes, we could at least make disk images or
archives containing a MacPorts installation or
bunch-of-installed-libraries-for-Windows, but not including the OS.
Just mount/unpack as /opt or C:\pg-packages or whatever, I guess, if
you can figure out how to get a VM running ... somewhere. Perhaps
there is some way to make project-owned resources (MacMinis, Windows
VMs) available to our community too, but IDK how that would work.

Some random half-baked thoughts about the ccache, browsing, etc problems:

1. Local qemu: we could use overlay images so that your downloaded
copy of X-with-postgres-dependencies.img remains read-only. Create a
new empty overlay image for each clean run, and if you need to inspect
logs, core files, you can just log in before the next run wipes it.
2. Local qemu: we could mount a separate disk image as /cache that
survives between runs and can be wiped any time.
3. Public CI system like GitHub actions: I suppose we could run our
own ccache, artefact, log hosting service that it could push to...
that was something I already wondered about under Cirrus due to
various disk space and retention problems... but I'm quite hesitant to
get tangled up in running "public" services and unsure how you'd
control access.

I would at least like to think about trying to make cfbot
capitalism-proof. I may be underestimating the difficulty, but I keep
wondering if cfbot should at least be able to do everything itself,
with some combination of local qemu, qemu-on-project-Mac-fleet, and
public cloud VMs controlled directly. It doesn't really *need* to
depend on ephemeral venture capital-powered CI companies, it was just
nice to make it use the exact same CI setup as you could use for
yourself in your GitHub account. I'm imagining that it would still
push branches to GitHub, since that's a nice interface to browse code
on, and I suppose it might even be possible to publish our own
minimalist GitHub plugin that allows cfbot to push its green/red
result indicators to it since that's clearly something that external
providers can do (as well as pushing them to the commitfest UI as
now). But if you clicked them, you'd be taken to a really primitive
cfbot web interface where you could browse logs and artefacts retained
for N days. In other words, an extremely cut down and limited
"let's-make-our-own-CI" project, which doesn't have to tackle the much
harder "let's-make-our-own-semi-public-CI-platform" project. I like
the idea of at least having such a mode as an insurance policy anyway,
but I'm not sure what nitty gritty details might make it hard to pull
off... In this thought experiment, people could continue to work
separately on making personal CI work in various ways, GitHub, GitLab,
whatever else, and local, and all ways of doing it would be using the
same scripts and VM images.

* ... and AFAIK we could add illumos to the set if we wanted, in the
past a couple of us tried to get that going but ran into ... I think
it was driver problems? ... when using GCP VMs, but it definitely
works in qemu VMs as that cross-platform-actions project shows. Every
OS project makes sure it can boot in qemu. Even AIX can boot in qemu,
if you have a license.