fad5a7101cc3dccbb525cfe9afc105aace8da88e ci: Add macOS cross task for arm64 (MarcoFalke)
fa8c750a0aff9c03270b71a91536639f3922eed8 ci: Refactor get_previous_releases step in win-test-cross task (MarcoFalke)
Pull request description:
Cross compiling to Intel macOS seems fine, but it would be good to cross compile to arm64-apple-darwin as well.
Further reading:
* https://en.wikipedia.org/wiki/Mac_transition_to_Apple_silicon#Timeline.
* It is harder to find native Intel macOS hardware (E.g. GitHub is in the process of dropping it: https://github.blog/changelog/2025-07-11-upcoming-changes-to-macos-hosted-runners-macos-latest-migration-and-xcode-support-policy-updates/#macos-13-is-closing-down)
ACKs for top commit:
Sjors:
utACK fad5a7101cc3dccbb525cfe9afc105aace8da88e
hodlinator:
crACK fad5a7101cc3dccbb525cfe9afc105aace8da88e
Tree-SHA512: ce96ac9f68f594584dc910555bd34590084e3e45ca02a22d4949e88bb569de3bf87ebf6b5c6718ae82d7750a98212b72f6dab80bddfc9652a57180fbdda97f42
444409ff2b78d8f3e541bd6e883af8da7adfd264 ci: Reduce Alpine musl task to md runner size (MarcoFalke)
fa6b2e9efece2d728bdc257c36c95db03e1a7bc4 ci: Turn centos config into alpine musl config (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/33437
Historically, the centos task was added to add CI coverage for old packages and 32-bit depends builds, but both are now covered by different tasks.
The CentOS task aligns with Ubuntu/Debian CI tasks in terms of libc usage, but (slightly) differs in package naming and update philosophy. I am not aware of the task ever discovering a centos-related issue, so it seems fine to recycle it into an Alpine Linux task.
The main difference would be that musl libc is now used. Also, busybox is used in Alpine, so in theory the busybox install could be removed from the arm CI task in the future.
Packaging considerations: All packages should roughly be the same (gcc remains at version 14, python remains at version 3.12, etc). Also, all packages are from the Alpine main track, coming with 2 years of support. The only exception is the py3-pip package (https://pkgs.alpinelinux.org/packages?name=py3-pip&branch=v3.22&repo=&arch=riscv64) from the community track, however, I don't expect any issues arising from that.
ACKs for top commit:
janb84:
reACK 444409ff2b78d8f3e541bd6e883af8da7adfd264
willcl-ark:
ACK 444409ff2b78d8f3e541bd6e883af8da7adfd264
Tree-SHA512: fd1a1da0fd766591e44a57dbdb84f9b3b47ca92113a429bba139ee5fef54714b8fe509c321e7b3a470c29b4af7d9eab9786e1660b9effb862ecea52824f458aa
156927903d64297500dd73380908c654b07bfb1a ci: Check windows manifests for all executables (Max Edwards)
e1a1b14c9359751a4d0117a27a303d1f1d3ed30f ci: use a more generic way of finding mt.exe (Max Edwards)
7ae0497eef8f5b37fc1184897a5bbc9f023dfa67 ci: remove 3rd party js from windows dll gha job (Max Edwards)
Pull request description:
The windows job uses the external dependency `ilammy/msvc-dev-cmd` which runs javascript. We use this to put various tools on the path such as `MSBuild.exe` and `mt.exe`. We can remove this dependency and use `vswhere.exe` directly to find these tools and create a "[Developer command prompt](https://github.com/microsoft/vswhere/wiki/Start-Developer-Command-Prompt#using-powershell)" as someone would on their dev machine.
While in this area of the code, this PR also runs some additional manifest checks on the windows binaries.
Fixes: #32508
ACKs for top commit:
davidgumberg:
crACK 156927903d64297
hebasto:
ACK 156927903d64297500dd73380908c654b07bfb1a.
Tree-SHA512: df640dff27579a1c95daddc5a5ba8fd655bbd0a6f2aff74d0f63439c7185c0b18a90abfee3f1f032fe833cd19b822ef71812f44b24c4c044222e46d01c271864
The other executables have manifests and these should be checked in
addition to bitcoind. Skipping fuzz.exe, bench_bitcoin.exe and
test_bitcoin-qt.exe as they do not have manifests.
This sets up a vs developer command prompt and should hopefully should
be more resilient to upstream changes
Co-authored-by: David Gumberg <davidzgumberg@gmail.com>
We can use vswhere.exe directly to create a vs developer
prompt and so can remove this third party dependency.
Co-authored-by: David Gumberg <davidzgumberg@gmail.com>
Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>
Add an optional matrix field allowing opt-out of configuring cirrus
GHA cache when not using cirrus runners.
This is not needed for the cirruslabs/[save|restore]-cache actions, as
they automatically fallback based on runner type.
fa8f081af31cd99155c7545643e7b10beb26714d ci: Checkout latest merged pulls (MarcoFalke)
Pull request description:
Currently, the `actions/checkout@v5` checks out pull requests merged against master, which is what we want.
However, it checks out ancient/stale merge commits on a re-run. This is documented (https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs):
> Re-run workflows [...] will also use the same GITHUB_SHA (commit SHA) and GITHUB_REF (git ref) of the original event that triggered the workflow run.
For example:
* https://github.com/bitcoin/bitcoin/actions/runs/17458152407/job/49579638898?pr=29641#step:9:914 compiles with IPC=ON, even though latest master is at ed2ff3c63d83e275b0785a695fa4db38870abf1a
* https://github.com/bitcoin/bitcoin/pull/32989#issuecomment-3133536724 (example explained in comment)
This is problematic, because:
* Unrelated CI failures and intermittent issues, which are fixed or worked around in latest master can not be cleaned by re-running the task. The author has to actively go out and (force-)push the branch, invalidating review.
* It is odd to have a recent CI run, but it uses code and config from the past.
* Detecting silent merge conflicts by re-running the CI task is impossible.
Fix all issues by checking out the latest merged state of the pull request. The behavior is unchanged for non-pull-request actions. This patch changes the "re-run" default behaviour. Forcing it to use the new state instead of running the old state again.
ACKs for top commit:
janb84:
re ACK fa8f081af31cd99155c7545643e7b10beb26714d
hebasto:
ACK fa8f081af31cd99155c7545643e7b10beb26714d.
Tree-SHA512: c22c6f837402f61ec46be46817473e1946424b5312e36ed0e246cadb1ca89c04163bb471f71c309765a3d327f198a83cd83679d231f03828a99a97562a622fdd
5eeb2facbbbbf68a2c30ef9e6747e39c85d7b116 ci: reduce runner sizes on various jobs (will)
Pull request description:
These jobs can likely use reduced runner sizes to avoid wasting our CPU quota, as much of the long-running part of the job is single-threaded.
This will also give us more (job) parallelisem from the same number of CPU that we are using.
Suggested in: https://github.com/bitcoin/bitcoin/pull/32989#discussion_r2321775620
ACKs for top commit:
kevkevinpal:
ACK [5eeb2fa](5eeb2facbb)
m3dwards:
ACK 5eeb2facbbbbf68a2c30ef9e6747e39c85d7b116
janb84:
ACK 5eeb2facbbbbf68a2c30ef9e6747e39c85d7b116
Tree-SHA512: 6fb0352bc40623dd63b9bd6169d753d1ec9667c272445fda7a2db8bbedfa35350a51d08c1adf3fa5e070e84855c3f491668726d3c7ded07a39f2f9c63edacefc
Set new `BitcoinTestFramework.binary_paths.bitcoin_bin` property with path to
the `bitcoin` wrapper binary. This allows new tests for `bitcoin-mine` in
#30437 and `bitcoin-cli` in #32297 to find the `bitcoin` binary and call
`bitcoin -m` to start nodes with IPC support. This way the new tests can run
whenever the ENABLE_IPC build option is enabled, instead of only running when
the `BITCOIN_CMD` environment variable is set to `bitcoin -m`
Previously jobs were running on a large multi-core server where 10 jobs
as default made sense (or may even have been on the low side).
Using hosted runners with fixed (and lower) numbers of vCPUs we should
adapt compilation to match the number of cpus we have dynamically.
This is cross-platform compatible with macos and linux only.
To remove multiple occurances of the respository name, against which we
compare `${{ github.repository }}` to check if we should use Cirrus
Runners, introduce a helper job which can check a single environment
variable and output this as an input to subsequent jobs.
Forks can maintain a trivial patch of their repo name against the
`REPO_USE_CIRRUS_RUNNERS` variable in ci.yml if they have Cirrus Runners
of their own, which will then enable cache actions and docker build
cache to use Cirrus Cache.
It's not possible to use `${{ env.USE_CIRRUS_RUNNERS }}` in the
`runs-on:` directive as the context is not supported by GitHub.
If it was, this job would no longer be necessary.
Whilst the action cirruslabs/actions/cache will automatically set this
host, the docker `gha` build cache backend will not be aware of it.
Set the value here, which will later be used in the docker build args to
enable docker build cache on the cirrus cache.
If set, Cirrus runners will be used on pushes to, and pull requests
against, this repository.
Forks can set this if they have their own cirrus runners.
941b8f54c0d35d3243bb6083f3b52681d1b9a555 ci: run get_previous_releases as part of test cross win job (Max Edwards)
5e2182140bcd26afbe89b13d1d83ece6d5a89731 test: increment mocked time for migrating wallet backups (Max Edwards)
5174565802f426db1e9c8200a9421f82f61a5c99 ci: disable feature_unsupported_utxo_db functional test (Max Edwards)
3dc90d69a64f8bb39af27fa755683589b1bc76a7 test: remove mempool.dat before copying (Max Edwards)
67a6b20d5030ee51bf6d5e0fd77c9bdc8bafa96b test: add windows support to get previous releases script (Max Edwards)
1a1b478ca31be1f670754a47da17863271e46b7b scripted-diff: rename tarball to archive (Max Edwards)
4f06dc848460c887ad8337702ed900ba78725906 test: remove building from source from get prev releases script (Max Edwards)
Pull request description:
This PR updates the `test/get_previous_releases.py` script to also work on Windows by changing to be pure python rather than using unix tools such as `curl` and `tar`.
This enables additional functional tests to run such as `wallet_migration.py`, `mempool_compatability.py` and `wallet_backwards_compatibility.py`.
Unfortunately `feature_unsupported_utxo_db.py` _could_ run but this test requires Bitcoin `v0.14.3` which will not run under windows with emojis in the data directory (as the functional test runner has by default) . This test could be run as it's own step in the ci workflow file and would pass but as it's quite an old version / feature I have assumed it's not worth worrying about and best just to exclude.
Two tests needed to be slightly modified to run under windows. Both were issues with trying to overwrite a file that already exists which windows seems to be more strict on than the unix based systems.
Finally, building from source has been dropped from the `get_previous_releases.py` script. This had not been updated after the move to cmake and so it was assumed that nobody could have been using that feature.
ACKs for top commit:
maflcko:
re-ACK 941b8f54c0d35d3243bb6083f3b52681d1b9a555 🍪
achow101:
ACK 941b8f54c0d35d3243bb6083f3b52681d1b9a555
hodlinator:
re-ACK 941b8f54c0d35d3243bb6083f3b52681d1b9a555
Tree-SHA512: 22933d0ec278b9b0ffcd2a8e90026e1a3631b00186e7f78bd65be925049021e319367d488c36a82ab526a07b264bac18c2777f87ca1174b231ed49fed56d11cb
This test required v0.14.3 which doesn't support special characters like
emojis in the datapath on windows. Functional test runner includes a few
emojis in it's default data directory.
