d03e3be246f64065002268e74ee9a834089de37a ci: check macos bundle structure and codesigning (fanquake)
66d80d57b48982d8301e21321a9d82780586908d macdeploy: use plugins dir to find plugins (fanquake)
ab137cbfe2763f2306e52c7c5f1860d87defc636 macdeploy: subprocess out to zip rather than shutil.make_archive (fanquake)
Pull request description:
Fix bundle format.
Add a CI check that codesigning works.
Fixes#34744.
ACKs for top commit:
Sjors:
tACK d03e3be246f64065002268e74ee9a834089de37a
hebasto:
ACK d03e3be246f64065002268e74ee9a834089de37a, tested on macOS Tahoe 26.3.1.
sedited:
ACK d03e3be246f64065002268e74ee9a834089de37a
Tree-SHA512: 5a7db896952edf338ff4fe8c934f1e1c992642850a99d5fafbb1212c6979601b3b72b6f3af880fb6f6ac8759cd4102e9f01792abb05410ceaf36cbffaec48e47
fafdb8f635bc157f55e23890264d12170ecd41ae ci: Allow running iwyu ci in worktree (MarcoFalke)
fab73e213dee1057e9e759133767b17ec5b1f6ab ci: Reject unsafe execution of shell scripts (MarcoFalke)
Pull request description:
Currently, the iwyu CI fails to run in a git-worktree, or git-archive. This is due to the use of `git diff`.
Fix this by force-initializing a dummy git repo with a single dummy commit.
It may be possible to detect when `git diff` is not available in the directory, and only apply the fallback when needed, but the git history is not needed and it is easier to unconditionally apply the git init.
ACKs for top commit:
willcl-ark:
reACK fafdb8f635bc157f55e23890264d12170ecd41ae
hebasto:
ACK fafdb8f635bc157f55e23890264d12170ecd41ae, I have reviewed the code and it looks OK. Tested on Fedora 43.
sedited:
ACK fafdb8f635bc157f55e23890264d12170ecd41ae
Tree-SHA512: 572f1e2b9e215c2804095382498abb5b8636e3a49d5ba2a736b975e06afa2881d815b854a8a593d0f187c7c6b55034688e11f46d6814edfe7c29505197e80b18
9f3752c43778b7803f4c6810aa570542847f3261 ci: use latest versions of lint deps (fanquake)
Pull request description:
Use the latest available versions, except for LIEF, which is changed with Guix.
ACKs for top commit:
hebasto:
ACK 9f3752c43778b7803f4c6810aa570542847f3261, I've verified the releases against https://pypi.org and https://github.com/becheran/mlc.
Tree-SHA512: e6ed79bb7dc8601ed0708eb7b53cbf4cf843b69829c073c41e9d97be690b4b2bf9ea5ecf250e05cbacba4ad35df06aa3e2cb2ff319145a34e1a7831cf182ec21
24699fec8422a4d9219f8c5272370351e7adea7f doc: Add initial asmap data documentation (Fabian Jahr)
bab085d282b1ad1790861d710fd570f8531c9364 ci: Use without embedded asmap build option in one ci job (Fabian Jahr)
e53934422a29bdcb022d32f8eb6e171218cd3a26 doc: Expand documentation on asmap feature and tooling (Fabian Jahr)
6244212a5532a8a625e344fdbc8144f4befdd385 init, net: Implement usage of binary-embedded asmap data (Fabian Jahr)
6202b50fb9003a4feadd879ae189ee6f730e8155 build: Generate ip_asn.dat.h during build process (Fabian Jahr)
634cd60dc8f646b25701c45ac35a1175ce4c4da9 build: Add embedded asmap data (Fabian Jahr)
Pull request description:
This is the final in a series of PRs that implement the necessary changes for embedding of asmap data into the binary. This last part add the initial asmap data, implements the build changes and adds further documentation.
Currently an asmap file needs to be acquired by there user from some location or the user needs to generate one themselves. Then they need to move the file to the right place in datadir or pass the path to the file as `-asmap=PATH` in order to use the asmap feature. The change here allows for builds to embed asmap data into the bitcoind binary which makes it possible to use the feature without handling of the asmap file by the user. If the user starts bitcoind with `-asmap` the embedded data will be used for bucketing of nodes.
The data lives in the repository at `src/node/data/ip_asn.dat` and can be replaced with a new version at any time. The idea is that the data should be updated with every release. By default the data at that location is embedded into the binary but there is also a build option to prevent this (`-DWITH_EMBEDDED_ASMAP=OFF`). In this case the original behavior of the `-asmap` option is maintained.
ACKs for top commit:
achow101:
ACK 24699fec8422a4d9219f8c5272370351e7adea7f
sipa:
ACK 24699fec8422a4d9219f8c5272370351e7adea7f
hodlinator:
ACK 24699fec8422a4d9219f8c5272370351e7adea7f
Tree-SHA512: c2e33dbeea387efdfd3d415432bf8fa64de80f272c1207015ea53b85bb77f5c29f1dae5644513a23c844a98fb0a4bb257bf765f38b15bfc4c41984f0315b4c6a
The shell scripts are inherently unsafe, because they will install new
software packages, modify global configuration settings, write to the
root / or $HOME, and possibly modify the git repo.
The only safe way to run them is through the CI system itself, that is
the ci_exec python function.
The ci_exec funtion ensures that the user has set up a sandbox
externally and set DANGER_RUN_CI_ON_HOST=1 at their own risk, or that a
sandbox was set up with the given container_id, in which case it is safe
to set DANGER_RUN_CI_ON_HOST=1 for that sandbox.
Also, it is safe to set DANGER_RUN_CI_ON_HOST=1 when building the
sandbox image in ci/test_imagefile.
Then, the two shell scripts can reject early if unsafe execution is
detected.
faba426b3b666c0e93e4349ba88deb79517534c6 lint: Flatten lint image entry points (MarcoFalke)
1111fff91c768d6893868032a0dfba02a9709ffc lint: Add missing --platform=linux to docker build command (MarcoFalke)
Pull request description:
Two fixups to the lint container:
* Add a missing `--platform=linux` to avoid running a non-native arch, like s390x, which happens with podman if such a container was most recently used.
* Flatten the entry points to remove the bash-based one:
Previously, an additional entry point into the container that spawned a bash was supported. The bash had an alias `lint` to run all lint scripts. However, such a use-case seems limited (because it only runs inside the container), inflexible (because it only allows running all lint scripts), and possibly brittle (because it can miss re-building the image when the cache is stale). So remove it and just offer the single entry point via the `./ci/lint.py` script.
If there is a use-case to skip the image building, it should be trivial to add an env var setting the the lint Python script like `DANGER_SKIP_IMAGE_RE_BUILD=1` (or so) in the future.
ACKs for top commit:
willcl-ark:
ACK faba426b3b666c0e93e4349ba88deb79517534c6
Tree-SHA512: 9afda16723c215602c6c42fa3a286d1828c887c8f6ff9512c8ec162ec8997789695f0c464d389cae94e67acf8b5e0f1a55e2ee0d60131a2eee091cf281f91514
d79249d2799e7f450d5708ca3366461f5f2069bc ci: add chimera Linux LTO CI job (fanquake)
Pull request description:
Adds a CI config based on using [Chimera Linux](https://chimera-linux.org/). This might be interesting for any of the following:
* Chimera is based on LLVM & musl libc - we test both of these in isolation, but not together.
* No GNU components. I don't think we have an existing Linux CI job that doesn't have a gcc/stdlibc++ install. This exercises the depends logic for a fully LLVM/Clang/lld only build, including building the native tools (related to #33902).
* We don't currently have a job with LTO enabled (here using CMakes `CMAKE_INTERPROCEDURAL_OPTIMIZATION`, which is `-flto=thin` for LLVM/Clang). I think this is worth having generally (we do use LTO in some other places, like oss-fuzz). If runtime is too much of an issue, then it could also be dropped. (Chimera itself is also compiled with LTO).
QT in depends doesn't build (#32744), so is excluded for now.
Chimera has pointed out at least a few quirks, i.e #34390, #34408 and https://github.com/bitcoin/bitcoin/pull/29963#discussion_r2707922298.
ACKs for top commit:
maflcko:
lgtm ACK d79249d2799e7f450d5708ca3366461f5f2069bc
hebasto:
ACK d79249d2799e7f450d5708ca3366461f5f2069bc.
Tree-SHA512: 1174a7462bf2e7433a2c27a6cf398e94b05db42bb414629c71cf9f9a297ca269e173ae1b7517b30510b494b4397f918eef706d3c75c4286767c5557aeb6db4c7
b65a3d80093b992a2574d5762a8cd03ce2eb4412 iwyu: Fix patch to prefer `<cstdint>` (Hennadii Stepanov)
Pull request description:
The goal of the [patch](https://github.com/bitcoin/bitcoin/blob/master/ci/test/01_iwyu.patch) is to suggest C++ headers rather than their C counterparts. However, for fixed width integer types, the patched IWYU currently suggests `<cinttypes>` where `<cstdint>` is sufficient.
This PR fixes this behavior.
ACKs for top commit:
maflcko:
lgtm ACK b65a3d80093b992a2574d5762a8cd03ce2eb4412
furszy:
utACK b65a3d80093b992a2574d5762a8cd03ce2eb4412
willcl-ark:
utACK b65a3d80093b992a2574d5762a8cd03ce2eb4412
Tree-SHA512: 695efdd44b92a642401738572e49c8b6591aa4463d387107fdf3d2f7c9c4b39f4097cb82413752caf9e8890dcca7246a894e562a1dd17023b05a7e455705beac
-Werror is added to the previous releases job, given it runs on Ubuntu
22.04, which uses an older CMake.
`--compile-no-warning-as-error` can be used, if needed, in future, to
suppress the `CMAKE_COMPILE_WARNING_AS_ERROR` behaviour from a CI
config.
CMAKE_COMPILE_WARNING_AS_ERROR was added to CMake in 3.24.
See https://cmake.org/cmake/help/latest/prop_tgt/COMPILE_WARNING_AS_ERROR.html.
Co-authored-by: willcl-ark <will8clark@gmail.com>
The goal of the patch is to suggest C++ headers rather than their C
counterparts. However, for fixed width integer types, the patched IWYU
currently suggests `<cinttypes>` where `<cstdint>` is sufficient.
This change fixes this behavior.
bbbb78a4f28fd2378342398ccae60995ae0e08d2 ci: Print verbose build error message in test-each-commit (MarcoFalke)
2222dadabbbd03be9b4b917583fd51b34857f40c ci: [refactor] Allow overwriting check option in run helper (MarcoFalke)
Pull request description:
Currently, the build error in the test-each-commit task is not too nice. E.g. https://github.com/bitcoin/bitcoin/actions/runs/21509735101/job/61973587699#step:8:10464:
```
...
[ 75%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/txvalidation_tests.cpp.o
[ 75%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/txvalidationcache_tests.cpp.o
[ 75%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/validation_block_tests.cpp.o
[ 75%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/validation_chainstate_tests.cpp.o
[ 75%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/validation_chainstatemanager_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/validation_flush_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/validation_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/wallet_test_fixture.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/db_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/coinselector_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/coinselection_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/feebumper_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/group_outputs_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/ismine_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/psbt_wallet_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/scriptpubkeyman_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/spend_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/wallet_rpc_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/wallet_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/wallet_transaction_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/walletdb_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/walletload_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/ipc/test/ipc_tests.cpp.o
[ 77%] Linking CXX executable ../../bin/test_bitcoin
[ 87%] Built target test_bitcoin
gmake: *** [Makefile:146: all] Error 2
Command '['cmake', '--build', 'ci_build', '-j', '4']' returned non-zero exit status 2.
error: cannot rebase: Your index contains uncommitted changes.
warning: execution failed: git merge --no-commit origin/master && python3 ./.github/ci-test-each-commit-exec.py && git reset --hard
and made changes to the index and/or the working tree.
You can fix the problem, and then run
git rebase --continue
Error: Process completed with exit code 1.
```
Fix it by just using the same approach that the other CI tasks are using:
01651324f4/ci/test/03_test_script.sh (L143-L146)
ACKs for top commit:
willcl-ark:
ACK bbbb78a4f28fd2378342398ccae60995ae0e08d2
Tree-SHA512: ae54ee62f53e060ed42f76ca59daf7a017bd12495e171efed03d1f5dda969db4f7e2e2c9ed7a178ff5fa9a5baa55ab6a7b30b3ab6b81d5279fe937006ac8228f
7528d18796a22c695aac8090f3ecd2ccc859aa68 ci: show more verbose ccache stats (will)
580e9eefe39f2f94695e1a22437c6ac47a1664e6 ci: bump CCACHE_MAXSIZE to 2G (will)
Pull request description:
Currently some CI jobs don't have great ccache hitrates which we should try to improve: https://willcl-ark.github.io/bitcoin-core-ci-stats/graph/ccache/
- bump ccache maxsize to 2GB in all jobs. We have 150GB shared cache to use, so this should be OK at maximum of 36GB total (current jobset).
- print more verbose ccache stats in the CI logs
The idea is that increasing the cache size to > 2x needed size should eliminate any cache thrashing which might be taking place on master builds when we save the cache. Additionally, larger caches result in more hits in general.
ACKs for top commit:
maflcko:
lgtm ACK 7528d18796a22c695aac8090f3ecd2ccc859aa68
Tree-SHA512: ae00a05159e3f38d24aebc50a2576c5f11241b1196058a4ca2f5f78909795b891bef20cdc4412f512a2ba09cc9ec65afa5132ac0509b54845dd84f933528500c
3c8f5e48f710313de78bcbfafd09fed71890d754 ci: Treat SHA1 LLVM signing key as warning (will)
Pull request description:
The current SHA1 LLVM signing key is considered not secure since
2026-02-01T00:00:00Z which makes this run fail when downloading
packages.
See: https://github.com/llvm/llvm-project/issues/153385
Apply the fix from the issue to temporarily to treat this error as a
warning, until the upstream key can be updated.
This PR should be reverted once the upstream key is updated.
ACKs for top commit:
hebasto:
ACK 3c8f5e48f710313de78bcbfafd09fed71890d754, tested by running the "iwyu" CI job locally on Ubuntu 25.10 after burning all podman's caches.
Tree-SHA512: fbccf98bfd73cb338670f1ceea994d277d746acbc88b9b90a403d9a59d82abda0f3ba34c4d484b70926340c2d0c873259f930c36ccd4f9d18bb1d22d49ee70c4
The current SHA1 LLVM signing key is considered not secure since
2026-02-01T00:00:00Z which makes this run fail when downloading
packages.
See: https://github.com/llvm/llvm-project/issues/153385
Apply the fix from the issue to temporarily to treat this error as a
warning, until the upstream key can be updated.
This PR should be reverted once the upstream key is updated.
With 15 runners we get 150GB of cache space to use, and we currently
have 18 jobs using ccache.
Although each run only generates ~ 200-300 MB of cache data on each run,
the small the small size may be contributing to poor hitrate.
Bump ccache to 2GB per job ~ --> 36GB of the total 150GB cache space to
try and reduce any thrashaing and generally increase hitrate.
efcbf794484ecc02cae05e520120df9d1aa8c93a ci, iwyu: Fix warnings in `src/zmq` and treat them as errors (Hennadii Stepanov)
Pull request description:
This PR [continues](https://github.com/bitcoin/bitcoin/pull/33725#issuecomment-3466897433) the ongoing effort to enforce IWYU warnings.
See [Developer Notes](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#using-iwyu).
Additionally, this adds a new include category to `src/.clang-format`.
ACKs for top commit:
maflcko:
review ACK efcbf794484ecc02cae05e520120df9d1aa8c93a 🐼
janb84:
re ACK efcbf794484ecc02cae05e520120df9d1aa8c93a
sedited:
ACK efcbf794484ecc02cae05e520120df9d1aa8c93a
Tree-SHA512: 5396719d4a9f7fff7b57be7284af5b25ff055edbaba417187e29106c9e310f19f361fbeea74e2448ef1e883a8658028762a38664858a863e5019fcb0cbb346a2
Also, use str(e) consistently in all run helpers.
This refactor does not change any behavior.
This can be reviewed by checking that all instances are exactly
identical code now:
$ git grep --function-context 'def run(cmd'
fdc9fe2da6a8640b11a2871f8b653764652f8c1f ci, iwyu: Fix warnings in `src/primitives` and treat them as errors (Hennadii Stepanov)
Pull request description:
This PR [continues](https://github.com/bitcoin/bitcoin/pull/33725#issuecomment-3466897433) the ongoing effort to enforce IWYU warnings.
See [Developer Notes](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#using-iwyu).
ACKs for top commit:
maflcko:
review ACK fdc9fe2da6a8640b11a2871f8b653764652f8c1f 📀
janb84:
ACK fdc9fe2da6a8640b11a2871f8b653764652f8c1f
sedited:
ACK fdc9fe2da6a8640b11a2871f8b653764652f8c1f
Tree-SHA512: d290545c7aab477b4a5bf121b694899a78e0526be72efa31fa4205b0fd840e6e8240d32f9134a18c9dc58c5f91e7847d7f20ca34f8d2edc4d541ac858ec0dccc
Otherwise, this may pick the wrong arch like s390x, if a such a podman
container was most recently used.
See also the CI_IMAGE_PLATFORM setting in the "other" CI, which does the
same.
c8abac994122b67de8145579489576db5df2b3d3 ci: mount .git dir rw (ci)
Pull request description:
On merges to master we set LINT_CI_SANITY_CHECK_COMMIT_SIG (when "GITHUB_REPOSITORY == bitcoin/bitcoin") which runs verify-commits.py.
This requires write access to the .git directory.
Make the mounted .git directory writable.
This is currently not run on PR branches or locally which caused a miss during review.
Ideally we can have the same checks running in PRs as on merges to master to avoid future discrepancies like this.
ACKs for top commit:
maflcko:
lgtm ACK c8abac994122b67de8145579489576db5df2b3d3
l0rinc:
untested code review ACK c8abac994122b67de8145579489576db5df2b3d3
Tree-SHA512: 7ae4f63227ecffe1dc9003454a7473d6d592550af2e1c899457f34a947e5604b04c13319fb8979f36789ae7787bed62066be60697d163ad5ebedde3fbe8ce45f
On merges to master we set LINT_CI_SANITY_CHECK_COMMIT_SIG (when
"GITHUB_REPOSITORY == bitcoin/bitcoin") which runs verify-commits.py.
This requires write access to the .git directory.
Make the mounted .git directory writable.
This is currently not run on PR branches or locally which caused a miss
during review.
ddae1b4efa568b6099d5bece941f8f2733faf29e ci: remove gnu-getopt usage (fanquake)
Pull request description:
This is used for argument parsing in the `retry` script, however we don't use the script with any arguments. So remove the unused code, and the dependency on `gnu-getopt`.
This came up in the context of adding new CI jobs, where gnu-getopt might not be available, or working properly. It seemed easier to just remove the unused code, than look for more workarounds.
ACKs for top commit:
maflcko:
review ACK ddae1b4efa568b6099d5bece941f8f2733faf29e 🔀
sedited:
ACK ddae1b4efa568b6099d5bece941f8f2733faf29e
Tree-SHA512: a73cf61fe0965127f87f1725b3a25a305ebfd354c318f5f44ecfa20da02ba72fef42dca656dae07f6e1ece956b9d7c58e99edb124d968a4bffb2ce6ac8fc018b
This is used for argument parsing in the retry script, however we don't
use the script with any arguments. So remove the unused code, and the
dependency on gnu-getopt.
This came up in the context of adding new CI jobs, where gnu-getopt
might not be available, or working properly. It seemed easier to just
remove the unused code, than look for more workarounds.
Add a ci/lint.py script to run the linter both locally or inside the CI
(replacing .github/ci-lint-exec.py) which supports running from a
worktree.
Determines whether we are in a worktree, and mounts the real `.git`
directory as a read-only volume if we are.
