We can use vswhere.exe directly to create a vs developer
prompt and so can remove this third party dependency.
Co-authored-by: David Gumberg <davidzgumberg@gmail.com>
Github-Pull: #32513
Rebased-From: 7ae0497eef8f5b37fc1184897a5bbc9f023dfa67
When using `docker buildx build` in conjunction with the `gha` backend
cache type, it's important to specify the URL and TOKEN needed to
authenticate.
On Cirrus runners this is working with only `ACTIONS_CACHE_URL` and
`ACTIONS_RUNTIME_TOKEN`, but this is not enough for the GitHub backend.
Fix this by exporting all `ACTIONS_*` variables.
This fixes cache restore/save on forks or where GH-hosted runners are
being used.
Github-Pull: #33508
Rebased-From: bc706955d740f8a59bec78e44d33e80d1cca373b
$FILE_ENV has a full relative path already, prepending with ci/test/
results in a non-existent path which means that DEPENDS_HASH was not
actually committing to the test's environment file.
Github-Pull: #33581
Rebased-From: ceeb53adcd0a6a87a65c8ebbb20472c15c502dfd
Github-Pull: #33319
Rebased-From: 5eeb2facbbbbf68a2c30ef9e6747e39c85d7b116
These jobs can use reduced runner size to avoid wasting CPU, as much of
the long-running part of the job is single-threaded.
Suggested in: https://github.com/bitcoin/bitcoin/pull/32989#discussion_r2321775620
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Github-Pull: #32989
Rebased-From: 3f339e99e00
Previously jobs were running on a large multi-core server where 10 jobs
as default made sense (or may even have been on the low side).
Using hosted runners with fixed (and lower) numbers of vCPUs we should
adapt compilation to match the number of cpus we have dynamically.
This is cross-platform compatible with macos and linux only.
Github-Pull: #32989
Rebased-From: cc1735d7771
To remove multiple occurances of the respository name, against which we
compare `${{ github.repository }}` to check if we should use Cirrus
Runners, introduce a helper job which can check a single environment
variable and output this as an input to subsequent jobs.
Forks can maintain a trivial patch of their repo name against the
`REPO_USE_CIRRUS_RUNNERS` variable in ci.yml if they have Cirrus Runners
of their own, which will then enable cache actions and docker build
cache to use Cirrus Cache.
It's not possible to use `${{ env.USE_CIRRUS_RUNNERS }}` in the
`runs-on:` directive as the context is not supported by GitHub.
If it was, this job would no longer be necessary.
Github-Pull: #32989
Rebased-From: 020069e6b71
Whilst the action cirruslabs/actions/cache will automatically set this
host, the docker `gha` build cache backend will not be aware of it.
Set the value here, which will later be used in the docker build args to
enable docker build cache on the cirrus cache.
Github-Pull: #32989
Rebased-From: fdf64e55324
Another action to reduce boilerplate in the main ci.yml file.
This action will set up a docker builder compatible with caching build
layers to a container registry using the `gha` build driver.
It will then configure the docker build cache args.
Github-Pull: #32989
Rebased-From: 33ba073df7a
If set, Cirrus runners will be used on pushes to, and pull requests
against, this repository.
Forks can set this if they have their own cirrus runners.
Github-Pull: #32989
Rebased-From: b232b0fa5e9
Add "Restore" and "Save" caching actions.
These actions reduce boilerplate in the main ci.yml configuration file.
These actions are implemented so that caches will be saved on `push`
only.
When a pull request is opened it will cache hit on the caches from the
lastest push, or in the case of depends will hit on any matching depends
hash, falling back to partial matches.
Depends caches are hashed using
`$(git ls-tree HEAD depends "ci/test/$FILE_ENV" | sha256sum | cut -d' ' -f1)`
and this hash is passed in as an input to the actions. This means we
direct cache hit in cases where depends would not be re-built, otherwise
falling back to a partial match.
Previous releases cache is hashed similarly to depends, but using the
test/get_previous_releases.py file.
The cirruslabs cache action will fallback transparently to GitHub's
cache in the case that the job is not being run on a Cirrus Runner,
making these compatible with running on forks (on free GH hardware).
This change is necessary for Windows GHA images, which provide
CMake >= 4.0.
Github-Pull: #32184
Rebased-From: ef00a28414daed2dd026b458082ed03fe9508074
This change:
1. Collects build artifacts in dedicated locations.
2. Allows running bitcoin-chainstate.exe with bitcoinkernel.dll directly
from the build tree on Windows.
8888ee4403fa62972c49e18752695d15fd32c0b0 ci: Allow build dir on CI host (MarcoFalke)
Pull request description:
This is required to pass cross builds on to a different machine after the build.
See for example https://github.com/bitcoin/bitcoin/pull/31176, but this pull will also allow someone to implement it outside this repo.
ACKs for top commit:
davidgumberg:
lgtm ACK 8888ee4403
hebasto:
re-ACK 8888ee4403fa62972c49e18752695d15fd32c0b0.
Tree-SHA512: a1e2c32bc1b95efbd0b48287ac5b49e0e1bacbf5a5800845be5352bbdd3e17fa478e90348b2e94e95cf3ae863cdf75ab444089376588f6f8eec438f73a4b5b97
According to the description for pkg-config, "pkgconf is a
replacement for pkg-config, providing additional functionality
while also maintaining compatibility. This package only provides
a dependency link to the pkgconf package to help with package
upgrades. It can be safely removed."
Thus several scripts and markdown files are updated.
b031b7910d62819443813b91705211c466933a53 [ci] Split out native fuzz jobs for macOS and windows (dergoegge)
Pull request description:
Split out two new CI jobs (for native macOS and windows) that run the fuzz tests on the qa-assets input corpora.
In both jobs the fuzz binary is built with `-DBUILD_FOR_FUZZING` to enable `Assume` assertions as well as `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION`.
ACKs for top commit:
maflcko:
re-lgtm ACK b031b7910d62819443813b91705211c466933a53
achow101:
ACK b031b7910d62819443813b91705211c466933a53
hebasto:
ACK b031b7910d62819443813b91705211c466933a53.
Tree-SHA512: 7d0dc5a9cb299f6f4596dd9a5526b6aaf82efc6eba308bdc9d8b0a45f79dea87204fb6cd4b2ea2a1bd952466b2e958d64021999296d110d7a83c1667f4de51fe
fafbf8acf419d5e2ca307e5804099361ca7471af Make G_FUZZING constexpr, require -DBUILD_FOR_FUZZING=ON to execute a fuzz target (MarcoFalke)
fae3cf0ffa619f8fe7e4ace1889fe7abbb53a532 ci: Temporarily disable macOS/Windows fuzz step (MarcoFalke)
Pull request description:
`g_fuzzing` is used inside `Assume` at runtime, causing significant overhead in hot paths. See https://github.com/bitcoin/bitcoin/issues/31178
One could simply remove the `g_fuzzing` check from the `Assume`, but this would make fuzzing a bit less useful. Also, it would be unclear if `g_fuzzing` adds a runtime overhead in other code paths today or in the future.
Fix all issues by making `G_FUZZING` equal to the build option `BUILD_FOR_FUZZING`, and for consistency in fuzzing, require it to be set when executing any fuzz target.
Fixes https://github.com/bitcoin/bitcoin/issues/31178
Temporarily this drops fuzzing from two CI tasks, but they can be re-added in a follow-up with something like https://github.com/bitcoin/bitcoin/pull/31073
ACKs for top commit:
marcofleon:
Tested ACK fafbf8acf419d5e2ca307e5804099361ca7471af
davidgumberg:
I still ACK fafbf8acf4 for fixing the regression measured in #31178.
ryanofsky:
Code review ACK fafbf8acf419d5e2ca307e5804099361ca7471af but approach -0, because this approach means libraries built for fuzz testing do not function correctly if used in a release, and libraries built for releases are mostly useless for fuzz testing. So I would like to at least consider other solutions to this problem even if we go with this one.
dergoegge:
utACK fafbf8acf419d5e2ca307e5804099361ca7471af
Tree-SHA512: 124fc2e8b35e0c4df414436556a7a0a36cd1bec4b3000b40dcf2ab8c85f32e0610bf7f70d2fd79223d62f3c3665b6c09da21241654c7b9859461b8ca340d5421
7a8a6a06676dcb4066cc81a4e6872281a09bb00d doc: Fix comment in `contrib/devtools/check-deps.sh` script (Hennadii Stepanov)
712d105e0935d31915c067bb90c54b71f6c5f4f5 depends, doc: Do not install Autotools packages (Hennadii Stepanov)
b786449e663b6534abd0239e38122b9d082b90d8 ci: Do not install Autotools packages (Hennadii Stepanov)
Pull request description:
This PR is a follow-up to https://github.com/bitcoin/bitcoin/pull/30752 and addresses https://github.com/bitcoin/bitcoin/pull/30752#discussion_r1758594864.
ACKs for top commit:
kevkevinpal:
ACK [7a8a6a0](7a8a6a0667)
Tree-SHA512: ac701b34ebf1621be355a5b576032d30a342a5667eff47d858587ac3c60011d77bd469e524d69dae90a932f14269227886e3d170a63a2e855b29ef4fa5b9e5a4
