48 Commits

Author SHA1 Message Date
MarcoFalke
fabce97b30
test: Remove gccbug_90348 test case
The test case no longer detects this specific issue for GCC versions
12.1+, as explained in the
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90348 thread and in this
compiler-explorer playground:
https://godbolt.org/z/Y48osrjM8

So remove the test case and update the -fstack-reuse=none cmake
docstring with the underlying affected GCC versions, and the bug URL.
2025-11-11 22:44:01 +01:00
merge-script
1cd8d9fe5c
Merge bitcoin/bitcoin#33445: ci: Update Clang in "tidy" job
5d784bebaff5e3acc0b5180ee51d9a16aec0e356 clang-tidy: Disable `ArrayBound` check in src/ipc and src/test (Hennadii Stepanov)
5efdb0ef305624e5f3666441e761c658f38a8b39 ci: Update Clang in "tidy" job (Hennadii Stepanov)

Pull request description:

  This PR:

  1. Updates to [IWYU 0.25](https://github.com/include-what-you-use/include-what-you-use/releases/tag/0.25), which is compatible with Clang 21.

  2. Fixes new "modernize-use-default-member-init" warnings. The warning in `interpreter.cpp` is a [false positive](https://github.com/llvm/llvm-project/issues/160394), so it has been suppressed.

ACKs for top commit:
  maflcko:
    review ACK 5d784bebaff5e3acc0b5180ee51d9a16aec0e356 🎒
  ryanofsky:
    Code review ACK 5d784bebaff5e3acc0b5180ee51d9a16aec0e356, just adding clang version comment since last review.

Tree-SHA512: a1d853675ec064170ee0f1cd16be6a900676588d4a1e7b5def8733933b140ba1a9520ec6f6a42bf7638b2ff7cf2fe4d5866d407f68b677b49d2bd68ff345f735
2025-11-04 10:16:06 +00:00
Hennadii Stepanov
5d784bebaf
clang-tidy: Disable ArrayBound check in src/ipc and src/test 2025-10-28 15:34:24 +00:00
ismaelsadeeq
6dfdd7e034 fees: refactor: rename policy_fee_tests.cpp to feerounder_tests.cpp
- Also remame the test suite name to match the new name.
2025-10-27 10:34:21 +01:00
merge-script
af78d36512
Merge bitcoin/bitcoin#32588: util: Abort on failing CHECK_NONFATAL in debug builds
fa37153288ca420420636046ef6b8c4ba7e5a478 util: Abort on failing CHECK_NONFATAL in debug builds (MarcoFalke)
fa0dc4bdffb06b6f0c192fe1aa02b4dfdcdc6e15 test: Allow testing of check failures (MarcoFalke)
faeb58fe668662d8262c4cc7c54ad2af756dbe3b refactor: Set G_ABORT_ON_FAILED_ASSUME when G_FUZZING_BUILD (MarcoFalke)

Pull request description:

  A failing `CHECK_NONFATAL` will throw an exception. This is fine and even desired in production builds, because the program may catch the exception and give the user a way to easily report the bug upstream.

  However, in debug development builds, exceptions for internal bugs are problematic:

  * The exception could accidentally be caught and silently ignored
  * The exception does not include a full stacktrace, possibly making debugging harder

  Fix all issues by turning the exception into an abort in debug builds.

  This can be tested by reverting the hunks to `src/rpc/node.cpp` and `test/functional/rpc_misc.py` and then running the functional or fuzz tests.

ACKs for top commit:
  achow101:
    ACK fa37153288ca420420636046ef6b8c4ba7e5a478
  ryanofsky:
    Code review ACK fa37153288ca420420636046ef6b8c4ba7e5a478, just catching subprocess.CalledProcessError in test fixing up a comment since last review
  stickies-v:
    ACK fa37153288ca420420636046ef6b8c4ba7e5a478

Tree-SHA512: 2d892b838ccef6f9b25a066e7c2f6cd6f5acc94aad1d91fce62308983bd3f5c5d724897a76de4e3cc5c3678ddadc87e2ee8c87362965373526038e598dfb0101
2025-10-24 04:41:24 +02:00
Pieter Wuille
2e09d66fbb tests: add unit tests for CBlockIndex::GetAncestor and LastCommonAncestor 2025-10-02 10:34:09 -04:00
Lőrinc
56791b5829
test: split out system_ram_tests to signal when total ram cannot be determined
when `GetTotalRAM` returns an `std::nullopt` now we're getting:
```
The following tests did not run:
        106 - system_ram_tests (Skipped)
```
2025-09-22 12:25:45 +02:00
Lőrinc
168360f4ae coins: warn on oversized -dbcache
Oversized allocations can cause out-of-memory errors or [heavy swapping](https://github.com/getumbrel/umbrel-os/issues/64#issuecomment-663637321), [grinding the system to a halt](https://x.com/murchandamus/status/1964432335849607224).

`LogOversizedDbCache()` now emits a startup warning if the configured `-dbcache` exceeds a cap derived from system RAM, using the same parsing/clamping as cache sizing via CalculateDbCacheBytes(). This isn't meant as a recommended setting, rather a likely upper limit.

Note that we're not modifying the set value, just issuing a warning.
Also note that the 75% calculation is rounded for the last two numbers since we have to divide first before multiplying, otherwise we wouldn't stay inside size_t on 32-bit systems - and this was simpler than casting back and forth.

We could have chosen the remaining free memory for the warning (e.g. warn if free memory is less than 1 GiB), but this is just a heuristic, we assumed that on systems with a lot of memory, other processes are also running, while memory constrained ones run only Core.

If total RAM < 2 GiB, cap is `DEFAULT_DB_CACHE` (`450 MiB`), otherwise it's 75% of total RAM.
The threshold is chosen to be close to values commonly used in [raspiblitz](https://github.com/raspiblitz/raspiblitz/blob/dev/home.admin/_provision.setup.sh#L98-L115) for common setups:

| Total RAM | `dbcache` (MiB) | raspiblitz % | proposed cap (MiB) |
|----------:|----------------:|-------------:|-------------------:|
|     1 GiB |             512 |        50.0% |               450* |
|     2 GiB |            1536 |        75.0% |               1536 |
|     4 GiB |            2560 |        62.5% |               3072 |
|     8 GiB |            4096 |        50.0% |               6144 |
|    16 GiB |            4096 |        25.0% |              12288 |
|    32 GiB |            4096 |        12.5% |              24576 |

[Umbrel issues](https://github.com/getumbrel/umbrel-os/issues/64#issuecomment-663816367) also mention 75% being the upper limit.

Starting `bitcoind` on an 8 GiB rpi4b with a dbcache of 7 GiB:
> ./build/bin/bitcoind -dbcache=7000

warns now as follows:
```
2025-09-07T17:24:29Z [warning] A 7000 MiB dbcache may be too large for a system memory of only 7800 MiB.
2025-09-07T17:24:29Z Cache configuration:
2025-09-07T17:24:29Z * Using 2.0 MiB for block index database
2025-09-07T17:24:29Z * Using 8.0 MiB for chain state database
2025-09-07T17:24:29Z * Using 6990.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
```

Besides the [godbolt](https://godbolt.org/z/EPsaE3xTj) reproducers for the new total memory method, we also tested the warnings manually on:
- [x] Apple M4 Max, macOS 15.6.1
- [x] Intel Core i9-9900K, Ubuntu 24.04.2 LTS
- [x] Raspberry Pi 4 Model B, Armbian Linux 6.12.22-current-bcm2711
- [x] Intel Xeon x64, Windows 11 Home Version 24H2, OS Build 26100.4351

Co-authored-by: stickies-v <stickies-v@protonmail.com>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: w0xlt <woltx@protonmail.com>
2025-09-17 11:36:21 -07:00
merge-script
f58de8749e
Merge bitcoin/bitcoin#32345: ipc: Handle unclean shutdowns better
2581258ec200efb173ea6449ad09b2e7f1cc02e0 ipc: Handle bitcoin-wallet disconnections (Ryan Ofsky)
216099591632dc8a57cc1a3b1ad08e909f8c73cc ipc: Add Ctrl-C handler for spawned subprocesses (Ryan Ofsky)
0c28068ceb7b95885a5abb2685a89bb7c03c1689 doc: Improve IPC interface comments (Ryan Ofsky)
7f65aac78b95357e00e1c0cd996f05e944ea9d2e ipc: Avoid waiting for clients to disconnect when shutting down (Ryan Ofsky)
6eb09fd6141f4c96dae3e1fe1a1f1946c91d0131 test: Add unit test coverage for Init and Shutdown code (Ryan Ofsky)
9a9fb19536fa2f89c3c96860c1882b79b68c9e64 ipc: Use EventLoopRef instead of addClient/removeClient (Ryan Ofsky)
e886c65b6b37aaaf5d22ca68bc14e55d8ec78212 Squashed 'src/ipc/libmultiprocess/' changes from 27c7e8e5a581..b4120d34bad2 (Ryan Ofsky)

Pull request description:

  This PR fixes various problems when IPC connections are broken or hang which were reported in https://github.com/bitcoin-core/libmultiprocess/issues/123, https://github.com/bitcoin-core/libmultiprocess/issues/176, and https://github.com/bitcoin-core/libmultiprocess/pull/182. The different fixes are described in commit messages.

  ---

  The first two commits of this PR update the libmultiprocess subtree including the following PRs:

  - https://github.com/bitcoin-core/libmultiprocess/pull/181
  - https://github.com/bitcoin-core/libmultiprocess/pull/179
  - https://github.com/bitcoin-core/libmultiprocess/pull/160
  - https://github.com/bitcoin-core/libmultiprocess/pull/184
  - https://github.com/bitcoin-core/libmultiprocess/pull/187
  - https://github.com/bitcoin-core/libmultiprocess/pull/186
  - https://github.com/bitcoin-core/libmultiprocess/pull/192

  The subtree changes can be verified by running `test/lint/git-subtree-check.sh src/ipc/libmultiprocess` as described in [developer notes](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#subtrees) and [lint instructions](https://github.com/bitcoin/bitcoin/tree/master/test/lint#git-subtree-checksh).

  The remaining commits are:

  - [`9a9fb19536fa` ipc: Use EventLoopRef instead of addClient/removeClient](9a9fb19536)
  - [`6eb09fd6141f` test: Add unit test coverage for Init and Shutdown code](6eb09fd614)
  - [`7f65aac78b95` ipc: Avoid waiting for clients to disconnect when shutting down](7f65aac78b)
  - [`0c28068ceb7b` doc: Improve IPC interface comments](0c28068ceb)
  - [`216099591632` ipc: Add Ctrl-C handler for spawned subprocesses](2160995916)
  - [`2581258ec200` ipc: Handle bitcoin-wallet disconnections](2581258ec2)

  The new commits depend on the subtree update, and because the subtree update includes an incompatible API change, the "Use EventLoopRef" commit needs to be part of the same PR to avoid breaking the build. The other commits also make sense to merge at the same time because the bitcoin & libmultiprocess changes were written and tested together.

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).

ACKs for top commit:
  Sjors:
    re-utACK 2581258ec200efb173ea6449ad09b2e7f1cc02e0
  josibake:
    code review ACK 2581258ec2
  pinheadmz:
    re-ACK 2581258ec200efb173ea6449ad09b2e7f1cc02e0

Tree-SHA512: 0095aa22d507803e2a2d46eff51fb6caf965cc0c97ccfa615bd97805d5d51e66a5b4b040640deb92896438b1fb9f6879847124c9d0e120283287bfce37b8d748
2025-08-18 20:19:19 +01:00
merge-script
2bb06bcaf2
Merge bitcoin/bitcoin#31679: cmake: Install internal binaries to <prefix>/libexec/
f49840dd902cd9b14b6aadb431b16a4aeb719c3f doc: Fix typo in files.md (Ryan Ofsky)
f5cf0b1ccc8fd426135809a8a4becdae2d797bb5 bitcoin wrapper: improve help output (Ryan Ofsky)
c810b168b89dc07017e9feaec1a8746a449a60b1 doc: Add description of installed files to files.md (Ryan Ofsky)
94ffd01a0294afbe045f1b17a77e4a3caf21e674 doc: Add release notes describing libexec/ binaries (Ryan Ofsky)
cd97905ebc564b8b095099a28d1d5437951927c4 cmake: Move internal binaries from bin/ to libexec/ (Ryan Ofsky)

Pull request description:

  This change moves binaries that are not typically invoked directly by users from the `bin/` directory to the `libexec/` directory in CMake installs and binary releases. The goal of the PR is to introduce a distinction between internal and external binaries so starting with #31802, we can use IPC to implement features in new binaries without adding those binaries to the CLI. The change also helps reduce clutter in `bin/`, making it easier for users to identify useful tools to run. Summary of changes:

  - For **source builds** (i.e. developer builds) — There are no changes.
  - For **source installs** (i.e. `cmake --install` result) — `test_bitcoin`, `test_bitcoin-qt`, and `bench_bitcoin` are installed in `${CMAKE_PREFIX_PATH}/libexec` instead of `${CMAKE_PREFIX_PATH}/bin`, so they are no longer on the system `PATH`. However, they can still be invoked from the `libexec/` directory, or from the CLI as `bitcoin test`, `bitcoin test-gui`, and `bitcoin bench`, respectively.
  - For **binary releases** — Since `test_bitcoin` is the only test binary enabled in releases, the only change is moving `test_bitcoin` from `bin/` to `libexec/`.

  <details><summary>Details</summary>
  <p>

   The table below shows the install location of each binary after this change, and the availability of each binary.

  | Binary               | Location     | Availability         | Change                        |
  |----------------------|--------------|----------------------|-------------------------------|
  | `bitcoin`            | `bin/`       | 📦 Binary release (since #31375) | Unchanged                     |
  | `bitcoin-cli`        | `bin/`       | 📦 Binary release     | Unchanged                     |
  | `bitcoind`           | `bin/`       | 📦 Binary release     | Unchanged                     |
  | `bitcoin-qt`         | `bin/`       | 📦 Binary release     | Unchanged                     |
  | `bitcoin-tx`         | `bin/`       | 📦 Binary release     | Unchanged                     |
  | `bitcoin-util`       | `bin/`       | 📦 Binary release     | Unchanged                     |
  | `bitcoin-wallet`     | `bin/`       | 📦 Binary release     | Unchanged                     |
  | `bench_bitcoin`      | `libexec/`   | 🛠 Source build only  | Moved from `bin/`             |
  | `bitcoin-chainstate` | `libexec/`   | 🛠 Source build only  | Newly installed (was built)   |
  | `bitcoin-gui`        | `libexec/`   | 🛠 Source build only (until #31802) | Moved from `bin/`             |
  | `bitcoin-node`       | `libexec/`   | 🛠 Source build only (until #31802) | Moved from `bin/`             |
  | `test_bitcoin`       | `libexec/`    | 📦 Binary release     | Moved from `bin/`             |
  | `test_bitcoin-qt`    | `libexec/`   | 🛠 Source build only  | Moved from `bin/`             |

  </p>
  </details>

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).

ACKs for top commit:
  l0rinc:
    re-ACK f49840dd902cd9b14b6aadb431b16a4aeb719c3f
  Sjors:
    re-ACK f49840dd902cd9b14b6aadb431b16a4aeb719c3f
  achow101:
    ACK f49840dd902cd9b14b6aadb431b16a4aeb719c3f
  janb84:
    re ACK f49840dd902cd9b14b6aadb431b16a4aeb719c3f
  BrandonOdiwuor:
    Tested ACK f49840dd902cd9b14b6aadb431b16a4aeb719c3f
  hodlinator:
    re-ACK f49840dd902cd9b14b6aadb431b16a4aeb719c3f
  willcl-ark:
    utACK f49840dd902cd9b14b6aadb431b16a4aeb719c3f

Tree-SHA512: 858a2e1a53db11ee3c5c759bfdeea566f242b9ce5e8a898fa435222e41662b8184577c0dc2c4c058294b4de41d8cb3ba3e5d24c748c280efa4a3f84e3ec4344d
2025-08-07 14:53:26 +01:00
Ryan Ofsky
6eb09fd614 test: Add unit test coverage for Init and Shutdown code
Currently this code is not called in unit tests. Calling should make it
possible to write tests for things like IPC exceptions being thrown during
shutdown.
2025-08-04 13:38:26 -04:00
Ava Chow
3b188b8b3d
Merge bitcoin/bitcoin#31576: test: Move script_assets_tests into its own suite
c40dbbbf77078bb86a652ca151b472e7bef61ae0 test: Move `script_assets_tests` into its own suite (Hennadii Stepanov)

Pull request description:

  This PR ensures that the `script_assets_tests` test case is explicitly reported as "Skipped" when it is not run, making it clearer when running the test suite with `ctest`:

  - on the master branch @ 9355578a77978a0c2f189bd7315a2883142d8119:
  ```
  $ env -u DIR_UNIT_TEST_DATA ctest --test-dir build -j 16 -R "^script_"
  Internal ctest changing into directory: /home/hebasto/git/bitcoin/build
  Test project /home/hebasto/git/bitcoin/build
      Start 87: script_tests
      Start 83: script_p2sh_tests
      Start 85: script_segwit_tests
      Start 86: script_standard_tests
      Start 84: script_parse_tests
  1/5 Test #84: script_parse_tests ...............   Passed    0.11 sec
  2/5 Test #86: script_standard_tests ............   Passed    0.11 sec
  3/5 Test #85: script_segwit_tests ..............   Passed    0.12 sec
  4/5 Test #83: script_p2sh_tests ................   Passed    0.12 sec
  5/5 Test #87: script_tests .....................   Passed    0.36 sec

  100% tests passed, 0 tests failed out of 5

  Total Test time (real) =   0.37 sec
  ```
  - with this PR:
  ```
  $ env -u DIR_UNIT_TEST_DATA ctest --test-dir build -j 16 -R "^script_"
  Internal ctest changing into directory: /home/hebasto/git/bitcoin/build
  Test project /home/hebasto/git/bitcoin/build
      Start 83: script_assets_tests
      Start 88: script_tests
      Start 84: script_p2sh_tests
      Start 86: script_segwit_tests
      Start 87: script_standard_tests
      Start 85: script_parse_tests
  1/6 Test #85: script_parse_tests ...............   Passed    0.11 sec
  2/6 Test #83: script_assets_tests ..............***Skipped   0.12 sec
  3/6 Test #86: script_segwit_tests ..............   Passed    0.11 sec
  4/6 Test #87: script_standard_tests ............   Passed    0.11 sec
  5/6 Test #84: script_p2sh_tests ................   Passed    0.12 sec
  6/6 Test #88: script_tests .....................   Passed    0.36 sec

  100% tests passed, 0 tests failed out of 6

  Total Test time (real) =   0.37 sec

  The following tests did not run:
   83 - script_assets_tests (Skipped)
  $ env DIR_UNIT_TEST_DATA=/home/hebasto/git/bitcoin/qa-assets/unit_test_data ctest --test-dir build -j 16 -R "^script_"
  Internal ctest changing into directory: /home/hebasto/git/bitcoin/build
  Test project /home/hebasto/git/bitcoin/build
      Start 83: script_assets_tests
      Start 88: script_tests
      Start 84: script_p2sh_tests
      Start 86: script_segwit_tests
      Start 87: script_standard_tests
      Start 85: script_parse_tests
  1/6 Test #85: script_parse_tests ...............   Passed    0.11 sec
  2/6 Test #87: script_standard_tests ............   Passed    0.11 sec
  3/6 Test #86: script_segwit_tests ..............   Passed    0.11 sec
  4/6 Test #84: script_p2sh_tests ................   Passed    0.12 sec
  5/6 Test #88: script_tests .....................   Passed    0.35 sec
  6/6 Test #83: script_assets_tests ..............   Passed    1.58 sec

  100% tests passed, 0 tests failed out of 6

  Total Test time (real) =   1.58 sec
  ```

ACKs for top commit:
  maflcko:
    re-ACK c40dbbbf77078bb86a652ca151b472e7bef61ae0 👈
  ajtowns:
    ACK c40dbbbf77078bb86a652ca151b472e7bef61ae0
  achow101:
    ACK c40dbbbf77078bb86a652ca151b472e7bef61ae0

Tree-SHA512: 25713e1c3b507b6f2a5fecc7b1ea285a6642b906c248769238a58fc0df48489ac5f7606778f9e3653b407b7f1d06563e1554d04321303b350c80eb888500cc5d
2025-07-25 14:48:14 -07:00
MarcoFalke
fa0dc4bdff
test: Allow testing of check failures
This allows specific tests to mock the check behavior to consistently
use exceptions instead of aborts for intentionally failing checks in all
build configurations.
2025-07-24 19:44:04 +02:00
Anthony Towns
06ab3a394a tests: speed up coins_tests by parallelizing
Updates the cmake logic to generate a separate test for each
BOOST_FIXTURE_TEST_SUITE declaration in a file, and splits coins_tests.cpp
into three separate suites so that they can be run in parallel. Also
updates the convention enforced by test/lint/lint-tests.py.
2025-07-22 12:56:26 +10:00
glozow
938e86f8fe txgraph: add unit test for TxGraph::Trim (tests)
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
2025-07-02 15:06:48 -04:00
Hennadii Stepanov
c40dbbbf77
test: Move script_assets_tests into its own suite
This change ensures that the `script_assets_tests` test case is
explicitly reported as "Skipped" when it is not run, making it clearer
when running the test suite with `ctest`.
2025-06-20 10:09:32 +01:00
Ryan Ofsky
cd97905ebc cmake: Move internal binaries from bin/ to libexec/
This change moves binaries that are not typically invoked directly by users
from the `bin/` directory to the `libexec/` directory in CMake installs and
binary releases. The goal is to simplify the contents of `bin/` for end users
while still making all binaries available when needed. After this change, the
binaries remaining in `bin/` are:

- bitcoin
- bitcoin-cli
- bitcoind
- bitcoin-qt
- bitcoin-tx
- bitcoin-util
- bitcoin-wallet

And the binaries that are moved to `libexec/` are:

- bench_bitcoin
- bitcoin-chainstate(*)
- bitcoin-gui(***)
- bitcoin-node(***)
- test_bitcoin(**)
- test_bitcoin-qt

(*) bitcoin-chainstate was previously missing an install rule and was actually
not installed even when it was enabled.

(**) test_bitcoin is the only libexec/ binary that is currently included in
bitcoin binary releases. The others are only installed when building from
source with relevant cmake options enabled.

(***) bitcoin-node and bitcoin-gui are not currently built by default or
included in binary releases but both of these changes are planned and
implemented in #31802
2025-05-29 07:51:08 -05:00
Hennadii Stepanov
282b4913c7 cmake: Add application manifests when cross-compiling for Windows
Windows application manifests provide several benefits. However, on the
master branch, the linker generates and embeds manifests only when
building with MSVC.

This change unifies manifest embedding for both native and
cross-compilation.
2025-05-15 13:57:57 +01:00
Ava Chow
5b8046a6e8
Merge bitcoin/bitcoin#30611: validation: write chainstate to disk every hour
e976bd3045010ee217aa0f2dca4c962aabb789d5 validation: add randomness to periodic write interval (Andrew Toth)
2e2f41068128c38120a5b44d24ee30f71970455a refactor: replace m_last_write with m_next_write (Andrew Toth)
b557fa7a175f139614932fbb3a4ad0af8271c73c refactor: rename fDoFullFlush to should_write (Andrew Toth)
d73bd9fbe483ad1397f62dc1d580314202351ace validation: write chainstate to disk every hour (Andrew Toth)
0ad7d7abdbcffc11a46413545a214a716f56dc95 test: chainstate write test for periodic chainstate flush (Andrew Toth)

Pull request description:

  Since #28233, periodically writing the chainstate to disk every 24 hours does not clear the dbcache. Since #28280, periodically writing the chainstate to disk is proportional only to the amount of dirty entries in the cache. Due to these changes, it is no longer beneficial to only write the chainstate to disk every 24 hours. The periodic flush interval was necessary because every write of the chainstate would clear the dbcache. Now, we can get rid of the periodic flush interval and simply write the chainstate along with blocks and block index at least every hour.

  Three benefits of doing this:
  1. For IBD or reindex-chainstate with a combination of large dbcache setting, slow CPU, slow internet speed/unreliable peers, it could be up to 24 hours until the chainstate is persisted to disk. A power outage or crash could potentially lose up to 24 hours of progress. If there is a very large amount of dirty cache entries, writing to disk when a flush finally does occur will take a very long time. Crashing during this window of writing can cause https://github.com/bitcoin/bitcoin/issues/11600. By syncing every hour in unison with the block index we avoid this problem. Only a maximum of one hour of progress can be lost, and the window for crashing during writing is much smaller. For IBD with lower dbcache settings, faster CPU, or better internet speed/reliable peers, chainstate writes are already triggered more often than every hour so this change will have no effect on IBD.
  2. Based on discussion in #28280, writing only once every 24 hours during long running operation of a node causes IO spikes. Writing smaller chainstate changes every hour like we do with blocks and block index will reduce IO spikes.
  3. Faster shutdown speeds. All dirty chainstate entries must be persisted to disk on shutdown. If we have a lot of dirty entries, such as when close to 24 hours or if we sync with a large dbcache, it can take a long time to shutdown. By keeping the chainstate clean we avoid this problem.

  Inspired by [this comment](https://github.com/bitcoin/bitcoin/pull/28280#issuecomment-2121088705).

  Resolves https://github.com/bitcoin/bitcoin/issues/11600

ACKs for top commit:
  achow101:
    ACK e976bd3045010ee217aa0f2dca4c962aabb789d5
  davidgumberg:
    utACK e976bd3045
  sipa:
    utACK e976bd3045010ee217aa0f2dca4c962aabb789d5
  l0rinc:
    ACK  e976bd3045010ee217aa0f2dca4c962aabb789d5

Tree-SHA512: 5bccd8f1dea47f9820a3fd32fe3bb6841c0167b3d6870cc8f3f7e2368f124af1a914bca6acb06889cd7183638a8dbdbace54d3237c3683f2b567eb7355e015ee
2025-05-01 12:11:55 -07:00
Ryan Ofsky
abdf3cb645 cmake: Fix warnings from boost headers
This change is technically not needed to add libmultiprocess as a subtree, but
it avoids a CI failure in followup PR #30975 which enables multiprocess build
option in more CI jobs. In that PR, the "macOS 14 native no depends job" fails
due to warnings in boost headers treated as errors, reported in
https://github.com/bitcoin/bitcoin/pull/30975#issuecomment-2623801480 and
https://github.com/chaincodelabs/libmultiprocess/issues/138
2025-04-02 08:41:16 -05:00
Ryan Ofsky
69f0d4adb7 scripted-diff: s/WITH_MULTIPROCESS/ENABLE_IPC/ in cmake
Rename WITH_MULTIPROCESS to ENABLE_IPC, because ENABLE_IPC is a more accurate
name for the feature. It controls whether the src/ipc/ directory is built and
whether IPC features like -ipcbind, -ipcconnect, and -ipcfd are available. It
does NOT currently enable multiprocess features which are implemented in #10102
building on top of the IPC features. It will also no longer (as of the next
commit), control whether a find_package call is made so the "WITH_" prefix is
also inappropriate.

-BEGIN VERIFY SCRIPT-
git grep -l WITH_MULTIPROCESS | xargs sed -i s/WITH_MULTIPROCESS/ENABLE_IPC/g
-END VERIFY SCRIPT-
2025-04-02 08:41:16 -05:00
Ryan Ofsky
f347d7980e
Merge bitcoin/bitcoin#31283: Add waitNext() to BlockTemplate interface
cadbd4137d84b71be26effd6a2ae177d5031345e miner: have waitNext return after 20 min on testnet (Sjors Provoost)
d4020f502a63cb4390ec241fc5f989e988afa022 Add waitNext() to BlockTemplate interface (Sjors Provoost)

Pull request description:

  This PR introduces `waitNext()`. It waits for either the tip to update or for fees at the top of the mempool to rise sufficiently. It then returns a new template, with which the caller can rinse and repeat.

  On testnet3 and testnet4 the difficulty drops after 20 minutes, so the second ensures that a new template is returned in that case.

  Alternative approach to #31003, suggested in https://github.com/bitcoin/bitcoin/issues/31109#issuecomment-2451942362

ACKs for top commit:
  ryanofsky:
    Code review ACK cadbd4137d84b71be26effd6a2ae177d5031345e. Main change since last review is adding back a missing `m_interrupt` check in the waitNext loop. Also made various code cleanups in both commits.
  ismaelsadeeq:
    Code review ACK cadbd4137d84b71be26effd6a2ae177d5031345e
  vasild:
    ACK cadbd4137d84b71be26effd6a2ae177d5031345e

Tree-SHA512: c5a40053723c1c1674449ba1e4675718229a2022c8b0a4853b12a2c9180beb87536a1f99fde969a0ef099bca9ac69ca14ea4f399d277d2db7f556465ce47de95
2025-03-12 15:20:48 -04:00
Hennadii Stepanov
a8c78a0574
cmake: Revamp handling of data files
This change introduces new functions `target_json_data_sources()` and
`target_raw_data_sources()`.
2025-02-21 11:11:29 +00:00
Sjors Provoost
cadbd4137d
miner: have waitNext return after 20 min on testnet
On testnet we need to create a min diff template after 20 min.
2025-02-19 17:20:58 +01:00
merge-script
73e2ec1373
Merge bitcoin/bitcoin#31844: cmake: add a component for each binary
9b033bebb18dfd609c02736292f37cc6589fcc8d cmake: rename Kernel component to bitcoinkernel for consistency (Cory Fields)
2e0c92558e96b43351312d8fd403a20b9795acd4 cmake: add and use install_binary_component (Cory Fields)
0264c5d86c74c25407f0a07326d6f8722665fe74 cmake: use per-target components for bitcoin-qt and bitcoin-gui (Cory Fields)
fb0546b1c5ebb858605bef4c9fa001782e0ab213 ci: don't try to install for a fuzz build (Cory Fields)

Pull request description:

  This makes it possible to build/install only the desired binaries regardless of the configuration.
  For consistency, the component names match the binary names. `Kernel` and `GUI` have been renamed.

  Additionally it fixes #31762 by installing only the manpages for the configured targets (and includes them in the component installs for each).

  Also fixes #31745.

  Alternative to #31765 which is (imo) more correct/thorough.

  Can be tested using (for ex):
  ```bash
  $ cmake -B build
  $ cmake --build build -t bitcoind -t bitcoin-cli
  $ cmake --install build --component bitcoind
  $ cmake --install build --component bitcoin-cli
  ```

ACKs for top commit:
  hebasto:
    ACK 9b033bebb18dfd609c02736292f37cc6589fcc8d.
  TheCharlatan:
    Re-ACK 9b033bebb18dfd609c02736292f37cc6589fcc8d
  stickies-v:
    re-ACK 9b033bebb18dfd609c02736292f37cc6589fcc8d

Tree-SHA512: fd4818e76f190dbeafbf0c246b466f829771902c9d6d7111ed917093b811c8a5536a4a45e20708f73e7f581d6cb77c8e61cfa69e065788dcf0886792f553a355
2025-02-14 14:19:12 +01:00
Cory Fields
2e0c92558e cmake: add and use install_binary_component
Add a separate component for each binary for fine-grained installation options.

Also install the man pages for only for the targets enabled.
2025-02-13 18:14:41 +00:00
laanwj
fc700bb47f test: Add tests for PCP and NATPMP implementations
Add a mock for a simple scriptable UDP server, and use this to test
various code paths (including successful mappings, timeouts and errors)
in the PCP and NATPMP implementations.
2025-01-16 15:56:02 +01:00
Andrew Toth
0ad7d7abdb
test: chainstate write test for periodic chainstate flush 2024-12-28 10:14:01 -05:00
Hennadii Stepanov
46e207d329 cmake: Link bitcoin_consensus as a library
The TARGET_OBJECTS generator expression was introduced in the staging
branch when we aimed to build the libbitcoinconsensus shared library.
However, `bitcoin_consensus` is a STATIC library, not an OBJECT library.

This change updates the build system to link `bitcoin_consensus`
normally to `test_bitcoin`, resolving linking issues when building with
clang-cl.
2024-12-15 15:37:14 +00:00
merge-script
8d340be924
Merge bitcoin/bitcoin#31181: cmake: Revamp FindLibevent module
5a96767e3f531ba9e8a676eec47727421f9f589f depends, libevent: Do not install *.pc files and remove patches for them (Hennadii Stepanov)
ffda355b5a2113fa0f7db8015f7b08bf1351e245 cmake, refactor: Move `HAVE_EVHTTP_...` to `libevent` interface (Hennadii Stepanov)
b619bdc3303217f4415342fe60e586e18fa48308 cmake: Revamp `FindLibevent` module (Hennadii Stepanov)

Pull request description:

  This PR generalizes the use of `find_package` / `pkg_check_modules`, prioritizing the former.

  Addresses https://github.com/bitcoin/bitcoin/pull/30903#issuecomment-2444700876:
  > We should also follow up with refactoring the libevent module, to more generically use CMake/pkg-config, rather than restricting the CMake usage to `vcpkg`. At that point, we'd likely be able to dump pkg-config for the depends path entirely.

  Similar to https://github.com/bitcoin/bitcoin/pull/30903.

ACKs for top commit:
  fanquake:
    ACK 5a96767e3f531ba9e8a676eec47727421f9f589f

Tree-SHA512: 181020c16ccd2821e718c73f264badcdc5e62980c4a8d9691e759efe2ea00da2326e26308d1dcfdeac01e9e27930428ecace9f36941deee951b751b138d7266c
2024-11-11 15:31:58 +00:00
merge-script
19f277711e
Merge bitcoin/bitcoin#26593: tracing: Only prepare tracepoint arguments when actually tracing
0de3e96e333090548a43e5e870c4cb8941d6baf1 tracing: use bitcoind pid in bcc tracing examples (0xb10c)
411c6cfc6c2e488e407f057b646730e63806ed8a tracing: only prepare tracepoint args if attached (0xb10c)
d524c1ec06643208c189089089e84f6e1cd0abad tracing: dedup TRACE macros & rename to TRACEPOINT (0xb10c)

Pull request description:

  Currently, if the tracepoints are compiled (e.g. in depends and release builds), we always prepare the tracepoint arguments regardless of the tracepoints being used or not. We made sure that the argument preparation is as cheap as possible, but we can almost completely eliminate any overhead for users not interested in the tracepoints (the vast majority), by gating the tracepoint argument preparation with an `if(something is attached to this tracepoint)`. To achieve this, we use the optional semaphore feature provided by SystemTap.

  The first commit simplifies and deduplicates our tracepoint macros from 13 TRACEx macros to a single TRACEPOINT macro. This makes them easier to use and also avoids more duplicate macro definitions in the second commit.

  The Linux tracing tools I'm aware of (bcc, bpftrace, libbpf, and systemtap) all support the semaphore gating feature. Thus, all existing tracepoints and their argument preparation is gated in the second commit. For details, please refer to the commit messages and the updated documentation in `doc/tracing.md`.

  Also adding unit tests that include all tracepoint macros to make sure there are no compiler problems with them (e.g. some varadiac extension not supported).

  Reviewers might want to check:
  - Do the tracepoints still work for you? Do the examples in `contrib/tracing/` run on your system (as bpftrace frequently breaks on every new version, please test master too if it should't work for you)? Do the CI interface tests still pass?
  - Is the new documentation clear?
  - The `TRACEPOINT_SEMAPHORE(event, context)` macros places global variables in our global namespace. Is this something we strictly want to avoid or maybe move to all `TRACEPOINT_SEMAPHORE`s to a separate .cpp file or even namespace? I like having the `TRACEPOINT_SEMAPHORE()` in same file as the `TRACEPOINT()`, but open for suggestion on alternative approaches.
  - Are newly added tracepoints in the unit tests visible when using `readelf -n build/src/test/test_bitcoin`? You can run the new unit tests with `./build/src/test/test_bitcoin --run_test=util_trace_tests* --log_level=all`.
  <details><summary>Two of the added unit tests demonstrate that we are only processing the tracepoint arguments when attached by having a test-failure condition in the tracepoint argument preparation. The following bpftrace script can be used to demonstrate that the tests do indeed fail when attached to the tracepoints.</summary>

  `fail_tests.bt`:

  ```c
  #!/usr/bin/env bpftrace

  usdt:./build/src/test/test_bitcoin:test:check_if_attached {
    printf("the 'check_if_attached' test should have failed\n");
  }

  usdt:./build/src/test/test_bitcoin:test:expensive_section {
    printf("the 'expensive_section' test should have failed\n");
  }
  ```

  Run the unit tests with `./build/src/test/test_bitcoin` and start `bpftrace fail_tests.bt -p $(pidof test_bitcoin)` in a separate terminal. The unit tests should fail with:

  ```
  Running 594 test cases...
  test/util_trace_tests.cpp(31): error: in "util_trace_tests/test_tracepoint_check_if_attached": check false has failed
  test/util_trace_tests.cpp(51): error: in "util_trace_tests/test_tracepoint_manual_tracepoint_active_check": check false has failed

  *** 2 failures are detected in the test module "Bitcoin Core Test Suite"
  ```

  </details>

  These links might provide more contextual information for reviewers:
  - [How SystemTap Userspace Probes Work by eklitzke](https://eklitzke.org/how-sytemtap-userspace-probes-work) (actually an example on Bitcoin Core; mentions that with semaphores "the overhead for an untraced process is effectively zero.")
  - [libbpf comment on USDT semaphore handling](1596a09b5d/src/usdt.c (L83-L92)) (can recommend the whole comment for background on how the tracepoints and tracing tools work together)
  - https://sourceware.org/systemtap/wiki/UserSpaceProbeImplementation#Semaphore_Handling

ACKs for top commit:
  willcl-ark:
    utACK 0de3e96e333090548a43e5e870c4cb8941d6baf1
  laanwj:
    re-ACK 0de3e96e333090548a43e5e870c4cb8941d6baf1
  jb55:
    utACK 0de3e96e333090548a43e5e870c4cb8941d6baf1
  vasild:
    ACK 0de3e96e333090548a43e5e870c4cb8941d6baf1

Tree-SHA512: 0e5e0dc5e0353beaf5c446e4be03d447e64228b1be71ee9972fde1d6fac3fac71a9d73c6ce4fa68975f87db2b2bf6eee2009921a2a145e24d83a475d007a559b
2024-11-11 10:33:28 +00:00
Hennadii Stepanov
b619bdc330
cmake: Revamp FindLibevent module
This change generalizes the use of `find_package` / `pkg_check_modules`,
prioritizing the former.
2024-11-06 15:40:34 +00:00
0xb10c
d524c1ec06
tracing: dedup TRACE macros & rename to TRACEPOINT
This deduplicates the TRACEx macros by using systemtaps STAP_PROBEV[0]
variadic macro instead of the DTrace compability DTRACE_PROBE[1] macros.
Bitcoin Core never had DTrace tracepoints, so we don't need to use the
drop-in replacement for it. As noted in pr25541[2], these macros aren't
compatibile with DTrace on macOS anyway.

This also renames the TRACEx macro to TRACEPOINT to clarify what the
macro does: inserting a tracepoint vs tracing (logging) something.

[0]: https://sourceware.org/git/?p=systemtap.git;a=blob;f=includes/sys/sdt.h;h=24d5e01c37805e55c36f7202e5d4e821b85167a1;hb=ecab2afea46099b4e7dfd551462689224afdbe3a#l407
[1]: https://sourceware.org/git/?p=systemtap.git;a=blob;f=includes/sys/sdt.h;h=24d5e01c37805e55c36f7202e5d4e821b85167a1;hb=ecab2afea46099b4e7dfd551462689224afdbe3a#l490
[2]: https://github.com/bitcoin/bitcoin/pull/25541/files#diff-553886c5f808e01e3452c7b21e879cc355da388ef7680bf310f6acb926d43266R30-R31

Co-authored-by: Martin Leitner-Ankerl <martin.ankerl@gmail.com>
2024-10-28 14:23:47 +01:00
glozow
699643f23a [unit test] MempoolRejectedTx 2024-10-24 21:23:56 -04:00
Ryan Ofsky
206c6e78ee build: Make bitcoin_ipc_test depend on bitcoin_ipc
This change is needed to allow generated capnp code in src/ipc/capnp/ to be
used in unit tests for better test coverage in upcoming commits.
2024-09-23 16:03:04 -04:00
Ryan Ofsky
e46bebb444
Merge bitcoin/bitcoin#30546: util: Use consteval checked format string in FatalErrorf, LogConnectFailure
fa5bc450d5d4c1d2daf7b205f1678402c3c21678 util: Use compile-time check for LogConnectFailure (MarcoFalke)
fa7087b896c0150c29d7a27c53e0533831a2bf3b util: Use compile-time check for FatalErrorf (MarcoFalke)
faa62c0112f2b7ab69c80a5178f5b79217bed0a6 util: Add ConstevalFormatString (MarcoFalke)
fae7b83eb58d22ed83878561603991131372cdd7 lint: Remove forbidden functions from lint-format-strings.py (MarcoFalke)

Pull request description:

  The `test/lint/lint-format-strings.py` was designed to count the number of format specifiers and assert that they are equal to the number of parameters passed to the format function. The goal seems reasonable, but the implementation has many problems:

  * It is written in Python, meaning that C++ code can not be parsed correctly. Currently it relies on brittle regex and string parsing.
  * Apart from the parsing errors, there are also many logic errors. For example, `count_format_specifiers` allows a mix of positional specifiers and non-positional specifiers, which can lead to runtime format bugs. Also, `count_format_specifiers` silently skipped over "special" format specifiers, which are valid in tinyformat, which again can lead to runtime format bugs being undetected.
  * The brittle logic has a history of breaking in pull requests that are otherwise fine. This causes the CI to fail and the pull request being blocked from progress until the bug in the linter is fixed, or the code is rewritten to work around the bug.
  * It is only run in the CI, or when the developer invokes the script. It would be better if the developer got the error message at compile-time, directly when writing the code.

  Fix all issues by using a `consteval` checked format string in `FatalErrorf` and `LogConnectFailure`.

  This is the first step toward https://github.com/bitcoin/bitcoin/issues/30530 and a follow-up will apply the approach to the other places.

ACKs for top commit:
  stickies-v:
    re-ACK fa5bc450d5d4c1d2daf7b205f1678402c3c21678
  l0rinc:
    ACK fa5bc450d5d4c1d2daf7b205f1678402c3c21678
  hodlinator:
    ACK fa5bc450d5d4c1d2daf7b205f1678402c3c21678
  ryanofsky:
    Code review ACK fa5bc450d5d4c1d2daf7b205f1678402c3c21678

Tree-SHA512: d6189096b16083143687ed1b1559cf4f92f97dd87bc5d00673e44f4fb9fce7bb7b215cfdfc39b6e6a24f0b75a79a03ededce966639e554f7172e1fc22cf015ae
2024-09-12 13:21:53 -04:00
MarcoFalke
faa62c0112
util: Add ConstevalFormatString
The type is used to wrap a format string once it has been compile-time
checked to contain the right number of format specifiers.
2024-09-12 15:00:53 +02:00
merge-script
85833cf05f
Merge bitcoin/bitcoin#30847: test: Drop no longer needed workarounds
5c80192ff6b982ee3a75be4142fe942b8206f2cd test: Drop no longer needed workarounds (Hennadii Stepanov)

Pull request description:

  This PR deletes the workarounds introduced in https://github.com/bitcoin/bitcoin/pull/16564 and https://github.com/bitcoin/bitcoin/pull/15382, as `ctest` skips these cases gracefully: 5c80192ff6/src/test/CMakeLists.txt (L201-L203)

ACKs for top commit:
  kevkevinpal:
    ACK [5c80192](5c80192ff6)
  fanquake:
    ACK 5c80192ff6b982ee3a75be4142fe942b8206f2cd. Looks correct:

Tree-SHA512: c47c606ecf7d64016b3c6353c3d4898350edc2caeac494dfd44484417f500a73f0c88c39f0f24651f3a02ef31ed9ca5c70d938bb9a8ca1eea54927e4d6a8fcd2
2024-09-12 11:28:27 +01:00
merge-script
db8350b0e3
Merge bitcoin/bitcoin#30803: build: Minor build system fixes and amendments
1cc93fe7b40f10a7d1d1189058af98a2bce31381 build: Delete dead code that implements `IF_CHECK_FAILED` option (Hennadii Stepanov)
341ad238091d4df520c70f1757b017e6f6620f24 build: Delete MSVC special case for `BUILD_FOR_FUZZING` option (Hennadii Stepanov)
fdad128b528bc8622bc6d8343026c28b18260f64 build: Stop enabling CMake's CMP0141 policy (Hennadii Stepanov)
b2a6f545b4f6e3442ae51f66a6f3c1de92d00a1b doc: Drop `ctest` command from Windows cross-compiling instructions (Hennadii Stepanov)
73b618582dcf06dd01be062fe0f81060cfcb48d8 build: Print `CMAKE_CXX_COMPILER_ARG1` in summary (Hennadii Stepanov)
f03c9420958de31fdfecec5fa3e23134aac61803 build, test: Add missed log options (Hennadii Stepanov)
6f2cb0eafdef81fb9464a4679c3a5905d19e5103 doc: Amend comment about ZeroMQ config files (Hennadii Stepanov)

Pull request description:

  This PR addresses the following comments:
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1742342524
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1728692369
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1736110362
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1742931121
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1747723657
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1742328675
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1723106474

ACKs for top commit:
  sipsorcery:
    tACK 1cc93fe7b40f10a7d1d1189058af98a2bce31381 (win11 msvc).
  maflcko:
    re-ACK 1cc93fe7b40f10a7d1d1189058af98a2bce31381

Tree-SHA512: a390797bb4d3b7eb9163653b6c9c324e7a01090f6cdda74df7349a24a5c4a2084e5912878747f56561315afc70cae9adb1c363f47ceb0af96004ea591d25171b
2024-09-12 10:30:06 +01:00
Hennadii Stepanov
5c80192ff6
test: Drop no longer needed workarounds
`ctest` skips "no test cases matching filter" tests gracefully.
2024-09-08 09:05:39 +01:00
Hennadii Stepanov
f03c942095
build, test: Add missed log options 2024-09-06 21:59:51 +01:00
Ryan Ofsky
73fe7d7230 multiprocess: Add unit tests for connect, serve, and listen functions 2024-09-06 09:08:10 -04:00
MarcoFalke
faecca9a85
test: Use span for raw data
This change allows to drop brittle sizeof calls in favor of the
std::span::size method.

Other improvements include:

* Use of a namespace to mark test and bench data
* Use of the modern std::byte
* Drop of a no longer used std::vector copy and the bench/data module
2024-09-05 10:57:19 +02:00
Hennadii Stepanov
973a3b0c5d
cmake: Implement install build target 2024-08-16 21:19:11 +01:00
Hennadii Stepanov
90cec4d251
cmake: Add MULTIPROCESS option 2024-08-16 21:19:10 +01:00
Hennadii Stepanov
d10c5c34c3
cmake: Add wallet functionality 2024-08-16 19:27:40 +01:00
Hennadii Stepanov
ab2e99b0d9
cmake: Create test suite for ctest 2024-08-16 19:27:39 +01:00
Hennadii Stepanov
959370bd76
cmake: Build test_bitcoin executable 2024-08-16 19:27:39 +01:00