Use the AFL++ shared memory ID environment variable to create
a deterministic datadir path. This prevents accumulation of stale
directories after a fuzz iteration crashes or times out. During
long fuzz campaigns, this accumulation has occasionally resulted
in running out of disk space.
Github-Pull: #34445
Rebased-From: d3e681bc06758fe0686cd96fcfd4a1c4c5af62b4
and_v() must never be 'd'. This is not a bug fix since this was
unreachable in valid Miniscripts: the first sub of an and_v() must be of
type V, which conflicts with (i.e. never has) property 'd'.
Github-Pull: #34434
Rebased-From: 4fab35cf88c048d2784fe6d71d3f83cc4e420879
removeprunedfunds removes all entries from mapTxSpends for the
inputs of the pruned tx. However, this is incorrect, because there could be
multiple entries from conflicting transactions (that shouldn't be
removed as well). This could lead to the wallet creating invalid
transactions, trying to double spend utxos.
The bug persists when the conflicting tx was mined, because
the wallet trusts its internal accounting instead of calling
AddToSpends again.
Github-Pull: #34358
Rebased-From: 1f60ca360eb83fa7982b1aac402eaaf477294197
The previous `assert` used `>=`, allowing `input_index == psbt.inputs.size()` and out-of-bounds access in `psbt.inputs[input_index]`.
Found during review: https://github.com/bitcoin/bitcoin/pull/31650#discussion_r2685892867
Github-Pull: #34272
Rebased-From: 2f5b1c5f80590ffa6b5a5bcfb21fddb1dc22e852
Because the default wallet has no name, the watch-only and solvables
wallets created during migration end up having no name either.
This fixes it by applying the same prefix name we use for the backup
file for an unnamed default wallet.
Before: watch-only wallet named "_watchonly"
After: watch-only wallet named "default_wallet_watchonly"
Github-Pull: #34156
Rebased-From: 82caa8193a3e36f248dcc949e0cd41def191efac
Right now, after migration the last message users see is "migration completed",
but the migration isn't actually finished yet. We still need to load the new wallets
to ensure consistency, and if that fails, the migration will be rolled back. This
can be confusing for users.
This change logs the post-migration loading step and if a wallet fails to load and
the migration will be rolled back.
Github-Pull: #34156
Rebased-From: d70b159c42008ac3b63d1c43d99d4f1316d2f1ef
When migrating any legacy unnamed wallet, a failed migration would
cause the cleanup logic to remove its parent directory. Since this
type of legacy wallet lives directly in the main '/wallets/' folder,
this resulted in unintentionally erasing all wallets, including the
backup file.
To be fully safe, we will no longer call `fs::remove_all`. Instead,
we only erase the individual db files we have created, leaving
everything else intact. The created wallets parent directories are
erased only if they are empty.
As part of this last change, `RestoreWallet` was modified to allow
an existing directory as the destination, since we no longer remove
the original wallet directory (we only remove the files we created
inside it). This also fixes the restore of top-level default wallets
during failures, which were failing due to the directory existence
check that always returns true for the /wallets/ directory.
This bug started after:
f6ee59b6e2
Previously, the `fs::copy_file` call was failing for top-level wallets,
which prevented the `fs::remove_all` call from being reached.
Github-Pull: #34156
Rebased-From: f4c7e28e80bf9af50b03a770b641fd309a801589
Track what RestoreWallet creates so only those files and directories
are removed during a failure and nothing else. Preexisting paths
must be left untouched.
Note:
Using fs::remove_all() instead of fs::remove() in RestoreWallet does
not cause any problems currently, but the change is necessary for the
next commit which extends RestoreWallet to work with existing directories,
which may contain files that must not be deleted.
Github-Pull: #34156
Rebased-From: 4ed0693a3f2a427ef9e7ad016930ec29fa244995
Signal m_tip_block_cv when Ctrl-C is pressed or SIGTERM is received, the same
way it is currently signalled when the `stop` RPC is called. This lets RPC
calls like `waitforblockheight` and IPC calls like `waitTipChanged` be
interrupted, instead of waiting for their original timeouts and delaying
shutdown.
Historical notes:
- The behavior where `stop` RPC signals `m_tip_block_cv`, but CTRL-C does not,
has been around since the condition variable was introduced in #30409
(7eccdaf16081d6f624c4dc21df75b0474e049d2b).
- The signaling was later moved without changing behavior in #30967
(5ca28ef28bcca1775ff49921fc2528d9439b71ab). This commit moves it again to
the Interrupt() function, which is probably the place it should have been
added initially, so it works for Ctrl-C shutdowns as well as `stop`
shutdowns.
- A Qt shutdown bug calling wait methods was fixed previously in #18452
(da73f1513a637a9f347b64de66564d6cdb2541f8), and this change updates that
fix to avoid the hang happening again in Qt.
Github-Pull: #33511
Rebased-From: c25a5e670b27d3b6eb958ce437dbe89678bd1511
- This method can be used to cancel a running
waitNext().
- This commit also adds a test case for interruptWait method
Github-Pull: #33676
Rebased-From: dcb56fd4cb59e6857c110dd87019459989dc1ec3
In `QSortFilterProxyModel`, `invalidateFilter()` is scheduled for
deprecation in Qt 6.13.
`beginFilterChange()` was introduced in Qt 6.9.
`endFilterChange()` was introduced in Qt 6.10.
Github-Pull: gui#899
Rebased-From: e15e8cbadad5ce1de41ebb817b87054f8b5192f2
Block template fees are calculated by looping over new_tmpl->vTxFees
and return (early) once the fee_threshold is exceeded.
This left an edge case when the mempool is empty, which this commit
fixes and adds a test for. It does so by using std::accumulate instead
of manual loops.
Also update interface_ipc.py to account for the new behavior.
Co-authored-by: Raimo33 <claudio.raimondi@protonmail.com>
Github-Pull: #33566
Rebased-From: 8f7673257a1a86717c1d83770dc857fc254df107
Without this change, logging (even if unused) may account for a
substantial portion of bitcoin-node's and/or client's runtime cpu usage, due
to libmultiprocess's expensive message serialization.
This (along with some recent upstream changes) avoids the overhead by opting
out of log handling for messages that we're not interested in.
Info, Warning, and Error are logged unconditionally to match our behavior
elsewhere. See BCLog::Logger::GetCategoryLogLevel .
Github-Pull: #33517
Rebased-From: 0626b90f507db68610a69feec86deb712dd095a1
a4f929696490 Merge bitcoin-core/libmultiprocess#224: doc: fix typos
f4344ae87da0 Merge bitcoin-core/libmultiprocess#222: test, ci: Fix threadsanitizer errors in mptest
1434642b3804 doc: fix typos
73d22ba2e930 test: Fix tsan race in thread busy test
b74e1bba014d ci: Use tsan-instrumented cap'n proto in sanitizers job
c332774409ad test: Fix failing exception check in new thread busy test
ca3c05d56709 test: Use KJ_LOG instead of std::cout for logging
7eb1da120ab6 ci: Use tsan-instrumented libcxx in sanitizers job
ec86e4336e98 Merge bitcoin-core/libmultiprocess#220: Add log levels and advertise them to users via logging callback
515ce93ad349 Logging: Pass LogData struct to logging callback
213574ccc43d Logging: reclassify remaining log messages
e4de0412b430 Logging: Break out expensive log messages and classify them as Trace
408874a78fdc Logging: Use new logging macros
67b092d835cd Logging: Disable logging if messsage level is less than the requested level
d0a1ba7ebf21 Logging: add log levels to mirror Core's
463a8296d188 Logging: Disable moving or copying Logger
83a2e10c0b03 Logging: Add an EventLoop constructor to allow for user-specified log options
58cf47a7fc8c Merge bitcoin-core/libmultiprocess#221: test default PassField impl handles output parameters
db03a663f514 Merge bitcoin-core/libmultiprocess#214: Fix crash on simultaneous IPC calls using the same thread
afcc40b0f1e8 Merge bitcoin-core/libmultiprocess#213: util+doc: Clearer errors when attempting to run examples + polished docs
6db669628387 test In|Out parameter
29cf2ada75ea test default PassField impl handles output parameters
1238170f68e8 test: simultaneous IPC calls using same thread
eb069ab75d83 Fix crash on simultaneous IPC calls using the same thread
ec03a9639ab5 doc: Precision and typos
2b4348193551 doc: Where possible, remove links to ryanofsky/bitcoin/
286fe469c9c9 util: Add helpful error message when failing to execute file
47d79db8a552 Merge bitcoin-core/libmultiprocess#201: bug: fix mptest hang, ProxyClient<Thread> deadlock in disconnect handler
f15ae9c9b9fb Merge bitcoin-core/libmultiprocess#211: Add .gitignore
4a269b21b8c8 bug: fix ProxyClient<Thread> deadlock if disconnected as IPC call is returning
85df96482c49 Use try_emplace in SetThread instead of threads.find
ca9b380ea91a Use std::optional in ConnThreads to allow shortening locks
9b0799113557 doc: describe ThreadContext struct and synchronization requirements
d60db601ed9b proxy-io.h: add Waiter::m_mutex thread safety annotations
4e365b019a9f ci: Use -Wthread-safety not -Wthread-safety-analysis
15d7bafbb001 Add .gitignore
fe1cd8c76131 Merge bitcoin-core/libmultiprocess#208: ci: Test minimum cmake version in olddeps job
b713a0b7bfbc Merge bitcoin-core/libmultiprocess#207: ci: output CMake version in CI script
0f580397c913 ci: Test minimum cmake version in olddeps job
d603dcc0eef0 ci: output CMake version in CI script
git-subtree-dir: src/ipc/libmultiprocess
git-subtree-split: a4f92969649018ca70f949a09148bccfeaecd99a
Not enforcing TRUC topology on reorg was the intended
behavior, but the appropriate bypass argument was not
checked.
This mistake means we could potentially invalidate a long
chain of perfectly incentive-compatible transactions that
were made historically, including subsequent non-TRUC
transactions, all of which may have been very high feerate.
Lastly, it wastes CPU cycles doing topology checks since
this behavior cannot actually enforce the topology in
general for the reorg setting.
Github-Pull: #33504
Rebased-From: 26e71c237d9d2197824b547f55ee3a0a60149f92
Using bypass_limits=true is essentially fuzzing part of a
reorg only, and results in TRUC invariants unable to be
checked. Remove most instances of bypassing limits, leaving
one harness able to do so.
Github-Pull: #33504
Rebased-From: bbe8e9063c15dc230553e0cbf16d603f5ad0e4cf
A target field was added to the getblock and getblockheader RPC calls in bitcoin#31583, but it mistakingly always used the tip value.
Because regtest does not have difficulty adjustment, a test is added for mainnet instead.
Github-Pull: #33446
Rebased-From: bf7996cbc3becf329d8b1cd2f1007fec9b3a3188
when `GetTotalRAM` returns an `std::nullopt` now we're getting:
```
The following tests did not run:
106 - system_ram_tests (Skipped)
```
GitHub-Pull: #33435
Rebased-From: 56791b582958e905e5ba5cbf172a8ea7dad1a8b0
This patch achieves two things:
1. Fix unused variable warning (https://github.com/bitcoin/bitcoin/pull/33333#discussion_r2362493046)
2. Enable GetTotalRAM() on other platforms where it was tested to work.
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
Github-Pull: #33435
Rebased-From: 337a6e738616781f81504275bac8ed7bcf8068df
Oversized allocations can cause out-of-memory errors or [heavy swapping](https://github.com/getumbrel/umbrel-os/issues/64#issuecomment-663637321), [grinding the system to a halt](https://x.com/murchandamus/status/1964432335849607224).
`LogOversizedDbCache()` now emits a startup warning if the configured `-dbcache` exceeds a cap derived from system RAM, using the same parsing/clamping as cache sizing via CalculateDbCacheBytes(). This isn't meant as a recommended setting, rather a likely upper limit.
Note that we're not modifying the set value, just issuing a warning.
Also note that the 75% calculation is rounded for the last two numbers since we have to divide first before multiplying, otherwise we wouldn't stay inside size_t on 32-bit systems - and this was simpler than casting back and forth.
We could have chosen the remaining free memory for the warning (e.g. warn if free memory is less than 1 GiB), but this is just a heuristic, we assumed that on systems with a lot of memory, other processes are also running, while memory constrained ones run only Core.
If total RAM < 2 GiB, cap is `DEFAULT_DB_CACHE` (`450 MiB`), otherwise it's 75% of total RAM.
The threshold is chosen to be close to values commonly used in [raspiblitz](https://github.com/raspiblitz/raspiblitz/blob/dev/home.admin/_provision.setup.sh#L98-L115) for common setups:
| Total RAM | `dbcache` (MiB) | raspiblitz % | proposed cap (MiB) |
|----------:|----------------:|-------------:|-------------------:|
| 1 GiB | 512 | 50.0% | 450* |
| 2 GiB | 1536 | 75.0% | 1536 |
| 4 GiB | 2560 | 62.5% | 3072 |
| 8 GiB | 4096 | 50.0% | 6144 |
| 16 GiB | 4096 | 25.0% | 12288 |
| 32 GiB | 4096 | 12.5% | 24576 |
[Umbrel issues](https://github.com/getumbrel/umbrel-os/issues/64#issuecomment-663816367) also mention 75% being the upper limit.
Starting `bitcoind` on an 8 GiB rpi4b with a dbcache of 7 GiB:
> ./build/bin/bitcoind -dbcache=7000
warns now as follows:
```
2025-09-07T17:24:29Z [warning] A 7000 MiB dbcache may be too large for a system memory of only 7800 MiB.
2025-09-07T17:24:29Z Cache configuration:
2025-09-07T17:24:29Z * Using 2.0 MiB for block index database
2025-09-07T17:24:29Z * Using 8.0 MiB for chain state database
2025-09-07T17:24:29Z * Using 6990.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
```
Besides the [godbolt](https://godbolt.org/z/EPsaE3xTj) reproducers for the new total memory method, we also tested the warnings manually on:
- [x] Apple M4 Max, macOS 15.6.1
- [x] Intel Core i9-9900K, Ubuntu 24.04.2 LTS
- [x] Raspberry Pi 4 Model B, Armbian Linux 6.12.22-current-bcm2711
- [x] Intel Xeon x64, Windows 11 Home Version 24H2, OS Build 26100.4351
Co-authored-by: stickies-v <stickies-v@protonmail.com>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: w0xlt <woltx@protonmail.com>
Github-Pull: #33333
Rebased-From: 168360f4ae47cbfdb30a2cc4704435bc67e12f16
Added a minimal system helper to query total physical RAM on [Linux/macOS/Windows](https://stackoverflow.com/a/2513561) (on other platforms we just return an empty optional).
The added test checks if the value is roughly correct by checking if the CI platforms are returning any value and if the value is at least 1 GiB and not more than 10 TiB.
The max value is only validated on 64 bits, since it's not unreasonable for 32 bits to have max memory, but on 64 bits it's likely an error.
https://learn.microsoft.com/en-us/windows/win32/api/sysinfoapi/ns-sysinfoapi-memorystatusex
> ullTotalPhys The amount of actual physical memory, in bytes.
https://man7.org/linux/man-pages/man3/sysconf.3.html:
> _SC_PHYS_PAGES The number of pages of physical memory. Note that it is possible for the product of this value and the value of _SC_PAGESIZE to overflow.
> _SC_PAGESIZE Size of a page in bytes. Must not be less than 1.
See https://godbolt.org/z/ec81Tjvrj for further details
Github-Pull: #33333
Rebased-From: 6c720459beead5c825b354a1d5c11969b6e3a170
In the presence of smaller transactions on the network, blocks can sustain a
higher relay rate than 7tx/second. In this event, the per-peer inventory queues
can grow too large.
This commit bumps the rate up to 14 tx/s (for inbound peers), increasing the
safety margin by a factor of 2.
Outbound peers continue to receive relayed transactions at 2.5x the rate of
inbound peers, for a rate of 35tx/second.
Co-Authored-By: Suhas Daftuar <sdaftuar@gmail.com>
Github-Pull: #28592
Rebased-From: b81f37031c8f2ccad9346f1b65ee0f8083c44796
Tor inbound connections do not reveal the peer's actual network address.
Therefore do not apply whitelist permissions to them.
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Github-Pull: #33395
Rebased-From: f563ce90818d486d2a199439d2f6ba39cd106352
mzumsande pointed out https://github.com/bitcoin/bitcoin/pull/32345#issuecomment-3286964369 that this test was causing a warning:
Warning: Disk space for "/tmp/test_common bitcoin/node_init_tests/init_test/bf78678cb7723a3e84b5/blocks" may not accommodate the block files. Approximately 810 GB of data will be stored in this directory.
Fix by setting regtest instead of mainnet network before running the test.
Github-Pull: #33391
Rebased-From: bdf01c6f61262cd6e211ead3c0dbc66ccb48b32f
m_cached_from_me is used to track whether a transaction is "from me", i.e. has
any inputs which belong to the wallet. This is held in memory only in
the same way that a transaction's balances are.
Github-Pull: #33268
Rebased-From: 113a4228229baedda2a730e097f2d59ad58a4b0d
Instead of checking whether the total amount of inputs known by the
wallet is greater than 0, we should be checking for whether the input is
known by the wallet. This enables us to determine whether a transaction
spends an of output with an amount of 0, which is necessary for marking
0-value dust outputs as spent.
Github-Pull: #33268
Rebased-From: 39a7dbdd277d1dea9a70314d8cc5ae057999ee88
790b440197bde322432a5bab161f1869b667e681 Fix benchmark CSV output (Hennadii Stepanov)
Pull request description:
The `SHA256AutoDetect` return output is used, among other use cases, to name benchmarks. Using a comma breaks the `bench_bitcoin` CSV output.
This PR replaces the comma with a semicolon, which fixes https://github.com/bitcoin/bitcoin/issues/33331.
ACKs for top commit:
Raimo33:
Code Review ACK 790b440197bde322432a5bab161f1869b667e681
l0rinc:
Code review ACK 790b440197bde322432a5bab161f1869b667e681
janb84:
code review ACK 790b440197bde322432a5bab161f1869b667e681
Tree-SHA512: 096bfa29a0639a4d97d510a3e2a15f071f384148c3035e4d0fc525794682e499c45a0d0c95728d5c78010098393b2c486a7fa9c21c1e2fbb600dea7c5638a55f
8b6264768030db1840041abeeaeefd6c227a2644 test: send duplicate blocktxn message in p2p_compactblocks.py (Eugene Siegel)
5e585a0fc4fd68dd7b4982054b34deae2e7aeb89 net: check for empty header before calling FillBlock (Eugene Siegel)
Pull request description:
This avoids an Assume crash if multiple blocktxn messages are received. The first call to `FillBlock` would make the header empty via `SetNull` and the call right before the second `FillBlock` would crash [here](689a321976/src/net_processing.cpp (L3333)) since `LookupBlockIndex` won't find anything. Fix that by checking for an empty header before the Assume.
ACKs for top commit:
instagibbs:
reACK 8b62647680
fjahr:
tACK 8b6264768030db1840041abeeaeefd6c227a2644
achow101:
ACK 8b6264768030db1840041abeeaeefd6c227a2644
mzumsande:
Code Review ACK 8b6264768030db1840041abeeaeefd6c227a2644
Tree-SHA512: d43a6f652161d4f7e6137f207a3e95259fc51509279d20347b1698c91179c39c8fcb75d2668b13a6b220f478a03578573208a415804be1d8843acb057fa1a73a
