This fixes ``Assertion `m_node.chainman' failed`` errors first reported
https://github.com/bitcoin/bitcoin/issues/33994#issuecomment-3602551596 when
IPC mining methods are called before ChainstateManager is loaded.
The fix works by making the `Init.makeMining` method block until chainstate
data is loaded.
Instead of having the InitAndLoadChainstate function delete and create the
KernelNotifications object each time it is called (it can be called twice when
reindexing) to clear cached state, create it just one time and add a
setChainstateLoaded() method to manage state as it is loaded and unloaded.
This refactoring should make sense by itself to be more explicit about how
KernelNotifications state is cleared, but it's also needed to make outside code
accessing KernelNotifications state (currently just mining code) safe during
node startup and shutdown so the KernelNofications mutex can be used for
synchronization and does not get recreated itself.
This parametrizes the cost model for the SFL algorithm with another
class. Right now, the behavior of that class matches the naive cost
model so far, but it will be replaced with a more advanced on in a
future commit.
The reason for abstracting this out is that it makes benchmarking for
creating such cost models easy, by instantiating the cost model class
with one that tracks time.
The index is now initialized after the setup phase (chain generation
and txs creation), since it doesn't participate on it at all.
This improves readability and splits setup from what we actually
want to check.
This also adds a check after Sync() to verify the index best block hash
matches the tip, so we know it fully synced before checking the
processed data. This will help catching errors as Sync() could have
aborted prematurely.
As a happy side effect, the SyncWithValidationInterfaceQueue() call at
the end of the test is no longer needed and has been removed.
There's no change in behavior. This is just a refactoring to avoid a minor
layer violation in init code. The node.init object is intended to return
interface pointers for code outside the node (like wallet and gui code), not
used by node itself to initialize its internal state.
(Motivation for this change is to introduce a MakeMining wait_loaded option in
an upcoming commit that can only be used internally and not set by external
clients.)
a28eedb8c29a6475c40b9d32b8de0ba017ea98b7 ci: use LLVM 22 in sanitizer tasks (fanquake)
Pull request description:
Clang/LLVM 22 is out, and upstream apt issues seem to be resolved. Use it in the sanitizer related tasks in CI. Not changing tidy here (needs more changes).
ACKs for top commit:
maflcko:
lgtm ACK a28eedb8c29a6475c40b9d32b8de0ba017ea98b7
hebasto:
ACK a28eedb8c29a6475c40b9d32b8de0ba017ea98b7.
Tree-SHA512: 311cca7d1ebc5769812e63c1946e8bad1bd2da728bfe3a4b91182eca8c9a9036f7506aff4ea99665982f57c14e4523b674a8689a2390e30936c16ae2548bc759
The error was added in commit 1ea7e45a1f445d32a2b690d52befb2e63418653b,
because there was an additional confusing `AssertionError: [node 0]
Error: no RPC connection` instead of just a single `FileNotFoundError:
[Errno 2] No such file or directory`.
This is no longer needed on current master.
Also, the test is incomplete, because it was just checking bitcoind and
bitcoin-cli, not any other missing binaries.
Also, after the previous commit, it would not work in combination with
--valgrind.
Instead of trying to make it complete, and work in all combinations,
just remove it, because the already existing error will be clear in any
case.
This can be tested via:
```sh
./test/get_previous_releases.py
mv releases releases_backup
# Confirm the test is skipped due to missing releases
./bld-cmake/test/functional/wallet_migration.py
# Confirm the test fails due to missing releases
./bld-cmake/test/functional/wallet_migration.py --previous-releases
mv releases_backup releases
mv ./releases/v28.2 ./releases/v28.2_backup
# Confirm the test fails with a single FileNotFoundError
./bld-cmake/test/functional/wallet_migration.py
mv ./releases/v28.2_backup ./releases/v28.2
# Confirm the test runs and passes
./bld-cmake/test/functional/wallet_migration.py
rm ./bld-cmake/bin/bitcoind
# Confirm the test fails with a single "No such file or directory",
# testing with and without --valgrind
./bld-cmake/test/functional/wallet_migration.py
./bld-cmake/test/functional/wallet_migration.py --valgrind
```
Prior to this commit, tool_bitcoin.py was failing:
```sh
$ ./bld-cmake/test/functional/tool_bitcoin.py --valgrind
TestFramework (ERROR): Unexpected exception
Traceback (most recent call last):
File "./test/functional/test_framework/test_framework.py", line 138, in main
self.setup()
~~~~~~~~~~^^
File "./test/functional/test_framework/test_framework.py", line 269, in setup
self.setup_network()
~~~~~~~~~~~~~~~~~~^^
File "./test/functional/tool_bitcoin.py", line 38, in setup_network
assert all(node.args[:len(node_argv)] == node_argv for node in self.nodes)
~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
```
This commit fixes this issue by running `bitcoin` under valgrind. Also,
it comes with other improvements:
* Drop the outdated valgrind 3.14 requirement, because there is no
distro that ships a version that old anymore.
* Drop the VALGRIND_SUPPRESSIONS_FILE env var handling, because it was
presumably never used since it was introduced. Also, the use-case
seems limited.
Review note:
The set_cmd_args was ignoring the --valgrind test option.
In theory, this could be fixed by refactoring Binaries::node_argv() to
be used here. However, for now, just re-implement the node_argv logic in
set_cmd_args to prepend the valgrind cmd.
fcaec2544b32226fd5357a88506fe080058d25bc doc: release note for IPC cooldown and interrupt (Sjors Provoost)
1e82fa498cf4881466f0539146c101242b9dc30d mining: add interrupt() (Sjors Provoost)
a11297a9048e0d910915e1a37b2be467c057a78d mining: add cooldown argument to createNewBlock() (Sjors Provoost)
Pull request description:
As reported in #33994, connected mining clients will receive a flood of new templates if the node is still going through IBD or catching up on the last 24 hours. This PR fixes that using an _optional_ cooldown mechanism, only applied to `createNewBlock()`.
First, cooldown waits for IBD. Then, as the tip keeps moving forward, it waits a few seconds to see if the tip updated. If so, it restarts the timer and waits again. The trade-offs for this mechanism are explained below.
Because this PR changes `createNewBlock()` from a method that returns quickly to one that can block for minutes, we rely on #34568 to fix a bug in our `.capnp` definition, adding the missing `context` to `createNewBlock` (and `checkBlock`).
The second commit then adds an `interrupt()` method so that clients can cleanly disconnect.
---
## Rationale
The cooldown argument is optional, and not used by internal non-IPC code, for two reasons:
1. The mechanism wreaks havoc on the functional test suite, which would require very careful mock time handling to work around. But that's pointless, because only IPC clients need it.
2. It needs to be optional for IPC clients too, because in some situations, like a signet with only one miner, waiting for IBD can mean being stuck forever.
The reason it's only applied to `createNewBlock()` is that this is the first method called by clients; `waitNext()` is a method on the interface returned by `createNewBlock()`, at which point the cooldown is done.
After IBD, we wait N seconds if the header is N blocks ahead of the tip, with a minimum of 3 and a maximum of 20 seconds. The minimum waiting time is short enough that it shouldn't be annoying or confusing for someone manually starting up a client. While the maximum should be harmless if it happens spuriously (which it shouldn't).
If the minimum wait is too short, clients get a burst of templates, as observed in the original issue. We can't entirely rule this out without a lot of additional complexity (like scanning our own log file for heuristics). This PR should make it a lot less likely, and thanks to the IBD wait also limit it to one day worth of blocks (`-maxtipage`).
Some test runs on an M4 MacBook Pro, where I had a node catch up on the last few days worth of blocks:
<img width="872" height="972" alt="Schermafbeelding 2026-02-04 om 18 21 17" src="https://github.com/user-attachments/assets/7902a0f2-0e0b-4604-9688-cec2da073261" />
As the chart shows, sometimes it takes longer than 3 seconds. But it turns out that in all those cases there were quite a few headers ahead of the tip. It also demonstrates that it's important to first wait for IBD, because it's less likely a random tip update takes longer than 20 seconds.
- modified sv2-apps: https://github.com/Sjors/sv2-apps/tree/2026/02/cooldown
- test script: https://gist.github.com/Sjors/feb6122c97acc2b9e6d66b168614609c#file-run_mainnet_pool_loop-zsh
- chart script: https://gist.github.com/Sjors/feb6122c97acc2b9e6d66b168614609c#file-tip_interval_charts-py
ACKs for top commit:
ryanofsky:
Code review ACK fcaec2544b32226fd5357a88506fe080058d25bc. Only changes since last review were removing two cooldown arguments from the mining IPC test to simplify it
enirox001:
ACK fcaec2544b
Tree-SHA512: 08b75470f7c5c80a583a2fdb918fad145e7d5377309e5c599f67fc0d0e3139d09881067ba50c74114f117e69da17ee50666838259491691c031b1feaf050853f
c462e54f9df431434e6480d8293060645468d3ab test: don't always assert NUM_PRIVATE_BROADCAST_PER_TX broadcasts (Vasil Dimov)
3710566305e569bed8458809f0dedc83420b7de2 test: move abortprivatebroadcast test at the end (Vasil Dimov)
Pull request description:
_test: move abortprivatebroadcast test at the end_
The piece of `p2p_private_broadcast.py` which tests the correctness of
`abortprivatebroadcast` issues a new `sendrawtransaction` call. That
call schedules up to 3 new connections: peer=13, peer=14 and possibly
peer=15 before it gets aborted.
These up to 3 in-the-process-of-opening private broadcast connections
have `CNode::m_connected` set early - when the `CNode` object is
created. Later in the test the mock time is advanced by 20 minutes and
those "old" connections pick a transaction for rebroadcast but that
triggers `PRIVATE_BROADCAST_MAX_CONNECTION_LIFETIME` immediately:
```
2026-02-21T13:28:14.209766Z [privbcast] [net.cpp:4006] [CNode] [net] Added connection peer=20
2026-02-21T13:28:14.309792Z (mocktime: 2026-02-21T13:48:14Z) [msghand] [net.cpp:4074] [PushMessage] [net] sending inv (37 bytes) peer=20
2026-02-21T13:28:14.309801Z (mocktime: 2026-02-21T13:48:14Z) [msghand] [net_processing.cpp:5745] [SendMessages] [privatebroadcast] Disconnecting: did not complete the transaction send within 180 seconds, peer=20
```
This prematurely stops the private broadcast connection and results in
a failure like:
```
AssertionError: ... not({} == {'ping': 1, 'tx': 1})
```
---
_test: don't always assert NUM_PRIVATE_BROADCAST_PER_TX broadcasts_
In `p2p_private_broadcast.py` in the function `check_broadcasts()` we
should assert that the broadcast was done to `broadcasts_to_expect`
peers, not to `NUM_PRIVATE_BROADCAST_PER_TX`. This is because in the
"Basic" test we check the first broadcast manually because it is done to
`nodes[1]` and then check the other two by
`check_broadcasts(..., NUM_PRIVATE_BROADCAST_PER_TX - 1, ...)`.
The first broadcast might not have fully concluded by the time we call
`check_broadcasts()` to check the remaining 2.
Demanding always `NUM_PRIVATE_BROADCAST_PER_TX` can lead to:
```
Traceback (most recent call last):
File "/home/vd/gh/bitcoin/bitcoin/test/functional/test_framework/test_framework.py", line 142, in main
self.run_test()
~~~~~~~~~~~~~^^
File "/tmp/build/clang22/test/functional/p2p_private_broadcast.py", line 347, in run_test
self.check_broadcasts("Basic", txs[0], NUM_PRIVATE_BROADCAST_PER_TX - 1, NUM_INITIAL_CONNECTIONS + 1)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/build/clang22/test/functional/p2p_private_broadcast.py", line 313, in check_broadcasts
assert_greater_than_or_equal(sum(1 for p in peers if "received" in p), NUM_PRIVATE_BROADCAST_PER_TX)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vd/gh/bitcoin/bitcoin/test/functional/test_framework/util.py", line 94, in assert_greater_than_or_equal
raise AssertionError("%s < %s" % (str(thing1), str(thing2)))
AssertionError: 2 < 3
```
ACKs for top commit:
l0rinc:
ACK c462e54f9df431434e6480d8293060645468d3ab
achow101:
ACK c462e54f9df431434e6480d8293060645468d3ab
andrewtoth:
ACK c462e54f9df431434e6480d8293060645468d3ab
Tree-SHA512: 0de8d0eae079eeedc3bfad39df8129a8fa0d7734bdc03b4fb3e520a2f13a187d68118ffc210556af125d634f0ff51a1b081b34a023ac68a1c6a0caf541cecb91
a9e59f7d955f995078b3e0bf3b527c03c74fef8d rpc: add optimal result to getmempoolinfo (Greg Sanders)
a3fb3dd55c2326452a5085add220bd3682052352 mempool: log if we detect a non-optimal mempool (Greg Sanders)
Pull request description:
Post-SFL #34023 I don't think we expect the mempool to be unordered for long periods of time. If we consider it likely to be a serious regression in production, it would be useful to expose the fact that the mempool is not known to be optimal.
1. do a MEMPOOL log after any `DoWork()` returns false, meaning non-optimal
2. expose it via getmempoolinfo, by calling `DoWork(0)`, which does nothing but return known-optimality
I'm not wedded to either approach, I just think something is better than nothing for the next release.
ACKs for top commit:
ajtowns:
ACK a9e59f7d955f995078b3e0bf3b527c03c74fef8d
ismaelsadeeq:
reACK a9e59f7d955f995078b3e0bf3b527c03c74fef8d [c89b93b95..a9e59f7d95](c89b93b958..a9e59f7d95) fixed typo, added more logging for block/reorg additions to mempool, and fixed brittle test case.
sedited:
ACK a9e59f7d955f995078b3e0bf3b527c03c74fef8d
sipa:
ACK a9e59f7d955f995078b3e0bf3b527c03c74fef8d
Tree-SHA512: 1560ad21cc1606df7279c102f35f61d4555c0ac920f02208b2a6eb89b14d7e22befb6d7f510a00a9074c2f9931f32e9af86bcea3a8dd9a1d947b0398c84666dd
Modify fuzzer logic to avoid setting `FRESH` for an outpoint that already exists unspent in the parent view, and ensure `FRESH` implies `DIRTY`.
This keeps cursor invariants realistic and lets `BatchWrite` failures expose real bugs without resetting state.
The coins view fuzzer can call `AddCoin` with `possible_overwrite=false` for an outpoint that already exists unspent in the view, which violates the `AddCoin` caller contract.
Derive `possible_overwrite` from `PeekCoin` so `possible_overwrite=false` is only used when the outpoint is absent.
This matches the approach used by the `coinscache_sim` fuzzer, which derives the overwrite flag from simulated state.
In validation, `AddCoins(check_for_overwrite=false)` is only used after BIP30 has already ensured the transaction does not overwrite any unspent outputs in the UTXO view.
The coins view fuzz target can call `AddCoins` with arbitrary txids, so using the `check_for_overwrite=false` fast path on non-coinbase transactions may violate the `AddCoin` caller contract and trigger logic errors.
Only use `check_for_overwrite=false` when we have first confirmed that none of the outputs are currently unspent.
Otherwise, fall back to `check_for_overwrite=true` so `AddCoins` determines overwrites via the view.
Introduce `TrySub(T&, U)` which subtracts an unsigned integral `U` from an unsigned integral `T`, returning `false` on underflow.
Use with `Assume(TrySub(...))` at coins cache accounting decrement sites so invariant violations fail immediately rather than silently wrapping.
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Co-authored-by: Pieter Wuille <pieter@wuille.net>
Grouped changes to improve the overall readability and maintainability of the test.
A lot more can be done, but this is a good first step.
1) Use for-loops instead of duplicating lines to perform the same checks for each
node.
2) The {'txid': x, 'vout': y} dict is repeated everywhere in the test, both as
input to gettxspendingprevout and as part of its result when an output has no
known spender, making the test tedious to read and maintain.
This introduces a prevout(txid, vout) query helper and an unspent_out(txid, vout)
result helper to reduce the repetition. These two helpers are intentionally kept
separate to make it immediately clear whether a dict is an input to
gettxspendingprevout or an assertion on its result.
3) The same repetition problem mentioned above applies to other gettxspendingprevout
possible results:
Spent outputs returns {'txid': x, 'vout': y, 'spendingtxid': z} and
Spent outputs when requesting spending tx returns {'txid': x, 'vout': y,
'spendingtxid': z, 'blockhash': w, 'spendingtx': v}
To fix it, this introduces:
- spent_out(txid, vout, spending_tx_id): for outputs with a known spender
- spent_out_in_block(txid, vout, spending_tx_id, blockhash, spending_tx): for
outputs spent in a confirmed block, when full tx data is requested
4) Rename overloaded confirmed_utxo variable (used in three different tests) to more
descriptive names: root_utxo, reorg_replace_utxo, reorg_cancel_utxo to clarify
their roles in each of the tests.
I2P addresses must use port=0, otherwise `bitcoind` refuses to connect.
The test `p2p_private_broadcast.py` cannot simulate connections to I2P
peers, so the I2P proxy is set to a dummy `127.0.0.1:1`. Still it is
good to pick I2P addresses and attempt connections to increase coverage.
However the test uses port=8333 for I2P addresses and thus the
connection attempts fail for the "wrong" reason:
```
Error connecting to ...i2p:8333, connection refused due to arbitrary port 8333
```
Using the proper port=0 makes the failures:
```
Error connecting to ...i2p:0: Cannot connect to 127.0.0.1:1
```
which will make possible simulated I2P connections once we have a test
I2P proxy.
If the following two events happen:
* (likely) the automatic 10 initial connections are not made to all
networks
* (unlikely) the network-specific logic kicks in almost immediately.
It is using exponential distribution with a mean of 5 minutes
(`rng.rand_exp_duration(EXTRA_NETWORK_PEER_INTERVAL)`).
So if both happen, then the 11th connection may not be the expected
private broadcast, but a network-specific connection.
Fix this by retrieving the connection type from
`destinations_factory()`. This is more flexible because it allows
connections to happen in any order and does not break if e.g. the 11th
connection is not the expected first private broadcast.
This also makes the test run faster:
before: 19-44 sec
now: 10-25 sec
because for example there is no need to wait for the initial 10
automatic outbound connections to be made in order to proceed.
Fixes: https://github.com/bitcoin/bitcoin/issues/34387
When `git range-diff` cannot match a commit between two versions of a branch, it shows the old commit as removed (`<`) and the new commit as added (`>`), and it does not show the patch contents.
This output is easy to misread as "no code changes" because the diff for that commit is effectively empty.
It really means the commits were considered unrelated and the new commit should be reviewed from scratch.
This is analogous to rename detection in `git diff`: if similarity is too low, a rename shows up as delete+add.
Example (exact SHAs from PR #34320):
B=ff338fdb53a66ab40a36e1277e7371941fc89840; A=dd76338a57b9b1169ac27f7b783d6d0d4c6e38ab; git fetch upstream $B $A
git range-diff 0ca4295f2e5f4443a1f8b3bae7cba0f6c054276f..$B 139aa4b27e4839291c83a04dcd1649c5595814ca..$A
This produced output like:
1: 4b32181dbb < -: ---------- test: add `HaveInputs` call-path unit tests
-: ---------- > 1: 277c57f0c5 test: add `HaveInputs` call-path unit tests
Even though the subject matches, the first commit had no matching diff shown.
That should be treated as "unmatched" rather than "unchanged".
If you expected a match, try increasing the creation factor so `git range-diff` searches harder:
git range-diff --creation-factor=95 <old_range> <new_range>
This avoids wasting work on calculating bloom filters that aren't
consumed during ibd and continuously re-calculated as now blocks get
validated.
Also update the functional test to document that transactions would now
be requested again once out of IBD.
Co-authored-by: Lőrinc <pap.lorinc@gmail.com>
This documents existing behaviour before the change in the following
commit: The bloom filter maintained by the txdownload manager tracks
recently confirmed transasctions even during ibd. If a peer sends an INV
once IBD is over it does not re-request them.
Co-authored-by: sedited <seb.kung@gmail.com>
In `p2p_private_broadcast.py` in the function `check_broadcasts()` we
should assert that the broadcast was done to `broadcasts_to_expect`
peers, not to `NUM_PRIVATE_BROADCAST_PER_TX`. This is because in the
"Basic" test we check the first broadcast manually because it is done to
`nodes[1]` and then check the other two by
`check_broadcasts(..., NUM_PRIVATE_BROADCAST_PER_TX - 1, ...)`.
The first broadcast might not have fully concluded by the time we call
`check_broadcasts()` to check the remaining 2.
Demanding always `NUM_PRIVATE_BROADCAST_PER_TX` can lead to:
```
Traceback (most recent call last):
File "/home/vd/gh/bitcoin/bitcoin/test/functional/test_framework/test_framework.py", line 142, in main
self.run_test()
~~~~~~~~~~~~~^^
File "/tmp/build/clang22/test/functional/p2p_private_broadcast.py", line 347, in run_test
self.check_broadcasts("Basic", txs[0], NUM_PRIVATE_BROADCAST_PER_TX - 1, NUM_INITIAL_CONNECTIONS + 1)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/build/clang22/test/functional/p2p_private_broadcast.py", line 313, in check_broadcasts
assert_greater_than_or_equal(sum(1 for p in peers if "received" in p), NUM_PRIVATE_BROADCAST_PER_TX)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vd/gh/bitcoin/bitcoin/test/functional/test_framework/util.py", line 94, in assert_greater_than_or_equal
raise AssertionError("%s < %s" % (str(thing1), str(thing2)))
AssertionError: 2 < 3
```
The piece of `p2p_private_broadcast.py` which tests the correctness of
`abortprivatebroadcast` issues a new `sendrawtransaction` call. That
call schedules up to 3 new connections: peer=13, peer=14 and possibly
peer=15 before it gets aborted.
These up to 3 in-the-process-of-opening private broadcast connections
have `CNode::m_connected` set early - when the `CNode` object is
created. Later in the test the mock time is advanced by 20 minutes and
those "old" connections pick a transaction for rebroadcast but that
triggers `PRIVATE_BROADCAST_MAX_CONNECTION_LIFETIME` immediately:
```
2026-02-21T13:28:14.209766Z [privbcast] [net.cpp:4006] [CNode] [net] Added connection peer=20
2026-02-21T13:28:14.309792Z (mocktime: 2026-02-21T13:48:14Z) [msghand] [net.cpp:4074] [PushMessage] [net] sending inv (37 bytes) peer=20
2026-02-21T13:28:14.309801Z (mocktime: 2026-02-21T13:48:14Z) [msghand] [net_processing.cpp:5745] [SendMessages] [privatebroadcast] Disconnecting: did not complete the transaction send within 180 seconds, peer=20
```
This prematurely stops the private broadcast connection and results in
a failure like:
```
AssertionError: ... not({} == {'ping': 1, 'tx': 1})
```
4b53cbd69220c1c786bb23a72c0b26a6f78a38f7 test: Test for musig() in various miniscript expressions (Ava Chow)
ec0f47b15cb3269015523e6fab8ae9241f4181a1 miniscript: Using Func and Expr when parsing keys, hashes, and locktimes (Ava Chow)
6fd780d4fbc497b657025afe48d0dfbf103ee120 descriptors: Increment key_exp_index in ParsePubkey(Inner) (Ava Chow)
b12281bd86e2298ba6cdd79d55c9d6e23e5136a5 miniscript: Use a reference to key_exp_index in KeyParser (Ava Chow)
ce4c66eb7c5e99e3df1c20d5c0ae8278a714b9f8 test: Test that key expression indexes match key count (Ava Chow)
Pull request description:
The miniscript parser currently only looks for the next `)` when parsing key, hash, and locktime expressions. This fails to parse when the expressions contain a nested expression. Currently, this is only possible with `musig()` inside of key expressions. However, this pattern can be generalized to handling hashes and locktimes, so I implemented those too.
Fixes#34076
ACKs for top commit:
rkrux:
ACK 4b53cbd69220c1c786bb23a72c0b26a6f78a38f7
sipa:
ACK 4b53cbd69220c1c786bb23a72c0b26a6f78a38f7
darosior:
Other than that, Approach ACK 4b53cbd69220c1c786bb23a72c0b26a6f78a38f7. That makes sense to me but i have not closely reviewed the code.
Tree-SHA512: 01040c7b07a59d8e3725ff11ab9543b256aea22535fb94059f490a5bb45319e859666af04c2f0a4edcb8cf1e6dfc7bd8a8271b21ad81143bafccd4d0a39cae9c
When running `enable_language()`, CMake checks the platform-specific
toolchain components, which includes `install_name_tool` for macOS.
We disable this check when cross-compiling on Linux because
`install_name_tool` is not used and our toolchain file does not provide
it.
However, the current workaround has been breaking CMake's assumptions
since commit eb8facd39606e9472f9ff6de40d7caf76767d0da (CMake 4.1.0,
backported to 4.0.5), which causes an error during configuration using
the Ninja generator.
This change fixes this behaviour by providing a stub executable instead
of `install_name_tool`, as we currently do in the depends build
subsystem.
fd06157d1465d93b960e8be6e8e419295abde9a1 test: Add coverage for restarted node without any block sync (Fabian Jahr)
3d7ab7ecb7dfcdfb8aaa45869388887b948841c8 rpc, test: Address feedback from #29668 (Fabian Jahr)
312919c9dd5dba7da20317604e1638bdc5010f14 test: Indices can not start based on block data without undo data (Fabian Jahr)
a9a3b29dd687b4c355e131fefc145e8e48b48b17 index: Check availability of undo data for indices (Fabian Jahr)
881ab4fc82fe3cf36b227cf1ba704448df160745 support multiple block status checks in CheckBlockDataAvailability (furszy)
Pull request description:
Currently, we check that `BLOCK_HAVE_DATA` is available for all blocks an index needs to sync during startup. However, for `coinstatsindex` and `blockfilterindex` we also need the undo data for these blocks. If that data is missing in the blocks, we are currently still starting to sync each of these indices and then crash later when we encounter the missing data.
This PR adds explicit knowledge of which block data is needed for each index and then checks its availability during startup before initializing the sync process on them.
This also addresses a few open comments from #29668 in the last commit.
ACKs for top commit:
achow101:
ACK fd06157d1465d93b960e8be6e8e419295abde9a1
furszy:
utACK fd06157d1465d93b960e8be6e8e419295abde9a1
sedited:
Re-ACK fd06157d1465d93b960e8be6e8e419295abde9a1
Tree-SHA512: e2ed81c93372b02daa8ddf2819df4164f96d92de05b1d48855410ecac78d5fcd9612d7f0e63a9d57d7e75a0b46e1bea278e43ea87f2693af0220d1f9c600e416
231dd04b8dcb85ed1bc9191b839fb6cd64acea86 build: define CMAKE_COMPILE_WARNING_AS_ERROR as a cache option (will)
Pull request description:
`CMAKE_COMPILE_WARNING_AS_ERROR` is not a cache variable by default in CMake, so it has no value in the configure summary when not set, and even when set cannot be toggled in `ccmake`. Define it as an `option()` to make it a cache BOOL with a default of OFF.
From the original MR to cmake, this was deliberately not set as a cache variable: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/7187 (see Brad King's reply to the collapsed comments from Marc Chevrier).
Most `CMAKE_*` variables which are expected to be toggled by users are (as far as I can research) cache variables by default. Those that are considered likely to be set by the project (e.g. `CMAKE_CXX_STANDARD` or `CMAKE_POSITION_INDEPENDENT_CODE`) are not, along with read-only variables, script/internal variables, platform sppecific variables, template variables. `CMAKE_COMPILE_WARNING_AS_ERROR` may be a slight outlier here.
I count ~ 600 documented CMAKE_* variables, of which ~ 60 are default cache variables.
I could only see a few of these like:
- CMAKE_COMPILE_WARNING_AS_ERROR
- CMAKE_CXX_STANDARD,
- CMAKE_POSITION_INDEPENDENT_CODE
- CMAKE_INTERPROCEDURAL_OPTIMIZATION
...that we (or any project) _might_ want to expose as user-togglable, and would have to add as an `option()` in CMakeLists.txt.
ACKs for top commit:
ajtowns:
Yeah, seems to work for me. ACK 231dd04b8dcb85ed1bc9191b839fb6cd64acea86
fanquake:
ACK 231dd04b8dcb85ed1bc9191b839fb6cd64acea86
hebasto:
ACK 231dd04b8dcb85ed1bc9191b839fb6cd64acea86, tested on Fedora 43.
Tree-SHA512: e380d79188859c97a9f7e45215a789daff55b8bff945a3d61fe4de3daae54bb7196aca0f6a6f1165979e787238dc26e94d217cff35f9642031ff72f4b39655e1
Both waitTipChanged() and createNewBlock() can take a long time to
return. Add a way for clients to interrupt them.
The new m_interrupt_mining is safely accessed with a lock on
m_tip_block_mutex, but it has no guard annotation. A more thorough
solution is discussed here:
https://github.com/bitcoin/bitcoin/pull/34184#discussion_r2743566474
At startup, if the needs to catch up, connected mining clients will
receive a flood of new templates as new blocks are connected.
Fix this by adding a cooldown argument to createNewBlock(). When set
to true, block template creation is briefly paused while the best
header chain is ahead of the tip.
This wait only happens when the best header extends the current tip,
to ignore competing branches.
Additionally, cooldown waits for isInitialBlockDownload() to latch to
false, which happens when there is less than a day of blocks left to sync.
When cooldown is false createNewBlock() returns immediately. The argument
is optional, because many tests are negatively impacted by this
mechanism, and single miner signets could end up stuck if no block
was mined for a day.
The getblocktemplate RPC also opts out, because it would add a delay
to each call.
Fixes#33994
We expect this to be rare in practice, and to not be the
usual state of the mempool. If we we detect non-optimal
ordering after a DoWork() invocation, allow this to be
observed in MEMPOOL logs.
79c934b51cdca75971aecca8d89586522a5a9733 cmake: Fix NetBSD-specific workaround for Boost (Hennadii Stepanov)
Pull request description:
The recently merged https://github.com/bitcoin/bitcoin/pull/34143 broke builds with depends on NetBSD due to a workaround introduced in 5a5ddbd78922236402df378c8588a7b0b3f83a13.
The upstream [bug](https://gnats.netbsd.org/59856) has been fixed, and the entire workaround can be removed once the fixed Boost package becomes generally available.
However, it seems prudent to amend the workaround now to have it workable in the 31.0 release.
Here are CI runs:
- broken: https://github.com/hebasto/bitcoin-core-nightly/actions/runs/21933683654
- fixed: https://github.com/hebasto/bitcoin-core-nightly/actions/runs/21933683654
ACKs for top commit:
fanquake:
ACK 79c934b51cdca75971aecca8d89586522a5a9733
Tree-SHA512: 360e4943fbedc7692fa2eba48f3375fc927d3aaef178dec6c0fafd3addfad0c89db9d1fe88e214313ebd285584bf8dc7d7b0a3a41d905c7f3d28293aa743405a
f700609e8ada3b48fd45ec19979cd72d943d47a6 doc: Release notes for mining IPC interface bump (Ryan Ofsky)
9453c153612ae9b30308362048099bc53afcde6f ipc mining: break compatibility with existing clients (version bump) (Sjors Provoost)
70de5cc2d205672743379f2e1a94290ee8b4b84b ipc mining: pass missing context to BlockTemplate methods (incompatible schema change) (Sjors Provoost)
2278f017afad4d2c570527b15df776ee64fc1ee2 ipc mining: remove deprecated methods (incompatible schema change) (Ryan Ofsky)
c6638fa7c5e97f9fd7a5ea8feb29f8caeac788bd ipc mining: provide default option values (incompatible schema change) (Ryan Ofsky)
a4603ac77412790b6498ab1750017e31353740bf ipc mining: declare constants for default field values (Ryan Ofsky)
ff995b50cf9e1ea521f3cf546339f05d10b79a4d ipc test: add workaround to block_reserved_weight exception test (Ryan Ofsky)
b970cdf20fce43fb58dde1cbf713e97ff21d7a2e test framework: expand expected_stderr, expected_ret_code options (Ryan Ofsky)
df53a3e5ec8781833c29682ff9e459fca489fa7b rpc refactor: stop using deprecated getCoinbaseCommitment method (Ryan Ofsky)
Pull request description:
This PR increments the field number of the `Init.makeMining` method and makes the old `makeMining` method return an error, so IPC mining clients not using the latest schema file will get an error and not be able to access the Mining interface.
Normally, there shouldn't be a need to break compatibility this way, but the mining interface has evolved a lot since it was first introduced, with old clients using the original methods less stable and performant than newer clients. So now is a good time to introduce a cutoff, drop deprecated methods, and stop supporting old clients which can't function as well.
Bumping the field number is also an opportunity to make other improvements that would be awkward to implement compatibly:
- Making Cap'n Proto default parameter and field values match default values of corresponding C++ methods and structs.
- Adding missing Context parameters to Mining.createNewBlock and checkBlock methods so these methods will be executed on separate execution threads and not block the Cap'n Proto event loop thread.
More details about these changes are in the commit messages.
ACKs for top commit:
Sjors:
ACK f700609e8ada3b48fd45ec19979cd72d943d47a6
enirox001:
ACK f700609e8ada3b48fd45ec19979cd72d943d47a6
ismaelsadeeq:
ACK f700609e8ada3b48fd45ec19979cd72d943d47a6
sedited:
ACK f700609e8ada3b48fd45ec19979cd72d943d47a6
Tree-SHA512: 0901886af00214c138643b33cec21647de5671dfff2021afe06d78dfd970664a844cde9a1e28f685bb27edccaf6e0c3f2d1e6bb4164bde6b84f42955946e366d
0b96b9c600e0dd946fd4d0e827e7f7cbef7a571a Minimize mempool lock, sync txo spender index only when and if needed (sstone)
3d82ec5bdd019cf1c048c41fe44faa855fcb8b53 Add a "tx output spender" index (sstone)
Pull request description:
This PR adds a new "tx output spender" index, which allows users to query which tx spent a given outpoint with the `gettxspendingprevout` RPC call that was added by https://github.com/bitcoin/bitcoin/pull/24408.
Such an index would be extremely useful for Lightning, and probably for most layer-2 protocols that rely on chains of unpublished transactions.
UPDATE: this PR is ready for review and issues have been addressed:
- using a watch-only wallet instead would not work if there is a significant number of outpoints to watch (see https://github.com/bitcoin/bitcoin/pull/24539#issuecomment-1276595646)
- this PR does not require `-txindex` anymore
We use a composite key with 2 parts (suggested by romanz): hash(spent outpoint) and tx position, with an empty value. Average composite key size is 15 bytes.
The spending tx can optionally be returned by `gettxspendingprevout` (even it `-txindex is not set`).
ACKs for top commit:
hodlinator:
re-ACK 0b96b9c600e0dd946fd4d0e827e7f7cbef7a571a
sedited:
Re-ACK 0b96b9c600e0dd946fd4d0e827e7f7cbef7a571a
fjahr:
ACK 0b96b9c600e0dd946fd4d0e827e7f7cbef7a571a
w0xlt:
reACK 0b96b9c600e0dd946fd4d0e827e7f7cbef7a571a
Tree-SHA512: 95c2c313ef4086e7d5bf1cf1a3c7b91cfe2bb1a0dcb4c9d3aa8a6e5bfde66aaca48d85a1f1251a780523c3e4356ec8a97fe6f5c7145bc6ccb6f820b26716ae01
* The bitcoin.conf related checks do not need any timeout, because the
logging happens in the main thread, before the node is fully started.
* The net thread related checks do need a timeout, because the threads
may be late to start after the node is fully started.
This is required to actually erase the orphan transaction when the
BlockConnected event is handled in the background validation interface
queue thread.
cae6d895f8a8cf5f57e05519536fda5d62b10841 fuzz: add target for CoinsViewOverlay (Andrew Toth)
86eda88c8e486eb1db724e60948f71349d050e1d fuzz: move backend mutating block to end of coins_view (Andrew Toth)
89824fb27b228a12d5c2f63106c2a4d793e73107 fuzz: pass coins_view_cache to TestCoinsView in coins_view (Andrew Toth)
73e99a59665551243d6dbe03a0e9baa9cab046b9 coins: don't mutate main cache when connecting block (Andrew Toth)
67c0d1798e6147f48d4bafc2c9e5ff30f2a62340 coins: introduce CoinsViewOverlay (Andrew Toth)
69b01af0eb9017a6ae7ca3134c9dcf89e74dbfa8 coins: add PeekCoin() (Andrew Toth)
Pull request description:
This is a slightly modified version of the first few commits of #31132, which can be merged as an independent change. It has a small benefit on its own, but will help in moving the parent PR forward.
When accessing coins via the `CCoinsViewCache`, methods like `GetCoin` can call `FetchCoin` which actually mutate `cacheCoins` internally to cache entries when they are pulled from the backing db. This is generally a performance improvement for single threaded access patterns, but it precludes us from accessing entries in a `CCoinsViewCache` from multiple threads without a lock.
Another aspect is that when we use the resettable `CCoinsViewCache` view backed by the main cache for use in `ConnectBlock()`, we will insert entries into the main cache even if the block is determined to be invalid. This is not the biggest concern, since an invalid block requires proof-of-work. But, an attacker could craft multiple invalid blocks to fill the main cache. This would make us `Flush` the cache more often than necessary. Obviously this would be very expensive to do on mainnet.
Introduce `CoinsViewOverlay`, a `CCoinsViewCache` subclass that reads coins without mutating the underlying cache via `FetchCoin()`.
Add `PeekCoin()` to look up a Coin through a stack of `CCoinsViewCache` layers without populating parent caches. This prevents the main cache from caching inputs pulled from disk for a block that has not yet been fully validated. Once `Flush()` is called on the view, these inputs will be added as spent to `coinsCache` in the main cache via `BatchWrite()`.
This is the foundation for async input fetching, where worker threads must not mutate shared state.
ACKs for top commit:
l0rinc:
ACK cae6d895f8a8cf5f57e05519536fda5d62b10841
sipa:
reACK cae6d895f8a8cf5f57e05519536fda5d62b10841
sedited:
Re-ACK cae6d895f8a8cf5f57e05519536fda5d62b10841
willcl-ark:
ACK cae6d895f8a8cf5f57e05519536fda5d62b10841
vasild:
Cursory ACK cae6d895f8a8cf5f57e05519536fda5d62b10841
ryanofsky:
Code review ACK cae6d895f8a8cf5f57e05519536fda5d62b10841. PR is basically back to the form I had acked the first time, implementing `PeekCoin()` by calling `GetCoin()`. This is not ideal because `PeekCoin()` is not supposed to modify caches and `GetCoin()` does that, but it at least avoids problems of the subsequent approach tried where `GetCoin()` calls `PeekCoin` and would result in bugs when subclasses implement `GetCoin` forgetting to override `PeekCoin`. Hopefully #34124 can clean all of this by making relevant methods pure virtual.
Tree-SHA512: a81a98e60ca9e47454933ad879840cc226cb3b841bc36a4b746c34b350e07c546cdb5ddc55ec1ff66cf65d1ec503d22201d3dc12d4e82a8f4d386ccc52ba6441
