b19caeea098f92a7f72aaeee49573358f4b153a3 doc: add release note for #31560 (named pipe support for `dumptxoutset` RPC) (Sebastian Falbesoner)
61a5460d0d6cd174d395c51333def798fe7442fe test: add test for utxo-to-sqlite conversion using named pipe (Sebastian Falbesoner)
2e8072edbeb20a8c05c0dbd06ca105bc4dd07b96 rpc: support writing UTXO set dump (`dumptxoutset`) to a named pipe (Sebastian Falbesoner)
Pull request description:
This PR slightly modifies the `dumptxoutset` RPC to allow writing the UTXO set dump into a [named pipe](https://askubuntu.com/a/449192), so that the output data can be consumed by another process, see #31373. Taking use of this with the utxo-to-sqlite.py tool (introduced in #27432), creating an UTXO set in SQLite3 format is possible on the fly. E.g. for signet:
```
$ mkfifo /tmp/utxo_fifo && ./build/bin/bitcoin-cli -signet dumptxoutset /tmp/utxo_fifo latest &
$ ./contrib/utxo-tools/utxo_to_sqlite.py /tmp/utxo_fifo ./utxo.sqlite
UTXO Snapshot for Signet at block hash 000000012711f0a4e741be4a22792982..., contains 61848352 coins
1048576 coins converted [1.70%], 2.800s passed since start
....
....
60817408 coins converted [98.33%], 159.598s passed since start
{
"coins_written": 61848352,
"base_hash": "000000012711f0a4e741be4a22792982370f51326db20fca955c7d45da97f768",
"base_height": 294305,
"path": "/tmp/utxo_fifo",
"txoutset_hash": "34ae7fe7af33f58d4b83e00ecfc3b9605d927f154e7a94401226922f8e3f534e",
"nchaintx": 28760852
}
TOTAL: 61848352 coins written to ./utxo.sqlite, snapshot height is 294305.
```
Note that the `dumptxoutset` RPC calculates an UTXO set hash as a first step before any data is emitted, so especially on mainnet it takes quite a while until the conversion starts and something is happening visibly.
ACKs for top commit:
ajtowns:
utACK b19caeea098f92a7f72aaeee49573358f4b153a3
sedited:
Re-ACK b19caeea098f92a7f72aaeee49573358f4b153a3
Tree-SHA512: 7101563d0dba15439cdef8c8fb535f8593d5a779ff04208e2d72382a3f99072db8eac3651d1b3fe72c5e1f03e164efb281c3030d45d0723b943ebbbcf2a841d6
22335474d768f99067856173ff2764b6db753f67 net: format peer+addr logs with `LogPeer` (Lőrinc)
e55ea534f74b2dd2a81e8f9972554303f7cf9c20 test: add pre-`LogPeer` net log assertion (Lőrinc)
736b17c0f0f23dca78e7266be6c98b24b0eda210 log: fix minor formatting in debug logs (Lőrinc)
9cf82bed3205d8405f56838d0e2ca0c72023a081 log: show placeholders for missing peer fields (Lőrinc)
Pull request description:
This is an alternative to #34293, but aims to address the remaining logging inconsistencies more broadly.
It extends the example fixed there to every instance, restores the original separator behavior, applies it consistently via a single helper, and adds tests for `logips` (covering both current and new behavior).
### Problem
After #28521 centralized peer address logging into `CNode::LogIP()`, the original comma separator before `peeraddr=` was lost, resulting in inconsistent formatting across net (and recent private broadcast) logs.
Some lines also had double spaces, empty fields, or mismatched format specifiers.
### Fix
Introduces `CNode::LogPeer(bool)` which always emits `peer=<id>` and, when `-logips=1`, appends `, peeraddr=<addr>`. This eliminates hand-rolled separators and makes peer identification predictable.
Minor issues (double spaces, empty placeholders, format specifiers) are fixed along the way in separate commits.
### Reproducer
Run with `-debug=net -logips=1` and observe peer log lines now show `peer=<id>, peeraddr=<addr>` (comma-separated). The new assertion in `feature_logging.py` automates this check.
ACKs for top commit:
naiyoma:
ACK 22335474d768f99067856173ff2764b6db753f67
vasild:
ACK 22335474d768f99067856173ff2764b6db753f67
sedited:
ACK 22335474d768f99067856173ff2764b6db753f67
Tree-SHA512: 562262a58c3042f139099ff4c41e3fc6a97505fe9603c2bf700a97fd0aa052954b47c14da0e50c1fc311db1ae6c04e6a92156c9b85e25c777a637b7766c1dafe
73e385311055eb6c0e3ddb2221f927a3872af1fb test: add test for rebroadcast of transaction received via p2p (Martin Zumsande)
Pull request description:
The wallet doesn't only rebroadcast transactions it created, but also relevant transactions received via p2p.
Since this is not self-evident, add test coverage for it.
ACKs for top commit:
maflcko:
re-ACK 73e385311055eb6c0e3ddb2221f927a3872af1fb 🦌
vasild:
ACK 73e385311055eb6c0e3ddb2221f927a3872af1fb
furszy:
ACK 73e385311055eb6c0e3ddb2221f927a3872af1fb
danielabrozzoni:
tACK 73e385311055eb6c0e3ddb2221f927a3872af1fb
Tree-SHA512: b5249bb5eb6a124a98030816319aa4364d7aee9c28ee28750ebba5fc8d6bfc9d7960a97ed638611f4560e051760ec4a7a75d302a4cb106dbfadfe11adc604f22
fa0587a306430b35d2852ee32f7eee2adb8c1e64 test: Remove fixed TODO in address_to_scriptpubkey (MarcoFalke)
Pull request description:
After commit d178082996dc3000f42816f89afcf3fa4d31e159 added the bech32 format, the TODO about adding other formats can be removed.
ACKs for top commit:
sedited:
ACK fa0587a306430b35d2852ee32f7eee2adb8c1e64
rkrux:
crACK fa0587a306
musaHaruna:
ACK [fa0587a](fa0587a306)
Tree-SHA512: 49a8a514bed11e945bd8f9b13b84ae14b4dbc8a7ebb7224b1746776d9dbf68abc3b53d67f1b7fff83bc4360b15324fb96611550f4aca808b16beb03bcbfd0a55
5b2c3960b929bd015911783fb59e3660b0810f03 test: clean up tx resurrection (re-org) test in feature_block.py (Sebastian Falbesoner)
Pull request description:
The following comment about ECDSA signatures created with the test framework not passing mempool policy has been obsolete for a long time (at least since 2019, see PR #15826):
8c07800b19/test/functional/feature_block.py (L1167-L1172)
so remove it. While at it, change the resurrected txs to be indeed standard valid, so the `acceptnonstdtxn=1` parameter can also be removed from the functional test.
Kudos to stratospher, who (IIRC) mentioned this outdated comment a while ago.
ACKs for top commit:
sedited:
ACK 5b2c3960b929bd015911783fb59e3660b0810f03
Bortlesboat:
Tested ACK 5b2c3960b929. Ran feature_block.py locally, passes. Nice cleanup — removing `-acceptnonstdtxn=1` by making the resurrection txs standard (P2PK) is the right approach. The old unsigned OP_TRUE workaround comment explains why this was needed, glad to see it gone.
stratospher:
ACK 5b2c3960. nice!
Tree-SHA512: 94517e432647a8e8db51fad90a26493c57ce9dfd924b17ce909ea40d50bc6279d974c6d2046b34f5f92b481ca5e0abdedaf412ce41b4cb904605c140a8dba583
20ae9b98eab20117344cf31f7cde39cadd70ca22 Extend functional test for setBlockIndexCandidates UB (marcofleon)
854a6d5a9a0e40329a2852efb2a8dfec4b54886e validation: fix UB in LoadChainTip (marcofleon)
9249e6089ec4e2eb63f0896961f04d9dbe14651a validation: remove LoadChainTip call from ActivateSnapshot (marcofleon)
Pull request description:
Addresses https://github.com/bitcoin/bitcoin/issues/34503. See this issue for more details as well.
Fixes a bug where, under certain conditions, `setBlockIndexCandidates` had blocks in it that were worse than the tip. The block index candidate set uses `nSequenceId` as a sort key, so modifying this field while blocks are in the set results in undefined behavior. This PR populates `setBlockIndexCandidates` after the `nSequenceId` modifications, avoiding the UB.
ACKs for top commit:
achow101:
ACK 20ae9b98eab20117344cf31f7cde39cadd70ca22
sedited:
Re-ACK 20ae9b98eab20117344cf31f7cde39cadd70ca22
sipa:
Code review ACK 20ae9b98eab20117344cf31f7cde39cadd70ca22
Tree-SHA512: 121c170bb70fb6365089d578db63c811e7926e129d7206e569947f7a1f6c5ddc8d5f4937b80f1ba1b7d7daa42789b143ca5b56f154b7ab968a1cd55f925f378d
cbdb891de23d98dae3cfb3c4d993d6111df0aaa4 test: Wait for txospender index to be synced in rpc_gettxspendingprevout (Ava Chow)
2db5c049bb3cf9a323c100ce9d4957a937c01624 test: Sync mempools after tx creation in rpc_gettxspendingprevout (Ava Chow)
Pull request description:
On slower runs, the txospender index may not be synced yet when the tests of its behavior begin, causing intermittent failures. Wait for them to be synced before starting the tests.
The tests also query mempool txs from other nodes, make sure mempools are synced before doing so.
The first commit with the following diff reproduces #34735:
```
diff --git a/src/index/txospenderindex.cpp b/src/index/txospenderindex.cpp
index d451bb1e0a4..e786f05a98c 100644
--- a/src/index/txospenderindex.cpp
+++ b/src/index/txospenderindex.cpp
@@ -129,6 +129,7 @@ static std::vector<std::pair<COutPoint, CDiskTxPos>> BuildSpenderPositions(const
bool TxoSpenderIndex::CustomAppend(const interfaces::BlockInfo& block)
{
+ UninterruptibleSleep(100ms);
WriteSpenderInfos(BuildSpenderPositions(block));
return true;
}
```
Fixes#34735
ACKs for top commit:
furszy:
ACK cbdb891de23d98dae3cfb3c4d993d6111df0aaa4
andrewtoth:
ACK cbdb891de23d98dae3cfb3c4d993d6111df0aaa4
sedited:
ACK cbdb891de23d98dae3cfb3c4d993d6111df0aaa4
rkrux:
crACK cbdb891de23d98dae3cfb3c4d993d6111df0aaa4
Tree-SHA512: ba1ab6216ac3b647a5f9e20f4899e51e643bf20829e698aa63c127c88ed4e33afa222bebeae1210fc869c46288e99c0b344330197913e244ffe1a6aca943568d
15c4889497b96037e41019a8f43090af841b36ec index: document TxoSpenderIndex::FindSpender (furszy)
f8b9595aaa966c373b02e6227dc799fed6d038ba test: Add missing txospenderindex coverage in feature_init (Fabian Jahr)
a1074d852a7a46b746fb4ed90d94cb4cc346f9b3 index, rpc, test: Misc formatting fixes (Fabian Jahr)
Pull request description:
This addresses my own comments in the last review of #24539: https://github.com/bitcoin/bitcoin/pull/24539#pullrequestreview-3829110465
The first commit fixes three small formatting errors.
The second commit adds some missing coverage in `feature_init` and refactors the code a bit as well so these misses don't happen so easily in the future.
The third commit is by furzy:
> TxoSpenderIndex::FindSpender returns an Expected<optional<TxoSpender>> but
the two levels of the return type were undocumented, making it unclear what a returned
nullopt means. So added doc clarifying each return case.
ACKs for top commit:
furszy:
ACK 15c4889497b96037e41019a8f43090af841b36ec
sedited:
ACK 15c4889497b96037e41019a8f43090af841b36ec
rkrux:
crACK 15c4889497b96037e41019a8f43090af841b36ec
Tree-SHA512: 2e0f060a54b558d2967ebae0835cf81bd86c2d8d983d670a48d1bee7d347f186623e75db7ae311ca1566807f715c1b3fa67cf734c9467d35e13b84d082f28253
5c005363a880c136cc44ff2456a402e398fcbf44 test: improve `wallet_backup` test (rkrux)
04d95157485e31b6a8a0f2eb2d7a65023b4199ac test: improve `wallet_assumeutxo` func test (rkrux)
Pull request description:
Relates to #34354
While the actual fix of the issue is in another PR, this one improves the
affected tests by trying to reduce the chain notifications that
need to be processed while simulating erroneous wallet restoration
scenarios.
ACKs for top commit:
maflcko:
lgtm ACK 5c005363a880c136cc44ff2456a402e398fcbf44
furszy:
ACK 5c005363a880c136cc44ff2456a402e398fcbf44
w0xlt:
ACK 5c005363a880c136cc44ff2456a402e398fcbf44
brunoerg:
code review ACK 5c005363a880c136cc44ff2456a402e398fcbf44
Tree-SHA512: 176e3ea8275c7aa082af695f5b76d82c079ff9a7178855b4ce95504edb8ce89b59a772e2d38dd43e997a5bea3d64be700b74cfec7bbc6992538f837877ab7222
Fix the from-disk subtest to use a separate node so it builds on a
clean genesis block, rather than the leftover chain from the
in-memory subtest.
Change from a two-way to a three-way block race. The UB in the old
LoadChainTip (mutating nSequenceId, a sort key, while the block is
in setBlockIndexCandidates) corrupts the internal tree structure,
resulting in a failed erase that leaves stale blocks in the set
alongside the tip. With only two competing blocks, this is caught
by libstdc++ but not by libc++. A three-way split triggers the bug
on both implementations.
To trigger CheckBlockIndex (where the crashing assertion is), replace
the restart loop with sending a new block after a single restart.
faa68ed4bdce6540f99d23c9b200ca575794a1b2 test: Fix intermittent issue in wallet_assumeutxo.py (MarcoFalke)
Pull request description:
The test has many issues:
* It fails intermittently, due to the use of `-stopatheight` (https://github.com/bitcoin/bitcoin/issues/34710)
* Using `-stopatheight` is expensive, because it requires two restarts, making the test slow
* The overwritten `def setup_network` does not store the extra args via the `add_nodes` call, so if code is added in the future to restart a node, it may not pick up its global extra args.
Fix all issues by:
* Adding and using a fast `dumb_sync_blocks` util helper to achieve what `-stopatheight` doesn't achieve
* Calling `self.add_nodes(self.num_nodes, self.extra_args)` in the overridden `setup_network`
Can be tested via this diff:
```diff
diff --git a/src/node/kernel_notifications.cpp b/src/node/kernel_notifications.cpp
index ab0e5ccb69..49384c10b8 100644
--- a/src/node/kernel_notifications.cpp
+++ b/src/node/kernel_notifications.cpp
@@ -61,2 +61,3 @@ kernel::InterruptResult KernelNotifications::blockTip(SynchronizationState state
if (m_stop_at_height && index.nHeight >= m_stop_at_height) {
+ LogInfo("Send shutdown signal after reaching stop height");
if (!m_shutdown_request()) {
diff --git a/src/util/tokenpipe.cpp b/src/util/tokenpipe.cpp
index c982fa6fc4..a5565ebd36 100644
--- a/src/util/tokenpipe.cpp
+++ b/src/util/tokenpipe.cpp
@@ -4,2 +4,3 @@
#include <util/tokenpipe.h>
+#include <util/time.h>
@@ -60,2 +61,3 @@ int TokenPipeEnd::TokenRead()
ssize_t result = read(m_fd, &token, 1);
+ UninterruptibleSleep(500ms);
if (result < 0) {
```
On master: The test fails
On this pull: The test passes
Fixes https://github.com/bitcoin/bitcoin/issues/34710
ACKs for top commit:
kevkevinpal:
ACK [faa68ed](faa68ed4bd)
achow101:
ACK faa68ed4bdce6540f99d23c9b200ca575794a1b2
w0xlt:
ACK faa68ed4bdce6540f99d23c9b200ca575794a1b2
Tree-SHA512: 6fcd52b6f6a5fa5a5e41e7b3cf5c733af62af9c60271e7d22c266aca90f2af67f91ffe80a3ed8b8d1a91d001700870f493207998bac988c4e3671a3a0dba7ba7
f51665bee72c26d3f3cc6813b6c02adad5f0af6a psbt: validate pubkeys in MuSig2 pubnonce/partial sig deserialization (tboy1337)
Pull request description:
The previous fix for invalid MuSig2 pubkeys (bitcoin/bitcoin#34010) only
addressed the PSBT_IN_MUSIG2_PARTICIPANT_PUBKEYS field. However, the
PSBT_IN_MUSIG2_PUB_NONCE and PSBT_IN_MUSIG2_PARTIAL_SIG fields also
deserialize pubkeys without validation, which could lead to crashes when
invalid pubkeys are processed.
This commit adds validation to the DeserializeMuSig2ParticipantDataIdentifier
function to ensure all pubkeys in MuSig2 pubnonce and partial signature
fields are fully valid elliptic curve points.
The fix:
- Validates both aggregate and participant pubkeys in MuSig2 pubnonce and
partial signature deserialization
- Throws std::ios_base::failure with descriptive error messages for invalid
pubkeys
- Prevents potential crashes from invalid elliptic curve points
- Maintains backward compatibility for valid PSBTs
This completes the fix for issues [#33999](https://github.com/bitcoin/bitcoin/issues/33999) and [#34201](https://github.com/bitcoin/bitcoin/issues/34201).
ACKs for top commit:
rkrux:
lgtm ACK f51665bee72c26d3f3cc6813b6c02adad5f0af6a
w0xlt:
ACK f51665bee7
darosior:
utACK f51665bee72c26d3f3cc6813b6c02adad5f0af6a
Tree-SHA512: 8454d77a05aa003a3121b0a5975e8a000125ee0d62343bfa625a75db113358ba7a210ae0376ca1666957b7de7005e06e5a54c95170430ee5e9e1416719b8af53
1c1de334e9c0e3b4c72c3f3fe45b1f4df5ce4502 test: avoid interface_ipc.py race and null pointer dereference (Ryan Ofsky)
Pull request description:
Avoid race condition in `run_deprecated_mining_test` caused by creating and immediately destroying an unused worker thread. This is leading to test failures reported by maflcko in #34711
ACKs for top commit:
optout21:
utACK 1c1de334e9c0e3b4c72c3f3fe45b1f4df5ce4502
l0rinc:
Tested ACK 1c1de334e9c0e3b4c72c3f3fe45b1f4df5ce4502
w0xlt:
ACK 1c1de334e9c0e3b4c72c3f3fe45b1f4df5ce4502
enirox001:
ACK 1c1de33
Tree-SHA512: d0af9676a46e991a3f4fda3795c02d1998d30de24991436b8ada425585c6699ff32a7057ca7a0ef2925f782fd3bf73e0381f5d4325e4f1c09f487fce1de49e45
Avoid race condition in run_deprecated_mining_test caused by creating and
immediately destroying an unused worker thread. This leads to test failures
reported by maflcko in #34711
da7f70a5322843b70f29456a8bc2227209a0718b test: use port 0 for I2P addresses in p2p_private_broadcast.py (Vasil Dimov)
a8ebcfd34c63f142064b4f5ef7d52299739d4cd6 test: let connections happen in any order in p2p_private_broadcast.py (Vasil Dimov)
67696b207f370e902c8d5fb765e4ff10f6c9e1b4 net: extend log message to include attempted connection type (Vasil Dimov)
Pull request description:
If the following two events happen:
* (likely) the automatic 10 initial connections are not made to all
networks
* (unlikely) the network-specific logic kicks in almost immediately.
It is using exponential distribution with a mean of 5 minutes
(`rng.rand_exp_duration(EXTRA_NETWORK_PEER_INTERVAL)`).
So if both happen, then the 11th connection may not be the expected
private broadcast, but a network-specific connection.
Fix this by retrieving the connection type from
`destinations_factory()`. This is more flexible because it allows
connections to happen in any order and does not break if e.g. the 11th
connection is not the expected first private broadcast.
This also makes the test run faster:
before: 19-44 sec
now: 10-25 sec
because for example there is no need to wait for the initial 10
automatic outbound connections to be made in order to proceed.
Fixes: https://github.com/bitcoin/bitcoin/issues/34387
ACKs for top commit:
achow101:
ACK da7f70a5322843b70f29456a8bc2227209a0718b
andrewtoth:
ACK da7f70a5322843b70f29456a8bc2227209a0718b
mzumsande:
Code Review ACK da7f70a5322843b70f29456a8bc2227209a0718b
Tree-SHA512: 7c293e59c15c148a438e0119343b05eb278205640658c99336d4caf4848c5bae92b48e15f325fa616cbc9d5f394649abfa02406a76e802cffbd3d312a22a6885
Reduce the number of blocks that need to be generated before pruning
the blockchain.
Unload the wallet that was restored in a prior test because it is not
needed anymore after the test.
Both the above steps should reduce the number of chain notifications
that need to be processed by the wallet(s) when an erroneous scenario
of restoring wallet is checked.
fa48f8c8655d93e78b32b560a870577909b666d3 test: Add missing timeout_factor to zmq socket (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/34189
Otherwise, the test may intermittently fail on slow CI systems that have `--timeout-factor=` properly set.
It can be tested by running `./bld-cmake/test/functional/interface_zmq.py --timeout-factor=10` with this diff:
```diff
diff --git a/src/validationinterface.cpp b/src/validationinterface.cpp
index c7be6abc3a..b14cf2aee6 100644
--- a/src/validationinterface.cpp
+++ b/src/validationinterface.cpp
@@ -166,2 +166,3 @@ void ValidationSignals::SyncWithValidationInterfaceQueue()
LOG_EVENT(fmt, local_name, __VA_ARGS__); \
+ UninterruptibleSleep(45ms); \
event(); \
diff --git a/test/functional/interface_zmq.py b/test/functional/interface_zmq.py
index 6717007626..eee377daea 100755
--- a/test/functional/interface_zmq.py
+++ b/test/functional/interface_zmq.py
@@ -176,3 +176,3 @@ class ZMQTest (BitcoinTestFramework):
for sub in subscribers:
- sub.socket.set(zmq.RCVTIMEO, recv_timeout*1000)
+ sub.socket.set(zmq.RCVTIMEO, int(recv_timeout * 1000))
@@ -271,3 +271,3 @@ class ZMQTest (BitcoinTestFramework):
[(topic, address) for topic in ["hashblock", "hashtx"]],
- recv_timeout=2) # 2 second timeout to check end of notifications
+ recv_timeout=0.2) # 2 second timeout to check end of notifications
self.disconnect_nodes(0, 1)
```
Before this pull: Test fails with `zmq.error.Again: Resource temporarily unavailable`
After this pull: Test passes
ACKs for top commit:
l0rinc:
code review ACK fa48f8c8655d93e78b32b560a870577909b666d3
achow101:
ACK fa48f8c8655d93e78b32b560a870577909b666d3
Tree-SHA512: 5ff8bdd807ff4b644daa634bb7b469da3da3915f58afba63a90e662df99cbebc86636e34e2b1b313c8629773aef2a239fb3025226a84d2ec22f6ecd4cea666c4
bbc8f1e0a7e5739f15b2e646a4ace338083309a3 ipc mining: Prevent ``Assertion `m_node.chainman' failed`` errors on early startup (Ryan Ofsky)
a7cabf92e4de83c87f6b9274ddd2fb70712d29f8 init refactor: Only initialize node.notifications one time (Ryan Ofsky)
c8e332cb33594cc307cdf89b6036a0e95c238cd8 init refactor: Remove node.init accesss in AppInitInterfaces (Ryan Ofsky)
Pull request description:
This fixes ``Assertion `m_node.chainman' failed`` errors first reported https://github.com/bitcoin/bitcoin/issues/33994#issuecomment-3602551596 when IPC mining methods are called before ChainstateManager is loaded.
The fix works by making the `Init.makeMining` method wait until chainstate data is loaded. It's probably the simplest possible fix but other alternatives like moving the wait to `Mining.createNewBlock` were discussed in the thread https://github.com/bitcoin/bitcoin/pull/34661#discussion_r2848176298 and could be implemented later without changes to clients.
ACKs for top commit:
Sjors:
utACK bbc8f1e0a7e5739f15b2e646a4ace338083309a3
ismaelsadeeq:
ACK bbc8f1e0a7e5739f15b2e646a4ace338083309a3
achow101:
ACK bbc8f1e0a7e5739f15b2e646a4ace338083309a3
Tree-SHA512: 3e2e4e28ccff364b2303efd06ce337a229c28609076638500acb29559f716a15ad99409c6970ce9ad91776d53e3f9d959f1bbbd144ea9a4a2fb578ddbf2da267
The wallet doesn't only rebroadcast transactions it created, but
also relevant transactions received via p2p. Since this is not
self-evident, add test coverage for it.
Normally, when a symlinked test script is executed directly (e.g.,
`./bld-cmake/test/functional/wallet_disable.py`), Python's default
behavior is to resolve the symlink of the script itself, setting
`sys.path[0]` to the directory containing the physical source file.
Consequently, the test framework util.py is imported from the source
tree, and `Path(__file__).parents[3]` correctly resolves to the source
root.
However, `feature_framework_testshell.py` is unique because it manually
inserts `Path(__file__).parent` into `sys.path`. That refers to the
build tree and when importing the test framework util.py, the
`Path(__file__).parents[3]` will incorrectly point to the build
directory instead of the source root.
Use `.resolve()` to ensure the Valgrind suppressions file path is always
calculated relative to the physical source file, regardless of how the
framework was imported.
8834e4e86c851c4eb01a7e978cdbde5b3cbd24cc test: remove appveyor reference in comment (Max Edwards)
Pull request description:
Appveyor is not longer used however the test still requires to check for permissions including 666 as otherwise the tests fail on Windows
Fixes: #32576
ACKs for top commit:
maflcko:
lgtm ACK 8834e4e86c851c4eb01a7e978cdbde5b3cbd24cc
hebasto:
ACK 8834e4e86c851c4eb01a7e978cdbde5b3cbd24cc.
Tree-SHA512: 655b44e52da5e0c6c11c79bb4f92c701c6e0e66dce8d7791ccf1d64e4561fe4d1d5f37c1317bead89c88e4d7208a278925168b419482a6be17abf93d0ebc5dfa
408d5b12e80151feded31c2a5509e2dc5f15edf3 test: include response body in non-JSON HTTP error msg (Matthew Zipkin)
9dc653b3b4f3049b0e742499b762f7c13bb006cc test: threadpool, add coverage for all Submit() errors (furszy)
ce2a984ee324d37ba1dd7c2c4e27e40e0508bedc test: cleanup, use HasReason in threadpool_tests.cpp (l0rinc)
d9c6769d0324b65121935b7c8a285c6421fe74a6 test: refactor, decouple HasReason from test framework machinery (furszy)
dbbb780af02d850a1f9257f18610cfb9de9cb828 test: move and simplify BOOST_CHECK ostream helpers (Hodlinator)
3b7cbcafcb9b318bf1fa00a3499f514c5ebe9bb6 test: ensure Stop() thread helps drain the queue (seduless)
ca101a2315774f0ed65da633ba99899fd0dad740 test: coverage for queued tasks completion after interrupt (furszy)
bf2c607aaa22d253b9367c11b0a198bd4244ad2f threadpool: active-wait during shutdown (furszy)
e88d2744301a434064714f0a21e1395d41ac3984 test: add threadpool Start-Stop race coverage (furszy)
8cd4a4363fb85f5487a19ace82aa0d12d5fab450 threadpool: guard against Start-Stop race (furszy)
9ff1e82e7dbdf31ddf1c534853da4581a1f41bd5 test: cleanup, block threads via semaphore instead of shared_future (l0rinc)
Pull request description:
A few follow-ups to #33689, includes:
1) `ThreadPool` active-wait during shutdown:
Instead of just waiting for workers to finish processing tasks, `Stop()` now helps them actively.
This speeds up the JSON-RPC and REST server shutdown, resulting in a faster node shutdown when many requests remain unhandled. This wasn't included in the original PR due to the behavior change this introduces.
2) Decouple `HasReason` from the unit test framework machinery
This avoids providing the entire unit test framework dependency to low-level tests that only require access to the `HasReason` utility class. Examples are: `reverselock_tests.cpp`, `sync_tests.cpp`, `util_check_tests.cpp`, `util_string_tests.cpp`, `script_parse_tests.cpp` and `threadpool_tests.cpp`. These tests no longer gain access to unnecessary components like the chainstate, node context, caches, etc. It includes l0rinc's `threadpool_tests.cpp` `HasReason` changes.
3) Include response body in non-JSON HTTP error messages
Straight from pinheadmz [comment](https://github.com/bitcoin/bitcoin/pull/33689#discussion_r2783817192), it makes debugging CI issues easier.
ACKs for top commit:
maflcko:
review ACK 408d5b12e80151feded31c2a5509e2dc5f15edf3 🕗
achow101:
ACK 408d5b12e80151feded31c2a5509e2dc5f15edf3
hodlinator:
re-ACK 408d5b12e80151feded31c2a5509e2dc5f15edf3
Tree-SHA512: 57aa0ef96886f32bf95a0bd7f87c878d31c9df9e34cb96de615eee703ce0824b5cfdf8f5c9cd19a3594559994295b5810c38c94f5efd6291cbbd83a95473357a
0a8d303d667cc10a68fd15d799039b9ae26c3315 test: fix test_limit_enforcement_package (Greg Sanders)
Pull request description:
The current test has a couple issues:
1) the parent_tx_good is regenerating the exact same transaction that is already in the cluster, so it's resulting in no replacements on submission
2) once fixed, the additional fee needs to be allocated to the parent transaction in the package, not the child. If the RBF fees are allocated to the child, this triggers the package RBF logic, which requires no in-mempool ancestors to be present.
Fix the bug and add a few assertions to protect against regressions.
ACKs for top commit:
bensig:
ACK 0a8d303d667cc10a68fd15d799039b9ae26c3315
achow101:
ACK 0a8d303d667cc10a68fd15d799039b9ae26c3315
sipa:
ACK 0a8d303d667cc10a68fd15d799039b9ae26c3315
Tree-SHA512: 0ba184d82edc5a502e9119a6876e80c4564c876fa51ee39293d47bd30c18bf3ded50fbd2f6f2a3394784fad05d8f6370a90682068b30358b077280abd2477252
e5f0613503b6973dbc886eba8e999f208d84853b net processing: Check if we are in ibd before processing block for txdownloadman (sedited)
ce8b692897f6aacbe936fe2220e85f23cd83cbf2 Add functional test exercising tx downloadman recently confirmed filter (Lőrinc)
Pull request description:
Calculating the rolling bloom filters for the txorphanage takes some CPU time from the scheduler thread. This can be observed for example in [this flamegraph](https://bitcoin-dev-tools.github.io/benchcoin/results/pr-172/20066462508/mainnet-default-instrumented-base-flamegraph.svg?x=920203898521&y=780), where handling the filter takes about 2.6% of total time (and most of the scheduler thread's time).
During ibd the entries in the tx download bloom filter are just continuously rolled over and aren't consumed, since no mempool entries are created by incoming transactions from peers during ibd. The mempool does accept transactions via RPC, or the wallet at the time, however these don't interact with the orphanage and the txdownloadman, because adding anything to those is guarded by IsInitialBlockDownload() checks as well.
We're usually latching ibd to false a few blocks before catching up to the tip, so this should also not significantly degrade the performance of the filter once fully caught up.
ACKs for top commit:
l0rinc:
ACK e5f0613503b6973dbc886eba8e999f208d84853b
instagibbs:
ACK e5f0613503b6973dbc886eba8e999f208d84853b
fjahr:
Code review ACK e5f0613503b6973dbc886eba8e999f208d84853b
Tree-SHA512: d667e677f5723c438cdf5b34f0f9c1ade7cc1b2e98530c23f14384514daa38217c4e7c3b756194b6831b590a487449c4514b52bf0fb461ae8083061722824270
33fbaed310a6a37d41d26af8fb34308d088d72c8 policy: don't CheckEphemeralSpends on reorg (Greg Sanders)
Pull request description:
Similar reasoning to https://github.com/bitcoin/bitcoin/pull/33504
During a deeper reorg it's possible that a long sequence of dust-having transactions that are connected in a linear fashion. On reorg, this could cause each subsequent "generation" to be rejected. These rejected transactions may contain a large amount of competitive fees via normal means.
PreCheck based `PreCheckEphemeralSpends` is left in place because we wouldn't have relayed them prior to the reorg.
ACKs for top commit:
darosior:
re-ACK 33fbaed310a
ismaelsadeeq:
reACK 33fbaed310a6a37d41d26af8fb34308d088d72c8
sedited:
ACK 33fbaed310a6a37d41d26af8fb34308d088d72c8
Tree-SHA512: cf0a6945066e9f5f8f9a847394c2c1225facf475a8aa4bc811b436513eff79c0a720d4ad21ba6b0f1cc4dfdd61cf46acb148333ac592b2ee252953732326ad1d
5cd57943b8adc76ed0b8a75a83f27bc0f971cbef test: verify node state after restart in assumeutxo (Yash Bhutwala)
Pull request description:
## Summary
This PR replaces the TODO comment in `wallet_assumeutxo.py` (line 242) with actual test assertions that verify node and wallet behavior after a restart during assumeutxo background sync.
## Changes
The new tests verify:
- Two chainstates exist (background validation not complete)
- Background chainstate is still at `START_HEIGHT`
- Snapshot chainstate has synced to at least `PAUSE_HEIGHT`
- Wallets cannot be loaded after restart (expected behavior)
- Wallet backup from before snapshot height cannot be restored
## Motivation
During implementation, I discovered that **wallets cannot be loaded after a node restart during assumeutxo background sync**. This is expected behavior because:
- The wallet loading code checks if required blocks are available for rescanning
- During assumeutxo background sync, blocks before the snapshot are not available
- This applies to all wallets, including watch-only wallets created at the snapshot height
This is a valuable test addition because it documents this expected behavior and ensures it doesn't regress. Users should be aware that if they restart their node during assumeutxo background sync, they won't be able to load their wallets until the background sync completes.
## Related
refs #28648
Addresses the TODO comment that was originally added as part of the assumeutxo wallet test implementation.
ACKs for top commit:
Bicaru20:
Code review ACK 5cd57943b8adc76ed0b8a75a83f27bc0f971cbef
achow101:
ACK 5cd57943b8adc76ed0b8a75a83f27bc0f971cbef
fjahr:
ACK 5cd57943b8adc76ed0b8a75a83f27bc0f971cbef
sedited:
ACK 5cd57943b8adc76ed0b8a75a83f27bc0f971cbef
polespinasa:
code lgtm ACK 5cd57943b8adc76ed0b8a75a83f27bc0f971cbef
Tree-SHA512: 4a125c5247168da2bbf4d855b4150ca453bb5e4cce1a62e633ce5e43acdc2c58883a6a94dcc46b38f8b4c44206fe42cec4db151a76aded53d8ea433ea5eb2562
e8f8b74a46aa075bf6c74c104fd572cc89d3b53b test: index, improve txospenderindex_initial_sync() test code (furszy)
ac3bea07cdceac9e316448a9a5f190848156efd5 test: improve rpc_gettxspendingprevout.py code (furszy)
Pull request description:
Fixes#34637.
Was reviewing #34637 and, while reading the new txospender index
test code for the first time, found it could use some cleanups. Finding
stuff in there is harder than it should be due to the amount of dup code.
The first commit cleans up `rpc_gettxspendingprevout.py` by introducing
helper functions to avoid repeating the same dicts everywhere, using
for-loops instead of duplicating the same checks for each node, and
renaming variables to better reflect what they actually represent.
The second commit reorganizes `txospenderindex_initial_sync()`
moving index initialization after the test setup phase, since the index
doesn't participate in it anyway. It adds a post-sync check to catch
cases where `Sync()` aborted prematurely.
Note:
This is just a pre-work for deeper index changes I'm cooking.
ACKs for top commit:
achow101:
ACK e8f8b74a46aa075bf6c74c104fd572cc89d3b53b
sedited:
Re-ACK e8f8b74a46aa075bf6c74c104fd572cc89d3b53b
w0xlt:
reACK e8f8b74a46aa075bf6c74c104fd572cc89d3b53b
Tree-SHA512: 3f7026712ab20a43f376afa28c683dcd5daec8ed1bbf1c36d7ec6bbf231f468d4de74efae4aa8295ff3afb83986286ccaf31c03b34e45fc9971652f064791ed0
This fixes ``Assertion `m_node.chainman' failed`` errors first reported
https://github.com/bitcoin/bitcoin/issues/33994#issuecomment-3602551596 when
IPC mining methods are called before ChainstateManager is loaded.
The fix works by making the `Init.makeMining` method block until chainstate
data is loaded.
The error was added in commit 1ea7e45a1f445d32a2b690d52befb2e63418653b,
because there was an additional confusing `AssertionError: [node 0]
Error: no RPC connection` instead of just a single `FileNotFoundError:
[Errno 2] No such file or directory`.
This is no longer needed on current master.
Also, the test is incomplete, because it was just checking bitcoind and
bitcoin-cli, not any other missing binaries.
Also, after the previous commit, it would not work in combination with
--valgrind.
Instead of trying to make it complete, and work in all combinations,
just remove it, because the already existing error will be clear in any
case.
This can be tested via:
```sh
./test/get_previous_releases.py
mv releases releases_backup
# Confirm the test is skipped due to missing releases
./bld-cmake/test/functional/wallet_migration.py
# Confirm the test fails due to missing releases
./bld-cmake/test/functional/wallet_migration.py --previous-releases
mv releases_backup releases
mv ./releases/v28.2 ./releases/v28.2_backup
# Confirm the test fails with a single FileNotFoundError
./bld-cmake/test/functional/wallet_migration.py
mv ./releases/v28.2_backup ./releases/v28.2
# Confirm the test runs and passes
./bld-cmake/test/functional/wallet_migration.py
rm ./bld-cmake/bin/bitcoind
# Confirm the test fails with a single "No such file or directory",
# testing with and without --valgrind
./bld-cmake/test/functional/wallet_migration.py
./bld-cmake/test/functional/wallet_migration.py --valgrind
```
