a240e150e837b5a95ed19765a2e8b7c5b6013f35 streams: remove AutoFile::Get() entirely (Pieter Wuille)
e624a9bef16b6335fd119c10698352b59bf2930a streams: cache file position within AutoFile (Pieter Wuille)
Pull request description:
Fixes#30833.
Instead of relying on frequent `ftell` calls (which appear to cause a significant slowdown on some systems) in XOR-enabled `AutoFile`s, cache the file position within `AutoFile` itself.
ACKs for top commit:
achow101:
ACK a240e150e837b5a95ed19765a2e8b7c5b6013f35
davidgumberg:
untested reACK a240e150e8
theStack:
Code-review ACK a240e150e837b5a95ed19765a2e8b7c5b6013f35
Tree-SHA512: fd3681edc018afaf955dc7a41a0c953ca80d46c1129e3c5b306c87c95aae93b2fe7b900794eb8b6f10491f9211645e7939918a28838295e6873eb226fca7006f
e4e3b44e9cc7227b3ad765397c884999f57bac2e net: call `Select` with reachable networks in `ThreadOpenConnections` (brunoerg)
829becd990b504a2e8a57fa8a6ff6ac6ae8ff900 addrman: change `Select` to support multiple networks (brunoerg)
f698636ec86c004ab331994559c163b7319e6423 net: add `All()` in `ReachableNets` (brunoerg)
Pull request description:
This PR changes addrman's `Select` to support multiple networks and change `ThreadOpenConnections` to call it with reachable networks. It can avoid unnecessary `Select` calls and avoid exceeding the max number of tries (100), especially when turning a clearnet + Tor/I2P/CJDNS node to Tor/I2P/CJDNS. Compared to #29330, this approach is "less aggresive". It does not add a new init flag and does not impact address relay.
I did an experiment of calling `Select` without passing a network until it finds an address from a network that compose 20% ~ 25% of the addrman (limited to 100 tries).
ACKs for top commit:
achow101:
ACK e4e3b44e9cc7227b3ad765397c884999f57bac2e
vasild:
ACK e4e3b44e9cc7227b3ad765397c884999f57bac2e
naumenkogs:
ACK e4e3b44e9cc7227b3ad765397c884999f57bac2e
Tree-SHA512: e8466b72b85bbc2ad8bfb14471eb27d2c50d4e84218f5ede2c15a6fa3653af61b488cde492dbd398f7502bd847e95bfee1abb7e01092daba2236d3ce3d6d2268
a97f43d63a6e835bae20b0bc5d536df98f55d8a0 fuzz: Add harness for p2p headers sync (marcofleon)
a0eaa4749fe0f755e113eee70dee1989bdc07ad5 Add FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION in PoW check (marcofleon)
a3f6f5acd89f2f5bb136ec247f259d212e8944d0 build: Automatically define FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION for fuzz builds (marcofleon)
0c02d4b2bdbc7a3fc3031a63b3b16bafa669d51c net_processing: Make MAX_HEADERS_RESULTS a PeerManager option (marcofleon)
Pull request description:
This PR reopens https://github.com/bitcoin/bitcoin/pull/28043. It's a regression fuzz test for https://github.com/bitcoin/bitcoin/pull/26355 and [a couple bugs](ed6cddd98e) that were addressed in https://github.com/bitcoin/bitcoin/pull/25717. This should help us move forward with the [removal of mainnet checkpoints](https://github.com/bitcoin/bitcoin/pull/25725).
It seems like the main concern in https://github.com/bitcoin/bitcoin/pull/28043 was the global mock function for proof of work. This PR aims to be an improvement by replacing the previous approach with a fuzz build configured using `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION`. This ensures that the simplified test code will never be in a release binary. If we agree this is the way to go, there are some other places (for future targets) where this method could be used.
In this target, PoW isn't being tested, so the goal is to bypass the check and let the fuzzer do its thing. In the other harnesses where PoW is actually being fuzzed, `CheckProofOfWork` is now `CheckProofOfWorkImpl`. So, the only change to that function is in the name.
More about `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` can be found at https://llvm.org/docs/LibFuzzer.html#fuzzer-friendly-build-mode and https://github.com/AFLplusplus/AFLplusplus/blob/stable/docs/fuzzing_in_depth.md#d-modifying-the-target.
ACKs for top commit:
naumenkogs:
ACK a97f43d63a6e835bae20b0bc5d536df98f55d8a0
dergoegge:
reACK a97f43d63a6e835bae20b0bc5d536df98f55d8a0
instagibbs:
tested ACK a97f43d63a6e835bae20b0bc5d536df98f55d8a0
brunoerg:
ACK a97f43d63a6e835bae20b0bc5d536df98f55d8a0
Tree-SHA512: 60b0bc6aadd8ca4c39db9cbba2da2debaaf68afcb6a8dd75c1ce48ca9e3996948fda8020930b6771a424e0f7c41b0b1068db4aa7dbe517f8fc152f1f712058ad
9ad2fe7e69e9e69949ebbb280a15756dc3301f09 clusterlin: only start/use search when enough iterations left (Pieter Wuille)
bd044356edb6c8978df600f33b0073f772ee047c clusterlin: improve heuristic to decide split transaction (optimization) (Pieter Wuille)
71f26293988019d2035bcc55af7b440b494b56bc clusterlin: include topological pot subsets automatically (optimization) (Pieter Wuille)
e20fda77a2da1da3deb273d83700a0b7139422ab clusterlin: reduce computation of unnecessary pot sets (optimization) (Pieter Wuille)
6060a948caf6dbc7505658d6cac750e25698eaf9 clusterlin bench: add example hard cluster benchmarks (Pieter Wuille)
2965fbf203f0b244814d7159381a2e9472bc1f97 clusterlin: track upper bound potential set for work items (optimization) (Pieter Wuille)
9e43e4ce109e98a1ea3f54bbb4de86bc1b92ae4f clusterlin: use feerate-sorted depgraph in SearchCandidateFinder (Pieter Wuille)
b80e6dfe780b3678bb41f2d9d63816f097529b2e clusterlin: add reordering support for DepGraph (Pieter Wuille)
85a285a306100d1815c4ad0f4e52ccbcae8b0fbc clusterlin: separate initial search entries per component (optimization) (Pieter Wuille)
e4faea9ca79ce1f277d8499962c7de64f26b0436 clusterlin bench: have low/high iter benchmarks instead of per-iter (Pieter Wuille)
Pull request description:
Part of cluster mempool: #30289
Depends on #30126, and was split off from it.
This improves the candidate search algorithm introduced in the previous PR with a variety of optimizations.
The resulting search algorithm largely follows Section 2 of [How to linearize your cluster](https://delvingbitcoin.org/t/how-to-linearize-your-cluster/303#h-2-finding-high-feerate-subsets-5), though with a few changes:
* Connected component analysis is performed inside the search algorithm (creating initial work items per component for each candidate), rather than once at a higher level. This duplicates some work but is significantly simpler in implementation.
* No ancestor-set based presplitting inside the search is performed; instead, the `best` value is initialized with the best topologically valid set known to the LIMO algorithm before search starts: the better one out of the highest-feerate remaining ancestor set, and the highest-feerate prefix of remaining transactions in `old_linearization`.
* Work items are represented using an included set *inc* and an undefined set *und*, rather than included and excluded.
* Potential sets *pot* are not computed for work items with empty *inc*.
At a high level, the only missing optimization from that post is bottleneck analysis; my thinking is that it only really helps with clusters that are already relatively cheap to linearize (doing so would need to be done at a higher level, not inside the search algorithm).
---
Overview of the impact of each commit here on linearize performance:
* **[clusterlin bench: have low/high iter benchmarks instead of per-iter](21a184db63)**: no impact
* **[separate initial search entries per component (optimization)](c84c5c86ba)**: reduce iterations, increase start-up cost
* **[add reordering support for DepGraph](019ff29609)**: no impact
* **[use feerate-sorted depgraph in SearchCandidateFinder](8e27dd5a22)**: typically reduce iterations, increase start-up cost
* **[track upper bound potential set for work items](781e0fb3aa)**: reduce iterations, increase cost per iteration
* **[reduce computation of unnecessary pot sets](9fe834fa97)**: reduce cost per iteration
* **[include topological pot subsets automatically](30612710a4)**: reduce iterations, increase cost per iteration
* **[improve heuristic to decide split transaction](1880c00ab1)**: typically reduce iterations, increase cost per iteration
* **[only start/use search when enough iterations left](12760a57b3)**: just account for start-up cost as equivalent iterations
ACKs for top commit:
sdaftuar:
ACK 9ad2fe7e69e9e69949ebbb280a15756dc3301f09
instagibbs:
reACK 9ad2fe7e69e9e69949ebbb280a15756dc3301f09
glozow:
reACK 9ad2fe7e69e, just have a question about the docs
Tree-SHA512: 108bcbb0676f36071eb83954059b5f3d6646c745015b644a2a5d7f5a8ac9424c2d01d339fa6318a3aff4cf313308e85bb80b0090899720a3fcba027b8025590a
282f0e92559da23e356504a564a0322b9888e50b Unit test runner documentation fix and improvements (Jon Atack)
Pull request description:
Running `test_bitcoin --help` prints the list of arguments that may be passed, not the list of tests, so fix that.
Improve the content and order of the unit test documentation.
ACKs for top commit:
pablomartin4btc:
re-ACK 282f0e92559da23e356504a564a0322b9888e50b
tdb3:
re ACK 282f0e92559da23e356504a564a0322b9888e50b
Tree-SHA512: 0d25108ab641bcd9b53f99d139afeec90a34f44d5b00c3c677f7539d87782440a28fadc348663b8c28ace43552834737b9c1e8f5517c68edc8547695a9cb5063
- Running `test_bitcoin --help` prints the list of arguments that may be passed,
not the list of tests, so fix that.
- Improve the content and order of the unit test documentation.
Automatically add topologically-valid subsets of the potential set pot
to inc. It can be proven that these must be part of the best reachable
topologically-valid set from that work item.
This is a crucial optimization that (apparently) reduces the maximum
number of iterations from ~2^(N-1) to ~sqrt(2^N).
Co-Authored-By: Suhas Daftuar <sdaftuar@gmail.com>
In each work item, keep track of a conservative overestimate of the best
possible feerate that can be reached from it, and then use these to avoid
exploring hopeless work items.
Add a DepGraph(depgraph, reordering) function that constructs a new DepGraph
corresponding to an old one, but with its transactions is a modified order
(given as a vector from old to new positions).
Also use this reordering feature inside DepGraphFormatter::Unser, which needs
a small modification so that its reordering mapping is old-to-new (rather than
the new-to-old it used before).
Before this commit, the worst case for linearization involves clusters which
break apart in several smaller components after the first candidate is
included in the output linearization.
Address this by never considering work items that span multiple components
of what remains of the cluster.
fa5bc450d5d4c1d2daf7b205f1678402c3c21678 util: Use compile-time check for LogConnectFailure (MarcoFalke)
fa7087b896c0150c29d7a27c53e0533831a2bf3b util: Use compile-time check for FatalErrorf (MarcoFalke)
faa62c0112f2b7ab69c80a5178f5b79217bed0a6 util: Add ConstevalFormatString (MarcoFalke)
fae7b83eb58d22ed83878561603991131372cdd7 lint: Remove forbidden functions from lint-format-strings.py (MarcoFalke)
Pull request description:
The `test/lint/lint-format-strings.py` was designed to count the number of format specifiers and assert that they are equal to the number of parameters passed to the format function. The goal seems reasonable, but the implementation has many problems:
* It is written in Python, meaning that C++ code can not be parsed correctly. Currently it relies on brittle regex and string parsing.
* Apart from the parsing errors, there are also many logic errors. For example, `count_format_specifiers` allows a mix of positional specifiers and non-positional specifiers, which can lead to runtime format bugs. Also, `count_format_specifiers` silently skipped over "special" format specifiers, which are valid in tinyformat, which again can lead to runtime format bugs being undetected.
* The brittle logic has a history of breaking in pull requests that are otherwise fine. This causes the CI to fail and the pull request being blocked from progress until the bug in the linter is fixed, or the code is rewritten to work around the bug.
* It is only run in the CI, or when the developer invokes the script. It would be better if the developer got the error message at compile-time, directly when writing the code.
Fix all issues by using a `consteval` checked format string in `FatalErrorf` and `LogConnectFailure`.
This is the first step toward https://github.com/bitcoin/bitcoin/issues/30530 and a follow-up will apply the approach to the other places.
ACKs for top commit:
stickies-v:
re-ACK fa5bc450d5d4c1d2daf7b205f1678402c3c21678
l0rinc:
ACK fa5bc450d5d4c1d2daf7b205f1678402c3c21678
hodlinator:
ACK fa5bc450d5d4c1d2daf7b205f1678402c3c21678
ryanofsky:
Code review ACK fa5bc450d5d4c1d2daf7b205f1678402c3c21678
Tree-SHA512: d6189096b16083143687ed1b1559cf4f92f97dd87bc5d00673e44f4fb9fce7bb7b215cfdfc39b6e6a24f0b75a79a03ededce966639e554f7172e1fc22cf015ae
1eac96a503b6bac3eaf5d0eb3d23ffde3bfbf9aa Compare FromUserHex result against other hex validators and parsers (Lőrinc)
19947863e16425e6a119e7a4819867292b1235ee Use BOOST_CHECK_EQUAL for optional, arith_uint256, uint256, uint160 (Lőrinc)
743ac30e349e181c26a2d2af0bcb93b0835ce521 Add std::optional support to Boost's equality check (Lőrinc)
Pull request description:
Enhanced `FromUserHex` coverage by:
* Added `std::optional` support to `BOOST_CHECK_EQUAL`, allowing direct comparisons of `std::optional<T>` with other `T` expected values.
* Increased fuzz testing for hex parsing to validate against other hex validators and parsers.
----
* Use BOOST_CHECK_EQUAL for https://github.com/bitcoin/bitcoin/pull/30569#discussion_r1706637780 arith_uint256, uint256, uint160
Example error before:
> unknown location:0: fatal error: in "validation_chainstatemanager_tests/chainstatemanager_args": std::bad_optional_access: bad_optional_access
test/validation_chainstatemanager_tests.cpp:781: last checkpoint
after:
> test/validation_chainstatemanager_tests.cpp:801: error: in "validation_chainstatemanager_tests/chainstatemanager_args": check set_opts({"-assumevalid=0"}).assumed_valid_block == uint256::ZERO has failed [std::nullopt != 0000000000000000000000000000000000000000000000000000000000000000]
ACKs for top commit:
stickies-v:
re-ACK 1eac96a503b6bac3eaf5d0eb3d23ffde3bfbf9aa
ryanofsky:
Code review ACK 1eac96a503b6bac3eaf5d0eb3d23ffde3bfbf9aa. Only changes since last review were auto type and fuzz test tweaks.
hodlinator:
ACK 1eac96a503b6bac3eaf5d0eb3d23ffde3bfbf9aa
Tree-SHA512: f1d2c65f0ee4e97830700be5b330189207b11ed0c89a8cebf0f97d43308402a6b3732e10130c79a0c044f7d2eeabfb5359990825aadf02c4ec19428dcd982b00
5c80192ff6b982ee3a75be4142fe942b8206f2cd test: Drop no longer needed workarounds (Hennadii Stepanov)
Pull request description:
This PR deletes the workarounds introduced in https://github.com/bitcoin/bitcoin/pull/16564 and https://github.com/bitcoin/bitcoin/pull/15382, as `ctest` skips these cases gracefully: 5c80192ff6/src/test/CMakeLists.txt (L201-L203)
ACKs for top commit:
kevkevinpal:
ACK [5c80192](5c80192ff6)
fanquake:
ACK 5c80192ff6b982ee3a75be4142fe942b8206f2cd. Looks correct:
Tree-SHA512: c47c606ecf7d64016b3c6353c3d4898350edc2caeac494dfd44484417f500a73f0c88c39f0f24651f3a02ef31ed9ca5c70d938bb9a8ca1eea54927e4d6a8fcd2
Example error before:
> unknown location:0: fatal error: in "validation_chainstatemanager_tests/chainstatemanager_args": std::bad_optional_access: bad_optional_access
test/validation_chainstatemanager_tests.cpp:781: last checkpoint
after:
> test/validation_chainstatemanager_tests.cpp:801: error: in "validation_chainstatemanager_tests/chainstatemanager_args": check set_opts({"-assumevalid=0"}).assumed_valid_block == uint256::ZERO has failed [std::nullopt != 0000000000000000000000000000000000000000000000000000000000000000]
Also added extra minimum_chainwork test to make it symmetric with assumevalid
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
43cd83b0c7ba436b8ffc83d8a65e935714dffe70 test: move uint256_tests/operator_with_self to arith_uint256_tests (stickies-v)
c6c994cb2b9af58b1c0e554255d1a3be785e8d56 test: remove test-only uint160S (stickies-v)
62cc4656e2bb38f80485a13d75b42add09a6b731 test: remove test-only uint256S (stickies-v)
adc00ad728dd54084671398f8fa5c12be92d2bab test: remove test-only arith_uint256S (stickies-v)
f51b237723b87db706cbce2939d20eb193332799 refactor: rpc: use uint256::FromHex for ParseHashV (stickies-v)
Pull request description:
_Continuation of #30569._
Since fad2991ba0, `uint256S()` has been [deprecated](fad2991ba0 (diff-800776e2dda39116e889839f69409571a5d397de048a141da7e4003bc099e3e2R138)) because it is less robust than the `base_blob::FromHex()` introduced in https://github.com/bitcoin/bitcoin/pull/30482. Specifically, it tries to recover from length-mismatches, recover from untrimmed whitespace, 0x-prefix and garbage at the end, instead of simply requiring exactly 64 hex-only characters. (see also https://github.com/bitcoin/bitcoin/pull/30532)
This PR removes `uint256S()` (and `uint160S()`) completely, with no non-test behaviour change.
Specifically, the main changes in this PR are:
- the (minimal) last non-test usage of `uint256S()` in `ParseHashV()` is removed without behaviour change, which can partially be verified by cherry-picking and/or modifying [this test commit](1f2b0fa86d)).
- the test usage of `uint{160,256}S()` is removed, largely replacing it with `uint{160,256}::FromHex()` where applicable, potentially modifying the test by removing non-hex characters or dropping the test entirely if removing non-hex characters makes it redundant
- the now unused `uint{160,256}S()` functions are removed completely.
- unit test coverage on converting `uint256` <-> `arith_uint256` through `UintToArith256()` and `ArithToUint256()` is beefed up, and `arith_uint256` tests are moved to `arith_uint256_tests.cpp`, removing the `uint256_tests.cpp` dependency on `uint256h`, mirroring how the code is structured.
_Note: `uint256::FromUserHex()` exists to more leniently construct uint256 from user input, allowing "0x" prefixes and too-short-input, as safer alternative to `uint256S()` where necessary._
ACKs for top commit:
l0rinc:
reACK 43cd83b0c7ba436b8ffc83d8a65e935714dffe70
hodlinator:
re-ACK 43cd83b0c7ba436b8ffc83d8a65e935714dffe70
ryanofsky:
Code review ACK 43cd83b0c7ba436b8ffc83d8a65e935714dffe70. Only code change is a small refactoring which looks good. The rest of the PR is all test changes, which I only lightly reviewed, but seem to be positive and do what's described
Tree-SHA512: 48147a4c6af671597df0f72c1b477ae4631cd2cae4645ec54d0e327611ff302c9899e344518c81242cdde82930f6ad23a3a7e6e0b80671816e9f457b9de90a5c
30073e6b3a24cbe417c45cd5df6a3a2de0251e9d multiprocess: Add -ipcbind option to bitcoin-node (Russell Yanofsky)
73fe7d723084653671f2178ea1177a8627edfafa multiprocess: Add unit tests for connect, serve, and listen functions (Ryan Ofsky)
955d4077aac621697246bcb20a854ba97e37c519 multiprocess: Add IPC connectAddress and listenAddress methods (Russell Yanofsky)
4da20434d4d68b7933e39aca36faa6fd2264cc45 depends: Update libmultiprocess library for CustomMessage function and ThreadContext bugfix (Ryan Ofsky)
Pull request description:
Add `-ipcbind` option to `bitcoin-node` to make it listen on a unix socket and accept connections from other processes. The default socket path is `<datadir>/node.sock`, but this can be customized.
This option lets potential wallet, gui, index, and mining processes connect to the node and control it. See examples in #19460, #19461, and #30437.
Motivation for this PR, in combination with #30510, is be able to release a bitcoin core node binary that can generate block templates for a separate Stratum v2 mining service, like the one being implemented in https://github.com/Sjors/bitcoin/pull/48, that connects over IPC.
Other things to know about this PR:
- While the `-ipcbind` option lets other processes to connect to the `bitcoin-node` process, the only thing they can actually do after connecting is call methods on the [`Init`](https://github.com/bitcoin/bitcoin/blob/master/src/ipc/capnp/init.capnp#L17-L20) interface which is currently very limited and doesn't do much. But PRs [#30510](https://github.com/bitcoin/bitcoin/pull/30510), [#29409](https://github.com/bitcoin/bitcoin/pull/29409), and [#10102](https://github.com/bitcoin/bitcoin/pull/10102) expand the `Init` interface to expose mining, wallet, and gui functionality respectively.
- This PR is not needed for [#10102](https://github.com/bitcoin/bitcoin/pull/10102), which runs GUI, node, and wallet code in different processes, because [#10102](https://github.com/bitcoin/bitcoin/pull/10102) does not use unix sockets or allow outside processes to connect to existing processes. [#10102](https://github.com/bitcoin/bitcoin/pull/10102) lets parent and child processes communicate over internal socketpairs, not externally accessible sockets.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
achow101:
ACK 30073e6b3a24cbe417c45cd5df6a3a2de0251e9d
TheCharlatan:
Re-ACK 30073e6b3a24cbe417c45cd5df6a3a2de0251e9d
itornaza:
Code review ACK 30073e6b3a24cbe417c45cd5df6a3a2de0251e9d
Tree-SHA512: 2b766e60535f57352e8afda9c3748a32acb5a57b2827371b48ba865fa9aa1df00f340732654f2e300c6823dbc6f3e14377fca87e4e959e613fe85a6d2312d9c8
Also moved the operators to the bottom of the file since they're less important and to group them together.
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: stickies-v <stickies-v@protonmail.com>
move/formatting-only change.
These tests do not cover uint256, so move them to the appropriate
test suite. Additionally, apply clang-format suggestions.
uint160S is a test-only function, and testing input that
is not allowed in uint160::FromHex() is superfluous.
Tests that can't use uint160::FromHex() because they use input
with non-hex digit characters are
a) modified by dropping the non-hex digit characters if that
provides useful test coverage.
b) dropped if the test without non-hex digit characters does
not provide useful test coverage, e.g. because it is now
duplicated.
uint256S was previously deprecated for being unsafe. All non-test
usage has already been removed in earlier commits.
1. Tests now use uint256::FromHex() or other constructors wherever
possible without further modification.
2. Tests that can't use uint256::FromHex() because they use input
with non-hex digit characters are
a) modified by dropping the non-hex digit characters if that
provides useful test coverage.
b) dropped if the test without non-hex digit characters does
not provide useful test coverage, e.g. because it is now
duplicated.
Additionally, use BOOST_CHECK_EQUAL where relevant on touched lines
to make error messages more readable.
Tests that are solely testing constructing from a hex string
are dropped, others are modified to use a uint256 constructor
or the arith_uint256 uint64_t constructor.
Since an arith_uint256 can not be constructed from a string
directly, we need to ensure that test coverage on
UintToArith256(uint256::FromHex()) is not reduced.
uint256::FromHex() already has good test coverage, but
the test coverage on UintToArith256() and ArithToUint256()
is increased in this commit by upgrading the `conversion`
test case.
Moreover, since `uint256.h` does not have any dependencies
on `arith_uint256.h`, the conversion tests are moved to
`arith_uint256_tests.cpp` so the dependency can be cleaned
up entirely in a future commit.
Add `-ipcbind` option to `bitcoin-node` to listen on an IPC socket and accept
connections from other processes. In the future, there will be an `-ipcconnect`
option added to `bitcoin-wallet` and `bitcoin-node` to allow wallet and gui
processes to connect to the node and access it.
Example usage:
src/bitcoin-node -regtest -debug -ipcbind=unix
src/bitcoin-wallet -regtest -ipcconnect=unix info
src/bitcoin-gui -regtest -ipcconnect=unix
src/bitcoin-mine -regtest -ipcconnect=unix
fadbcd51fc77a3f4e877851463f3c7425fb751d2 bench: Remove redundant logging benchmarks (MarcoFalke)
fa8dd952e279a87f6027ddd2e2119bf2ae2f9943 bench: Use LogInfo instead of the deprecated alias LogPrintf (MarcoFalke)
Pull request description:
`LogPrint*ThreadNames` is redundant with `LogWith(out)ThreadNames`,
because they all measure toggling the thread names (and check that it
has no effect on performance).
Fix it by removing the redundant ones. This also allows to drop a deprecated logging alias.
ACKs for top commit:
stickies-v:
ACK fadbcd51fc77a3f4e877851463f3c7425fb751d2
Tree-SHA512: 4fe137f374aa4ee1aa0e1da4a1f9839c0e52c23dbb93198ecafee98de39d311cc47304bba4191f3807aa00c51b1eae543e3f270f03d341c84910e5e341a1d475
fa84f9decd224ea1c25dc7095bad70a48fa1a534 test: Pin and document TEST_DIR_PATH_ELEMENT (MarcoFalke)
2222f7a87404078984c7189768a3422deb114302 test: Rename SeedRand::SEED to FIXED_SEED for clarity (MarcoFalke)
Pull request description:
Two small test changes:
* A refactor to update the name and documentation around `SeedRand::FIXED_SEED`.
* A change to extract and document `TEST_DIR_PATH_ELEMENT`, and to change its value to better match the `TMPDIR_PREFIX` in functional tests. The value previously included `PACKAGE_NAME`, which is cute, but doesn't explain why it was used (to include a space). So just use `test_common bitcoin` to achieve the same with less effort.
ACKs for top commit:
hodlinator:
ACK fa84f9decd224ea1c25dc7095bad70a48fa1a534
ryanofsky:
Code review ACK fa84f9decd224ea1c25dc7095bad70a48fa1a534
Tree-SHA512: eb35d6598bb08f9b996e3a4762d8f26b2441c0ca00780798e473015af735dfc9997120895a922b94d4b6ada45adadba4a686e9cf9c285ddf688848e764c64840
faecca9a85c1c1917d024f55cca34d19cc94f3b9 test: Use span for raw data (MarcoFalke)
fac973647d69dd14089cee9972e8dfa0074c8a97 test: Use string_view for json_tests (MarcoFalke)
Pull request description:
The build system converts raw data into a C++ header file for tests.
This change modernizes the code to use the convenience wrappers `std::span` and `std::string_view`, so that redundant copies can be avoided.
ACKs for top commit:
fjahr:
re-ACK faecca9a85
TheCharlatan:
ACK faecca9a85c1c1917d024f55cca34d19cc94f3b9
stickies-v:
ACK faecca9a85c1c1917d024f55cca34d19cc94f3b9
hebasto:
ACK faecca9a85c1c1917d024f55cca34d19cc94f3b9, I have reviewed the code and the generated headers.
Tree-SHA512: 1f4951c54aff11ba27c41fb70f2821bdb79e06ca0abae734b970bd0d64dda9d8cced824a891fd51b3e9d4e5715ee9eb49ed5d369010a45eca7c3bec9f8641235
This change allows to drop brittle sizeof calls in favor of the
std::span::size method.
Other improvements include:
* Use of a namespace to mark test and bench data
* Use of the modern std::byte
* Drop of a no longer used std::vector copy and the bench/data module
LogPrint*ThreadNames is redundant with LogWith(out)ThreadNames, because
they all measure toggling the thread names (and check that it has no
effect on performance).
This also allows to remove unused and deprecated macros.
ae48a22a3df086fb59843b7b814619ed5df7557b test: fixing failing system_tests/run_command under some Locales (Jadi)
Pull request description:
the run_command test under system_tests fails if the locale is anything
other than English ones because results such as "No such file or directory"
will be different under Non-English locales.
On the old version, a `ls nonexistingfile` was used to generate the error
output which is not ideal. In the current version we are using a Python one-liner
to generate a non 0 zero return value and "err" on stderr and check the
expected value against this.
fixes https://github.com/bitcoin/bitcoin/issues/30608
ACKs for top commit:
maflcko:
review ACK ae48a22a3df086fb59843b7b814619ed5df7557b
achow101:
ACK ae48a22a3df086fb59843b7b814619ed5df7557b
hebasto:
ACK ae48a22a3df086fb59843b7b814619ed5df7557b, tested on Ubuntu 24.04 by switching locale.
Tree-SHA512: af7522ddcd786fa4a6832c8336ca89d8ff05f49ff963cbe1a96653b0edf29e0f950a032f23d742b16b3895e90cf5117b5f6a95464268dec67039df166d7d8639
01960c53c7d71c70792abe19413315768dc2275a fuzz: make FuzzedDataProvider usage deterministic (Martin Leitner-Ankerl)
Pull request description:
There exist many usages of `fuzzed_data_provider` where it is evaluated directly in the function call.
Unfortunately, [the order of evaluation of function arguments is unspecified](https://en.cppreference.com/w/cpp/language/eval_order), and a simple example shows that it can differ e.g. between clang++ and g++: https://godbolt.org/z/jooMezWWY
When the evaluation order is not consistent, the same fuzzing/random input will produce different output, which is bad for coverage/reproducibility. This PR fixes all these cases I have found where unspecified evaluation order could be a problem.
Finding these has been manual work; I grepped the sourcecode for these patterns, and looked at each usage individually. So there is a chance I missed some.
* `fuzzed_data_provider`
* `.Consume`
* `>Consume`
* `.rand`
I first discovered this in https://github.com/bitcoin/bitcoin/pull/29013#discussion_r1420236394. Note that there is a possibility that due to this fix the evaluation order is now different in many cases than when the fuzzing corpus has been created. If that is the case, the fuzzing corpus will have worse coverage than before.
Update: In list-initialization the order of evaluation is well defined, so e.g. usages in `initializer_list` or constructors that use `{...}` is ok.
ACKs for top commit:
achow101:
ACK 01960c53c7d71c70792abe19413315768dc2275a
vasild:
ACK 01960c53c7d71c70792abe19413315768dc2275a
ismaelsadeeq:
ACK 01960c53c7d71c70792abe19413315768dc2275a
Tree-SHA512: e56d087f6f4bf79c90b972a5f0c6908d1784b3cfbb8130b6b450d5ca7d116c5a791df506b869a23bce930b2a6977558e1fb5115bb4e061969cc40f568077a1ad
c8e6771af002eaf15567794fcdc57fdb0e3fb140 test: restrict multiple CLI arguments (naiyoma)
8838c4f171f3c3e568d237b216167fddb521d0a7 common/args.h: automate check for multiple cli commands (naiyoma)
Pull request description:
This PR is a continuation of the validation suggested [here](https://github.com/bitcoin/bitcoin/pull/27815) to ensure that only one Request Handler can be specified at a time.
ACKs for top commit:
stratospher:
reACK c8e6771.
achow101:
ACK c8e6771af002eaf15567794fcdc57fdb0e3fb140
tdb3:
cr re ACK c8e6771af002eaf15567794fcdc57fdb0e3fb140
Tree-SHA512: f4fe036fee342a54f1a7ac702ac35c40bf3d420fde6ab16313a75327292d5ee5c8ece1be9f852a13fcf73da1148b143b37b4894e294052abdde6eefb2e8c6f3f
66d13c870284327abc89d36c0b5cc5f58e96f570 test: add check that large txs aren't put into orphanage (Sebastian Falbesoner)
ed7d2246661ec1789b7db0f21668270f0681ea4a test: add `BulkTransaction` helper to unit test transaction utils (Sebastian Falbesoner)
Pull request description:
This PR adds test coverage for the following check in `TxOrphanage::AddTx`, where large orphan txs are ignored in order to avoid memory exhaustion attacks:
5abb9b1af4/src/txorphanage.cpp (L22-L34)
Note that this code-path isn't reachable under normal circumstances, as txs larger than `MAX_STANDARD_TX_WEIGHT` are already rejected earlier in the course of doing the mempool standardness checks (see `MemPoolAccept::PreChecks` -> `IsStandardTx` -> `reason = "tx-size";`), so this is only relevant if tx standardness rules are disabled via `-acceptnonstdtxns=1`. The ignore path is checked ~~by asserting the debug log, which is ugly, but as far as I know there is currently no way to access the orphanage entries from the outside~~ via unit test that checks the return value of `AddTx`. As an alternative to adding test coverage, one might consider removing this check altogether (or replacing it with an `Assume`), as it's redundant as explained above.
ACKs for top commit:
maflcko:
review ACK 66d13c870284327abc89d36c0b5cc5f58e96f570
glozow:
ACK 66d13c870284327abc89d36c0b5cc5f58e96f570
tdb3:
re-ACK 66d13c870284327abc89d36c0b5cc5f58e96f570
Tree-SHA512: 88e8254ab5fca70c387a5992649ea6a704a65162999be972cc86bd74fc26c5f0f1e13e04856708d07ad5524cb77c0918e19663db92b3593e842469dfe04af6a1
8888beea8d477b1d4a2dfd2a0bb5f686de62f3ff scripted-diff: fuzz: Rename fuzz_seed_corpus to fuzz_corpora (MarcoFalke)
Pull request description:
Now that cmake was a breaking change for all fuzz scripts, it seems fine to bundle it with another breaking change to rename the fuzz corpora directory, as discussed and approved in https://github.com/bitcoin-core/qa-assets/issues/200:
* The word "seed" in the old name doesn't really apply. In reality it is a collection of fuzz input seeds, as well as fuzz inputs.
* The rename will also allow in the future (when there is a need and desire) to provide a minimal set of possibly hand-crafted or otherwise non-fuzz-generated fuzz seed inputs to some fuzz targets (and possibly store them in a separate folder and validate that their format is still accurate and matches the fuzz target code).
* Finally, "corpus" is renamed to corpora, to clarify that the folder holds the fuzz inputs for several fuzz targets.
ACKs for top commit:
brunoerg:
ACK 8888beea8d477b1d4a2dfd2a0bb5f686de62f3ff
marcofleon:
ACK 8888beea8d477b1d4a2dfd2a0bb5f686de62f3ff
Tree-SHA512: abc693ca5d946850f04b6349e2a98f8fbc2ba9991be5a025bc0f357e341cbe7510f2f5f0e47b997d07136736d818df361270f372b8fb70860995a0605ca81e4d
the run_command test under system_tests fails if the locale is anything
other than English ones because results such as "No such file or directory"
will be different under Non-English locales.
On the old version, a `ls nonexistingfile` was used to generate the error
output which is not ideal. In the current version we are using a Python one-liner
to generate a non 0 zero return value and "err" on stderr and check the
expected value against this.
fixes#30608
a2955f09792b6232f3a45aa44a498b466279a8b7 validation: Use span for ImportBlocks paths (TheCharlatan)
20515ea3f5bd426f6e3746cf5cddd2324dacae31 validation: Use span for CalculateClaimedHeadersWork (TheCharlatan)
52575e96e72a0402c448f86728b2e84836b1e817 validation: Use span for ProcessNewBlockHeaders (TheCharlatan)
Pull request description:
Makes it friendlier for potential future users of the kernel library if they do not store the headers in a std::vector, but can guarantee contiguous memory.
Take this opportunity to also change the argument of ImportBlocks previously taking a `std::vector` to a `std::span`.
ACKs for top commit:
stickies-v:
re-ACK a2955f09792b6232f3a45aa44a498b466279a8b7 - no changes except further walking the ~file~ path of modernizing variable names.
maflcko:
ACK a2955f09792b6232f3a45aa44a498b466279a8b7 🕑
achow101:
ACK a2955f09792b6232f3a45aa44a498b466279a8b7
danielabrozzoni:
ACK a2955f09792b6232f3a45aa44a498b466279a8b7
Tree-SHA512: 8b07f4ad26e270b65600d1968cd78847b85caca5bfbb83fd9860389f26656b1d9a40b85e0990339f50403d18cedcd2456990054f3b8b0bedce943e50222d2709
