This adds a large simulation fuzz test for all TxOrphanage public interface
functions, using a mix of comparison with expected behavior (in case it is
fully specified), and testing of properties exhibited otherwise.
This is preparation for the simulation fuzz test added in a later commit. Since
AddChildrenToWorkSet consumes randomness, there is no way for the simulator to
exactly predict its behavior. By returning the set of made-reconsiderable announcements
instead, the simulator can instead test that it is *a* valid choice, and then
apply it to its own data structures.
For the default number of peers (125), allows each to relay a default
descendant package (up to 25-1=24 can be missing inputs) of small (9
inputs or fewer) transactions out of order.
This limit also gives acceptable bounds for worst case LimitOrphans iterations.
Functional tests aren't changed to check for larger cap because it would
make the runtime too long.
Also deletes the now-unused DEFAULT_MAX_ORPHAN_TRANSACTIONS.
This is largely a reimplementation using boost::multi_index_container.
All the same public methods are available. It has an index by outpoint,
per-peer tracking, peer worksets, etc.
A few differences:
- Limits have changed: instead of a global limit of 100 unique orphans,
we have a maximum number of announcements (which can include duplicate
orphans) and a global memory limit which scales with the number of
peers.
- The maximum announcements limit is 100 to match the original limit,
but this is actually a stricter limit because the announcement count
is not de-duplicated.
- Eviction strategy: when global limits are reached, a per-peer limit
comes into play. While limits are exceeded, we choose the peer whose
“DoS score” (max usage / limit ratio for announcements and memory
limits) is highest and evict announcements by entry time, sorting
non-reconsiderable ones before reconsiderable ones. Since announcements
are unique by (wtxid, peer), as long as 1 announcement remains for a
transaction, it remains in the orphanage.
- This eviction strategy means no peer can influence the eviction of
another peer’s orphans.
- Also, since global limits are a multiple of per-peer limits, as long
as a peer does not exceed its limits, its orphans are protected from
eviction.
- Orphans no longer expire, since older announcements are generally
removed before newer ones.
- GetChildrenFromSamePeer returns the transactions from newest to
oldest.
Co-authored-by: Pieter Wuille <pieter@wuille.net>
Move towards a model where TxOrphanage is initialized with limits that
it remembers throughout its lifetime.
Remove the param. Limiting by number of unique orphans will be removed
in a later commit.
Now that -maxorphantx is gone, this does not change the node behavior.
The parameter is only used in tests.
c18bf0bd9be632e54591493a5309e9ed4020f2e5 refactor: cleanup index logging (Sjors Provoost)
Pull request description:
This PR removes the use of `__func__` from index logging, since we have `-logsourcelocations`.
It also improves readability by putting `GetName()` in a more logical place.
Before
> coinstatsindex: best block of the index not found. Please rebuild the index.
After:
> best block of coinstatsindex not found. Please rebuild the index.
I found myself maintaining this commit as part of https://github.com/Sjors/bitcoin/pull/86, but since that might never land here, it seemed better to split it into its own PR (or get rid of it).
ACKs for top commit:
l0rinc:
Lightweight code review ACK c18bf0bd9be632e54591493a5309e9ed4020f2e5
maflcko:
review ACK c18bf0bd9be632e54591493a5309e9ed4020f2e5 🚣
Tree-SHA512: 755948371e3ff7a5515b63ce48075631ec7868d69c3c1469176d5be0e8b28e1c071e206ae3f7320f87d8c441f815894acfef61621f05795b5ff6b8a5a3031e3b
node1 (with 24 blocks) causes node0 (with 6 blocks) to silently
reorg. so move the subtest to a point before the 20 blocks are
generated so that node1's state doesn't cause node0 to silently
reorg.
44f3bae300dcafbe53f9b07e6cc22a112833e579 depends: Force `CMAKE_EXPORT_NO_PACKAGE_REGISTRY=TRUE` (Hennadii Stepanov)
Pull request description:
When using CMake policies 3.14 and below, the `export(PACKAGE)` command by default populates the user package registry, which is stored outside the build tree. Setting the `CMAKE_EXPORT_NO_PACKAGE_REGISTRY` variable disables this side effect.
In CMake 3.15 and later, this behavior is disabled by default, and the variable has no effect.
This PR forces `CMAKE_EXPORT_NO_PACKAGE_REGISTRY=TRUE` globally, rather than managing it for each dependency package individually rather. It may be reverted once all CMake-based packages have updated to policies 3.15 or newer.
Fixes https://github.com/bitcoin/bitcoin/issues/32938.
ACKs for top commit:
fanquake:
ACK 44f3bae300dcafbe53f9b07e6cc22a112833e579
Tree-SHA512: 0aac398b7182e80185b064d59f81aece4d8477a609fad9cc3fee317da2aff43b66ef7db1efec0135b4f0feaad23b1db664e33bd035fe659712c5b2a9bf2d6fb6
fad191ff48b15832a90c19d560a7c0525c146be3 ci: Avoid cd into build dir (MarcoFalke)
Pull request description:
Changing into the build dir is confusing and brittle, because the following commands implicitly assume it. So they could break on unrelated changes.
The changes are required for stuff like:
* cmake presets (see https://github.com/bitcoin/bitcoin/pull/30871#issuecomment-2344031208)
* meta ci tests (like https://github.com/bitcoin/bitcoin/pull/32874)
So remove the `cd` and just make the build dir explicit.
ACKs for top commit:
hebasto:
ACK fad191ff48b15832a90c19d560a7c0525c146be3, I have reviewed the code and it looks OK.
Tree-SHA512: a88a9341445ffe28a0dac3815f235ec8eb0459d10a91a80829fd3184762d3c807d0f68c56243b20c04a6efa5becd8a7fad568f43c2b1e6af1ff8ba07b140ef87
12a6959892cb24b940b3579828f2066651572153 cmake: Drop no longer necessary "cmakeMinimumRequired" object (Hennadii Stepanov)
Pull request description:
The minimum required CMake version is 3.22:6a13a6106e/CMakeLists.txt (L10)
ACKs for top commit:
fanquake:
ACK 12a6959892cb24b940b3579828f2066651572153 - has been unneeded since it was introduced (minimum was already 3.22).
Tree-SHA512: 26f97662bfe52986e19e38dbf4ab8e1e7558bc78c3a65593cbecd1f35887bba7a9f7d8a3d08ccfab8396f41c2334cdad5b0e503999a759cfa158d3bb8d0d14d7
84ef5524d5abaf07cc9970f42e74ee15bd381e3d fix spelling in tor.md docs (stutxo)
Pull request description:
This PR is to fix some spelling mistakes i found of the word occurrences! there are two occurrences of this mistake.
thanks!
ACKs for top commit:
maflcko:
lgtm ACK 84ef5524d5abaf07cc9970f42e74ee15bd381e3d
willcl-ark:
ACK 84ef5524d5abaf07cc9970f42e74ee15bd381e3d
delta1:
ACK 84ef5524d5abaf07cc9970f42e74ee15bd381e3d
Tree-SHA512: 4ba71b772fdc8cf36ada7493d29fb5b312a7a6085099347162eb3495db4de984b0417b7861f2927c617cbd552741356e26688479601bdf7e835c15e097aa28f3
8f766f39df3e312f79f461b2accc2f9c90fa6338 ci: enable -Werror=dev (fanquake)
7b420ca8341a68e196a4f6fbd03104dc27b166ee guix: configure with -Werror=dev (fanquake)
44097ddb19139d07a905d1238fe67e53f8343d54 cmake: enable -Werror=dev in dev-mode preset (fanquake)
Pull request description:
Pass `-Werror=dev` in the CI, Guix and the `dev-mode` preset.
https://cmake.org/cmake/help/latest/manual/cmake.1.html#cmdoption-cmake-Werror:
> Make developer warnings errors.
> Make warnings that are meant for the author of the CMakeLists.txt files errors. By default this will also turn on deprecated warnings as errors.
Pulled out of #32865.
ACKs for top commit:
Sjors:
re-ACK 8f766f39df3e312f79f461b2accc2f9c90fa6338
hebasto:
ACK 8f766f39df3e312f79f461b2accc2f9c90fa6338, tested on Ubuntu 24.04.
Tree-SHA512: 0fa321b77d2519b5249d90590664c4e5938ac86209b068658647adf97ab55ea4d54c913aae2f622385fe2f41d7c851cd5d7371905fdad38b66cb124371e16ac7
Expiry is going away in a later commit.
This is only an RPC change. Behavior of the orphanage does not change.
Note that getorphantxs is marked experimental.
a60f863d3e276534444571282f432b913d3967db scripted-diff: Replace GenTxidVariant with GenTxid (marcofleon)
c8ba1995986323cd9e76097acc1f15eed7c60943 Remove old GenTxid class (marcofleon)
072a198ea4bc9f1e8449cd31e55d397b75ce4ad5 Convert remaining instances of GenTxid to GenTxidVariant (marcofleon)
1b528391c79497ae19f7e481439e350533c7cd1a Convert `txrequest` to GenTxidVariant (marcofleon)
bde4579b0780aa3754af35beffbcfeb31f28045b Convert `txdownloadman_impl` to GenTxidVariant (marcofleon)
c876a892ec0b04851bea0a688d7681b6aaca4cb7 Replace GenTxid with Txid/Wtxid overloads in `txmempool` (marcofleon)
de858ce2bea83c53635dee9a49c8c273a12440dd move-only: make GetInfo a private CTxMemPool member (stickies-v)
eee473d9f3019a0ea4ebbc9c41781813ad574a86 Convert `CompareInvMempoolOrder` to GenTxidVariant (marcofleon)
243553d59071f3e43a42f3809706790495b17ffc refactor: replace get_iter_from_wtxid with GetIter(const Wtxid&) (stickies-v)
fcf92fd640eae60d1f601136a4e1c9de8ccb68b5 refactor: make CTxMemPool::GetIter strongly typed (marcofleon)
11d28f21bb8f0c3094934b3fef45871f73bb216a Implement GenTxid as a variant (marcofleon)
Pull request description:
Part of the [type safety refactor](https://github.com/bitcoin/bitcoin/pull/32189).
This PR changes the GenTxid class to a variant, which holds both Txids and Wtxids. This provides compile-time type safety and eliminates the manual type check (bool m_is_wtxid). Variables that can be either a Txid or a Wtxid are now using the new GenTxid variant, instead of uint256.
ACKs for top commit:
w0xlt:
ACK a60f863d3e
dergoegge:
Code review ACK a60f863d3e276534444571282f432b913d3967db
maflcko:
review ACK a60f863d3e276534444571282f432b913d3967db 🎽
theStack:
Code-review ACK a60f863d3e276534444571282f432b913d3967db
Tree-SHA512: da9b73b7bdffee2eb9281a409205519ac330d3336094d17681896703fbca8099608782c9c85801e388e4d90af5af8abf1f34931f57bbbe6e9674d802d6066047
94931656b52fddb7c03b29685844d85c008cf6ca cmake: Use newer signature of `qt6_add_lrelease` when available (Hennadii Stepanov)
Pull request description:
See Qt docs here: https://doc.qt.io/qt-6/qtlinguist-cmake-qt-add-lrelease.html.
Fixes https://github.com/bitcoin/bitcoin/issues/32710.
ACKs for top commit:
fanquake:
ACK 94931656b52fddb7c03b29685844d85c008cf6ca
Tree-SHA512: bf0320306967164374499dd0be122473799e830fdff5e070ef13f87af3c14a3b799d90afb423881edd7eea17c13d27af8ced381bbb3cd149353b31b3990dde67
fa894b0f3e13dcc55fd42cec2c56d4aa2115194d log: Properly log warnings with warn loglevel in addrdb (MarcoFalke)
Pull request description:
The logging in addrdb is confusing, because it uses `LogPrintf` (info level) to log warnings.
Fix this by properly using the `warn` level, where needed. Also, drop unused trailing `\n` while touching the lines.
ACKs for top commit:
stickies-v:
ACK fa894b0f3e13dcc55fd42cec2c56d4aa2115194d
dergoegge:
utACK fa894b0f3e13dcc55fd42cec2c56d4aa2115194d
Tree-SHA512: 96d3823623ea8e1698e8cb541ca97cbab7b2a9934b2f894884171045abbca7be796f07965082e997001c97d06d1e0c4d13b29354eb4fe71c3a2ee680eada5516
- don't log function name
- take into account that GetName() always ends with " index"
- replace deprecated LogPrintf with LogInfo
- remove trailing \n
- adjusted log level where needed
fa8862723c14cdd471bbffc7a4897ece2e6d8a7f fuzz: CheckGlobals in init (MarcoFalke)
fa26bfde988b1940e6b0d21accc60fbc4e45f9b1 test: Avoid resetting mocktime in testing setup (MarcoFalke)
fa6b45fa8ec8248544d22ba8429be8f6df19024a Add SetMockTime for time_point types (MarcoFalke)
Pull request description:
(Tracking issue https://github.com/bitcoin/bitcoin/issues/29018)
During fuzzing, `AppInitParameterInteraction` may actually disable a previously set mocktime. This is confusing and can also cause non-determinism.
Fix this issue, by
* fixing the erroneous `-mocktime` parsing in `AppInitParameterInteraction`.
* adding the missing `SetMockTime` calls to the affected fuzz init functions.
* adding a `CheckGlobals` to the fuzz init, to prevent this issue in the future.
This can be tested by
* Cherry-picking the `CheckGlobals`-commit onto current master and observing a fuzz failure in the touched fuzz targets.
* Reverting the touched fuzz fixups and observing a fuzz failure for each target.
ACKs for top commit:
w0xlt:
ACK fa8862723c
dergoegge:
utACK fa8862723c14cdd471bbffc7a4897ece2e6d8a7f
Tree-SHA512: 5a9400f0467c82fa224713af4cc2b525afbefefc7c3f419077110925ad7af6c7fda3dcd2b50f7facf0ee7df2547c6ac20336906d707adcdfd1d652a9d9a735fe
Starting with version 13.x, the mingw headers will define the value of
NTDDI_VERSION, based on the value of _WIN32_WINNT, if that version is <
Windows 10. Given that libevent was undefining our _WIN32_WINNT, and
redefining it to a value < Windows 10 (0x0501), NTDDI_VERSION was also
being defined to that value, leading to functions not being exposed in
the mingw-w64 headers; see here:
9c2668ef77/mingw-w64-headers/include/iphlpapi.h (L36-L41).
Imports a commit from usptream (a14ff91254f40cf36e0fee199e26fb11260fab49).
Fixes#32707.
f43571010e3853e83a21aa4774b1c8da47b5d961 Resolve guix non-determinism with emplace_back instead of push_back (Ava Chow)
Pull request description:
For some reason, building x86_64-w64-mingw32 on x86_64 and aarch64 results in a single instruction difference which can be traced down to prevector.h:174. The ultimate caller of this is the copy constructor for a prevector that ends up being called by std::vector::push_back in walletmodel.cpp:183. By replacing the push_back with an emplace_back, somehow this non-determinism goes away.
Closes#32923
ACKs for top commit:
l0rinc:
code review ACK f43571010e3853e83a21aa4774b1c8da47b5d961
Sjors:
utACK f43571010e3853e83a21aa4774b1c8da47b5d961
maflcko:
lgtm ACK f43571010e3853e83a21aa4774b1c8da47b5d961
Tree-SHA512: 5bf0571f32cb72efc0c533e16d2704cfc3a79bcef2943f0892743572808610fb00ca8ab41223897536f8e5090bf4030735be910942de8116652d02bc3f231e2e
83bb41455715a9e05320ba791987204031626c10 test: less ambiguous error if bitcoind is missing (Sjors Provoost)
Pull request description:
Before this change, when a functional test is run without building the source, the error message suggested that previous release binaries were missing.
When no previous release version is set, make the error message more specifically about bitcoind.
To test, try this before and after:
```sh
git clean -dfx
cmake -B build
build/test/functional/mining_basic.py
cmake --build build
build/test/functional/mining_basic.py
build/test/functional/wallet_backwards_compatibility.py
test/get_previous_releases.py
build/test/functional/wallet_backwards_compatibility.py
```
ACKs for top commit:
achow101:
ACK 83bb41455715a9e05320ba791987204031626c10
janb84:
ACK 83bb41455715a9e05320ba791987204031626c10
w0xlt:
ACK 83bb414557
Tree-SHA512: c6df65019de99d6c214951cf70944c4ddca9b635c5ab60ac2c47e4589478e9c65d5e079c394ace9b470a7eaeea3c9cf68b7246dd413e802c4a1e071913a7fc32
150b5c99ca11885ef15d04139d919d734e2c211a wallet: replace `reload_wallet` with inline functionality (rkrux)
0f86da382d3fdca6fc69ff277794acbc3f1e928d wallet: remove dead code in legacy wallet migration (rkrux)
Pull request description:
A discussion on a previous [PR 32481](https://github.com/bitcoin/bitcoin/pull/32481#discussion_r2145152084) related to legacy wallet dead
code removal made me realize that checking if the legacy
wallet was loaded prior to the start of the migration is not
required ever since legacy wallets can't be loaded in the first
place. I also verified that the `load_on_start` persistent
setting can also not cause the legacy wallets to be loaded, which
further makes the case for removal of the above mentioned checks
during migration.
The current test coverage also shows these lines uncovered.
ACKs for top commit:
achow101:
ACK 150b5c99ca11885ef15d04139d919d734e2c211a
furszy:
ACK 150b5c99ca11885ef15d04139d919d734e2c211a
Tree-SHA512: 9bc7043cac1f4051228557208895e43648de3c7ffae6860c0676d1aa2db3a8ed3a09d1f9defacd96ca50bbb9699ba86652ccb0c5e55cc88be248a1fe727c13d9
d7fca5c171f450621112457ea6f7c99b2e21d354 clusterlin: add big comment explaning the relation between tests (Pieter Wuille)
b64e61d2de65026c46f70cb91e3cb2213c336be0 clusterlin: abstract try-permutations into ExhaustiveLinearize function (Pieter Wuille)
1fa55a64ed18a0a3defbed33f6bd355929f67b48 clusterlin tests: verify that chunks are minimal (Pieter Wuille)
da23ecef29b7e6a1e03de9697a6bdf6fbe944ef7 clusterlin tests: support non-empty ReadTopologicalSubset() (Pieter Wuille)
94f3e17c33e6fe63c4f791de7aacfb912059e394 clusterlin tests: compare with fuzz-provided linearizations (Pieter Wuille)
5f92ebee0d2401d0d7b5c3466d66a4b09e51ccb0 clusterlin tests: compare with fuzz-provided topological sets (Pieter Wuille)
6e37824ac390835d3d9c82d197f58e992ccc584f clusterlin tests: optimize clusterlin_simple_linearize (Pieter Wuille)
98c1c88b6f8d928b3893e2910ce0c400a6fd1928 clusterlin tests: separate testing of SimpleLinearize and Linearize (Pieter Wuille)
10e90f7aef9cff6fa63dc15adbe22e68f76aac58 clusterlin tests: make SimpleCandidateFinder always find connected (Pieter Wuille)
a38c38951e10a61af80e3bca1e1ae04de978d5c0 clusterlin tests: separate testing of Search- and SimpleCandidateFinder (Pieter Wuille)
77a432ee704b4a83d56135ed10cf2adf4b3c18af clusterlin tests: count SimpleCandidateFinder iterations better (Pieter Wuille)
Pull request description:
Part of the cluster mempool project: #30289
The current cluster linearization fuzz tests contain two tests which combine testing of production code with testing of the test code itself:
* `clusterlin_search_finder`: establishes the correctness of `SearchCandidateFinder` by comparing against both `SimpleCandidateFinder` and `ExhaustiveCandidateFinder` (which is even more simple than `SimpleCandidateFinder`). If `SimpleCandidateFinder` works correctly, then this comparison with `ExhaustiveCandidateFinder` is redundant. If it isn't, we ought to find that in a test specific to `SimpleCandidateFinder` rather than as a side-effect of testing `SearchCandidateFinder`. Split this functionality out into a new `clusterlin_simple_finder`.
* `clusterlin_linearize`: establishes the correctness of `Linearize` by comparing against both `SimpleLinearize` and literally every valid linearization for the cluster. Again, if `SimpleLinearize` works correctly, then this comparison with all valid linearizations is redundant, and if it isn't we should find it in a test for `SimpleLinearize`. Do so by splitting off that functionality into `clusterlin_simple_linearize`.
After that, a few general improvements to the affected tests are made (comparing with linearizations and subsets read from the fuzz input, plus a performance improvement).
ACKs for top commit:
marcofleon:
Re ACK d7fca5c171f450621112457ea6f7c99b2e21d354
ismaelsadeeq:
re-ACK d7fca5c171f450621112457ea6f7c99b2e21d354
monlovesmango:
ACK d7fca5c171f450621112457ea6f7c99b2e21d354
Tree-SHA512: 33cb76bd9b9547a5f3ee231fa452e928f064ad03af98e3d9e64246eb972f2b026c13e7367257ccdac1ae57982ee8ef98c907684588ecbb4bc4c82cbec160b3e8
For some reason, building x86_64-w64-mingw32 on x86_64 and aarch64
results in a single instruction difference which can be traced down to
prevector.h:174. The ultimate caller of this is the copy constructor for
a prevector that ends up being called by std::vector::push_back in
walletmodel.cpp:183. By replacing the push_back with an emplace_back,
somehow this non-determinism goes away.
When using CMake policies 3.14 and below, the `export(PACKAGE)` command
by default populates the user package registry, which is stored outside
the build tree. Setting the `CMAKE_EXPORT_NO_PACKAGE_REGISTRY` variable
disables this side effect.
In CMake 3.15 and later, this behavior is disabled by default, and the
variable has no effect.
