1555 Commits

Author SHA1 Message Date
fanquake
2835158be0
fuzz: add cstdlib to FuzzedDataProvider
Same as https://github.com/llvm/llvm-project/pull/113951.

Avoids compile failures under clang-20 &
`D_LIBCPP_REMOVE_TRANSITIVE_INCLUDES`:
```bash
In file included from /bitcoin/src/test/fuzz/addition_overflow.cpp:5:
/bitcoin/src/test/fuzz/FuzzedDataProvider.h:209:5: error: use of undeclared identifier 'abort'
  209 |     abort();
      |     ^
/bitcoin/src/test/fuzz/FuzzedDataProvider.h:250:5: error: use of undeclared identifier 'abort'
  250 |     abort();
```

Github-Pull: bitcoin/bitcoin#31448
Rebased-From: bb7e686341e437b2e7aae887827710918c00ae0f
2024-12-11 11:41:27 -05:00
MarcoFalke
446f5d20d6
refactor: Drop deprecated space in operator""_mst
Github-Pull: #31267
Rebased-From: faf21625652fd0d4bbf9b86fd9ebedb5857505ea
2024-12-02 14:20:14 +00:00
Martin Zumsande
1d0411dc8f
addrman, refactor: introduce user-defined type for internal nId
This makes it easier to track which spots refer to an nId
(as opposed to, for example, bucket index etc. which also use int)

Co-authored-by: Pieter Wuille <pieter@wuille.net>

Github-Pull: #30568
Rebased-From: 051ba3290e30e210bfc50dea974063053313ad3e
2024-11-04 16:27:06 +00:00
Pieter Wuille
ff95cb31da
streams: remove AutoFile::Get() entirely
Co-Authored-By: David Gumberg <davidzgumberg@gmail.com>

Github-Pull: bitcoin/bitcoin#30884
Rebased-From: a240e150e837b5a95ed19765a2e8b7c5b6013f35
2024-09-16 23:10:17 -04:00
merge-script
6d546336e8
Merge bitcoin/bitcoin#30651: fuzz: remove repeated word in note
3f05a1068d10ffe0f2859cd20c5fc9bc8efa1c70 remove repeated word in note (sunerok)

Pull request description:

  Fix typo.

ACKs for top commit:
  maflcko:
    ACK 3f05a1068d10ffe0f2859cd20c5fc9bc8efa1c70
  danielabrozzoni:
    ACK 3f05a1068d10ffe0f2859cd20c5fc9bc8efa1c70

Tree-SHA512: 709d96ed18608c0ea788b4f0696abad79ab1b81c4f266487d16bbe4cfca5b99b8f7f9a58f830866db9695aa3aebcc6442098b1533d85507729af99709a53d26a
2024-08-24 18:56:24 +01:00
merge-script
6441c77e97
Merge bitcoin/bitcoin#30687: test: replace deprecated secp256k1 context flags usage
60055f1abc4b4ad5f66a2fcf2e61c65efc777036 test: replace deprecated secp256k1 context flags usage (Sebastian Falbesoner)

Pull request description:

  The flags `SECP256K1_CONTEXT_{SIGN,VERIFY}` have been marked as deprecated since libsecp256k1 version 0.2 (released in December 2022), with the recommendation to use SECP256K1_CONTEXT_NONE instead, see https://github.com/bitcoin-core/secp256k1/pull/1126 and 1988855079/CHANGELOG.md (L132). Note that in contrast to other deprecated functions/variables, these defines don't have a deprecated attribute and hence don't lead to a compiler warning (see https://github.com/bitcoin-core/secp256k1/pull/1126#discussion_r922105271), so they are not easily detected.

ACKs for top commit:
  TheCharlatan:
    ACK 60055f1abc4b4ad5f66a2fcf2e61c65efc777036
  ismaelsadeeq:
    utACK 60055f1abc4b4ad5f66a2fcf2e61c65efc777036
  tdb3:
    light CR and test ACK 60055f1abc4b4ad5f66a2fcf2e61c65efc777036

Tree-SHA512: d93cf49e018a58469620c0d2f50242141f22dabc70afb2a7cd64e416f4f55588714510ae5a877376dd1e6b6f7494261969489af4b18a1c9dff0d0dfdf93f1fa8
2024-08-24 18:53:41 +01:00
Ava Chow
60b816439e
Merge bitcoin/bitcoin#30644: fuzz: Faster utxo_snapshot fuzz target
fa899fb7aa8a14acecadd8936ad5824fa0f697ff fuzz: Speed up utxo_snapshot fuzz target (MarcoFalke)
fa386642b4dfd88f74488c288c7886494d69f4ed fuzz: Speed up utxo_snapshot by lazy re-init (MarcoFalke)
fa645c7a861ffa83a53a459263b6a620defe31f9 fuzz: Remove unused DataStream object (MarcoFalke)
fae8c73d9e4eba4603447bb52b6e3e760fbf15f8 test: Disallow fee_estimator construction in ChainTestingSetup (MarcoFalke)

Pull request description:

  Two commits to speed up unit and fuzz tests.

  Can be tested by running the fuzz target and looking at the time it took, or by looking at the flamegraph. For example:

  ```
  FUZZ=utxo_snapshot perf record -g --call-graph dwarf ./src/test/fuzz/fuzz  -runs=100
  hotspot ./perf.data

ACKs for top commit:
  TheCharlatan:
    Re-ACK fa899fb7aa8a14acecadd8936ad5824fa0f697ff
  marcofleon:
    Re ACK fa899fb7aa8a14acecadd8936ad5824fa0f697ff
  brunoerg:
    ACK fa899fb7aa8a14acecadd8936ad5824fa0f697ff

Tree-SHA512: d3a771bb12d7ef491eee61ca47325dd1cea5c20b6ad42554babf13ec98d03bef8e7786159d077e59cc7ab8112495037b0f6e55edae65b871c7cf1708687cf717
2024-08-20 22:41:53 -04:00
Sebastian Falbesoner
60055f1abc test: replace deprecated secp256k1 context flags usage
The flags SECP256K1_CONTEXT_{SIGN,VERIFY} have been deprecated since
libsecp256k1 version 0.2 (released in December 2022), with the
recommendation to use SECP256K1_CONTEXT_NONE instead.
2024-08-21 00:41:56 +02:00
MarcoFalke
fa899fb7aa
fuzz: Speed up utxo_snapshot fuzz target
This speeds up the fuzz target, which allows "valid" inputs. It does not
affect the "INVALID" fuzz target.
2024-08-20 07:54:04 +02:00
MarcoFalke
fa386642b4
fuzz: Speed up utxo_snapshot by lazy re-init
The re-init is expensive, so skip it if there is no need.

Also, add an even faster fuzz target utxo_snapshot_invalid, which does
not need any re-init at all.
2024-08-16 08:06:59 +02:00
MarcoFalke
fa645c7a86
fuzz: Remove unused DataStream object 2024-08-15 08:53:04 +02:00
sunerok
3f05a1068d remove repeated word in note 2024-08-13 15:44:46 -04:00
glozow
7583eac43c
Merge bitcoin/bitcoin#30617: net: Clarify that m_addr_local is only set once
fa6fe432075df5e0eceb1ccd85038159cc820ccc net: Clarify that m_addr_local is only set once (MarcoFalke)

Pull request description:

  The function is supposed to be only called once when the version msg arrives (a single time). Calling it twice would be an internal logic bug. However, the `LogError` in this function has many issues:

  * If the error happens in tests, as is the case for the buggy fuzz test, it will go unnoticed
  * It is dead code, unless a bug is introduced to execute it

  Fix all issues by using `Assume(!m_addr_local.IsValid())` instead. Idea taken from https://github.com/bitcoin/bitcoin/pull/30364#discussion_r1680530382

ACKs for top commit:
  achow101:
    ACK fa6fe432075df5e0eceb1ccd85038159cc820ccc
  mzumsande:
    utACK fa6fe432075df5e0eceb1ccd85038159cc820ccc
  glozow:
    ACK fa6fe432075df5e0eceb1ccd85038159cc820ccc

Tree-SHA512: 8c1e8c524768f4f36cc50110ae54ee423e057a963ff78f736f3bf92df1ce5af28e3e0149153780897944e1d5c22ddbca9dac9865d9f4d44afffa152bc8559405
2024-08-13 10:19:19 +01:00
Ava Chow
ee57737bd6
Merge bitcoin/bitcoin#28209: fuzz: a target for the block index database
86b38529d5014612c3e7bb59fdc4dad3bff2aa64 qa: a fuzz target for the block index database (Antoine Poinsot)

Pull request description:

  This introduces a small fuzz target for `CBlockTreeDB` which asserts a few invariants by using an in-memory LevelDb.

ACKs for top commit:
  achow101:
    ACK 86b38529d5014612c3e7bb59fdc4dad3bff2aa64
  TheCharlatan:
    Re-ACK 86b38529d5014612c3e7bb59fdc4dad3bff2aa64
  maflcko:
    review ACK 86b38529d5014612c3e7bb59fdc4dad3bff2aa64 🥒
  brunoerg:
    utACK 86b38529d5014612c3e7bb59fdc4dad3bff2aa64

Tree-SHA512: ab75b4ae1c7e0a4b15f8a6ceffdf509fbc79833e6ea073ecef68558d53b83663d1b30362aaa2d77c22b8890a572f5b1d4b1c5abbca483c8c8f9b1fb5b276a59a
2024-08-12 16:06:38 -04:00
glozow
257a4fda3c
Merge bitcoin/bitcoin#30616: test: remove ExtractDestination false assertion for ANCHOR script
a4f2b185732649eeea4a042cebd90d0e0e12cc92 [test]: remove `ExtractDestination` false assertion for `ANCHOR` script (ismaelsadeeq)

Pull request description:

  This PR fixes #30615

  `ExtractDestination` returns `true` when `TxoutType` is `ANCHOR` see https://github.com/bitcoin/bitcoin/issues/30615#issuecomment-2277538703

ACKs for top commit:
  maflcko:
    review ACK a4f2b185732649eeea4a042cebd90d0e0e12cc92
  instagibbs:
    ACK a4f2b185732649eeea4a042cebd90d0e0e12cc92
  theStack:
    utACK a4f2b185732649eeea4a042cebd90d0e0e12cc92
  BrandonOdiwuor:
    Code Review ACK a4f2b185732649eeea4a042cebd90d0e0e12cc92
  glozow:
    ACK a4f2b185732649eeea4a042cebd90d0e0e12cc92

Tree-SHA512: 6120494fe888acf26b252d4aadc01dc256e294ea5e4c954fd9b4694be27dc35cf0e33e3b3bcb012fb4abe1cab0b1d0d515db226fa771e791e0fe7efbcbd8834d
2024-08-12 09:29:54 +01:00
Ava Chow
9a696397e7
Merge bitcoin/bitcoin#30598: assumeutxo: Drop block height from metadata
00618e8745192d209c23e3ae873c077e58168957 assumeutxo: Drop block height from metadata (Fabian Jahr)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/30514 which has more context and shows how the issue can be reproduced. Since the value in question is removed, there is no test to add to reproduce anything.

  This is an alternative approach to #30516 with much of the [code being suggested there](https://github.com/bitcoin/bitcoin/pull/30516#discussion_r1689146902).

ACKs for top commit:
  maflcko:
    re-ACK 00618e8745192d209c23e3ae873c077e58168957 🎌
  achow101:
    ACK 00618e8745192d209c23e3ae873c077e58168957
  theStack:
    Code-review ACK 00618e8745192d209c23e3ae873c077e58168957
  ismaelsadeeq:
    Re-ACK 00618e8745192d209c23e3ae873c077e58168957
  mzumsande:
    ACK 00618e8745192d209c23e3ae873c077e58168957

Tree-SHA512: db9575247bae838ad7742a27a216faaf55bb11e022f9afdd05752bb09bbf9614717d0ad64304ff5722a16bf41d8dea888af544e4ae26dcaa528c1add0269a4a8
2024-08-09 16:20:00 -04:00
MarcoFalke
fa6fe43207
net: Clarify that m_addr_local is only set once
Includes a rename from addrLocal to m_addr_local to match the name of
its corresponding Mutex.
2024-08-09 14:05:04 +02:00
ismaelsadeeq
a4f2b18573 [test]: remove ExtractDestination false assertion for ANCHOR script 2024-08-09 11:18:45 +01:00
merge-script
24ced52744
Merge bitcoin/bitcoin#28687: C++20 std::views::reverse
2925bd537cbd8c70594e23f6c4298b7101f7f73d refactor: use c++20 std::views::reverse instead of reverse_iterator.h (stickies-v)

Pull request description:

  C++20 introduces [`std::ranges::views::reverse`](https://en.cppreference.com/w/cpp/ranges/reverse_view), which allows us to drop our own `reverse_iterator.h` implementation and also makes it easier to chain views (even though I think we currently don't use this).

ACKs for top commit:
  achow101:
    ACK 2925bd537cbd8c70594e23f6c4298b7101f7f73d
  maflcko:
    ACK 2925bd537cbd8c70594e23f6c4298b7101f7f73d 🎷

Tree-SHA512: 567666ec44af5d1beb7a271836bcc89c4c577abc77f522fcc18bc6d4de516ae9b0df766d0bfa6dd217569e6878331c2aee1d9815620860375e3510dad7fed476
2024-08-09 09:51:19 +01:00
Fabian Jahr
00618e8745
assumeutxo: Drop block height from metadata
The Snapshot format version is updated to 2 to indicate this change.

Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
2024-08-08 23:55:06 +02:00
Ava Chow
27a770b34b
Merge bitcoin/bitcoin#28280: Don't empty dbcache on prune flushes: >30% faster IBD
589db872e116779ab9cae693171ac8a8c02d9923 validation: don't erase coins cache on prune flushes (Andrew Toth)
0e8918755f725b6269ed2be5a0b46f1611233515 Add linked-list test to CCoinsViewCache::SanityCheck (Pieter Wuille)
05cf4e18758618bb493d26014d3a9c89bf28d898 coins: move Sync logic to CoinsViewCacheCursor (Andrew Toth)
7825b8b9aeceb4ff607650cdc9c49e5de9c7719f coins: pass linked list of flagged entries to BatchWrite (Andrew Toth)
a14edada8a051e280af6fedd5130be40247e2d7a test: add cache entry linked list tests (Andrew Toth)
24ce37cb867b95e86d9fd4e50858d64ee8a59abf coins: track flagged cache entries in linked list (Andrew Toth)
58b7ed156d5993b69375bb455b03bd8b17f64fa4 coins: call ClearFlags in CCoinsCacheEntry destructor (Andrew Toth)
8bd3959feaa0e71585bc5e0976f584fb06ee6d14 refactor: require self and sentinel parameters for AddFlags (Andrew Toth)
75f36d241d2a344c5c4ce2c80250bdde91b3295e refactor: add CoinsCachePair alias (Andrew Toth)
f08faeade2f99ae1de6f3c481697541cc16186c7 refactor: move flags to private uint8_t and rename to m_flags (Andrew Toth)
4e4fb4cbabcc10e723534f5f80f3e3cf09f6ca50 refactor: disallow setting flags in CCoinsCacheEntry constructors (Andrew Toth)
8737c0cefa6ec49a4d17d9bef9e5e1a7990af1ac refactor: encapsulate flags setting with AddFlags and ClearFlags (Andrew Toth)
9715d3bf1e4013476539f1523a6b54d2055c32c6 refactor: encapsulate flags get access for all other checks (Andrew Toth)
df34a94e57659c31873c26c86a8de5a032400061 refactor: encapsulate flags access for dirty and fresh checks (Andrew Toth)

Pull request description:

  Since https://github.com/bitcoin/bitcoin/pull/17487 we no longer need to clear the coins cache when syncing to disk. A warm coins cache significantly speeds up block connection, and only needs to be fully flushed when nearing the `dbcache` limit.

  For frequent pruning flushes there's no need to empty the cache and kill connect block speed. However, simply using `Sync` in place of `Flush` actually slows down a pruned full IBD with a high `dbcache` value. This is because as the cache grows, sync takes longer since every coin in the cache is scanned to check if it's dirty. For frequent prune flushes and a large cache this constant scanning starts to really slow IBD down, and just emptying the cache on every prune becomes faster.

  To fix this, we can add two pointers to each cache entry and construct a doubly linked list of dirty entries. We can then only iterate through all dirty entries on each `Sync`, and simply clear the pointers after.

  With this approach a full IBD with `dbcache=16384` and `prune=550` was 32% faster than master. For default `dbcache=450` speedup was ~9%. All benchmarks were run with `stopatheight=800000`.

  |  | prune | dbcache | time | max RSS | speedup |
  |-----------:|----------:|------------:|--------:|-------------:|--------------:|
  | master | 550 | 16384 | 8:52:57 | 2,417,464k | - |
  | branch | 550 | 16384 | 6:01:00 | 16,216,736k | 32% |
  | branch | 550 | 450 | 8:05:08 | 2,818,072k | 8.8% |
  | master | 10000 | 5000 | 8:19:59 | 2,962,752k | - |
  | branch | 10000 | 5000| 5:56:39 | 6,179,764k | 28.8% |
  | master | 0 | 16384 | 4:51:53 | 14,726,408k | - |
  | branch | 0 | 16384 | 4:43:11 | 16,526,348k | 2.7% |
  | master | 0 | 450 | 7:08:07 | 3,005,892k | - |
  | branch | 0 | 450 | 6:57:24 | 3,013,556k |2.6%|

  While the 2 pointers add memory to each cache entry, it did not slow down IBD. For non-pruned IBD results were similar for this branch and master. When I performed the initial IBD, the full UTXO set could be held in memory when using the max `dbcache` value. For non-pruned IBD with max `dbcache` to tip ended up using 12% more memory, but it was also 2.7% faster somehow. For smaller `dbcache` values the `dbcache` limit is respected so does not consume more memory, and the potentially more frequent flushes were not significant enough to cause any slowdown.

  For reviewers, the commits in order do the following:
  First 4 commits encapsulate all accesses to `flags` on cache entries, and then the 5th makes `flags` private.
  Commits `refactor: add CoinsCachePair alias` to `coins: call ClearFlags in CCoinsCacheEntry destructor` create the linked list head nodes and cache entry self references and pass them into `AddFlags`.
  Commit `coins: track flagged cache entries in linked list` actually adds the entries into a linked list when they are flagged DIRTY or FRESH and removes them from the linked list when they are destroyed or the flags are cleared manually. However, the linked list is not yet used anywhere.
  Commit `test: add cache entry linked list tests` adds unit tests for the linked list.
  Commit `coins: pass linked list of flagged entries to BatchWrite` uses the linked list to iterate through DIRTY entries instead of using the entire coins cache.
  Commit `validation: don't erase coins cache on prune flushes` uses `Sync` instead of `Flush` for pruning flushes, so the cache is no longer cleared.

  Inspired by [this comment](https://github.com/bitcoin/bitcoin/pull/15265#issuecomment-457720636).

  Fixes https://github.com/bitcoin/bitcoin/issues/11315.

ACKs for top commit:
  paplorinc:
    ACK 589db872e116779ab9cae693171ac8a8c02d9923
  sipa:
    reACK 589db872e116779ab9cae693171ac8a8c02d9923
  achow101:
    ACK 589db872e116779ab9cae693171ac8a8c02d9923
  mzumsande:
    re-ACK 589db872e116779ab9cae693171ac8a8c02d9923

Tree-SHA512: 23b2bc01c83edacb5b39aa60bb0b766de9a74ce17f0c59bf13b97b4328a7b758ad9aff6581c3ca88e2973f7658380651530d497444f48d6e22ea0bfc51cc921d
2024-08-07 20:06:39 -04:00
Ryan Ofsky
870447fd58
Merge bitcoin/bitcoin#30212: rename TransactionError:ALREADY_IN_CHAIN
e9de0a76b99fd4708295e1178f6c079db92e7f36 doc: release note for 30212 (willcl-ark)
87b188052528e97729a85d9701864eaff1ea5ec6 rpc: clarify ALREADY_IN_CHAIN rpc errors (willcl-ark)

Pull request description:

  Closes: #19363

  Renaming this error improves clarity around the returned error both internally and externally when a transactions' outputs are already found in the utxo set (`TransactionError::ALREADY_IN_CHAIN -> TransactionError::ALREADY_IN_UTXO_SET`)

ACKs for top commit:
  tdb3:
    ACK e9de0a76b99fd4708295e1178f6c079db92e7f36
  ismaelsadeeq:
    ACK e9de0a76b99fd4708295e1178f6c079db92e7f36
  ryanofsky:
    Code review ACK e9de0a76b99fd4708295e1178f6c079db92e7f36.

Tree-SHA512: 7d2617200909790340951fe56a241448f9ce511900777cb2a712e8b9c0778a27d1f912b460f82335844224f1abb4322bc898ca076440959edade55c082a09237
2024-08-06 11:31:03 -04:00
josibake
59c0ece0a7
fuzz: replace hardcoded numbers for bech32 limits
Use bech32::CharLimit::BECH32 and bech32::CHECKSUM_SIZE instead of
hardcoded values. This is a follow-up fix for #34007
(where this file was missed)
2024-08-06 11:03:31 +02:00
Andrew Toth
7825b8b9ae
coins: pass linked list of flagged entries to BatchWrite
BatchWrite now iterates through the linked
list of flagged entries instead of the entire
coinsCache map.

Co-Authored-By: l0rinc <pap.lorinc@gmail.com>
2024-08-05 19:43:56 -04:00
stickies-v
2925bd537c
refactor: use c++20 std::views::reverse instead of reverse_iterator.h
Use std::ranges::views::reverse instead of the implementation in
reverse_iterator.h, and remove it as it is no longer used.
2024-08-06 00:23:38 +01:00
willcl-ark
87b1880525
rpc: clarify ALREADY_IN_CHAIN rpc errors
When using `sendrawtransaction` the ALREADY_IN_CHAIN error help string
may be confusing.

Rename TransactionError::ALREADY_IN_CHAIN to
TransactionError::ALREADY_IN_UTXO_SET and update the rpc help string.

Remove backwards compatibility alias as no longer required.
2024-08-05 15:45:58 +01:00
Hodlinator
c06f2368e2
refactor: Hand-replace some uint256S -> uint256
chainparams.cpp - workaround for MSVC bug triggering C7595 - Calling consteval constructors in initializer lists fails, but works on GCC (13.2.0) & Clang (17.0.6).
2024-08-05 14:51:47 +02:00
glozow
1a19a4d960
Merge bitcoin/bitcoin#29656: chainparams: Change nChainTx type to uint64_t
bf0efb4fc72d3c49a2c498c944e55466dfa046dc scripted-diff: Modernize naming of nChainTx and nTxCount (Fabian Jahr)
72e5d1be1f4491565249d43e836ee42cfd858866 test: Add basic check for nChainTx type (Fabian Jahr)
dc2938e9799d79696d1db2438ef33d90542d984b chainparams: Change nChainTx to uint64_t (Fabian Jahr)

Pull request description:

  This picks up the work from #29331 and closes #29258.

  This simply changes the type and addresses the comments from #29331 by changing the type in all relevant places and removing unnecessary casts. This also adds an extremely simple unit test.

  Additionally this modernizes the name of `nChainTx` which helps reviewers check all use of the symbol and can make silent merge conflicts.

ACKs for top commit:
  maflcko:
    only rebase in scripted-diff, re-ACK bf0efb4fc72d3c49a2c498c944e55466dfa046dc 🔈
  glozow:
    reACK bf0efb4fc72 via range-diff

Tree-SHA512: ee4020926d0800236fe655d0c7b127215ab36b553b04d5f91494f4b7fac6e1cfe7ee298b07c0983db5a3f4786932acaa54f5fd2ccd45f2fcdcfa13427358dc3b
2024-08-05 10:00:25 +01:00
glozow
bba01ba18d
Merge bitcoin/bitcoin#30285: cluster mempool: merging & postprocessing of linearizations
bbcee5a0d67db46526ba29a1a4a7c590d303de03 clusterlin: improve rechunking in LinearizationChunking (optimization) (Pieter Wuille)
04d7a04ea426dd0a69b61e3b887867b0277d84d1 clusterlin: add MergeLinearizations function + fuzz test + benchmark (Pieter Wuille)
4f8958d7563ae2d0d359ec1e6885f8cb5e40a5e0 clusterlin: add PostLinearize + benchmarks + fuzz tests (Pieter Wuille)
0e2812d2938b933debffba5b873637fa1d348b81 clusterlin: add algorithms for connectedness/connected components (Pieter Wuille)
0e52728a2d6ccafcfecfefbb5a0432a9881d8e0d clusterlin: rename Intersect -> IntersectPrefixes (Pieter Wuille)

Pull request description:

  Part of cluster mempool: #30289

  Depends on #30126, and was split off from it. #28676 depends on this.

  This adds the algorithms for merging & postprocessing linearizations.

  The `PostLinearize(depgraph, linearization)` function performs an in-place improvement of `linearization`, using two iterations of the [Linearization post-processing](https://delvingbitcoin.org/t/linearization-post-processing-o-n-2-fancy-chunking/201/8) algorithm. The first running from back to front, the second from front to back.

  The `MergeLinearizations(depgraph, linearization1, linearization2)` function computes a new linearization for the provided cluster, given two existing linearizations for that cluster, which is at least as good as both inputs. The algorithm is described at a high level in [merging incomparable linearizations](https://delvingbitcoin.org/t/merging-incomparable-linearizations/209).

  For background and references, see [Introduction to cluster linearization](https://delvingbitcoin.org/t/introduction-to-cluster-linearization/1032).

ACKs for top commit:
  sdaftuar:
    ACK bbcee5a0d67db46526ba29a1a4a7c590d303de03
  glozow:
    code review ACK bbcee5a0d67
  instagibbs:
    ACK bbcee5a0d6

Tree-SHA512: d2b5a3f132d1ef22ddf9c56421ab8b397efe45b3c4c705548dda56f5b39fe4b8f57a0d2a4c65b338462d80bb5b9b84a9a39efa1b4f390420a8005ce31817774e
2024-08-05 09:42:22 +01:00
Fabian Jahr
bf0efb4fc7
scripted-diff: Modernize naming of nChainTx and nTxCount
-BEGIN VERIFY SCRIPT-
sed -i 's/nChainTx/m_chain_tx_count/g' $(git grep -l 'nChainTx' ./src)
sed -i 's/nTxCount/tx_count/g' $(git grep -l 'nTxCount' ./src)
-END VERIFY SCRIPT-
2024-08-04 14:24:43 +02:00
glozow
2aff9a36c3
Merge bitcoin/bitcoin#30352: policy: Add PayToAnchor(P2A), OP_1 <0x4e73> as a standard output script for spending
75648cea5a9032b3d388cbebacb94d908e08924e test: add P2A ProduceSignature coverage (Greg Sanders)
7998ce6b20fba62c022228355907b612ba6692e1 Add release note for P2A output feature (Greg Sanders)
71c9b02a04742eeecab14aae4697b1a3eb51ff7f test: add P2A coverage for decodescript (Greg Sanders)
1349e9ec1558484f2912a2444c410170fcec8745 test: Add anchor mempool acceptance test (Greg Sanders)
9d892099378b2ad5f52220403bdeae43c61d6955 policy: stop 3rd party wtxid malleability of anchor spend (Greg Sanders)
b60aaf8b239978947d2b0e3f56e7d8a4092d7570 policy: make anchor spend standard (Greg Sanders)
455fca86cfada1823aa28615b5683f9dc73dbb9a policy: Add OP_1 <0x4e73> as a standard output type (Greg Sanders)

Pull request description:

  This is a sub-feature taken out of the original proposal for ephemeral anchors #30239

  This PR makes *spending* of `OP_1 <0x4e73>` (i.e. `bc1pfeessrawgf`) standard. Creation of this output type is already standard.

  Any future witness output types are considered relay-standard to create, but not to spend. This preserves upgrade hooks, such as a completely new output type for a softfork such as BIP341.  It also gives us a bit of room to use a new output type for policy uses.

  This particular sized witness program has no other known use-cases (https://bitcoin.stackexchange.com/a/110664/17078), s it affords insufficient cryptographic security for a secure commitment to data, such as a script or a public key. This makes this type of output "keyless", or unauthenticated.

  As a witness program, the `scriptSig` of the input MUST be blank, by BIP141. This helps ensure txid-stability of the spending transaction, which may be required for smart contracting wallets. If we do not use segwit, a miner can simply insert an `OP_NOP` in the `scriptSig` without effecting the result of program execution.

  An additional relay restriction is to disallow non-empty witness data, which an adversary may use to penalize the "honest" transactor when RBF'ing the transaction due to the incremental fee requirement of RBF rules.

  The intended use-case for this output type is to "anchor" the transaction with a spending child to bring exogenous CPFP fees into the transaction package, encouraging the inclusion of the package in a block. The minimal size of creation and spending of this output makes it an attractive contrast to outputs like `p2sh(OP_TRUE)` and `p2wsh(OP_TRUE)` which
  are significantly larger in vbyte terms.

  Combined with TRUC transactions which limits the size of child transactions significantly, this is an attractive option for presigned transactions that need to be fee-bumped after the fact.

ACKs for top commit:
  sdaftuar:
    utACK 75648cea5a9032b3d388cbebacb94d908e08924e
  theStack:
    re-ACK 75648cea5a9032b3d388cbebacb94d908e08924e
  ismaelsadeeq:
    re-ACK 75648cea5a9032b3d388cbebacb94d908e08924e via [diff](e7ce6dc070..75648cea5a)
  glozow:
    ACK 75648cea5a9032b3d388cbebacb94d908e08924e
  tdb3:
    ACK 75648cea5a9032b3d388cbebacb94d908e08924e

Tree-SHA512: d529de23d20857e6cdb40fa611d0446b49989eaafed06c28280e8fd1897f1ed8d89a4eabbec1bbf8df3d319910066c3dbbba5a70a87ff0b2967d5205db32ad1e
2024-08-02 15:49:44 +01:00
Andrew Toth
8bd3959fea
refactor: require self and sentinel parameters for AddFlags
No behavior change. Prepares for adding the CoinsCachePairs to
a linked list when flagged.
2024-08-02 00:29:42 -04:00
Andrew Toth
f08faeade2
refactor: move flags to private uint8_t and rename to m_flags
No behavior change. This prepares to add CCoinsCacheEntrys
to a doubly linked list when a flag is added.
2024-08-01 23:36:00 -04:00
Andrew Toth
8737c0cefa
refactor: encapsulate flags setting with AddFlags and ClearFlags
No behavior change. This prepares moving the cache entry
flags field to private access.
2024-08-01 23:36:00 -04:00
Andrew Toth
df34a94e57
refactor: encapsulate flags access for dirty and fresh checks
No behavior change. This prepares moving the cache entry flags field
to private access.

Co-Authored-By: l0rinc <pap.lorinc@gmail.com>
2024-08-01 23:36:00 -04:00
Pieter Wuille
04d7a04ea4 clusterlin: add MergeLinearizations function + fuzz test + benchmark 2024-08-01 16:03:34 -04:00
Pieter Wuille
4f8958d756 clusterlin: add PostLinearize + benchmarks + fuzz tests 2024-08-01 16:02:09 -04:00
Pieter Wuille
0e2812d293 clusterlin: add algorithms for connectedness/connected components
Add utility functions to DepGraph for finding connected components.
2024-08-01 15:43:59 -04:00
Pieter Wuille
0e52728a2d clusterlin: rename Intersect -> IntersectPrefixes
This makes it clearer what the function does.
2024-08-01 14:07:54 -04:00
glozow
ebd82fa9fa
Merge bitcoin/bitcoin#30532: refactor: remove deprecated TxidFromString() in favour of transaction_identifier::FromHex()
f553e6d86fe114e90585ea6d4b8e345a209cae5d refactor: remove TxidFromString (stickies-v)
285ab50ace4c04209f331ccaf121152b977cc490 test: replace WtxidFromString with Wtxid::FromHex (stickies-v)
9a0b2a69c4d836d7382f6fe7703de40288fc7421 fuzz: increase FromHex() coverage (stickies-v)
526a87ba6b4f20592ad3c090b82e83ecff2107fc test: add uint256::FromHex unittest coverage (stickies-v)

Pull request description:

  Since fab6ddbee6, `TxidFromString()` has been deprecated because it is less robust than the `transaction_identifier::FromHex()` introduced in [the same PR](https://github.com/bitcoin/bitcoin/pull/30482). Specifically, it tries to recover from length-mismatches, recover from untrimmed whitespace, 0x-prefix and garbage at the end, instead of simply requiring exactly 64 hex-only characters.

  In this PR, `TxidFromString` is removed completely to clean up the code and prevent further unsafe usage. Unit and fuzz test coverage on `uint256::FromHex()` and functions that wrap it is increased.

  Note: `TxidFromSring` allowed users to prefix strings with "0x", this is no longer allowed for `transaction_identifier::FromHex()`, so a helper function for input validation may prove helpful in the future _(this overlaps with the `uint256::uint256S()` vs `uint256::FromHex()` future cleanup)_. It is not relevant to this PR, though, besides the fact that this unused (except for in tests) functionality is removed.

  The only users of `TxidFromString` are:
  - `test`, where it is straightforward to drop in the new `FromHex()` methods without much further concern
  - `qt` coincontrol. There is no need for input validation here, but txids are not guaranteed to be 64 characters. This is already handled by the existing code, so again, using `FromHex()` here seems quite straightforward.

  Addresses @maflcko's suggestion: https://github.com/bitcoin/bitcoin/pull/30482#discussion_r1691826934

  Also removes `WtxidFromString()`, which is a test-only helper function.

  ### Testing GUI changes

  To test the GUI coincontrol affected lines, `regtest` is probably the easiest way to quickly get some test coins, you can use e.g.

  ```
  alias cli="./src/bitcoin-cli -regtest"
  cli createwallet "coincontrol"
  # generate 10 spendable outputs on 1 address
  cli generatetoaddress 10 $(cli -rpcwallet=coincontrol getnewaddress)
  # generate 10 spendable outputs on another address
  cli generatetoaddress 10 $(cli -rpcwallet=coincontrol getnewaddress)
  # make previous outputs spendable
  cli generatetoaddress 100 $(cli -rpcwallet=coincontrol getnewaddress)
  ```

ACKs for top commit:
  maflcko:
    re-ACK f553e6d86fe114e90585ea6d4b8e345a209cae5d 🔻
  hodlinator:
    ACK f553e6d86fe114e90585ea6d4b8e345a209cae5d
  paplorinc:
    ACK f553e6d86fe114e90585ea6d4b8e345a209cae5d
  TheCharlatan:
    Nice, ACK f553e6d86fe114e90585ea6d4b8e345a209cae5d

Tree-SHA512: c1c7e6ea4cbf05cf660ba178ffc4f35f0328f7aa6ad81872e2462fb91a6a22e4681ff64b3d0202a5a9abcb650c939561585cd309164a69ab6081c0765ee271ef
2024-08-01 12:02:52 +01:00
stickies-v
9a0b2a69c4
fuzz: increase FromHex() coverage 2024-07-31 16:47:38 +01:00
dergoegge
afd237bb5d [fuzz] Harness for version handshake 2024-07-31 13:25:52 +01:00
Greg Sanders
455fca86cf policy: Add OP_1 <0x4e73> as a standard output type
These outputs are called anchors, and allow
key-less anchor spends which are vsize-minimized
versus keyed anchors which require larger outputs
when creating and inputs when spending.
2024-07-30 14:06:58 -04:00
Ryan Ofsky
123888dcb8
Merge bitcoin/bitcoin#30447: fuzz: Deglobalize signature cache in sigcache test
fae0db0360466aed536f4ce96d357cf579100080 fuzz: Deglobalize signature cache in sigcache test (TheCharlatan)

Pull request description:

  The body of the fuzz test should ideally be a pure function. If data is persisted in the cache over many iterations, and there is a crash, reproducing it from the input might be difficult. Solve this by getting rid of the global state. This is a follow-up from #30425.

ACKs for top commit:
  dergoegge:
    utACK fae0db0360466aed536f4ce96d357cf579100080
  ryanofsky:
    Code review ACK fae0db0360466aed536f4ce96d357cf579100080

Tree-SHA512: 93dcbb9f2497f13856970469042d6870f04de10fe206827a8db1aae7fc8f3ac7fd900bee7945b5fe4c9e33883268dabb15be7e7bc91cf353ffc0d118cd60e97d
2024-07-26 07:41:10 -04:00
Pieter Wuille
28549791b3 clusterlin: permit passing in existing linearization to Linearize
This implements the LIMO algorithm for linearizing by improving an existing
linearization. See
https://delvingbitcoin.org/t/limo-combining-the-best-parts-of-linearization-search-and-merging
for details.
2024-07-25 10:16:40 -04:00
Pieter Wuille
97d98718b0 clusterlin: add LinearizationChunking class
It encapsulates a given linearization in chunked form, permitting arbitrary
subsets of transactions to be removed from the linearization. Its purpose
is adding the Intersect function, which is a crucial operation that will
be used in a further commit to make Linearize improve existing linearizations.
2024-07-25 10:16:40 -04:00
Pieter Wuille
d5918dc3c6 clusterlin: randomize the SearchCandidateFinder search order
To make search non-deterministic, change the BFS logic from always picking
the first queue item to randomly picking the first or second queue item.
2024-07-25 10:16:40 -04:00
Pieter Wuille
46aad9b099 clusterlin: add Linearize function
This adds a first version of the overall linearization interface, which given
a DepGraph constructs a good linearization, by incrementally including good
candidate sets (found using AncestorCandidateFinder and SearchCandidateFinder).
2024-07-25 10:16:37 -04:00
Pieter Wuille
ee0ddfe4f6 clusterlin: add chunking algorithm
A fuzz test is added which verifies various of its expected properties, including
correctness
2024-07-25 10:16:37 -04:00
Pieter Wuille
2a41f151af clusterlin: add SearchCandidateFinder class
Similar to AncestorCandidateFinder, this encapsulates the state needed for
finding good candidate sets using a search algorithm.
2024-07-25 10:16:37 -04:00