Ava Chow 7b66815b16
Merge bitcoin/bitcoin#30110: refactor: TxDownloadManager + fuzzing
0f4bc635854597e15ea6968767fc4e5cf5bdd790 [fuzz] txdownloadman and txdownload_impl (glozow)
699643f23a1bd0346e36bd90c83ba1b0b0a5c3fe [unit test] MempoolRejectedTx (glozow)
fa584cbe727b62853a410623b3d7c738e11cbffd [p2p] add TxDownloadOptions bool to make TxRequestTracker deterministic (glozow)
f803c8ce8dd88d9d0fd7857f63d76045b1e2bcaa [p2p] filter 1p1c for child txid in recent rejects (glozow)
5269d57e6d78e90baa0b40629f60a2d1d63e2992 [p2p] don't process orphan if in recent rejects (glozow)
2266eba43a973345351f2b0a8296523fb7de5576 [p2p] don't find 1p1cs for reconsiderable txns that are AlreadyHaveTx (glozow)
fa7027d0fc1fb2eb4148ba9741e1736f61d7e164 [refactor] add CheckIsEmpty and GetOrphanTransactions, remove access to TxDownloadMan internals (glozow)
969b07237b990b7eb6f3d24914ccc872202d8a0f [refactor] wrap {Have,Get}TxToReconsider in txdownload (glozow)
f150fb94e7dbb3c1f4fca32a0abf063943ca676d [refactor] make AlreadyHaveTx and Find1P1CPackage private to TxDownloadImpl (glozow)
1e08195135bc54f7a8b28560ae10943b1fef0d83 [refactor] move new tx logic to txdownload (glozow)
257568eab5baba07571fe2c68759e843d215d4a9 [refactor] move invalid package processing to TxDownload (glozow)
c4ce0c1218d0a3a2e9b22701f26391b8a9107196 [refactor] move invalid tx processing to TxDownload (glozow)
c6b21749ca0aea70908773d865e67511ca141ae6 [refactor] move valid tx processing to TxDownload (glozow)
a8cf3b6e845741e4b992beced564397779bfb7da [refactor] move Find1P1CPackage to txdownload (glozow)
f497414ce76a4cf44fa669e3665746cc17710fc6 [refactor] put peerman tasks at the end of ProcessInvalidTx (glozow)
6797bc42a762f431a986852fa74b1775aea8ba38 [p2p] restrict RecursiveDynamicUsage of orphans added to vExtraTxnForCompact (glozow)
798cc8f5aac9bf2111ea88d4a4c3817d34e089e2 [refactor] move Find1P1CPackage into ProcessInvalidTx (glozow)
416fbc952b209817a37e76c09fff5d17be7a72d0 [refactor] move new orphan handling to ProcessInvalidTx (glozow)
c8e67b9169bddc0bdfefa10e9cf7f9c22847e237 [refactor] move ProcessInvalidTx and ProcessValidTx definitions down (glozow)
3a41926d1b59dc9bbabc38cdc461c169426d94e7 [refactor] move notfound processing to txdownload (glozow)
042a97ce7fc672021cdb1dee62a550ef19c208fb [refactor] move tx inv/getdata handling to txdownload (glozow)
58e09f244b4bf07d31bc8dd4e939c2dc4dc74f3a [p2p] don't log tx invs when in IBD (glozow)
288865338f50d5b00758236aa4a59546a41c88c1 [refactor] rename maybe_add_extra_compact_tx to first_time_failure (glozow)
f48d36cd97e9b27dfa105c35e0fe67cba47056d1 [refactor] move peer (dis)connection logic to TxDownload (glozow)
f61d9e4b4b80842d520c490a1012044c0816679a [refactor] move AlreadyHaveTx to TxDownload (glozow)
84e4ef843db3443278d6eb70ff89fa254fcc6631 [txdownload] add read-only reference to mempool (glozow)
af918349de52e654927d50279de64f548a8b53d6 [refactor] move ValidationInterface functions to TxDownloadManager (glozow)
f6c860efb1221e1eadc3acebd6b0b885b9cc291a [doc] fix typo in m_lazy_recent_confirmed_transactions doc (glozow)
5f9004e1550f726ca9dc9a08c865fa8f2e4b92e8 [refactor] add TxDownloadManager wrapping TxOrphanage, TxRequestTracker, and bloom filters (glozow)

Pull request description:

  Part of #27463.

  This PR does 3 things:

  (1) It modularizes transaction download logic into a `TxDownloadManager`. Transaction download logic refers to the process of deciding what transactions to request, download, and validate.[1] There should be no behavior changes. Using `--color_moved=dimmed_zebra -w` may help.
  (2) It adds unit and fuzz (🪄) testing for transaction download.
  (3) It makes a few small behavioral changes:
  - Stop (debug-only) logging tx invs during IBD
  - Just like all other transactions, require orphans have RecursiveDynamicUsage < 100k before adding to vExtraTxnForCompact
  - Don't return a 1p1c that contains a parent or child in recent rejects. Don't process any orphan already in recent rejects. These cases should not happen in actual node operation; it's just to allow tighter sanity checks during fuzzing.

  There are several benefits to this interface, such as:
  - Unit test coverage and fuzzing for logic that currently isn't feasible to test as thoroughly (without lots of overhead) and/or currently only lightly tested through `assert_debug_log` (not good) in functional tests.
  - When we add more functionality (e.g. package relay messages, more robust orphan handling), the vast majority of it will be within `TxDownloadManager` instead of `PeerManager`, making it easier to review and test. See #28031 for what this looks like.
  - `PeerManager` will no longer know anything about / have access to `TxOrphanage`, `TxRequestTracker` or the rejection caches. Its primary interface with `TxDownloadManager` would be much simpler:
      - Passing on  `ValidationInterface` callbacks
      - Telling `txdownloadman` when a peer {connects, disconnects}
      - Telling `txdownloadman`when a {transaction, package} is {accepted, rejected} from mempool
      - Telling `txdownloadman` when invs, notfounds, and txs are received.
      - Getting instructions on what to download.
      - Getting instructions on what {transactions, packages, orphans} to validate.
      - Get whether a peer `HaveMoreWork` for the `ProessMessages` loop
  - (todo) Thread-safety can be handled internally.

  [1]: This module is concerned with tx *download*, not upload. It excludes transaction announcements/gossip which happens after we download/accept a transaction. Txreconciliation (erlay) is excluded from this module, as it only relates to deciding which `inv`s to send or helping the other peer decide which `inv`s to send. It is independent from this logic.

ACKs for top commit:
  achow101:
    light ACK 0f4bc635854597e15ea6968767fc4e5cf5bdd790
  theStack:
    ACK 0f4bc635854597e15ea6968767fc4e5cf5bdd790
  instagibbs:
    reACK 0f4bc635854597e15ea6968767fc4e5cf5bdd790
  naumenkogs:
    ACK 0f4bc635854597e15ea6968767fc4e5cf5bdd790

Tree-SHA512: 84ab8ef8a0fc705eb829d7f7d6885f28944aaa42b03172f256a42605677b3e783919bb900d4e3b8589f85a0c387dfbd972bcd61d252d44a88c6aaa90e4bf920f
2024-10-29 14:41:12 -04:00
..
2023-12-06 15:44:38 +01:00
2024-03-18 10:32:00 -04:00
2023-11-30 11:28:19 +01:00
2024-06-12 15:21:31 +02:00
2023-11-07 10:21:51 +09:00
2024-08-13 15:44:46 -04:00
2023-11-30 11:28:19 +01:00
2024-10-02 18:23:18 -04:00
2023-11-30 11:28:19 +01:00
2023-11-30 11:28:19 +01:00
2023-11-30 11:28:19 +01:00