19425 Commits

Author SHA1 Message Date
MarcoFalke
a273e3c58a
Merge bitcoin/bitcoin#21934: RPC/blockchain: getblockchaininfo: Include versionbits signalling details during LOCKED_IN
2b19f3443efc9e7868746ea1c603b1027d822f32 RPC/blockchain: getblockchaininfo: Include versionbits signalling details during LOCKED_IN (Luke Dashjr)

Pull request description:

  While the signal has no effect during `LOCKED_IN`, the bit is still defined and recommended for measuring uptake. Makes sense to expose statistics too.

ACKs for top commit:
  prayank23:
    ACK 2b19f3443e
  Sjors:
    tACK 2b19f34
  theStack:
    Tested ACK 2b19f3443efc9e7868746ea1c603b1027d822f32
  MarcoFalke:
    review-only ACK 2b19f3443efc9e7868746ea1c603b1027d822f32

Tree-SHA512: a9bb5adb21992586119cbb5f87e5348eabcab11d5a3bf769b00b69e466589a669846e503f8384fa8927fd77da0c2d64a54f13a7a55a62980046d70f8255ddf47
2021-07-21 09:50:21 +02:00
MarcoFalke
458d6ac23b
Merge bitcoin/bitcoin#22407: rpc: Return block time in getblockchaininfo
20edf4bcf61e9fa310c3d7f3cac0c80a04df5364 rpc: Return block time in getblockchaininfo (João Barbosa)

Pull request description:

  Return tip time in `getblockchaininfo`, for some use cases this can save a call to `getblock`.

ACKs for top commit:
  naumenkogs:
    ACK 20edf4bcf61e9fa310c3d7f3cac0c80a04df5364
  theStack:
    re-ACK 20edf4bcf61e9fa310c3d7f3cac0c80a04df5364
  0xB10C:
    ACK 20edf4bcf61e9fa310c3d7f3cac0c80a04df5364
  kristapsk:
    ACK 20edf4bcf61e9fa310c3d7f3cac0c80a04df5364
  Zero-1729:
    re-ACK 20edf4bcf61e9fa310c3d7f3cac0c80a04df5364

Tree-SHA512: 29a920cfff1ef53e0af601c3f93f8f9171f3be47fc84b0fa293cb865b824976e8c1510b17b27d17daf0b8e658dd77d9dc388373395f0919fc4a23cd5019642d5
2021-07-21 09:47:35 +02:00
fanquake
0fffd6c4fb
Merge bitcoin/bitcoin#22505: addrman: Remove unused test_before_evict argument from Good()
f036dfbb692c4d44d0f59194d089ed0aa1096347 [addrman] Remove unused test_before_evict argument from Good() (John Newbery)

Pull request description:

  This has never been used in the public interface method since it was
  introduced in #9037.

ACKs for top commit:
  lsilva01:
    Tested ACK f036dfbb69 on Ubuntu 20.04.
  theStack:
    Code-review ACK f036dfbb692c4d44d0f59194d089ed0aa1096347

Tree-SHA512: 98145d9596b4ae1f354cfa561be1a54c6b8057c920e0ac3d4c1d42c9326b2dad2d44320f4171bb701d97088b216760cca8017b84c8b5dcd2b1dc8f158f28066d
2021-07-21 12:32:44 +08:00
MarcoFalke
951850bebf
Merge bitcoin/bitcoin#22371: Move pblocktree global to BlockManager
faa54e375782b21cbc2761c763128131c569e903 Move pblocktree global to BlockManager (MarcoFalke)
fa27f03b4943540aa2eab283d4cf50ad4a1a01f8 Move LoadBlockIndexDB to BlockManager (MarcoFalke)

Pull request description:

  The block tree db is used within BlockManager to write and read the block index, so make the db global a member variable of BlockManager.

ACKs for top commit:
  jamesob:
    ACK faa54e375782b21cbc2761c763128131c569e903 ([`jamesob/ackr/22371.1.MarcoFalke.move_pblocktree_global_t`](https://github.com/jamesob/bitcoin/tree/ackr/22371.1.MarcoFalke.move_pblocktree_global_t))
  theStack:
    re-ACK faa54e375782b21cbc2761c763128131c569e903 🥧
  ryanofsky:
    Code review ACK faa54e375782b21cbc2761c763128131c569e903. I was thinking this looked like a change Carl would like, so no surprised he [Mega-acked](https://github.com/bitcoin/bitcoin/pull/22371#pullrequestreview-696450475)

Tree-SHA512: 1b7badbf503d53f5d4dbd9ed8f2e5c1ebfe48102665197048cc9e37bc87b5cec5f2277f3aae9f73a1095bfe879b19d288286ca3daa28031f5f1b64b1184439a9
2021-07-20 17:37:29 +02:00
John Newbery
f036dfbb69 [addrman] Remove unused test_before_evict argument from Good()
This has never been used in the public interface method since it was
introduced in #9037.
2021-07-20 16:17:51 +01:00
MarcoFalke
9faa4b68db
Merge bitcoin/bitcoin#22232: refactor: Pass interpreter flags as uint32_t instead of signed int
fa621ededdfe31a200b77a8787de7e3d2e667aec refactor: Pass script verify flags as uint32_t (MarcoFalke)

Pull request description:

  The flags are cast to unsigned in the interpreter anyway, so avoid the confusion (and fuzz crashes) by just passing them as unsigned from the beginning.

  Also, the flags are often inverted bit-wise with the `~` operator, which also works on signed integers, but might cause confusion as the sign bit is flipped.

  Fixes #22233

ACKs for top commit:
  theStack:
    Concept and code review ACK fa621ededdfe31a200b77a8787de7e3d2e667aec
  kristapsk:
    ACK fa621ededdfe31a200b77a8787de7e3d2e667aec
  jonatack:
    ACK fa621ededdfe31a200b77a8787de7e3d2e667aec

Tree-SHA512: ea0720f32f823fa7f075309978672aa39773c6019d12b6c1c9d611fc1983a76115b7fe2a28d50814673bb6415c311ccc05b99d6e871575fb6900faf75ed17769
2021-07-20 15:36:23 +02:00
fanquake
42af9596ce
Merge bitcoin/bitcoin#22499: Update assumed chain params
eeddd1c8fa96cf546b0bf92063cefa4fd8c6b415 Update assumed chain params (Sriram)

Pull request description:

  Update the relevant variables in `src/chainparams.cpp` for `mainnet`, `testnet`, and `signet` as given [here](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#before-branch-off).

  To review this PR, check out [this guide](https://github.com/fanquake/core-review/blob/master/update-assumevalid.md).

  Note: added a 10% overhead to the base value of `mainnet` in `m_assumed_blockchain_size`

ACKs for top commit:
  MarcoFalke:
    ACK eeddd1c8fa96cf546b0bf92063cefa4fd8c6b415, checked against my node 🌮
  bfolkens:
    ACK eeddd1c - checked against `mainnet`
  achow101:
    Code Review ACK eeddd1c8fa96cf546b0bf92063cefa4fd8c6b415
  0xB10C:
    ACK mainnet, testnet, and signet eeddd1c8fa96cf546b0bf92063cefa4fd8c6b415
  jamesob:
    ACK eeddd1c8fa96cf546b0bf92063cefa4fd8c6b415 ([`jamesob/ackr/22499.1.sriramdvt.update_assumed_chain_par`](https://github.com/jamesob/bitcoin/tree/ackr/22499.1.sriramdvt.update_assumed_chain_par))
  darosior:
    ACK eeddd1c8fa96cf546b0bf92063cefa4fd8c6b415 mainnet and testnet

Tree-SHA512: 0ab19d2acc6a854c6aa38fba199d61c68cec40f005d1d54341ea32b59aae9b7d1aabfd21d7c0bc79f54be99d3e71d1d727196cab88f370259fd2c6e002d3e43c
2021-07-20 21:09:58 +08:00
MarcoFalke
539023ab41
Merge bitcoin/bitcoin#22492: wallet: Reorder locks in dumpwallet to avoid lock order assertion
9b85a5e2f7e003ca8621feaac9bdd304d19081b4 tests: Test for dumpwallet lock order issue (Andrew Chow)
25d99e6511d8c43b2025a89bcd8295de755346a7 Reorder dumpwallet so that cs_main functions go first (Andrew Chow)

Pull request description:

  When a wallet is loaded which has an unconfirmed transaction in the mempool, it will end up establishing the lock order of cs_wallet -> cs_main -> cs_KeyStore. If `dumpwallet` is used on this wallet, then a lock order of cs_wallet -> cs_KeyStore -> cs_main will be used, which causes a lock order assertion. This PR fixes this by reordering `dumpwallet` and `GetKeyBirthTimes` (only used by `dumpwallet`). Specifically, in both functions, the function calls which lock cs_main are done prior to locking cs_KeyStore. This avoids the lock order issue.

  Additionally, I have added a test case to `wallet_dump.py`. Of course testing this requires `--enable-debug`.

  Fixes #22489

ACKs for top commit:
  MarcoFalke:
    review ACK 9b85a5e2f7e003ca8621feaac9bdd304d19081b4 🎰
  ryanofsky:
    Code review ACK 9b85a5e2f7e003ca8621feaac9bdd304d19081b4. Nice to reduce lock scope, and good test!
  prayank23:
    tACK 9b85a5e2f7
  lsilva01:
    Tested ACK 9b85a5e2f7 under the same conditions reported in issue #22489 and the `dumpwallet` command completed successfully.

Tree-SHA512: d370a8f415ad64ee6a538ff419155837bcdbb167e3831b06572562289239028c6b46d80b23d227286afe875d9351f3377574ed831549ea426fb926af0e19c755
2021-07-20 15:04:07 +02:00
fanquake
8ed8164e6f
Merge bitcoin/bitcoin#22261: [p2p/mempool] Two small fixes to node broadcast logic
5a77abd4e657458852875a07692898982f4b1db5 [style] Clean up BroadcastTransaction() (John Newbery)
7282d4c0363ab5152baa34af626cb49afbfddc32 [test] Allow rebroadcast for same-txid-different-wtxid transactions (glozow)
cd48372b67d961fe661990a2c6d3cc3d91478924 [mempool] Allow rebroadcast for same-txid-different-wtxid transactions (John Newbery)
847b6ed48d7bacec9024618922e9b339d2d97676 [test] Test transactions are not re-added to unbroadcast set (Duncan Dean)
2837a9f1eaa2c6bf402d1d9891d9aa84c4a56033 [mempool] Only add a transaction to the unbroadcast set when it's added to the mempool (John Newbery)

Pull request description:

  1. Only add a transaction to the unbroadcast set when it's added to the mempool

      Currently, if BroadcastTransaction() is called to rebroadcast a
      transaction (e.g. by ResendWalletTransactions()), then we add the
      transaction to the unbroadcast set. That transaction has already been
      broadcast in the past, so peers are unlikely to request it again,
      meaning RemoveUnbroadcastTx() won't be called and it won't be removed
      from m_unbroadcast_txids.

      Net processing will therefore continue to attempt rebroadcast for the
      transaction every 10-15 minutes. This will most likely continue until
      the node connects to a new peer which hasn't yet seen the transaction
      (or perhaps indefinitely).

      Fix by only adding the transaction to the broadcast set when it's added to the mempool.

  2. Allow rebroadcast for same-txid-different-wtxid transactions

      There is some slightly unexpected behaviour when:

      - there is already transaction in the mempool (the "mempool tx")
      - BroadcastTransaction() is called for a transaction with the same txid
        as the mempool transaction but a different witness (the "new tx")

      Prior to this commit, if BroadcastTransaction() is called with
      relay=true, then it'll call RelayTransaction() using the txid/wtxid of
      the new tx, not the txid/wtxid of the mempool tx. For wtxid relay peers,
      in SendMessages(), the wtxid of the new tx will be taken from
      setInventoryTxToSend, but will then be filtered out from the vector of
      wtxids to announce, since m_mempool.info() won't find the transaction
      (the mempool contains the mempool tx, which has a different wtxid from
      the new tx).

      Fix this by calling RelayTransaction() with the wtxid of the mempool
      transaction in this case.

  The third commit is a comment/whitespace only change to tidy up the BroadcastTransaction() function.

ACKs for top commit:
  duncandean:
    reACK 5a77abd
  naumenkogs:
    ACK 5a77abd4e657458852875a07692898982f4b1db5
  theStack:
    re-ACK 5a77abd4e657458852875a07692898982f4b1db5
  lsilva01:
    re-ACK 5a77abd4e6

Tree-SHA512: d1a46d32a9f975220e5b432ff6633fac9be01ea41925b4958395b8d641680500dc44476b12d18852e5b674d2d87e4d0160b4483e45d3d149176bdff9f4dc8516
2021-07-20 20:57:58 +08:00
fanquake
e4487fd5bb
Merge bitcoin/bitcoin#22096: p2p: AddrFetch - don't disconnect on self-announcements
5730a43703f7e5a5ca26245ba3b55fbdd027d0b6 test: Add functional test for AddrFetch connections (Martin Zumsande)
c34ad3309f93979b274a37de013502b05d25fad8 net, rpc: Enable AddrFetch connections for functional testing (Martin Zumsande)
533500d9072b7d5a36a6491784bdeb9247e91fb0 p2p: Add timeout for AddrFetch peers (Martin Zumsande)
b6c5d1e450dde6a54bd785504c923adfb45c7060 p2p: AddrFetch - don't disconnect on self-announcements (Martin Zumsande)

Pull request description:

  AddrFetch connections (old name: oneshots) are intended to be short-lived connections on which we ask a peer for addresses via `getaddr` and disconnect after receiving them.

  This is done by disconnecting after receiving the first `addr`. However, it is no longer working as intended, because nowadays, the first `addr` a typical bitcoin core node sends is its self-announcement.
  So we'll disconnect before the peer gets a chance to answer our `getaddr`.

  I checked that this affects both `-seednode` peers specified manually, and DNS seeds when AddrFetch is used as a fallback if DNS doesn't work for us.

  The current behavior of getting peers via AddrFetch when starting with an empty addrman would be to connect to the peer, receive its self-announcement and add it to addrman, disconnect, reconnect to the same peer again as a full outbound (no other addresses in addrman) and then receive more `addr`. This is silly and not in line with AddrFetch peer being intended to be short-lived peers. 

  Fix this by only disconnecting after receiving an `addr` message of size > 1.

  [Edit] As per review discussion, this PR now also adds a timeout after which we disconnect if we haven't received any suitable `addr`, and a functional test.

ACKs for top commit:
  amitiuttarwar:
    reACK 5730a43703f7e5a5ca26245ba3b55fbdd027d0b6
  naumenkogs:
    ACK 5730a43703f7e5a5ca26245ba3b55fbdd027d0b6
  jnewbery:
    ACK 5730a43703

Tree-SHA512: 8a81234f37e827705138eb254223f7f3b3bf44a06cb02126fc7990b0d231b9bd8f07d38d185cc30d55bf35548a6fdc286b69602498d875b937e7c58332158bf9
2021-07-20 20:27:21 +08:00
João Barbosa
20edf4bcf6 rpc: Return block time in getblockchaininfo 2021-07-20 10:43:26 +01:00
fanquake
d542603c5a
Merge bitcoin/bitcoin#22502: scripted-diff: Revert "fuzz: Add Temporary debug assert for oss-fuzz issue"
facd56750c8a6aee88eeef75d8c8233778d35757 scripted-diff: Revert "fuzz: Add Temporary debug assert for oss-fuzz issue" (MarcoFalke)

Pull request description:

  No longer needed, as it wouldn't help to debug this issue. See https://github.com/bitcoin/bitcoin/pull/22472#issuecomment-882692900

ACKs for top commit:
  fanquake:
    ACK facd56750c8a6aee88eeef75d8c8233778d35757

Tree-SHA512: 13352b3529c43d6e65ab127134b32158d3072dc2fbbb326fea9adfeada5a8610d0477ea75748b8b68e7abb3b9869a989df3a3169e92bdd458053d64bae6ed379
2021-07-20 10:46:56 +08:00
fanquake
624a193330
Merge bitcoin/bitcoin#22497: scripted-diff: remove ResetI2PPorts() (revert e0a2b390c14)
d4b67c8ebc2bb7488bcaaccc3a801cdef1cf1678 scripted-diff: remove ResetI2PPorts() (revert e0a2b390c14) (Vasil Dimov)

Pull request description:

  `CAddrMan::ResetI2PPorts()` was temporary. Remove it:
  * it has partially achieved its goal: probably ran on about half of the
    I2P nodes
  * it is hackish, deemed risky and two bugs where found in it:
    https://github.com/bitcoin/bitcoin/issues/22467
    https://github.com/bitcoin/bitcoin/issues/22470

  -BEGIN VERIFY SCRIPT-
  git show e0a2b390c144e123e2fc8a289fdff36815476964 |git apply -R
  -END VERIFY SCRIPT-

  Fixes https://github.com/bitcoin/bitcoin/issues/22467
  Fixes https://github.com/bitcoin/bitcoin/issues/22470

ACKs for top commit:
  laanwj:
    ACK d4b67c8ebc2bb7488bcaaccc3a801cdef1cf1678
  MarcoFalke:
    review ACK d4b67c8ebc2bb7488bcaaccc3a801cdef1cf1678 😲
  jonatack:
    ACK d4b67c8ebc2bb7488bcaaccc3a801cdef1cf1678 per IRC discussions https://www.erisian.com.au/bitcoin-core-dev/log-2021-07-16.html#l-212 and https://www.erisian.com.au/bitcoin-core-dev/log-2021-07-19.html#l-210

Tree-SHA512: 60d8f0ea0f66a8fcedfcb9c8944a419b974b15509b54ddfeec58db49ae9418e6916df712bba3fbd6b29497d85f7951fb9aa2e48eb9c59f88d09435685bd00b4c
2021-07-20 09:02:34 +08:00
MarcoFalke
facd56750c
scripted-diff: Revert "fuzz: Add Temporary debug assert for oss-fuzz issue"
-BEGIN VERIFY SCRIPT-
git show faf1af58f85da74f94c6b5f6910c7faf7b47cc88 | git apply --reverse
-END VERIFY SCRIPT-
2021-07-19 19:12:54 +02:00
Andrew Chow
25d99e6511 Reorder dumpwallet so that cs_main functions go first
DEBUG_LOCKORDER expects cs_wallet, cs_main, and cs_KeyStore to be
acquired in that order. However dumpwallet would take these in the order
cs_wallet, cs_KeyStore, cs_main. So when configured with
`--enable-debug`, it is possible to hit the lock order assertion when
using dumpwallet.

To fix this, cs_wallet and cs_KeyStore are no longer locked at the same
time. Instead cs_wallet will be locked first. Then the functions which
lock cs_main will be run. Lastly cs_KeyStore will be locked afterwards.
This avoids the lock order issue.

Furthermore, since GetKeyBirthTimes (only used by dumpwallet) also uses
a function that locks cs_main, and itself also locks cs_KeyStore, the
same reordering is done here.
2021-07-19 12:25:11 -04:00
Sriram
eeddd1c8fa Update assumed chain params
Note: 10% overhead to the base value of `mainnet` in `m_assumed_blockchain_size`
2021-07-19 19:34:30 +05:30
Vasil Dimov
d4b67c8ebc
scripted-diff: remove ResetI2PPorts() (revert e0a2b390c14)
`CAddrMan::ResetI2PPorts()` was temporary. Remove it:
* it has partially achieved its goal: probably ran on about half of the
  I2P nodes
* it is hackish, deemed risky and two bugs where found in it
  https://github.com/bitcoin/bitcoin/issues/22467
  https://github.com/bitcoin/bitcoin/issues/22470

-BEGIN VERIFY SCRIPT-
git show e0a2b390c144e123e2fc8a289fdff36815476964 |git apply -R
-END VERIFY SCRIPT-

Fixes https://github.com/bitcoin/bitcoin/issues/22467
Fixes https://github.com/bitcoin/bitcoin/issues/22470
2021-07-19 14:33:21 +02:00
MarcoFalke
54e31742d2
Merge bitcoin/bitcoin#22455: addrman: detect on-disk corrupted nNew and nTried during unserialization
816f29eab296ebec2da8f8606ad618609e3ba228 addrman: detect on-disk corrupted nNew and nTried during unserialization (Vasil Dimov)

Pull request description:

  Negative `nNew` or `nTried` are not possible during normal operation.
  So, if we read such values during unserialize, report addrman
  corruption.

  Fixes https://github.com/bitcoin/bitcoin/issues/22450

ACKs for top commit:
  MarcoFalke:
    cr ACK 816f29eab296ebec2da8f8606ad618609e3ba228
  jonatack:
    ACK 816f29eab296ebec2da8f8606ad618609e3ba228
  lsilva01:
    Code Review ACK 816f29eab2.  This change provides a more accurate description of the error.

Tree-SHA512: 01bdd72d2d86a0ef770a319fee995fd1e147b24a8db84ddb8cd121688e7f94fed73fddc0084758e7183c4f8d08e971f0b1b224f5adb10928a5aa4dbbc8709d74
2021-07-19 14:25:53 +02:00
W. J. van der Laan
d3474b8df2
Merge bitcoin/bitcoin#22387: Rate limit the processing of rumoured addresses
a4bcd687c934d47aa3922334e97e579caf5f8124 Improve tests using statistics (John Newbery)
f424d601e1b6870e20bc60f5ccba36d2e210377b Add logging and addr rate limiting statistics (Pieter Wuille)
b4ece8a1cda69cc268d39d21bba59c51fa2fb9ed Functional tests for addr rate limiting (Pieter Wuille)
5648138f5949013331c017c740646e2f4013bc24 Randomize the order of addr processing (Pieter Wuille)
0d64b8f709b4655d8702f810d4876cd8d96ded82 Rate limit the processing of incoming addr messages (Pieter Wuille)

Pull request description:

  The rate at which IP addresses are rumoured (through ADDR and ADDRV2 messages) on the network seems to vary from 0 for some non-participating nodes, to 0.005-0.025 addr/s for recent Bitcoin Core nodes. However, the current codebase will happily accept and process an effectively unbounded rate from attackers. There are measures to limit the influence attackers can have on the addrman database (bucket restrictions based on source IPs), but still - there is no need to permit them to feed us addresses at a rate that's orders of magnitude larger than what is common on the network today, especially as it will cause us to spam our peers too.

  This PR implements a [token bucket](https://en.wikipedia.org/wiki/Token_bucket) based rate limiter, allowing an average of 0.1 addr/s per connection, with bursts up to 1000 addresses at once. Whitelisted peers as well as responses to GETADDR requests are exempt from the limit. New connections start with 1 token, so as to not interfere with the common practice of peers' self-announcement.

ACKs for top commit:
  laanwj:
    ACK a4bcd687c934d47aa3922334e97e579caf5f8124
  vasild:
    ACK a4bcd687c934d47aa3922334e97e579caf5f8124
  jnewbery:
    ACK a4bcd687c934d47aa3922334e97e579caf5f8124
  jonatack:
    ACK a4bcd687c934d47aa3922334e97e579caf5f8124

Tree-SHA512: b757de76ad78a53035b622944c4213b29b3b55d3d98bf23585afa84bfba10808299d858649f92269a16abfa75eb4366ea047eae3216f7e2f6d3c455782a16bea
2021-07-19 12:42:07 +02:00
Samuel Dobson
e8f85e0e86
Merge bitcoin/bitcoin#22421: Make IsSegWitOutput return true for taproot outputs
8465978f235e2e43feb5dabe2a4d61026343b6ab Make IsSegWitOutput return true for taproot outputs (Pieter Wuille)

Pull request description:

  This fixes a bug: currently `utxoupdatepsbt` will not fill in UTXO data for PSBTs spending taproot outputs.

ACKs for top commit:
  achow101:
    Code Review ACK 8465978f235e2e43feb5dabe2a4d61026343b6ab
  jonatack:
    ACK 8465978f235e2e43feb5dabe2a4d61026343b6ab
  meshcollider:
    utACK 8465978f235e2e43feb5dabe2a4d61026343b6ab

Tree-SHA512: 2f8f873450bef4b5a4ce5962a231297b386c6b1445e69ce5f36ab28eca7343be3a11bc09c38534b0f75e6f99ba15d78d3ba5d484f6c63e5a9775e1f3f55a74e0
2021-07-18 20:07:52 +12:00
MarcoFalke
0eea1dfe80
Merge bitcoin/bitcoin#22445: fuzz: Move implementations of non-template fuzz helpers from util.h to util.cpp
a2aca207b1ad00ec05d7533dbd75bbff830e1d75 Move implementations of non-template fuzz helpers (Sriram)

Pull request description:

  There are 78 cpp files that include `util.h` (`grep -iIr "#include <test/fuzz/util.h>" src/test/fuzz | wc -l`). Modifying the implementation of a fuzz helper in `src/test/fuzz/util.h` will cause all fuzz tests to be recompiled. Keeping the declarations of these non-template fuzz helpers in `util.h` and moving their implementations to `util.cpp` will skip the redundant recompilation of all the fuzz tests, and builds these helpers only once in `util.cpp`.

  Functions moved from `util.h` to `util.cpp`:
  - `ConsumeTxMemPoolEntry`
  - `ContainsSpentInput`
  - `ConsumeNetAddr`
  - Methods of `FuzzedFileProvider::(open, read, write, seek, close)`

ACKs for top commit:
  MarcoFalke:
    review ACK a2aca207b1ad00ec05d7533dbd75bbff830e1d75 🍂

Tree-SHA512: e7037ebb86d0fc56048e4f3d8733eefc21da11683b09d2b22926bda410719628d89c52ddd9b4c18aa243607a66fdb4d13a63e62ca010e66b3ec9174fd18107f0
2021-07-18 09:46:06 +02:00
Samuel Dobson
5341c3b1b3
Merge bitcoin/bitcoin#22461: wallet: Change ScriptPubKeyMan::Upgrade default to True
5012a7912ee9fa35bc417cb073eebffd85f36c6c Test that descriptor wallet upgrade does nothing (Andrew Chow)
48bd7d3b7737656052d2c745ed40c7f6670842cf Change ScriptPubKeyMan::Upgrade to default to return true (Andrew Chow)

Pull request description:

  When adding a new ScriptPubKeyMan, it's likely that there will be nothing for `Upgrade` to do. If it is called (via `upgradewallet`), then it should do nothing, successfully. This PR changes the default `ScriptPubKeyMan::Upgrade` function so that it returns a success instead of failure when doing nothing.

  Fixes #22460

ACKs for top commit:
  jonatack:
    ACK 5012a7912ee9fa35bc417cb073eebffd85f36c6c
  meshcollider:
    utACK 5012a7912ee9fa35bc417cb073eebffd85f36c6c

Tree-SHA512: 578c6521e997f7bb5cc44be2cfe9e0a760b6bd4aa301026a6b8b3282e8757473e4cb9f68b2e79dacdc2b42dddae718450072e0a38817df205dfea177a74d7f3d
2021-07-18 19:33:10 +12:00
fanquake
b5889611c7
Merge bitcoin/bitcoin#22234: build: Mark print-% target as phony.
fb7be92b094477131140b58a4e3ae98366b93e76 Mark print-% target as phony. (Dmitry Goncharov)

Pull request description:

  .PHONY does not take patterns (such as print-%) as prerequisites.
  Have print-% depend on force and mark force as phony.

  This change ensures print-% rule works even when there is a file that matches the target.

  ```
  $ # on master
  $ make print-host
  host=x86_64-pc-linux-gnu
  $ touch print-host
  $ make print-host
  make: 'print-host' is up to date.
  $
  $ git co mark_print_as_phony
  Switched to branch 'mark_print_as_phony'
  $ make print-host
  host=x86_64-pc-linux-gnu
  $ touch force
  $ make print-host
  host=x86_64-pc-linux-gnu
  ```

ACKs for top commit:
  hebasto:
    ACK fb7be92b094477131140b58a4e3ae98366b93e76, tested on Linux Mint 20.2 (x86_64).

Tree-SHA512: b89ae66aa8c7aa6a7ab5f0956f9eb3b3ef9d56994b60dc2a97d498d4c1bba537845c190723e8a10310280b1b35df2cd935cc30aeb76735cac2dc621ad7823772
2021-07-18 13:41:24 +08:00
fanquake
6baabc4d1d
Merge bitcoin/bitcoin#21430: build: Add -Werror=implicit-fallthrough compile flag
3c4c8e79baf02af97ba1502189f649b04ef2198d build: Add -Werror=implicit-fallthrough compile flag (Hennadii Stepanov)
014110c47d94ece6e3e655cdbf02ed8c91c7a5cf Use C++17 [[fallthrough]] attribute, and drop -Wno-implicit-fallthrough (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK 3c4c8e79baf02af97ba1502189f649b04ef2198d - looks ok to me now. Checked that warnings occur in our code & leveldb by removing a `[[fallthrough]]` or `FALLTHROUGH_INTENDED`.
  jarolrod:
    ACK 3c4c8e79baf02af97ba1502189f649b04ef2198d
  theStack:
    ACK 3c4c8e79baf02af97ba1502189f649b04ef2198d

Tree-SHA512: 4dce91f0f26b8a3de09bd92bb3d7e1995e078e3a8b3ff861c4fbf6c0b32b2327d063633b07b89c4aa94a1141d7f78d46d9d43ab8df865273e342693ad30645b6
2021-07-18 11:06:10 +08:00
Pieter Wuille
8465978f23 Make IsSegWitOutput return true for taproot outputs 2021-07-15 17:08:52 -07:00
Jon Atack
e49d50cf40
bench: fix 32-bit narrowing warning in bench/peer_eviction.cpp 2021-07-15 23:05:10 +02:00
Pieter Wuille
f424d601e1 Add logging and addr rate limiting statistics
Includes logging improvements by Vasil Dimov and John Newbery.
2021-07-15 13:03:20 -07:00
Pieter Wuille
5648138f59 Randomize the order of addr processing 2021-07-15 12:59:23 -07:00
Pieter Wuille
0d64b8f709 Rate limit the processing of incoming addr messages
While limitations on the influence of attackers on addrman already
exist (affected buckets are restricted to a subset based on incoming
IP / network group), there is no reason to permit them to let them
feed us addresses at more than a multiple of the normal network
rate.

This commit introduces a "token bucket" rate limiter for the
processing of addresses in incoming ADDR and ADDRV2 messages.
Every connection gets an associated token bucket. Processing an
address in an ADDR or ADDRV2 message from non-whitelisted peers
consumes a token from the bucket. If the bucket is empty, the
address is ignored (it is not forwarded or processed). The token
counter increases at a rate of 0.1 tokens per second, and will
accrue up to a maximum of 1000 tokens (the maximum we accept in a
single ADDR or ADDRV2). When a GETADDR is sent to a peer, it
immediately gets 1000 additional tokens, as we actively desire many
addresses from such peers (this may temporarily cause the token
count to exceed 1000).

The rate limit of 0.1 addr/s was chosen based on observation of
honest nodes on the network. Activity in general from most nodes
is either 0, or up to a maximum around 0.025 addr/s for recent
Bitcoin Core nodes. A few (self-identified, through subver) crawler
nodes occasionally exceed 0.1 addr/s.
2021-07-15 12:52:38 -07:00
Andrew Chow
48bd7d3b77 Change ScriptPubKeyMan::Upgrade to default to return true
If a ScriptPubKeyMan does not implement Upgrade, then using upgraewallet
will fail unexpectedly. By changing the default to return true, then
this error can be avoided. This is still correct because a successful
upgrade can be that nothing happened.
2021-07-15 12:33:16 -04:00
W. J. van der Laan
a88fa1a555
Merge bitcoin/bitcoin#22211: net: relay I2P addresses even if not reachable (by us)
7593b06bd1262f438bf34769ecc00e9c22328e56 test: ensure I2P addresses are relayed (Vasil Dimov)
e7468139a1dddd4946bc70697ec38816b3fa8f9b test: make CAddress in functional tests comparable (Vasil Dimov)
33e211d2a442e4555ff3401f92af4ee2f7552568 test: implement ser/unser of I2P addresses in functional tests (Vasil Dimov)
86742811ce3662789ac85334008090a3b54babe3 test: use NODE_* constants instead of magic numbers (Vasil Dimov)
ba45f0270815d54ae3290efc16324c2ff1984565 net: relay I2P addresses even if not reachable (by us) (Vasil Dimov)

Pull request description:

  Nodes that can reach the I2P network (have set `-i2psam=`) will relay
  I2P addresses even without this patch. However, nodes that can't reach
  the I2P network will not. This was done as a precaution in
  https://github.com/bitcoin/bitcoin/pull/20119 before anybody could
  connect to I2P because then, for sure, it would have been useless.

  Now, however, we have I2P support and a bunch of I2P nodes, so get all
  nodes on the network to relay I2P addresses to help with propagation,
  similarly to what we do with Tor addresses.

ACKs for top commit:
  jonatack:
    ACK 7593b06bd1262f438bf34769ecc00e9c22328e56
  naumenkogs:
    ACK 7593b06bd1262f438bf34769ecc00e9c22328e56.
  laanwj:
    Code review ACK 7593b06bd1262f438bf34769ecc00e9c22328e56
  kristapsk:
    ACK 7593b06bd1262f438bf34769ecc00e9c22328e56. Code looks correct, tested that functional test suite passes and also that `test/functional/p2p_addrv2_replay.py` fails if I undo changes in `IsRelayable()`.

Tree-SHA512: c9feec4a9546cc06bc2fec6d74f999a3c0abd3d15b7c421c21fcf2d610eb94611489e33d61bdcd5a4f42041a6d84aa892f7ae293b0d4f755309a8560b113b735
2021-07-15 16:53:34 +02:00
W. J. van der Laan
21998bc028
Merge bitcoin/bitcoin#22284: p2p, refactor: performance improvements to ProtectEvictionCandidatesByRatio()
b1d905c225e87a4a289c0cd3593c6c21cea3fba7 p2p: earlier continuation when no remaining eviction candidates (Vasil Dimov)
c9e8d8f9b168dec2bc7b845da38449e96708cf8e p2p: process more candidates per protection iteration (Jon Atack)
02e411ec456af80d1da76085a814c68bb3aca6de p2p: iterate eviction protection only on networks having candidates (Jon Atack)
5adb06457403f8c1d874e9c6748ecbb78ef8fa2b bench: add peer eviction protection benchmarks (Jon Atack)
566357f8f7471f74729297868917aa32f6d3c390 refactor: move GetRandomNodeEvictionCandidates() to test utilities (Jon Atack)

Pull request description:

  This follow-up to #21261 improves `ProtectEvictionCandidatesByRatio()` for better performance.

  Benchmarks are added; the performance improvement is between 2x and 5x for the benchmarked cases (CPU 2.50GHz, Turbo off, performance mode, Debian Clang 11 non-debug build).

  ```
  $ ./src/bench/bench_bitcoin -filter="EvictionProtection*.*"
  ```

  The refactored code is well-covered by existing unit tests and also a fuzzer.

  - `$ ./src/test/test_bitcoin -t net_peer_eviction_tests`
  - `$ FUZZ=node_eviction ./src/test/fuzz/fuzz ../qa-assets/fuzz_seed_corpus/node_eviction`

ACKs for top commit:
  klementtan:
    Tested and code review ACK b1d905c2.
  vasild:
    ACK b1d905c225e87a4a289c0cd3593c6c21cea3fba7
  jarolrod:
    ACK b1d905c225e87a4a289c0cd3593c6c21cea3fba7

Tree-SHA512: a3a6607b9ea2fec138da9780c03f63e177b6712091c5a3ddc3804b896a7585216446310280791f5e20cc023d02d2f03a4139237e12b5c1d7f2a1fa1011610e96
2021-07-15 14:49:45 +02:00
MarcoFalke
faa54e3757
Move pblocktree global to BlockManager 2021-07-15 13:54:09 +02:00
MarcoFalke
fa27f03b49
Move LoadBlockIndexDB to BlockManager 2021-07-15 13:52:41 +02:00
Vasil Dimov
816f29eab2
addrman: detect on-disk corrupted nNew and nTried during unserialization
Negative `nNew` or `nTried` are not possible during normal operation.
So, if we read such values during unserialize, report addrman
corruption.

Fixes https://github.com/bitcoin/bitcoin/issues/22450
2021-07-15 13:40:29 +02:00
MarcoFalke
c0224bc962
Merge bitcoin/bitcoin#22415: Make m_mempool optional in CChainState
ceb7b35a39145717e2d9d356fd382bd1f95d2a5a refactor: move UpdateTip into CChainState (James O'Beirne)
4abf0779d6594e97222279110c328b75b5f3db7b refactor: no mempool arg to GetCoinsCacheSizeState (James O'Beirne)
46e3efd1e4ae2f058ecfffdaee7e882c4305eb35 refactor: move UpdateMempoolForReorg into CChainState (James O'Beirne)
617661703ac29e0744f21de74501d033fdc53ff6 validation: make CChainState::m_mempool optional (James O'Beirne)

Pull request description:

  Make `CChainState::m_mempool` optional by making it a pointer instead of a reference. This will allow a simplification to assumeutxo semantics (see https://github.com/bitcoin/bitcoin/pull/15606#pullrequestreview-692965905) and help facilitate the `-nomempool` option.

ACKs for top commit:
  jnewbery:
    ACK ceb7b35a39145717e2d9d356fd382bd1f95d2a5a
  naumenkogs:
    ACK ceb7b35a39145717e2d9d356fd382bd1f95d2a5a
  ryanofsky:
    Code review ACK ceb7b35a39145717e2d9d356fd382bd1f95d2a5a (just minor style and test tweaks since last review)
  lsilva01:
    Code review ACK and tested on Signet ACK ceb7b35a39
  MarcoFalke:
    review ACK ceb7b35a39145717e2d9d356fd382bd1f95d2a5a 😌

Tree-SHA512: cc445ad33439d5918cacf80a6354eea8f3d33bb7719573ed5b970fad1a0dab410bcd70be44c862b8aba1b71263b82d79876688c553e339362653dfb3d8ec81e6
2021-07-15 13:40:03 +02:00
MarcoFalke
97153a7026
Merge bitcoin/bitcoin#22385: refactor: Use DeploymentEnabled to hide VB deployments
fa5658ed077bfb02b6281d642dc649abdb99b6ee Use DeploymentEnabled to hide VB deployments (MarcoFalke)
fa11fecf0dac44846a08e1b325547641f2eca957 doc: Move buried deployment doc to the enum that enumerates them (MarcoFalke)

Pull request description:

  Plus a doc commit.

ACKs for top commit:
  jnewbery:
    utACK fa5658ed077bfb02b6281d642dc649abdb99b6ee
  ajtowns:
    utACK fa5658ed077bfb02b6281d642dc649abdb99b6ee

Tree-SHA512: 2aeceee0674feb603d76656eff40695b7d7305de309f837bbb6a8c1dbb1d0b962b741f06ab7b9a8b1dbd1964c9c0c9aa5dc9588fd8e6d896e620b69e08eedbaa
2021-07-15 08:34:40 +02:00
Sriram
a2aca207b1 Move implementations of non-template fuzz helpers
Moved implementations of `ConsumeTxMemPoolEntry`, `ContainsSpentInput`, `ConsumeNetAddr`, and the methods(open, read, write, seek, close) of FuzzedFileProvider from test/fuzz/util.h to test/fuzz/util.cpp.
2021-07-14 18:45:53 +05:30
James O'Beirne
ceb7b35a39
refactor: move UpdateTip into CChainState
Makes sense and saves on arguments.

Co-authored-by: John Newbery <john@johnnewbery.com>
2021-07-13 11:16:37 -04:00
James O'Beirne
4abf0779d6
refactor: no mempool arg to GetCoinsCacheSizeState
Unnecessary argument since we can make use of this->m_mempool

Co-authored-by: John Newbery <john@johnnewbery.com>
2021-07-13 11:16:30 -04:00
James O'Beirne
46e3efd1e4
refactor: move UpdateMempoolForReorg into CChainState
Allows fewer arguments and simplification of call sites.

Co-authored-by: John Newbery <john@johnnewbery.com>
2021-07-13 11:12:16 -04:00
James O'Beirne
617661703a
validation: make CChainState::m_mempool optional
Since we now have multiple chainstate objects, only one of them is active at any given
time. An active chainstate has a mempool, but there's no point to others having one.

This change will simplify proposed assumeutxo semantics. See the discussion here:
https://github.com/bitcoin/bitcoin/pull/15606#pullrequestreview-692965905

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2021-07-13 11:11:35 -04:00
W. J. van der Laan
d8f1e1327f
Merge bitcoin/bitcoin#22112: Force port 0 in I2P
4101ec9d2e05a35c35f587a28f1feee6cebcc61b doc: mention that we enforce port=0 in I2P (Vasil Dimov)
e0a2b390c144e123e2fc8a289fdff36815476964 addrman: reset I2P ports to 0 when loading from disk (Vasil Dimov)
41cda9d075ebcab1dbb950160ebe9d0ba7b5745e test: ensure I2P ports are handled as expected (Vasil Dimov)
4f432bd738c420512a86a51ab3e00323f396b89e net: do not connect to I2P hosts on port!=0 (Vasil Dimov)
1f096f091ebd88efb18154b8894a38122c39624f net: distinguish default port per network (Vasil Dimov)
aeac3bce3ead1f24ca782079ef0defa86fd8cb98 net: change I2P seeds' ports to 0 (Vasil Dimov)
38f900290cc3a839e99bef13474d35e1c02e6b0d net: change assumed I2P port to 0 (Vasil Dimov)

Pull request description:

  _This is an alternative to https://github.com/bitcoin/bitcoin/pull/21514, inspired by https://github.com/bitcoin/bitcoin/pull/21514#issuecomment-815049933. They are mutually exclusive. Just one of them should be merged._

  Change assumed ports for I2P to 0 (instead of the default 8333) as this is closer to what actually happens underneath with SAM 3.1 (https://github.com/bitcoin/bitcoin/pull/21514#issuecomment-812632520, https://github.com/bitcoin/bitcoin/pull/21514#issuecomment-816564719).

  Don't connect to I2P peers with advertised port != 0 (we don't specify a port to our SAM 3.1 proxy and it always connects to port = 0).

  Note, this change:
  * Keeps I2P addresses with port != 0 in addrman and relays them to others via P2P gossip. There may be non-bitcoin-core-22.0 peers using SAM 3.2 and for them such addresses may be useful.
  * Silently refuses to connect to I2P hosts with port != 0. This is ok for automatically chosen peers from addrman. Not so ok for peers provided via `-addnode` or `-connect` - a user who specifies `foo.b32.i2p:1234` (non zero port) may wonder why "nothing is happening".

  Fixes #21389

ACKs for top commit:
  laanwj:
    Code review ACK 4101ec9d2e05a35c35f587a28f1feee6cebcc61b
  jonatack:
    re-ACK 4101ec9d2e05a35c35f587a28f1feee6cebcc61b per `git range-diff efff9c3 0b0ee03 4101ec9`, built with DDEBUG_ADDRMAN, did fairly extensive testing on mainnet both with and without a peers.dat / -dnsseeds=0 to test boostrapping.

Tree-SHA512: 0e3c019e1dc05e54f559275859d3450e0c735596d179e30b66811aad9d5b5fabe3dcc44571e8f7b99f9fe16453eee393d6e153454dd873b9ff14907d4e6354fe
2021-07-13 14:52:41 +02:00
fanquake
aa72ffb1c2
init: remove straggling boost thread_group code
boost::thread_group usage was removed in #21016.
2021-07-12 21:46:59 +08:00
fanquake
839f5d06d6
Merge bitcoin/bitcoin#22432: doc: fix incorrect testmempoolaccept doc
9169be09f49c82fece034285e92f8ffa41e19ee2 fix incorrect testmempoolaccept doc (glozow)

Pull request description:

  Sorry, I somehow missed this...

ACKs for top commit:
  jnewbery:
    Tested ACK 9169be09f49c82fece034285e92f8ffa41e19ee2

Tree-SHA512: d44f81655669e338af298b7b5d616eb4ca15cbaac667c49251408cb92cee2fb9f440fcfbbac6a17744f24ceeafaf6cea6b9c49a37a464f7eaeeda6e655a56f7a
2021-07-12 20:31:29 +08:00
glozow
9169be09f4 fix incorrect testmempoolaccept doc 2021-07-12 10:57:52 +01:00
W. J. van der Laan
842e2a9c54
Merge bitcoin/bitcoin#20234: net: don't bind on 0.0.0.0 if binds are restricted to Tor
2feec3ce3130961f98ceb030951d0e46d3b9096c net: don't bind on 0.0.0.0 if binds are restricted to Tor (Vasil Dimov)

Pull request description:

  The semantic of `-bind` is to restrict the binding only to some address.
  If not specified, then the user does not care and we bind to `0.0.0.0`.
  If specified then we should honor the restriction and bind only to the
  specified address.

  Before this change, if no `-bind` is given then we would bind to
  `0.0.0.0:8333` and to `127.0.0.1:8334` (incoming Tor) which is ok -
  the user does not care to restrict the binding.

  However, if only `-bind=addr:port=onion` is given (without ordinary
  `-bind=`) then we would bind to `addr:port` _and_ to `0.0.0.0:8333` in
  addition.

  Change the above to not do the additional bind: if only
  `-bind=addr:port=onion` is given (without ordinary `-bind=`) then bind
  to `addr:port` (only) and consider incoming connections to that as Tor
  and do not advertise it. I.e. a Tor-only node.

ACKs for top commit:
  laanwj:
    Code review ACK 2feec3ce3130961f98ceb030951d0e46d3b9096c
  jonatack:
    utACK 2feec3ce3130961f98ceb030951d0e46d3b9096c per `git diff a004833 2feec3c`
  hebasto:
    ACK 2feec3ce3130961f98ceb030951d0e46d3b9096c, tested on Linux Mint 20.1 (x86_64):

Tree-SHA512: a04483af601706da928958b92dc560f9cfcc78ab0bb9d74414636eed1c6f29ed538ce1fb5a17d41ed82c9c9a45ca94899d0966e7ef93da809c9bcdcdb1d1f040
2021-07-12 10:08:22 +02:00
Martin Zumsande
c34ad3309f net, rpc: Enable AddrFetch connections for functional testing
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-07-12 02:16:45 +02:00
John Newbery
5a77abd4e6 [style] Clean up BroadcastTransaction() 2021-07-09 18:21:36 +01:00
John Newbery
cd48372b67 [mempool] Allow rebroadcast for same-txid-different-wtxid transactions
This commit fixes some slightly unexpected behaviour when:

- there is already transaction in the mempool (the "mempool tx")
- BroadcastTransaction() is called for a transaction with the same txid
  as the mempool transaction but a different witness (the "new tx")

Prior to this commit, if BroadcastTransaction() is called with
relay=true, then it'll call RelayTransaction() using the txid/wtxid of
the new tx, not the txid/wtxid of the mempool tx. For wtxid relay peers,
in SendMessages(), the wtxid of the new tx will be taken from
setInventoryTxToSend, but will then be filtered out from the vector of
wtxids to announce, since m_mempool.info() won't find the transaction
(the mempool contains the mempool tx, which has a different wtxid from
the new tx).

Fix this by calling RelayTransaction() with the wtxid of the mempool
transaction in this case.
2021-07-09 17:24:08 +01:00