While limitations on the influence of attackers on addrman already
exist (affected buckets are restricted to a subset based on incoming
IP / network group), there is no reason to permit them to let them
feed us addresses at more than a multiple of the normal network
rate.
This commit introduces a "token bucket" rate limiter for the
processing of addresses in incoming ADDR messages. Every connection
gets an associated token bucket. Processing an address in an ADDR
message from non-whitelisted peers consumes a token from the bucket.
If the bucket is empty, the address is ignored (it is not processed,
stored or forwarded). The token counter increases at a rate of 0.1
tokens per second, and will accrue up to a maximum of 1000 tokens
(the maximum we accept in a single ADDR message).
When a GETADDR is sent to a peer, it immediately gets 1000 additional
tokens, as we actively desire many addresses from such peers (this
may temporarily cause the token count to exceed 1000). This is
similar to allowing bursts.
The rate limit of 0.1 addr/s was copied from Bitcoin Core.
Backported from: 0d64b8f7, 5648138f, f424d601 and d930c7f5
Original authors: Pieter Wuille <pieter@wuille.net>, and
Jon Atack <jon@atack.com>
While looking at the network code, we realized this code is unused and
largely unmaintained. Given issue #2231, and its removal in 1.21, it's
time to remove it now.
This has the nice benefit of removing a single point of failure/control
(because it depends on a keypair to send alerts).
This code restores the pre-alert behavior before the merge of PR #1470.
Throughout the code, we use mockable time in places where we want
to test things that are subject to timing constraints, but don't
want to wait for great amounts of time when we use the regtest
network to ensure nothing gets broken. Mockable time is a system
time override that is only enabled on chains that have the
fMineBlocksOnDemand parameter set (currently: regtest).
Currently, only time expressed in seconds is able to be mocked, but
we have a couple of places where time is evaluated in microseconds,
one of them being the time between sending out addr messages.
This introduces a mockable time in microseconds and refactors the
time evaluation for addr messages to be mocked, so that we can
more reliably (and faster) test that.
Other protocol features may want to use this too, but currently the
sending of addr messages is the only place we test. Bitcoin Core
nowadays has a much better time system that we inherit in future
versions, but for the current scope I found this not worth the
effort of backporting, as these would impact a much larger part of
the code base.
Inspired by: 1a8f0d5a from Amiti Uttarwar
The largest sensible size for a locator is log in the number of blocks.
But, as noted by Coinr8d on BCT a maximum size message could encode a
hundred thousand locators. If height were used to limit the messages
that could open new attacks where peers on long low diff forks would
get disconnected and end up stuck.
Ideally, nodes first first learn to limit the size of locators they
send before limiting what would be processed, but common implementations
back off with an exponent of 2 and have an implicit limit of 2^32
blocks, so they already cannot produce locators over some size.
This sets the limit to an absurdly high amount of 101 in order to
maximize compatibility with existing software.
Cherry-picked from: e254ff5d
The timeout window for block downloads scales proportionally to the target
spacing for the chain, as set in chainparams.cpp. This causes issues on regtest
because the spacing is set to 1 second, allowing insufficient time for very
large blocks to sync when requested in batch, preventing success of the pruning
qa test.
We fix this by introducing a minimum multiplier (in seconds) that will be used
instead of the target block spacing whenever the latter is lower. With a value
of 10 seconds, pruning tests pass.
Introduces a counter for getheader requests that have been sent to
a peer but are pending response, reducing the number of parallel
requests a node pushes out to its peers when needing to sync large
amounts of headers. All getheader requests are serialized during
initial sync, except when a non-connecting header is received,
allowing the node to resolve issues with peers sending faulty
blocks using the DoS mechanism, and when we get an inv for a block
that we do not know, because it's possible we're only connected to
legacy nodes that do not implement header announcement properly.
Checking scrypt PoW is expensive and needless in this case. All block
headers are already checked when they are accepted, and they will be
checked again on the receiving end.
* Enable full block tests
* Fix invalidblocktest
* Move watch only address funding to immediately before it's used, so node 0 doesn't spend the output before it checks it later.
* Fix `fundrawtransaction` tests and sanitize fee calculation at the same time
* Correct resolution of chain parameters when validating tx inputs, especially from previous coinbase transactions
* Set block versions on full block tests so that the generated blocks are AuxPoW compatible
* Modify chain consensus parameters to be height aware
* Correct implementation of simplified rewards in parameters
* Correct max money
* Use base block version in IsSuperMajority() instead of full version
* Correct mining of blocks in AuxPoW tests
* Add in missing pre-AuxPoW consensus checks
Changes are as below:
Wrap CBlockHeader::nVersion into a new class (CBlockVersion). This allows to take care of interpreting the field into a base version, auxpow flag and the chain ID.
Update getauxblock.py for new 'generate' RPC call.
Add 'auxpow' to block JSON.
Accept auxpow as PoW verification.
Add unit tests for auxpow verification.
Add check for memory-layout of CBlockVersion.
Weaken auxpow chain ID checks for the testnet.
Allow Params() to overrule when to check the auxpow chain ID and for legacy blocks. Use this to disable the checks on testnet.
Introduce CPureBlockHeader.
Split the block header part that is used by auxpow and the "real" block header (that uses auxpow) to resolve the cyclic dependency between the two.
Differentiate between uint256 and arith_uint256.
This change was done upstream, modify the auxpow code.
Add missing lock in auxpow_tests.
Fix REST header check for auxpow headers.
Those can be longer, thus take that into account. Also perform the check actually on an auxpow header.
Correctly set the coinbase for getauxblock results.
Call IncrementExtraNonce in getauxblock so that the coinbase is actually initialised with the stuff it should be. (BIP30 block height and COINBASE_FLAGS.)
Implement getauxblock plus regression test.
Turn auxpow test into FIXTURE test.
This allows using of the Params() calls.
Move CMerkleTx code to auxpow.cpp.
Otherwise we get linker errors when building without wallet.
Fix rebase with BIP66.
Update the code to handle BIP66's nVersion=3.
Enforce that auxpow parent blocks have no auxpow block version.
This is for compatibility with namecoind. See also https://github.com/namecoin/namecoin/pull/199.
Move auxpow-related parameters to Consensus::Params.
d943491 qa: add a test to detect leaky p2p messages (Cory Fields)
8650bbb qa: Expose on-connection to mininode listeners (Matt Corallo)
5b5e4f8 qa: mininode learns when a socket connects, not its first action (Matt Corallo)
cbfc5a6 net: require a verack before responding to anything else (Cory Fields)
8502e7a net: parse reject earlier (Cory Fields)
c45b9fb net: correctly ban before the handshake is complete (Cory Fields)
7a8c251901 made this logic hard to follow. After that change, messages would
not be sent to a peer via SendMessages() before the handshake was complete, but
messages could still be sent as a response to an incoming message.
For example, if a peer had not yet sent a verack, we wouldn't notify it about
new blocks, but we would respond to a PING with a PONG.
This change makes the behavior straightforward: until we've received a verack,
never send any message other than version/verack/reject.
The behavior until a VERACK is received has always been undefined, this change
just tightens our policy.
This also makes testing much easier, because we can now connect but not send
version/verack, and anything sent to us is an error.
Prior to this change, all messages were ignored until a VERSION message was
received, as well as possibly incurring a ban score.
Since REJECT messages can be sent at any time (including as a response to a bad
VERSION message), make sure to always parse them.
Moving this parsing up keeps it from being caught in the
if (pfrom->nVersion == 0) check below.
7a8c251901 made a change to avoid getting into SendMessages() until the
version handshake (VERSION + VERACK) is complete. That was done to avoid
leaking out messages to nodes who could connect, but never bothered sending
us their version/verack.
Unfortunately, the ban tally and possible disconnect are done as part of
SendMessages(). So after 7a8c251901, if a peer managed to do something
bannable before completing the handshake (say send 100 non-version messages
before their version), they wouldn't actually end up getting
disconnected/banned. That's fixed here by checking the banscore as part of
ProcessMessages() in addition to SendMessages().
These are (afaik) all long-standing races or concurrent accesses. Going
forward, we can clean these up so that they're not all individual atomic
accesses.
- Reintroduce cs_vRecv to guard receive-specific vars
- Lock vRecv/vSend for CNodeStats
- Make some vars atomic.
- Only set the connection time in CNode's constructor so that it doesn't change
0729102 Net: pass interruptMsgProc as const where possible (Jorge Timón)
fc7f2ff Net: Make CNetMsgMaker more const (Jorge Timón)
d45955f Net: CConnman: Make some methods const (Jorge Timón)
This avoids having some vars set if the version negotiation fails.
Also copy it all into CNode at the same site. nVersion and
fSuccessfullyConnected are set last, as they are the gates for the other vars.
Make them atomic for that reason.
This adds a comment to the new logic for setting HB peers based
on block validation (and aligns the code below to reflect the comment).
It's not obvious why we're checking mapBlocksInFlight. Add a comment to
explain.
The old Bitcoin alert system has long since been retired.
( See also: https://bitcoin.org/en/alert/2016-11-01-alert-retirement )
This change causes each node to send any old peers that
it connects with a copy of the final alert.
The alert it hardcode cancels all other alerts including
other final alerts.
This forces the message handling thread to make another full
iteration of SendMessages prior to going back to sleep, ensuring
we announce the new block to all peers before sleeping.
02ee4eb Make most_recent_compact_block a pointer to a const (Matt Corallo)
73666ad Add comment to describe callers to ActivateBestChain (Matt Corallo)
962f7f0 Call ActivateBestChain without cs_main/with most_recent_block (Matt Corallo)
0df777d Use a temp pindex to avoid a const_cast in ProcessNewBlockHeaders (Matt Corallo)
c1ae4fc Avoid holding cs_most_recent_block while calling ReadBlockFromDisk (Matt Corallo)
9eb67f5 Ensure we meet the BIP 152 old-relay-types response requirements (Matt Corallo)
5749a85 Cache most-recently-connected compact block (Matt Corallo)
9eaec08 Cache most-recently-announced block's shared_ptr (Matt Corallo)
c802092 Relay compact block messages prior to full block connection (Matt Corallo)
6987219 Add a CValidationInterface::NewPoWValidBlock callback (Matt Corallo)
180586f Call AcceptBlock with the block's shared_ptr instead of CBlock& (Matt Corallo)
8baaba6 [qa] Avoid race in preciousblock test. (Matt Corallo)
9a0b2f4 [qa] Make compact blocks test construction using fetch methods (Matt Corallo)
8017547 Make CBlockIndex*es in net_processing const (Matt Corallo)
e60360e net: remove cs_vRecvMsg (Cory Fields)
991955e net: add a flag to indicate when a node's send buffer is full (Cory Fields)
c6e8a9b net: add a flag to indicate when a node's process queue is full (Cory Fields)
4d712e3 net: add a new message queue for the message processor (Cory Fields)
c5a8b1b net: rework the way that the messagehandler sleeps (Cory Fields)
c72cc88 net: remove useless comments (Cory Fields)
ef7b5ec net: Add a simple function for waking the message handler (Cory Fields)
f5c36d1 net: record bytes written before notifying the message processor (Cory Fields)
60befa3 net: handle message accounting in ReceiveMsgBytes (Cory Fields)
56212e2 net: set message deserialization version when it's actually time to deserialize (Cory Fields)
0e973d9 net: remove redundant max sendbuffer size check (Cory Fields)
6042587 net: wait until the node is destroyed to delete its recv buffer (Cory Fields)
f6315e0 net: only disconnect if fDisconnect has been set (Cory Fields)
5b4a8ac net: make GetReceiveFloodSize public (Cory Fields)
e5bcd9c net: make vRecvMsg a list so that we can use splice() (Cory Fields)
53ad9a1 net: fix typo causing the wrong receive buffer size (Cory Fields)
vRecvMsg is now only touched by the socket handler thread.
The accounting vars (nRecvBytes/nLastRecv/mapRecvBytesPerMsgCmd) are also
only used by the socket handler thread, with the exception of queries from
rpc/gui. These accesses are not threadsafe, but they never were. This needs to
be addressed separately.
Also, update comment describing data flow
