Minimizes the use of c_str() in netbase interfaces, by using
std::string when we're passing arguments instead, and only
converting to a C-style string when interfacing with
getaddrinfo.
Introduces attributes.h for definition of NODISCARD macro
Introduces utilstring.h for definition of ValidAsCString()
Backported from: 9cc0230c (partial), d945c6f5 and 9574de8
Original Author: practicalswift <practicalswift@users.noreply.github.com>
While limitations on the influence of attackers on addrman already
exist (affected buckets are restricted to a subset based on incoming
IP / network group), there is no reason to permit them to let them
feed us addresses at more than a multiple of the normal network
rate.
This commit introduces a "token bucket" rate limiter for the
processing of addresses in incoming ADDR messages. Every connection
gets an associated token bucket. Processing an address in an ADDR
message from non-whitelisted peers consumes a token from the bucket.
If the bucket is empty, the address is ignored (it is not processed,
stored or forwarded). The token counter increases at a rate of 0.1
tokens per second, and will accrue up to a maximum of 1000 tokens
(the maximum we accept in a single ADDR message).
When a GETADDR is sent to a peer, it immediately gets 1000 additional
tokens, as we actively desire many addresses from such peers (this
may temporarily cause the token count to exceed 1000). This is
similar to allowing bursts.
The rate limit of 0.1 addr/s was copied from Bitcoin Core.
Backported from: 0d64b8f7, 5648138f, f424d601 and d930c7f5
Original authors: Pieter Wuille <pieter@wuille.net>, and
Jon Atack <jon@atack.com>
Each node keeps a registry of manually added nodes (through the
addnode parameter, rpc call or UI) but there are currently no
limits imposed on that usage, which is a bit sloppy and can lead
to situations where memory is being used for storing addresses
that are never connected to, because the maximum number of
connections used for addnode entries is hardcoded as 8. This
could prevent smaller systems that host nodes (like those
running on an ARM SoC) to optimally use the available memory.
This enhancement limits the addnode functionality as follows:
1. Whenever over 799 nodes are added to the registry, require
the user to remove an entry before a new one can be added
2. Disallow very large addresses (more than 256 characters).
This limit provides for at least 4 levels of subdomains as
specified under RFC1035.
See https://datatracker.ietf.org/doc/html/rfc1035#section-2.3.1
The algorithm here is important and took some time to get right. Instead
of comparing whether the current number of connected nodes minus the
number of unevictable nodes is greater than the number of max
connections, check that:
* there are any evictable nodes (connected nodes minus unevictable
nodes)
* there are more nodes connected than requested (connected nodes minus
max connections)
While we could wait for nodes to disconnect organically, it's more
important to run the eviction logic frequently enough that we can tell
when it will have an effect.
Whitelisted connections and protected inbound connections are
unevictable, and max connections should account for inbound connections.
Because the evictor will never evict protected inbound connections, the
maximum connection count should always be at least as large as the
protected connection count.
Note that the tests for this use a delay and test that the delay has not
expired. This helps improve determinism in the testing. Otherwise, a
strict test for a fixed number of disconnections is susceptible to
things like CPU jitter, especially when running through CI.
Patrick ran this test for 1000 runs on busy CPUs and saw no failures.
This uses the constant in src/net.h for the minimum allowed number of
connections.
The limit of new max connections is silently capped to the number of
available file descriptors. This value is not exposed in the UI or
RPC messages so as not to leak interesting or important system details.
The floor of maximum connections is set to the number of connections
required for this node to operate.
This makes the best attempt to reduce the number of connections by
reusing the eviction logic. This reuses the core of that algorithm at
the expense of deterministic behavior. In other words, starting with a
max connections value of 125 and then changing the max connections to 3
does not mean that this will immediately evict 122 connections.
Eventually the number of connections will reduce to the limit.
While the body of the condition will add latency to network processing,
the integer comparison between max connections and size of the
connections vector should be quick in most cases.
Note the extraction of connection disconnect/delete helpers so as to
reuse the same logic in multiple places. While this may not be strictly
necessary for the algorithm, it reduces the possibility that this entire
loop will get stuck doing busy work when trying to evict connections to
get under the maximum threshold.
Initialize CConnman byte counters during construction, so that
GetTotalBytesRecv() and GetTotalBytesSent() methods don't return
garbage before Start() is called, in QT.
Backported from: 8313fa8e
Original author: Russell Yanofsky <russ@yanofsky.org>
Introduces a counter for getheader requests that have been sent to
a peer but are pending response, reducing the number of parallel
requests a node pushes out to its peers when needing to sync large
amounts of headers. All getheader requests are serialized during
initial sync, except when a non-connecting header is received,
allowing the node to resolve issues with peers sending faulty
blocks using the DoS mechanism, and when we get an inv for a block
that we do not know, because it's possible we're only connected to
legacy nodes that do not implement header announcement properly.
also once half of all outgoing nodes have our preferred flags, require only
minimal flags from the rest.
Github-Pull: #10441
Rebased-From: b6fbfc228236947eaea5c14dda299f5a01810e92
Previously if we didn't have any local addresses, GetLocalAddress would return
0.0.0.0 and then we'd swap in a peer's notion of our address in AdvertiseLocal,
but then nServices would never get set.
Github-Pull: #10424
Rebased-From: 307013469f9a3b8f13d3eb9dbeea419a55148493
We previously would block waiting for a CSemaphoreGrant in
ThreadOpenAddedConnections, when we did not need to. This would
block as the posts in CConnman shutdown were both to the wrong
semaphore and in the wrong location.
Github-Pull: #9953
Rebased-From: e007b243c4840e44857b5ccf686ed35899e44af0
66f861a Add a test for P2P inactivity timeouts (Matt Corallo)
b436f92 qa: Expose on-connection to mininode listeners (Matt Corallo)
8aaba7a qa: mininode learns when a socket connects, not its first action (Matt Corallo)
2cbd119 Disconnect peers which we do not receive VERACKs from within 60 sec (Matt Corallo)
These are (afaik) all long-standing races or concurrent accesses. Going
forward, we can clean these up so that they're not all individual atomic
accesses.
- Reintroduce cs_vRecv to guard receive-specific vars
- Lock vRecv/vSend for CNodeStats
- Make some vars atomic.
- Only set the connection time in CNode's constructor so that it doesn't change
618ee92 Further-enforce lockordering by enforcing directly after TRY_LOCKs (Matt Corallo)
2a962d4 Fixup style a bit by moving { to the same line as if statements (Matt Corallo)
8465631 Always enforce lock strict lock ordering (try or not) (Matt Corallo)
fd13eca Lock cs_vSend and cs_inventory in a consistent order even in TRY (Matt Corallo)
0729102 Net: pass interruptMsgProc as const where possible (Jorge Timón)
fc7f2ff Net: Make CNetMsgMaker more const (Jorge Timón)
d45955f Net: CConnman: Make some methods const (Jorge Timón)
Since ForEach* are can be used to send messages to all nodes, the caller may
end up sending a message before the version handshake is complete. To limit
this, filter out these nodes. While we're at it, may as well filter out
disconnected nodes as well.
Delete unused methods rather than updating them.
Once the CNode has been added to vNodes, it is possible that it is
disconnected+deleted in the socket handler thread. However, after
that we now call InitializeNode, which accesses the pnode.
helgrind managed to tickle this case (somehow), but I suspect it
requires in immensely braindead scheduler.
2366180 Do not add to vNodes until fOneShot/fFeeler/fAddNode have been set (Matt Corallo)
3c37dc4 Ensure cs_vNodes is held when using the return value from FindNode (Matt Corallo)
5be0190 Delete some unused (and broken) functions in CConnman (Matt Corallo)
The use of mocktime in test logic means that comparisons between
GetTime() and GetTimeMicros()/1000000 are unreliable since the former
can use mocktime values while the latter always gets the system clock;
this changes the networking code's inactivity checks to consistently
use the system clock for inactivity comparisons.
Also remove some hacks from setmocktime() that are no longer needed,
now that we're using the system clock for nLastSend and nLastRecv.
Technically cs_sendProcessing is entirely useless now because it
is only ever taken on the one MessageHandler thread, but because
there may be multiple of those in the future, it is left in place
cs_vSend is used for two purposes - to lock the datastructures used
to queue messages to place on the wire and to only call
SendMessages once at a time per-node. I believe SendMessages used
to access some of the vSendMsg stuff, but it doesn't anymore, so
these locks do not need to be on the same mutex, and also make
deadlocking much more likely.
e60360e net: remove cs_vRecvMsg (Cory Fields)
991955e net: add a flag to indicate when a node's send buffer is full (Cory Fields)
c6e8a9b net: add a flag to indicate when a node's process queue is full (Cory Fields)
4d712e3 net: add a new message queue for the message processor (Cory Fields)
c5a8b1b net: rework the way that the messagehandler sleeps (Cory Fields)
c72cc88 net: remove useless comments (Cory Fields)
ef7b5ec net: Add a simple function for waking the message handler (Cory Fields)
f5c36d1 net: record bytes written before notifying the message processor (Cory Fields)
60befa3 net: handle message accounting in ReceiveMsgBytes (Cory Fields)
56212e2 net: set message deserialization version when it's actually time to deserialize (Cory Fields)
0e973d9 net: remove redundant max sendbuffer size check (Cory Fields)
6042587 net: wait until the node is destroyed to delete its recv buffer (Cory Fields)
f6315e0 net: only disconnect if fDisconnect has been set (Cory Fields)
5b4a8ac net: make GetReceiveFloodSize public (Cory Fields)
e5bcd9c net: make vRecvMsg a list so that we can use splice() (Cory Fields)
53ad9a1 net: fix typo causing the wrong receive buffer size (Cory Fields)
