800 Commits

Author SHA1 Message Date
fanquake
a62f5ee86c
Merge bitcoin/bitcoin#27675: p2p: Drop m_recently_announced_invs bloom filter
fb02ba3c5f5bcd96b5e3622ef001b8e57ce63fc0 mempool_entry: improve struct packing (Anthony Towns)
1a118062fbc4ec8f645f4ec4298d869a869c3344 net_processing: Clean up INVENTORY_BROADCAST_MAX constants (Anthony Towns)
6fa49937e488d0924044786c76b42324b659f351 test: Check tx from disconnected block is immediately requestable (glozow)
e4ffabbffacc4b890d393aafcc8286916ef887d8 net_processing: don't add txids to m_tx_inventory_known_filter (Anthony Towns)
6ec1809d33bfc42b80cb6f35625dccd56be8d507 net_processing: drop m_recently_announced_invs bloom filter (Anthony Towns)
a70beafdb22564043dc24fc98133fdadbaf77d8a validation: when adding txs due to a block reorg, allow immediate relay (Anthony Towns)
1e9684f39fba909b3501e9402d5b61f4bf744ff2 mempool_entry: add mempool entry sequence number (Anthony Towns)

Pull request description:

  This PR replaces the `m_recently_announced_invs` bloom filter with a simple sequence number tracking the mempool state when we last considered sending an INV message to a node. This saves 33kB per peer (or more if we raise the rate at which we relay transactions over the network, in which case we would need to increase the size of the bloom filter proportionally).

  The philosophy here (compare with #18861 and #19109) is that we consider the rate limiting on INV messages to only be about saving bandwidth and not protecting privacy, and therefore after you receive an INV message, it's immediately fair game to request any transaction that was in the mempool at the time the INV message was sent. We likewise consider the BIP 133 feefilter and BIP 37 bloom filters to be bandwidth optimisations here, and treat transactions as requestable if they would have been announced without those filters. Given that philosophy, tracking the timestamp of the last INV message and comparing that against the mempool entry time allows removal of each of `m_recently_announced_invs`, `m_last_mempool_req` and `UNCONDITIONAL_RELAY_DELAY` and associated logic.

ACKs for top commit:
  naumenkogs:
    ACK fb02ba3c5f5bcd96b5e3622ef001b8e57ce63fc0
  amitiuttarwar:
    review ACK fb02ba3c5f5
  glozow:
    reACK fb02ba3c5f5bcd96b5e3622ef001b8e57ce63fc0

Tree-SHA512: cbba5ee04c86df26b6057f3654c00a2b45ec94d354f4f157a769cecdaa0b509edaac02b3128afba39b023e82473fc5e28c915a787f84457ffe66638c6ac9c2d4
2023-08-17 10:52:06 +01:00
Andrew Chow
f3c9078b4c Clean up things that include script/standard.h
Remove standard.h from files that don't use anything in it, and include
it in files that do.
2023-08-14 17:38:27 -04:00
glozow
0d9a13ddd8
Merge bitcoin/bitcoin#28149: net processing: clamp PeerManager::Options user input
547fa52443cbb5e8ccfee993486f5ced8cdbb33b net processing: clamp -blockreconstructionextratxn to uint32_t bounds (stickies-v)
e451d1e3c66350017da195335f428a96fdc7d840 net processing: clamp -maxorphantx to uint32_t bounds (stickies-v)
aa89e04e07ca9ff51b1d7d310a11821c6ad963cf doc: document PeerManager::Options members (stickies-v)

Pull request description:

  Avoid out-of-bounds user input for `PeerManager::Options` by safely clamping `-maxorphantx` and `-blockreconstructionextratxn`, and avoid platform-specific behaviour by changing `PeerManager::Options::max_extra_txs` from `size_t` to a `uint32_t`. Addresses https://github.com/bitcoin/bitcoin/pull/27499#pullrequestreview-1544114932.

  Also documents all `PeerManager::Options` members, addressing https://github.com/bitcoin/bitcoin/pull/27499#discussion_r1272302469.

ACKs for top commit:
  dergoegge:
    Code review ACK 547fa52443cbb5e8ccfee993486f5ced8cdbb33b
  glozow:
    reACK 547fa52443cbb5e8ccfee993486f5ced8cdbb33b

Tree-SHA512: 042d47b35bb8a7b29ef3dadd4c0c5d26f13a8f174f33687855d603c19f8de0fcbbda94418453331e149885412d4edd5f402d640d938f6d94b4dcf54e2fdbbcc9
2023-08-09 14:26:03 +02:00
fanquake
b565485c24
Merge bitcoin/bitcoin#28186: kernel: Prune leveldb headers
d8f1222ac50f089a0af29eaf8ce0555bad8366ef refactor: Correct dbwrapper key naming (TheCharlatan)
be8f159ac59b9e700cbd3314ed71ebf39bd5b67a build: Remove leveldb from BITCOIN_INCLUDES (TheCharlatan)
c95b37d641b1eed4a62d55ca5342a6ed8c7a1ce7 refactor: Move CDBWrapper leveldb members to their own context struct (TheCharlatan)
c534a615e93452a5f509aaf5f68c600391a98d6a refactor: Split dbwrapper CDBWrapper::EstimateSize implementation (TheCharlatan)
586448888b72f7c87db4dcd30fc4e4044afae13b refactor: Move HandleError to dbwrapper implementation (TheCharlatan)
dede0eef7adb7413f62f5abd68cac8e01635ba4a refactor: Split dbwrapper CDBWrapper::Exists implementation (TheCharlatan)
a5c2eb57484314b04ec94523d14e0ef0c6c46d4f refactor: Fix logging.h includes (TheCharlatan)
84058e0eed9c05bc30984b39131e88ad1425628f refactor: Split dbwrapper CDBWrapper::Read implementation (TheCharlatan)
e4af2408f2ac59788567b6fc8cb3a68fc43da9fe refactor: Pimpl leveldb::Iterator for CDBIterator (TheCharlatan)
ef941ff1281e76308c3e746e592375bec023e9e4 refactor: Split dbwrapper CDBIterator::GetValue implementation (TheCharlatan)
b7a1ab5cb4e60230f62c94efb3a10d07c9af4883 refactor: Split dbwrapper CDBIterator::GetKey implementation (TheCharlatan)
d7437908cdf242626263ba9d5541addcddadc594 refactor: Split dbwrapper CDBIterator::Seek implementation (TheCharlatan)
ea8135de7e617259cda3fc7b1c8e7569d454fd57 refactor: Pimpl leveldb::batch for CDBBatch (TheCharlatan)
b9870c920dc475ec759eaf7339ea42aecba92138 refactor: Split dbwrapper CDBatch::Erase implementation (TheCharlatan)
532ee812a499e13b123af6b8415d8de1f3804f0f refactor: Split dbwrapper CDBBatch::Write implementation (TheCharlatan)
afc534df9adbf5599b286b5dc3531a4b9ac2d056 refactor: Wrap DestroyDB in dbwrapper helper (TheCharlatan)

Pull request description:

  Leveldb headers are currently included in the `dbwrapper.h` file and thus available to many of Bitcoin Core's source files. However, leveldb-specific functionality should be abstracted by the `dbwrapper` and does not need to be available to the rest of the code. Having leveldb included in a widely-used header such as `dbwrapper.h` bloats the entire project's header tree.

  The `dbwrapper` is a key component of the libbitcoinkernel library. Future users of this library would not want to contend with  having the leveldb headers exposed and potentially polluting their project's namespace.

  For these reasons, the leveldb headers are removed from the `dbwrapper` by moving leveldb-specific code to the implementation file and creating a [pimpl](https://en.cppreference.com/w/cpp/language/pimpl) where leveldb member variables are indispensable. As a final step, the leveldb include flags are removed from the `BITCOIN_INCLUDES` and moved to places where the dbwrapper is compiled.

  ---

  This pull request is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587), and more specifically its stage 1 step 3 "Decouple most non-consensus headers from libbitcoinkernel".

ACKs for top commit:
  stickies-v:
    re-ACK d8f1222ac5
  MarcoFalke:
    ACK d8f1222ac50f089a0af29eaf8ce0555bad8366ef  🔠

Tree-SHA512: 0f58309be165af0162e648233451cd80fda88726fc10c0da7bfe4ec2ffa9afe63fbf7ffae9493698d3f39653b4ad870c372eee652ecc90ab1c29d86c387070f3
2023-08-07 22:31:46 +02:00
fanquake
624333455a
Merge bitcoin/bitcoin#26296: ci: Integrate bitcoin-tidy clang-tidy plugin
1c976c691cc4b20f43071aabf36c7afed1571057 tidy: Integrate bicoin-tidy clang-tidy plugin (fanquake)
7de23cceb8ac13fcc709453ef0fa14fb93c460b0 refactor: fix unterminated LogPrintf()s (fanquake)
0a1029aa2946787bfe91a84de882c2dd0de076e9 lint: remove  /* Continued */ markers from codebase (fanquake)
910007995d8603ffc466878856227153a638caff lint: remove lint-logs.py (fanquake)
d86a83d6b8587b0971e66c6910af23dd8c042969 lint: drop DIR_IWYU global (fanquake)

Pull request description:

  Demo of integrating the [bitcoin-tidy](https://github.com/theuni/bitcoin-tidy-plugin), [clang-tidy plugin](https://clang.llvm.org/extra/clang-tidy/) written by theuni into our tidy CI job.

  The plugin currently has a single check, `bitcoin-unterminated-logprintf`. This would replace our current Python driven, `git-grep`-based, `.cpp` file only, lint-logs linter.

ACKs for top commit:
  TheCharlatan:
    ACK 1c976c691cc4b20f43071aabf36c7afed1571057
  theuni:
    ACK 1c976c691cc4b20f43071aabf36c7afed1571057
  MarcoFalke:
    re-ACK 1c976c691cc4b20f43071aabf36c7afed1571057  👠

Tree-SHA512: 725b45c70e431d48e6f276671e05c694e10b6047cae1a31906ac3ee9093bc8105fb226b36a5bac6709557526ca6007222112d66aecec05a574434edc4897e4b8
2023-08-07 17:14:07 +02:00
fanquake
be44332803
Merge bitcoin/bitcoin#28191: refactor: Remove unused MessageStartChars parameters from BlockManager methods
fa69e3a95c452c2ba3221b17c19fba5993b5d073 Remove unused MessageStartChars parameters from BlockManager methods (MarcoFalke)

Pull request description:

  Seems odd to expose these for mocking, when it is not needed.

  Fix this by removing the the unused parameters and use the already existing member field instead.

ACKs for top commit:
  Empact:
    utACK fa69e3a95c452c2ba3221b17c19fba5993b5d073
  dergoegge:
    utACK fa69e3a95c452c2ba3221b17c19fba5993b5d073

Tree-SHA512: 7814e9560abba8d9c0926bcffc70f92e502d22f543af43671248f6fcd1433f35238553c0f05123fde6d8e0f80261af0ab0500927548115153bd68d57fe2da746
2023-08-07 10:57:39 +02:00
TheCharlatan
a5c2eb5748
refactor: Fix logging.h includes
These were uncovered as missing by the next commit.
2023-08-05 10:42:56 +02:00
fanquake
0a1029aa29
lint: remove /* Continued */ markers from codebase 2023-08-03 17:52:24 +01:00
Anthony Towns
1e9684f39f mempool_entry: add mempool entry sequence number 2023-08-03 13:42:45 +10:00
MarcoFalke
fae405556d
scripted-diff: Rename CBlockTreeDB -> BlockTreeDB
-BEGIN VERIFY SCRIPT-
 sed -i 's|CBlockTreeDB|BlockTreeDB|g' $( git grep -l CBlockTreeDB )
-END VERIFY SCRIPT-
2023-08-02 07:49:32 +02:00
MarcoFalke
faf63039cc
Fixup style of moved code
Can be reviewed with --word-diff-regex=.
2023-08-01 15:27:51 +02:00
MarcoFalke
fa65111b99
move-only: Move CBlockTreeDB to node/blockstorage
The block index (CBlockTreeDB) is required to write and read blocks, so
move it to blockstorage. This allows to drop the txdb.h include from
`node/blockstorage.h`.

Can be reviewed with:

--color-moved=dimmed-zebra  --color-moved-ws=ignore-all-space
2023-08-01 15:27:33 +02:00
Ryan Ofsky
f4f1d6d230
Merge bitcoin/bitcoin#27746: Rework validation logic for assumeutxo
a733dd79e29068ad1e0532ac42a45188a040a7b9 Remove unused function `reliesOnAssumedValid` (Suhas Daftuar)
d4a11abb1972b54f0babdccfbb2fde97ab885933 Cache block index entry corresponding to assumeutxo snapshot base blockhash (Suhas Daftuar)
3556b850221bc0e597d7dd749d4d47ab58dc8083 Move CheckBlockIndex() from Chainstate to ChainstateManager (Suhas Daftuar)
0ce805b632dcb98944a931f758f76f530f5ce5f2 Documentation improvements for assumeutxo (Ryan Ofsky)
768690b7ce551cd403f8e2a099372915f6022ad4 Fix initialization of setBlockIndexCandidates when working with multiple chainstates (Suhas Daftuar)
d43a1f1a2fa35d377c7a9ad7ab92d1ae325bde3d Tighten requirements for adding elements to setBlockIndexCandidates (Suhas Daftuar)
d0d40ea9a6478d81d7531b7cfc52a8bdaa0883d6 Move block-storage-related logic to ChainstateManager (Suhas Daftuar)
3cfc75366e6596942cbc84f354f42dfd7fc5c073 test: Clear block index flags when testing snapshots (Suhas Daftuar)
272fbc370c4e133d31d9f1d34e327cc265c5fad2 Update CheckBlockIndex invariants for chains based on an assumeutxo snapshot (Suhas Daftuar)
10c05710ce1602d932037f72dc6c4bbc3f6f34ba Add wrapper for adding entries to a chainstate's block index candidates (Suhas Daftuar)
471da5f6e74bac71aeffe2ebc5faff145a6cbcea Move block-arrival information / preciousblock counters to ChainstateManager (Suhas Daftuar)
1cfc887d00c5d1d4281107e3b3ff4641c6c34631 Remove CChain dependency in node/blockstorage (Suhas Daftuar)
fe86a7cd480b32463da900db764d2d11a2bea095 Explicitly track maximum block height stored in undo files (Suhas Daftuar)

Pull request description:

  This PR proposes a clean up of the relationship between block storage and the chainstate objects, by moving the decision of whether to store a block on disk to something that is not chainstate-specific.  Philosophically, the decision of whether to store a block on disk is related to validation rules that do not require any UTXO state; for anti-DoS reasons we were using some chainstate-specific heuristics, and those have been reworked here to achieve the proposed separation.

  This PR also fixes a bug in how a chainstate's `setBlockIndexCandidates` was being initialized; it should always have all the HAVE_DATA block index entries that have more work than the chain tip.  During startup, we were not fully populating `setBlockIndexCandidates` in some scenarios involving multiple chainstates.

  Further, this PR establishes a concept that whenever we have 2 chainstates, that we always know the snapshotted chain's base block and the base block's hash must be an element of our block index. Given that, we can establish a new invariant that the background validation chainstate only needs to consider blocks leading to that snapshotted block entry as potential candidates for its tip. As a followup I would imagine that when writing net_processing logic to download blocks for the background chainstate, that we would use this concept to only download blocks towards the snapshotted entry as well.

ACKs for top commit:
  achow101:
    ACK a733dd79e29068ad1e0532ac42a45188a040a7b9
  jamesob:
    reACK a733dd79e29068ad1e0532ac42a45188a040a7b9 ([`jamesob/ackr/27746.5.sdaftuar.rework_validation_logic`](https://github.com/jamesob/bitcoin/tree/ackr/27746.5.sdaftuar.rework_validation_logic))
  Sjors:
    Code review ACK a733dd79e29068ad1e0532ac42a45188a040a7b9.
  ryanofsky:
    Code review ACK a733dd79e29068ad1e0532ac42a45188a040a7b9. Just suggested changes since the last review. There are various small things that could be followed up on, but I think this is ready for merge.

Tree-SHA512: 9ec17746f22b9c27082743ee581b8adceb2bd322fceafa507b428bdcc3ffb8b4c6601fc61cc7bb1161f890c3d38503e8b49474da7b5ab1b1f38bda7aa8668675
2023-07-31 16:18:20 -04:00
MarcoFalke
fa69e3a95c
Remove unused MessageStartChars parameters from BlockManager methods 2023-07-31 14:32:57 +02:00
stickies-v
547fa52443
net processing: clamp -blockreconstructionextratxn to uint32_t bounds
Also changes max_extra_txs into a uint32_t to avoid platform-specific
behaviour
2023-07-25 21:51:20 +01:00
stickies-v
e451d1e3c6
net processing: clamp -maxorphantx to uint32_t bounds 2023-07-25 21:50:37 +01:00
stickies-v
5f41afcc46
refactor: set ignore_incoming_txs in ApplyArgsManOptions
Refactor to consistently use ApplyArgsManOptions to set all PeerManager::Options,
including ignore_incoming_txs.
2023-07-25 14:34:06 +01:00
dergoegge
23c7b51ddd [net processing] Move -capturemessages to PeerManager::Options 2023-07-24 18:35:30 +02:00
dergoegge
bd59bda26b [net processing] Move -blockreconstructionextratxn to PeerManager::Options 2023-07-24 18:35:30 +02:00
dergoegge
567c4e0b6a [net processing] Move -maxorphantx to PeerManager::Options 2023-07-24 18:35:30 +02:00
dergoegge
fa9e6d80d1 [net processing] Move -txreconciliation to PeerManager::Options 2023-07-24 18:35:28 +02:00
dergoegge
8b87725921 [net processing] Introduce PeerManager options 2023-07-24 18:30:59 +02:00
Suhas Daftuar
d0d40ea9a6 Move block-storage-related logic to ChainstateManager
Separate the notion of which blocks are stored on disk, and what data is in our
block index, from what tip a chainstate might be able to get to. We can use
chainstate-agnostic data to determine when to store a block on disk (primarily,
an anti-DoS set of criteria) and let the chainstates figure out for themselves
when a block is of interest for being a candidate tip.

Note: some of the invariants in CheckBlockIndex are modified, but more work is
needed (ie to move CheckBlockIndex to ChainstateManager, as most of what
CheckBlockIndex is doing is checking the consistency of the block index, which
is outside of Chainstate).
2023-07-21 10:09:44 -04:00
Martin Zumsande
ad66ca1e47 init: abort loading of blockindex in case of missing height.
If a height is missing we are facing a non-contiguous block index db, and could previously
hit an assert in GetAncestor() called from BuildSkip() instead of returning an error.
2023-07-18 11:29:40 -04:00
Suhas Daftuar
471da5f6e7 Move block-arrival information / preciousblock counters to ChainstateManager
Block arrival information (and the preciousblock RPC, a related concept) are
both chainstate-agnostic, so these are moved to ChainstateManager. This should
just be a refactor, without any observable behavior changes.
2023-07-14 17:09:06 -04:00
Suhas Daftuar
1cfc887d00 Remove CChain dependency in node/blockstorage 2023-07-14 14:54:57 -04:00
Suhas Daftuar
fe86a7cd48 Explicitly track maximum block height stored in undo files
When writing a new block to disk, if we have filled up the current block file,
then we flush and truncate that block file (to free allocated but unused
space) before advancing to the next one. When this happens, we have to
determine whether to also flush and truncate the corresponding undo file.

Undo data is only written when blocks are connected, not when blocks are
received. Thus it's possible that the corresponding undo file already has all
the data it will ever have, and we should flush/truncate it as we advance
files; or it's possible that there is more data we expect to write, and should
therefore defer flush/truncation until undo data is later written.

Prior to this commit, we made the determination of whether the undo file was
full of all requisite data by comparing against the chain tip. This patch
replaces that dependence on validation data structures by instead just tracking
the highest height of any block written in the undo file as we go.
2023-07-14 14:47:00 -04:00
Ryan Ofsky
31eca93a9e kernel: Remove StartShutdown calls from validation code
This change drops the last kernel dependency on shutdown.cpp. It also adds new
hooks for libbitcoinkernel applications to be able to interrupt kernel
operations when the chain tip changes.

This is a refactoring that does not affect behavior. (Looking at the code it
can appear like the new break statement in the ActivateBestChain function is a
change in behavior, but actually the previous StartShutdown call was indirectly
triggering a break before, because it was causing m_chainman.m_interrupt to be
true. The new code just makes the break more obvious.)
2023-07-11 12:30:56 -04:00
TheCharlatan
462390c85f
refactor: Move stopafterblockimport handling out of blockstorage
This has the benefit of moving the StartShutdown call out of the
blockstorage file and thus out of the kernel's responsibility. The user
can now decide if he wants to start shutdown / interrupt after a block
import or not.
2023-07-11 12:00:57 +02:00
furszy
ca91c244ef
index: verify blocks data existence only once
At present, during init, we traverse the chain (once per index)
to confirm that all necessary blocks to sync each index up to
the current tip are present.

To make the process more efficient, we can fetch the oldest block
from the indexers and perform the chain data existence check from
that point only once.

This also moves the pruning violation check to the end of the
'loadinit' thread, which is where the reindex, block loading and
chain activation processes happen.

Making the node's startup process faster, allowing us to remove
the global g_indexes_ready_to_sync flag, and enabling the
execution of the pruning violation verification even when the
reindex or reindex-chainstate flags are enabled (which has being
skipped so far).
2023-07-10 10:50:50 -03:00
furszy
2ec89f1970
refactor: simplify pruning violation check
By generalizing 'GetFirstStoredBlock' and implementing
'CheckBlockDataAvailability' we can dedup code and
avoid repeating work when multiple indexes are enabled.
E.g. get the oldest block across all indexes and
perform the pruning violation check from that point
up to the tip only once (this feature is being introduced
in a follow-up commit).

This commit shouldn't change behavior in any way.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-07-10 10:50:50 -03:00
furszy
c82ef91eae
make GetFirstStoredBlock assert that 'start_block' always has data
And transfer the responsibility of verifying whether 'start_block'
has data or not to the caller.

This is because the 'GetFirstStoredBlock' function responsibility
is to return the first block containing data. And the current
implementation can return 'start_block' when it has no data!. Which
is misleading at least.

Edge case behavior change:
Previously, if the block tip lacked data but all preceding blocks
contained data, there was no prune violation. And now, such
scenario will result in a prune violation.
2023-07-10 10:47:17 -03:00
furszy
225e213110
refactor: init indexes, decouple 'Start()' from the creation step
No behavior change.

The goal here is to group indexes, so we can perform the same
initialization and verification process equally for all of them.

The checks performed inside `StartIndexes` will be expanded
in the subsequent commits.
2023-07-07 19:31:27 -03:00
furszy
04575106b2
scripted-diff: rename 'loadblk' thread name to 'initload'
The thread does not only load blocks, it loads the mempool and,
in a future commit, will start the indexes as well.

Also, renamed the 'ThreadImport' function to 'ImportBlocks'
And the 'm_load_block' class member to 'm_thread_load'.

-BEGIN VERIFY SCRIPT-

sed -i "s/ThreadImport/ImportBlocks/g" $(git grep -l ThreadImport -- ':!/doc/')
sed -i "s/loadblk/initload/g" $(git grep -l loadblk -- ':!/doc/release-notes/')
sed -i "s/m_load_block/m_thread_load/g" $(git grep -l m_load_block)

-END VERIFY SCRIPT-
2023-07-07 19:31:27 -03:00
furszy
ed4462cc78
init: start indexes sync earlier
The mempool load can take a while, and it is not
needed for the indexes' synchronization.

Also, having the mempool load function call
inside 'blockstorage.cpp' wasn't structurally
correct.
2023-07-07 19:31:26 -03:00
TheCharlatan
6eb33bd0c2
kernel: Add fatalError method to notifications
FatalError replaces what previously was the AbortNode function in
shutdown.cpp.

This commit is part of the libbitcoinkernel project and further removes
the shutdown's and, more generally, the kernel library's dependency on
interface_ui with a kernel notification method. By removing interface_ui
from the kernel library, its dependency on boost is reduced to just
boost::multi_index. At the same time it also takes a step towards
de-globalising the interrupt infrastructure.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
2023-06-28 09:52:33 +02:00
TheCharlatan
7320db96f8
kernel: Add flushError method to notifications
This is done in addition with the following commit. Both have the goal
of getting rid of direct calls to AbortNode from kernel code. This extra
flushError method is added to notify specifically about errors that
arrise when flushing (syncing) block data to disk. Unlike other
instances, the current calls to AbortNode in the blockstorage flush
functions do not report an error to their callers.

This commit is part of the libbitcoinkernel project and further removes
the shutdown's and, more generally, the kernel library's dependency on
interface_ui with a kernel notification method. By removing interface_ui
from the kernel library, its dependency on boost is reduced to just
boost::multi_index. At the same time it also takes a step towards
de-globalising the interrupt infrastructure.
2023-06-28 09:52:32 +02:00
TheCharlatan
edb55e2777
kernel: Pass interrupt reference to chainman
This and the following commit seek to decouple the libbitcoinkernel
library from the shutdown code. As a library, it should it should have
its own flexible interrupt infrastructure without relying on node-wide
globals.

The commit takes the first step towards this goal by de-globalising
`ShutdownRequested` calls in kernel code.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
2023-06-28 09:52:27 +02:00
Andrew Chow
caff95a023
Merge bitcoin/bitcoin#27896: Remove the syscall sandbox
32e2ffc39374f61bb2435da507f285459985df9e Remove the syscall sandbox (fanquake)

Pull request description:

  After initially being merged in #20487, it's no-longer clear that an internal syscall sandboxing mechanism is something that Bitcoin Core should have/maintain, especially when compared to better maintained/supported alterantives, i.e [firejail](https://github.com/netblue30/firejail).

  There is more related discussion in #24771.

  Note that given where it's used, the sandbox also gets dragged into the kernel.

  If it's removed, this should not require any sort of deprecation, as this was only ever an opt-in, experimental feature.

  Closes #24771.

ACKs for top commit:
  davidgumberg:
     crACK 32e2ffc393
  achow101:
    ACK 32e2ffc39374f61bb2435da507f285459985df9e
  dergoegge:
    ACK 32e2ffc39374f61bb2435da507f285459985df9e

Tree-SHA512: 8cf71c5623bb642cb515531d4a2545d806e503b9d57bfc15a996597632b06103d60d985fd7f843a3c1da6528bc38d0298d6b8bcf0be6f851795a8040d71faf16
2023-06-27 18:19:21 -04:00
Andrew Chow
6a473373d4
Merge bitcoin/bitcoin#27862: validation: Stricter assumeutxo error handling when renaming chainstates
1c7d08b9acd33aff343228ada7e058e606cb1062 validation: Stricter assumeutxo error handling in InvalidateCoinsDBOnDisk (Ryan Ofsky)
9047337d369d800e6eca4d3b686139073a8e8905 validation: Stricter assumeutxo error handling in LoadChainstate (Ryan Ofsky)

Pull request description:

  There are two places in assumeutxo code where it is calling `AbortNode` to trigger asynchronous shutdowns without returning errors to calling functions.

  One case, in `LoadChainstate`, happens when snapshot validation succeeds, and there is an error trying to replace the background chainstate with the snapshot chainstate.

  The other case, in `InvalidateCoinsDBOnDisk`, happens when snapshot validatiion fails, and there is an error trying to remove the snapshot chainstate.

  In both cases the node is being forced to shut down, so it makes sense for these functions to raise errors so callers can know that an error happened without having to infer it from the shutdown state.

  Noticed these cases while reviewing #27861, which replaces the `AbortNode` function with a `FatalError` function.

ACKs for top commit:
  achow101:
    ACK 1c7d08b9acd33aff343228ada7e058e606cb1062
  TheCharlatan:
    ACK 1c7d08b9acd33aff343228ada7e058e606cb1062
  jamesob:
    ACK 1c7d08b9acd33aff343228ada7e058e606cb1062 ([`jamesob/ackr/27862.1.ryanofsky.validation_stricter_assu`](https://github.com/jamesob/bitcoin/tree/ackr/27862.1.ryanofsky.validation_stricter_assu))

Tree-SHA512: fb1dcde3fa0e77b4ba0c48507d289552b939c2866781579c8e994edc209abc3cd29cf81c89380057199323a8eec484956abb1fd3a43c957ecd0e7f7bbfd63fd8
2023-06-22 13:20:36 -04:00
fanquake
32e2ffc393
Remove the syscall sandbox
After initially being merged in #20487, it's no-longer clear that an
internal syscall sandboxing mechanism is something that Bitcoin Core
should have/maintain, especially when compared to better
maintained/supported alterantives, i.e firejail.

Note that given where it's used, the sandbox also gets dragged into the
kernel.

There is some related discussion in #24771.

This should not require any sort of deprecation, as this was only ever
an opt-in, experimental feature.

Closes #24771.
2023-06-16 10:38:19 +01:00
Ryan Ofsky
9047337d36 validation: Stricter assumeutxo error handling in LoadChainstate
Make LoadChainstate return an explicit error when snapshot validation succeeds,
but there is an error trying to replace the background chainstate with the
snapshot chainstate. Previously in this case LoadChainstate would trigger a
shutdown and return INTERRUPTED, now it will return an actual error code.

There's no real change to behavior other than error message being formatted a
little differently.

Motivation for this change is to replace error handling via callbacks with
error handling via return value ahead of
https://github.com/bitcoin/bitcoin/pull/27861
2023-06-15 15:11:32 -04:00
Jon Atack
daa5a658c0 refactor: rename BCLog::BLOCKSTORE to BLOCKSTORAGE
so the enum name is the same as its value, like the other BCLog enums.
2023-06-15 10:27:56 -06:00
Ryan Ofsky
c92fd63886
Merge bitcoin/bitcoin#27708: Return EXIT_FAILURE on post-init fatal errors
61c569ab6069d04079a0831468eb713983919636 refactor: decouple early return commands from AppInit (furszy)
4927167f855f8ed3bbf6d2766f61229f742e632a gui: return EXIT_FAILURE on post-init fatal errors (furszy)
3b2c61e8198bcefb1c2343caff1d705951026cc4 Return EXIT_FAILURE on post-init fatal errors (furszy)
3c06926cf21dcca3074ef51506f556b2286c299b refactor: index: use `AbortNode` in fatal error helper (Sebastian Falbesoner)
9ddf7e03a35592617a016418fd320cc93c8d1abd move ThreadImport ABC error to use AbortNode (furszy)

Pull request description:

  It seems odd to return `EXIT_SUCCESS` when the node aborted execution due a fatal internal error
  or any post-init problem that triggers an unrequested shutdown.

  e.g. blocks or coins db I/O errors, disconnect block failure, failure during thread import (external
  blocks loading process error), among others.

ACKs for top commit:
  TheCharlatan:
    ACK 61c569ab6069d04079a0831468eb713983919636
  ryanofsky:
    Code review ACK 61c569ab6069d04079a0831468eb713983919636
  pinheadmz:
    ACK 61c569ab6069d04079a0831468eb713983919636
  theStack:
    Code-review ACK 61c569ab6069d04079a0831468eb713983919636

Tree-SHA512: 18a59c3acc1c6d12cbc74a20a401e89659740c6477fccb59070c9f97922dfe588468e9e5eef56c5f395762187c34179a5e3954aa5b844787fa13da2e666c63d3
2023-06-12 12:54:49 -04:00
fanquake
361a0c00b3
Merge bitcoin/bitcoin#27783: Add public Boost headers explicitly
2484cacb7a6367b24e924dba0825c843b1dfc1c3 Add public Boost headers explicitly (Hennadii Stepanov)
fade2adb5bb4ce9753e7f25da5fb1521f2f503ec test: Avoid `BOOST_ASSERT` macro (Hennadii Stepanov)

Pull request description:

  To check symbols in the code base, run:
  ```
  git grep boost::multi_index::identity
  git grep boost::multi_index::indexed_by
  git grep boost::multi_index::tag
  git grep boost::make_tuple
  ```

  Hoping on the absence of conflicts with top-prio PRs :)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 2484cacb7a6367b24e924dba0825c843b1dfc1c3
  TheCharlatan:
    ACK 2484cacb7a6367b24e924dba0825c843b1dfc1c3

Tree-SHA512: d122ab028eee76ee1c4609ed51ec8db0c8c768edcc2ff2c0e420a48e051aa71e99748cdb5d22985ae6d97c808c77c1a27561f0715f77b256f74c1c310b37694c
2023-06-12 16:53:16 +01:00
furszy
4927167f85
gui: return EXIT_FAILURE on post-init fatal errors 2023-06-10 11:10:29 -03:00
furszy
3b2c61e819
Return EXIT_FAILURE on post-init fatal errors
It seems odd to return `EXIT_SUCCESS` when the node aborted
execution due a fatal internal error or any post-init problem
that triggers an unrequested shutdown.

e.g. blocks or coins db I/O errors, disconnect block failure,
failure during thread import (external blocks loading process
error), among others.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-06-09 17:52:23 -03:00
Ryan Ofsky
153a6882f4
Merge bitcoin/bitcoin#27576: kernel: Remove args, settings, chainparams, chainparamsbase from kernel library
db77f87c6365cb5f414036d6bfb1a12705772028 scripted-diff: move settings to common namespace (TheCharlatan)
c27e4bdc35bc7cedd1ee07e98a52c230241120d1 move-only: Move settings to the common library (TheCharlatan)
c2dae5d7d89634fbd771755ce3909719f5462f63 kernel: Remove chainparams, chainparamsbase, args, settings from kernel library (TheCharlatan)
05870b1c92f39d90e5ba6e0caf2f6c2b37955528 refactor: Remove gArgs access from validation.cpp (TheCharlatan)
8789b11114b4bd6c7ee727dffbc75a6bdf20dd27 refactor: Add path argument to FindSnapshotChainstateDir (TheCharlatan)
ef95be334f3aec671346372b64606e0fd390979a refactor: Add stop_at_height option in ChainstateManager (TheCharlatan)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/27587 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel".

  ---

  This completes the removal of the node's chainparams, chainparamsbase, args and settings files and their respective classes from the kernel library. This is the last pull request in a long series working towards decoupling the `ArgsManager` and the `gArgs` global from kernel code. These prior pull requests are: https://github.com/bitcoin/bitcoin/pull/26177 https://github.com/bitcoin/bitcoin/pull/27125 https://github.com/bitcoin/bitcoin/pull/25527 https://github.com/bitcoin/bitcoin/pull/25487 https://github.com/bitcoin/bitcoin/pull/25290

ACKs for top commit:
  MarcoFalke:
    lgtm ACK db77f87c6365cb5f414036d6bfb1a12705772028 🍄
  hebasto:
    ACK db77f87c6365cb5f414036d6bfb1a12705772028, I have reviewed the code and it looks OK.
  ryanofsky:
    Code review ACK db77f87c6365cb5f414036d6bfb1a12705772028. Looks great!

Tree-SHA512: cbfbd705d056f2f10f16810d4f869eb152362fff2c5ddae5e1ac6785deae095588e52ad48b29d921962b085e51de1e0ecab6e50f46149ffe3c16250608a2c93a
2023-06-09 14:58:49 -04:00
furszy
9ddf7e03a3
move ThreadImport ABC error to use AbortNode
'StartShutdown' should only be used for user requested
shutdowns. Internal errors that cause a shutdown should
use 'AbortNode'.
2023-06-08 16:38:36 -03:00
Murch
5d718f6913
Mitigate timeout in CalculateTotalBumpFees
The slow fuzz seed described in #27799 was just slower than expected,
not an endless loop. Ensuring that every anscestor is only processed
once speeds up the termination of the graph traversal.

Fixes #27799
2023-06-01 18:04:44 -04:00