Because the default wallet has no name, the watch-only and solvables
wallets created during migration end up having no name either.
This fixes it by applying the same prefix name we use for the backup
file for an unnamed default wallet.
Before: watch-only wallet named "_watchonly"
After: watch-only wallet named "default_wallet_watchonly"
Github-Pull: bitcoin/bitcoin#34156
Rebased-From: 82caa8193a3e36f248dcc949e0cd41def191efac
Right now, after migration the last message users see is "migration completed",
but the migration isn't actually finished yet. We still need to load the new wallets
to ensure consistency, and if that fails, the migration will be rolled back. This
can be confusing for users.
This change logs the post-migration loading step and if a wallet fails to load and
the migration will be rolled back.
Github-Pull: bitcoin/bitcoin#34156
Rebased-From: d70b159c42008ac3b63d1c43d99d4f1316d2f1ef
When migrating any legacy unnamed wallet, a failed migration would
cause the cleanup logic to remove its parent directory. Since this
type of legacy wallet lives directly in the main '/wallets/' folder,
this resulted in unintentionally erasing all wallets, including the
backup file.
To be fully safe, we will no longer call `fs::remove_all`. Instead,
we only erase the individual db files we have created, leaving
everything else intact. The created wallets parent directories are
erased only if they are empty.
As part of this last change, `RestoreWallet` was modified to allow
an existing directory as the destination, since we no longer remove
the original wallet directory (we only remove the files we created
inside it). This also fixes the restore of top-level default wallets
during failures, which were failing due to the directory existence
check that always returns true for the /wallets/ directory.
This bug started after:
f6ee59b6e2
Previously, the `fs::copy_file` call was failing for top-level wallets,
which prevented the `fs::remove_all` call from being reached.
Github-Pull: bitcoin/bitcoin#34156
Rebased-From: f4c7e28e80bf9af50b03a770b641fd309a801589
Track what RestoreWallet creates so only those files and directories
are removed during a failure and nothing else. Preexisting paths
must be left untouched.
Note:
Using fs::remove_all() instead of fs::remove() in RestoreWallet does
not cause any problems currently, but the change is necessary for the
next commit which extends RestoreWallet to work with existing directories,
which may contain files that must not be deleted.
Github-Pull: bitcoin/bitcoin#34156
Rebased-From: 4ed0693a3f2a427ef9e7ad016930ec29fa244995
Not enforcing TRUC topology on reorg was the intended
behavior, but the appropriate bypass argument was not
checked.
This mistake means we could potentially invalidate a long
chain of perfectly incentive-compatible transactions that
were made historically, including subsequent non-TRUC
transactions, all of which may have been very high feerate.
Lastly, it wastes CPU cycles doing topology checks since
this behavior cannot actually enforce the topology in
general for the reorg setting.
Github-Pull: #33504
Rebased-From: 26e71c237d9d2197824b547f55ee3a0a60149f92
Using bypass_limits=true is essentially fuzzing part of a
reorg only, and results in TRUC invariants unable to be
checked. Remove most instances of bypassing limits, leaving
one harness able to do so.
Github-Pull: #33504
Rebased-From: bbe8e9063c15dc230553e0cbf16d603f5ad0e4cf
Let's say an attacker wants to use/exhaust the network's bandwidth, and
has the choice between renting resources from a commercial provider and
getting the network to "spam" itself it by sending unconfirmed
transactions. We'd like the latter to be more expensive than the former.
The bandwidth for relaying a transaction across the network is roughly
its serialized size (plus relay overhead) x number of nodes. A 1000vB
transaction is 1000-4000B serialized. With 100k nodes, that's 0.1-0.4GB
If the going rate for commercial services is 10c/GB, that's like 1-4c per kvB
of transaction data, so a 1000vB transaction should pay at least $0.04.
At a price of 120k USD/BTC, 100sat is about $0.12. This price allows us
to tolerate a large decrease in the conversion rate or increase in the
number of nodes.
Github-Pull: #33106
Rebased-From: 6da5de58cabc4133c379baa50845e30e5bc6b3e4
Use a virtual size of 1000 to keep precision when using a feerate
(which is rounded to the nearest satoshi per kvb) that isn't just an
integer.
Github-Pull: #33106
Rebased-From: 457cfb61b5323a13218b3cfb5a6a6d8b3a7c5f7f
Back when we implemented coin age priority as a miner policy, miners
mempools might admit transactions paying very low fees, but then want to
set a higher fee for block inclusion. However, since coin age priority
was removed in v0.15, the block assembly policy is solely based on fees,
so we do not need to apply minimum feerate rules in multiple places. In
fact, the block assembly policy ignoring transactions that are added to
the mempool is likely undesirable as we waste resources accepting and
storing this transaction.
Instead, rely on mempool policy to enforce a minimum entry feerate to
the mempool (minrelaytxfee). Set the minimum block feerate to the
minimum non-zero amount (1sat/kvB) so it collects everything it finds in
mempool into the block.
Github-Pull: #33106
Rebased-From: 5f2df0ef78be7b24798d0983c9b962740608f1f4
The padding method used matches the one used in MiniWallet,
`MiniWallet._bulk_tx`.
Github-Pull: #30784
Rebased-From: ed7d2246661ec1789b7db0f21668270f0681ea4a
Tor inbound connections do not reveal the peer's actual network address.
Therefore do not apply whitelist permissions to them.
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Github-Pull: #33395
Rebased-From: f563ce90818d486d2a199439d2f6ba39cd106352
The `SHA256AutoDetect` return output is used, among other use cases, to
name benchmarks. Using a comma breaks the CSV output.
This change replaces the comma with a semicolon, which fixes the issue.
Github-Pull: #33340
Rebased-From: 790b440197bde322432a5bab161f1869b667e681
The getpeerinfo docs incorrectly specified the ping durations as
milliseconds. This was incorrectly changed in a3789c700b5a43efd4b366b4241ae840d63f2349
(released in v25; master since Sept. 2022). The correct duration unit
is seconds.
Also, remove the documentation of the getpeerinfo RPC response from the
ping RPC since it's incomplete. Better to just reference the getpeerinfo
RPC and it's documenation for this.
Github-Pull: #33133
Rebased-From: 1252eeb997df2eb12c33d92eb1a5c9d6643a67ff
The mutex (required by TestBlockValidity) must be held after creating
the block, until TestBlockValidity is called. Otherwise, it is possible
that the chain advances in the meantime and leads to a crash in
TestBlockValidity:
Assertion failed: pindexPrev && pindexPrev == chainstate.m_chain.Tip() (validation.cpp: TestBlockValidity: 4338)
The diff can be reviewed with the git options
--ignore-all-space --function-context
Github-Pull: 31563
Rebased-From: fa62c8b1f04a5386ffa171aeff713d55bd874cbe
After port collisions are no longer tolerated but lead to
a startup failure in v28.0, local setups of multiple nodes,
each with a different -port value would not be possible anymore
due to collision of the onion default port - even if the nodes
were using tor or not interested in receiving onion inbound connections.
Fix this by deriving the onion listening port to be -port + 1.
(idea by vasild / laanwj)
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Github-Pull: bitcoin/bitcoin#31223
Rebased-From: 0e2b12b92a28a2949e75bf50f31563f52e647d6e
The current code does not have a bug, but is implicitly casting -1 to
65535 and the sanitizer has no way to know whether we intend that or
not.
```
FUZZ=bitset src/test/fuzz/fuzz /tmp/fuz
error: implicit conversion from type 'int' of value -1 (32-bit, signed)
to type 'value_type' (aka 'unsigned short') changed the value to 65535
(16-bit, unsigned)
Base64: Qv7bX/8=
```
Github-Pull: bitcoin/bitcoin#31431
Rebased-From: edb41e4814ccc2c06a5694b2d2632dbbd22bc0cf
Same as https://github.com/llvm/llvm-project/pull/113951.
Avoids compile failures under clang-20 &
`D_LIBCPP_REMOVE_TRANSITIVE_INCLUDES`:
```bash
In file included from /bitcoin/src/test/fuzz/addition_overflow.cpp:5:
/bitcoin/src/test/fuzz/FuzzedDataProvider.h:209:5: error: use of undeclared identifier 'abort'
209 | abort();
| ^
/bitcoin/src/test/fuzz/FuzzedDataProvider.h:250:5: error: use of undeclared identifier 'abort'
250 | abort();
```
Github-Pull: bitcoin/bitcoin#31448
Rebased-From: bb7e686341e437b2e7aae887827710918c00ae0f
With nId being incremented for each addr received,
an attacker could cause an overflow in the past.
(https://bitcoincore.org/en/2024/07/31/disclose-addrman-int-overflow/)
Even though that attack was made infeasible by
rate-limiting (PR #22387), to be on the safe side change the
type to an int64_t.
Github-Pull: #30568
Rebased-From: 51f7668d31e2624e41c7ce77fe33162802808f3f
This makes it easier to track which spots refer to an nId
(as opposed to, for example, bucket index etc. which also use int)
Co-authored-by: Pieter Wuille <pieter@wuille.net>
Github-Pull: #30568
Rebased-From: 051ba3290e30e210bfc50dea974063053313ad3e
Same as in `DecodeSecret`, we should also clear out the secret data from
the vector resulting from the Base58Check parsing for xprv keys. Note
that the if condition is needed in order to avoid UB, see #14242 (commit
d855e4cac8303ad4e34ac31cfa7634286589ce99).
Github-Pull: #31166
Rebased-From: 559a8dd9c0aafcecf00f9ccd9aabe5720bcebe8c
The comparison of m_best_invalid with the tip of the respective chainstate
makes no sense for the background chainstate, and can lead to incorrect
error messages.
Github-Pull: bitcoin/bitcoin#30962
Rebased-From: c0a0c72b4d68a4f0c53c2c4b95f4d6e399f8e4ee
The recent translations from Transifex.com 28.x fetched with the
bitcoin-maintainer-tools/update-translations.py tool.
Github-Pull: bitcoin/bitcoin#30899
Rebased-From: ae0529576147a1a5bee992574e2cefc8a1fa37d0
The crash occurs because 'WalletController::removeAndDeleteWallet' is called
twice for the same wallet model: first in the GUI's button connected function
'WalletController::closeWallet', and then again when the backend emits the
'WalletModel::unload' signal.
This causes the issue because 'removeAndDeleteWallet' inlines an
erase(std::remove()). So, if 'std::remove' returns an iterator to the end
(indicating the element wasn't found because it was already erased), the
subsequent call to 'erase' leads to an undefined behavior.
Github-Pull: bitcoin-core/gui#835
Rebased-From: a965f2bc07a3588f8c2b8d6a542961562e3f5d0e
Because AssumeUTXO nodes prioritize tip synchronization, they relay their local
address through the network before completing the background chain sync.
This, combined with the advertising of full-node service (NODE_NETWORK), can
result in an honest peer in IBD connecting to the AssumeUTXO node (while syncing)
and requesting an historical block the node does not have. This behavior leads to
an abrupt disconnection due to perceived unresponsiveness (lack of response)
from the AssumeUTXO node.
This lack of response occurs because nodes ignore getdata requests when they do
not have the block data available (further discussion can be found in PR 30385).
Fix this by refraining from signaling full-node service support while the
background chain is being synced. During this period, the node will only
signal 'NODE_NETWORK_LIMITED' support. Then, full-node ('NODE_NETWORK')
support will be re-enabled once the background chain sync is completed.
Github-Pull: bitcoin/bitcoin#30807
Rebased-From: 6d5812e5c852c233bd7ead2ceef051f8567619ed
the run_command test under system_tests fails if the locale is anything
other than English ones because results such as "No such file or directory"
will be different under Non-English locales.
On the old version, a `ls nonexistingfile` was used to generate the error
output which is not ideal. In the current version we are using a Python one-liner
to generate a non 0 zero return value and "err" on stderr and check the
expected value against this.
fixes#30608
Github-Pull: #30788
Rebased-From: ae48a22a3df086fb59843b7b814619ed5df7557b
b061b3510585a1fe113cc9d1af65852b155aba45 seeds: Regenerate mainnet seeds (virtu)
02dc45c506f78eae96b5fe8e8e4899b45811da05 seeds: Pull nodes from Luke's seeder (virtu)
7a2068a0ff9eec2bab436b47eba37fd34b71bba4 seeds: Pull nodes from virtu's crawler (virtu)
Pull request description:
This builds on #30008 and adds data [exported](https://github.com/virtu/seed-exporter) by [my crawler](https://github.com/virtu/p2p-crawler) an additional source for seed nodes. Data covers all supported network types.
[edit: Added Luke's seeder as input as well.]
### Motivation
- Further decentralizes the seed node selection process (in the long term potentially enabling an _n_-source threshold for nodes to prevent a single source from entering malicious nodes)
- No longer need to manually curate seed node list for any network type: See last paragraph of OP in #30008. My crawler has been [discovering the handful of available cjdns nodes](https://21.ninja/reachable-nodes/nodes-by-net-type/) for around two months, all but one of which meet the reliability criteria.
- Alignment of uptime requirements for Onion and I2P nodes with those of clearnet nodes to 50%: If I'm reading the code correctly, seeders appear to optimize for up-to-dateness by using [lower connection timeouts](3c1a63c672/src/crawl.rs (L349)) than [Bitcoin Core](bc87ad9854/src/netbase.cpp (L40C27-L40C48)) to maximize throughput. Since my crawler does not have the same timeliness requirements, it opts for accuracy by using generous timeouts. As a result, its data contains additional eligible Onion (and other darknet nodes), as is shown in the histogram below. Around 4500 Onion nodes are discovered so far (blue); my data adds ~6400 more (orange); ~ 1500 nodes take longer than the default 20-second Bitcoin Core timeout and won't qualify as "good".
Here's the current results with 512 nodes for all networks except cjdns:
<details>
<summary>Using the extra data</summary>
```
IPv4 IPv6 Onion I2P CJDNS Pass
10335 2531 11545 1589 10 Initial
10335 2531 11545 1589 10 Skip entries with invalid address
5639 1431 11163 1589 8 After removing duplicates
5606 1417 11163 1589 8 Enforce minimal number of blocks
5606 1417 11163 1589 8 Require service bit 1
4873 1228 11163 1589 8 Require minimum uptime
4846 1225 11161 1588 8 Require a known and recent user agent
4846 1225 11161 1588 8 Filter out hosts with multiple bitcoin ports
512 512 512 512 8 Look up ASNs and limit results per ASN and per net
```
</details>
<details>
<summary>Before</summary>
```
IPv4 IPv6 Onion I2P CJDNS Pass
5772 1323 443 0 2 Initial
5772 1323 443 0 2 Skip entries with invalid address
4758 1110 443 0 2 After removing duplicates
4723 1094 443 0 2 Enforce minimal number of blocks
4723 1094 443 0 2 Require service bit 1
3732 867 443 0 2 Require minimum uptime
3718 864 443 0 2 Require a known and recent user agent
3718 864 443 0 2 Filter out hosts with multiple bitcoin ports
512 409 443 0 2 Look up ASNs and limit results per ASN and per net
```
</details>
### To dos
- [x] Remove manual nodes and update README
- [x] Mark nodes with connection times exceeding Bitcoin Core's default as bad in [exporter](https://github.com/virtu/seed-exporter): [done](https://github.com/virtu/seed-exporter/pull/12)
- [x] Regenerate mainnet seeds
- [x] Rebase, then remove WIP label once #30008 gets merged
ACKs for top commit:
achow101:
ACK b061b3510585a1fe113cc9d1af65852b155aba45
fjahr:
utACK b061b3510585a1fe113cc9d1af65852b155aba45
Tree-SHA512: 63e86220787251c7e8d2d5957bad69352e19ae17d7b9b2d27d8acddfec5bdafe588edb68d77d19c57f25f149de723e2eeadded0c8cf13eaca22dc33bd8cf92a0
1b41d45d462d856a9d0b44ae0039bbb2cd78407c wallet: bugfix: ensure atomicity in settings updates (ismaelsadeeq)
Pull request description:
This PR fixes#30620.
As outlined in the issue, creating two wallets with `load_on_startup=true` simultaneously results in only one wallet being added to the startup file.
The current issue arises because the wallet settings update process involves:
1. Obtaining the settings value while acquiring the settings lock.
2. Modifying the settings value.
3. Overwriting the settings value while acquiring the settings lock again.
This sequence is not thread-safe. Different threads could modify the same base value simultaneously, overwriting data from other workers without realizing it.
The PR attempts to fix this by modifying the chain interface's `updateRwSetting` method to accept a function that will be called with the settings reference. This function will either update or delete the setting and return an enum indicating whether the settings need to be overwritten in this or not.
Additionally, this PR introduces two new methods to the chain interface:
- `overwriteRwSetting`: This method replaces the setting with a new value.
Used in `VerifyWallets`
- `deleteRwSettings`: This method completely erases a specified setting.
This method is currently used only in `overwriteRwSetting`.
These changes ensure that updates are race-free across all clients.
ACKs for top commit:
achow101:
ACK 1b41d45d462d856a9d0b44ae0039bbb2cd78407c
furszy:
self-code-ACK 1b41d45d46
Tree-SHA512: 50cda612b782aeb5e03e2cf63cc44779a013de1c535b883b57af4de22f24b0de80b4edecbcda235413baec0a12bdf0e5750fb6731c9e67d32e742d8c63f08c13
Regenerate mainnet seeds from new sources without the need for hardcoded
data. Result has 512 nodes from each network type except cjdns, for
which only eight nodes were found that match the seed node criteria.
41ad84a00c20f54b520aab7f6f975231da0ee2d0 seeds: Use fjahr's more up to date asmap (Ava Chow)
d8fd1e0fafa144a9ff96fc646cf9f21e220d5cd6 seeds: Fixed seeds update (Ava Chow)
f1f24d72141dcd2955420195135cabe5092017ff seeds: Add testnet4 fixed seeds file (Ava Chow)
8ace71c73750e3b537784178f3fc299447c461ed seeds: Remove manual onion and i2p seeds (Ava Chow)
ed5b86cbe47676276f8ff1a48001d5ecd560e153 seeds: Add testnet instructions (Ava Chow)
0676515397fcc8fb580973047e60279ce65bec48 seeds: Also pull from achow101 seeder (Ava Chow)
5bab3175a663610070c1000dd4211a58490e5023 makeseeds: Configurable minimum blocks for testnet4's smaller chain (Ava Chow)
d2465dfac68f96ffdaad88a0bd4891ed37cbfdfc makeseeds: Shuffle ips after parsing (Ava Chow)
af550b3a0fd406f175f197ea9867b41ff4e97af4 makeseeds: Support CJDNS (Ava Chow)
d5a8c4c4bd76f296f4d744184dc80a6a6a0731bd makeseeds: Update user agent regex (Ava Chow)
Pull request description:
The [DNS seeder](https://github.com/achow101/dnsseedrs) that I wrote collects statistics on node reliability in the same way that sipa's seeder does, and also outputs this information in the same file format. Thus it can also be used in our fixed seeds update scripts. My seeder additionally crawls onion v3, i2p, and cjdns, so will now be able to set those fixed seeds automatically rather than curating manual lists.
In doing this update, I've found that `makeseeds.py` is missing newer versions from the regex as well as cjdns support; both of these have been updated.
I also noticed that the testnet fixed seeds are all manually curated and sipa's seeder does not appear to publish any testnet data. Since I am also running the seeder for testnet, I've added the commands to generate testnet fixed seeds from my seeder's data too.
Lastly, I've updated all of the fixed seeds. However, since my seeder has not found any cjdns nodes that met the reliability criteria (possibly due to connectivity issues present in those networks), I've left the previous manual seeds for that network.
ACKs for top commit:
fjahr:
re-ACK 41ad84a00c20f54b520aab7f6f975231da0ee2d0
virtu:
ACK [41ad84a](41ad84a00c)
Tree-SHA512: 6ba0141f053d9d6ae7d8c9574f061be38f3e65b28de1d6660c1885ab942623b5a0ec70754b4fcfc5d98fe970f5f179a940d5880b5061ed698f7932500e01d3ee
- Settings updates were not thread-safe, as they were executed in
three separate steps:
1) Obtain settings value while acquiring the settings lock.
2) Modify settings value.
3) Overwrite settings value while acquiring the settings lock.
This approach allowed concurrent threads to modify the same base value
simultaneously, leading to data loss. When this occurred, the final
settings state would only reflect the changes from the last thread
that completed the operation, overwriting updates from other threads.
Fix this by making the settings update operation atomic.
- Add test coverage for this behavior.
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
60055f1abc4b4ad5f66a2fcf2e61c65efc777036 test: replace deprecated secp256k1 context flags usage (Sebastian Falbesoner)
Pull request description:
The flags `SECP256K1_CONTEXT_{SIGN,VERIFY}` have been marked as deprecated since libsecp256k1 version 0.2 (released in December 2022), with the recommendation to use SECP256K1_CONTEXT_NONE instead, see https://github.com/bitcoin-core/secp256k1/pull/1126 and 1988855079/CHANGELOG.md (L132). Note that in contrast to other deprecated functions/variables, these defines don't have a deprecated attribute and hence don't lead to a compiler warning (see https://github.com/bitcoin-core/secp256k1/pull/1126#discussion_r922105271), so they are not easily detected.
ACKs for top commit:
TheCharlatan:
ACK 60055f1abc4b4ad5f66a2fcf2e61c65efc777036
ismaelsadeeq:
utACK 60055f1abc4b4ad5f66a2fcf2e61c65efc777036
tdb3:
light CR and test ACK 60055f1abc4b4ad5f66a2fcf2e61c65efc777036
Tree-SHA512: d93cf49e018a58469620c0d2f50242141f22dabc70afb2a7cd64e416f4f55588714510ae5a877376dd1e6b6f7494261969489af4b18a1c9dff0d0dfdf93f1fa8
59ff17e5af4e382cbe16f183767beef1bdcd9131 miner: adjust clock to timewarp rule (Sjors Provoost)
e929054e12210353812f440c685a23329e7040f7 Add timewarp attack mitigation test (Sjors Provoost)
e85f386c4b157b7d1ac16aface9bd2c614e62b46 consensus: enable BIP94 on regtest (Sjors Provoost)
dd154b05689c60fad45df0df6d31cec12e09ab21 consensus: lower regtest nPowTargetTimespan to 144 (Sjors Provoost)
Pull request description:
Because #30647 reduced the timewarp attack threshold from 7200s to 600s, our miner code will fail to propose a block template (on testnet4) if the last block of the previous period has a timestamp two hours in the future. This PR fixes that and also adds a test.
The non-test changes in the last commit should be in v28, otherwise miners have to patch it themselves. If necessary I can split that out into a separate PR, but I prefer to get the tests in as well.
In order to add the test, we activate BIP94 on regtest.
In order for the test to run faster, we reduce its difficulty retarget period to 144, the same number that's already used for softfork activation logic. Regtest does not actually adjust its difficulty, so this change has no effect (except for `getnetworkhashps`, see commit).
An alternative approach would be to run this test on testnet4, by hardcoding its first 2015 in the test suite. But since the timewarp mitigation is a serious candidate for a future mainnet softfork, it seems better to just deploy it on regtest.
The next commits add a test and fix the miner code.
The `MAX_TIMEWARP` constant is moved to `consensus.h` so both validation and miner code have access to it.
ACKs for top commit:
achow101:
ACK 59ff17e5af4e382cbe16f183767beef1bdcd9131
fjahr:
ACK 59ff17e5af4e382cbe16f183767beef1bdcd9131
glozow:
ACK 59ff17e5af4e382cbe16f183767beef1bdcd9131
Tree-SHA512: 50af9fdcba9b0d5c57e1efd5feffd870bd11b5318f1f8b0aabf684657f2d33ab108d5f00b1475fe0d38e8e0badc97249ef8dda20c7f47fcc1698bc1008798830
