b78990734621b8fe46c68a6e7edaf1fbd2f7d351 wallet: migration, avoid creating spendable wallet from a watch-only legacy wallet (furszy)
e86d71b749c08bde6002b9aa2baee824975a518a wallet: refactor, dedup wallet re-loading code (furszy)
1de423e0a08bbc63eed36c8772e9ef8b48e80fb8 wallet: introduce method to return all db created files (furszy)
d04f6a97ba9a55aa9455e1a805feeed4d630f59a refactor: remove sqlite dir path back-and-forth conversion (furszy)
Pull request description:
Currently, the migration process creates a brand-new descriptor wallet with no
connection to the user's legacy wallet when the legacy wallet lacks key material
and contains only watch-only scripts. This behavior is not aligned with user
expectations. If the legacy wallet contains only watch-only scripts, the migration
process should only generate a watch-only wallet instead.
TODO List:
* Explain that `migratewallet` renames the watch-only after migration, and
also that the wallet will not have keys enabled.
ACKs for top commit:
achow101:
ACK b78990734621b8fe46c68a6e7edaf1fbd2f7d351
pablomartin4btc:
tACK b78990734621b8fe46c68a6e7edaf1fbd2f7d351
rkrux:
LGTM ACK b78990734621b8fe46c68a6e7edaf1fbd2f7d351
Tree-SHA512: 1d583ac4b206fb477e9727daf4b5ad9c3e18b12d40e1ab4a61e8565da44c3d0327c892b51cf47b4894405d122e414cefb6b6366c357e02a74a7ca96e06762d83
6efbd1e1dcdfbe9eae2d5c22abab3ee616a75ff2 refactor: CTransaction equality should consider witness data (Cory Fields)
cbf9b2dab1d8800d63d65904ccfd64e1e439e510 mempool: codify existing assumption about duplicate txids during removal (Cory Fields)
e9331cd6ab2c756c56e8b27a2de2a6d4884c0c06 wallet: IsEquivalentTo should strip witness data in addition to scriptsigs (Cory Fields)
Pull request description:
I stumbled upon the `CTransaction` comparison operators while refactoring some nearby code. I found it surprising and not at all obvious that two transactions would test equal even if their witness data differed. It seems like an unnecessary potential footgun. Fix that by comparing against wtxid rather than txid.
Outside of tests, there were only 3 users of these functions in the code-base:
- Its use in the mempool has been replaced with an explicit txid comparison, as that's a tighter constraint and matches the old behavior. glozow suggested also upgrading this to an `Assume()`.
- Its use in the wallet was accidentally doing the correct thing by ignoring witness data. I've changed that to an explicit witness removal so that `IsEquivalentTo` continues to work as-intended.
- Its use in `getrawtransaction` is indifferent to the change.
ACKs for top commit:
maflcko:
review ACK 6efbd1e1dcdfbe9eae2d5c22abab3ee616a75ff2 🦋
achow101:
ACK 6efbd1e1dcdfbe9eae2d5c22abab3ee616a75ff2
glozow:
ACK 6efbd1e1dcdfbe9eae2d5c22abab3ee616a75ff2
Tree-SHA512: 89be424889f49e7e26dd2bdab7fbc8b2def59bf002ae8b94989b349ce97245f007d6c96e409a626cbf0de9df83ae2485b4815b40a70f7aa5b6c720eb34a6c017
8cc9845b8ddf4f93a02c622e7df8d1095dc1a640 wallet, rpc: Use `OUTPUT_TYPES` to describe the output types instead of hardcoding them (w0xlt)
Pull request description:
Follow-up to https://github.com/bitcoin/bitcoin/pull/32429, built on top of it.
This PR addresses the https://github.com/bitcoin/bitcoin/pull/32429#discussion_r2076251627 that the RPC documentation does not use `OUTPUT_TYPES`, but rather hardcodes them, as is already the case for the `getnewaddress` command.
So here the output types are changed from `std::string` to `std::string_view` so that the values are known at compile time or during the early stages of program startup, before main() execution.
It also updates `wallet/rpc/addresses.cpp` to write the RPC docs according to `OUTPUT_TYPES` instead of using hardcoded version.
It also updates the documentation in outputtypes.h, adding Doxygen comments,
ACKs for top commit:
maflcko:
lgtm ACK 8cc9845b8ddf4f93a02c622e7df8d1095dc1a640
achow101:
ACK 8cc9845b8ddf4f93a02c622e7df8d1095dc1a640
Tree-SHA512: e86d813d6d158dd2f6c62519a7ecaa878f2e4f686b5bae82028a106bd6671a13b10fb366f9bb7b94974777217e1852f38e8aa05bba00cd27f94f4412167a3562
When a legacy wallet has been migrated to contain descriptors, but
before the transactions have been updated to match, we need to recompute
the wallet TXOs so that the transaction update will work correctly.
Instead of searching mapWallet for the preselected inputs, search
m_txos.
wallet_fundrawtransaction.py spends external inputs and needs the change
output to also belong to the test wallet for the oversized tx test.
Instead of iterating every transaction and every output stored in wallet
when trying to figure out what outputs can be spent, iterate the TXO set
which should be a lot smaller.
Since we track the outputs owned by the wallet with m_txos, we can now
calculate the balance of the wallet by iterating m_txos and summing up
the amounts of the unspent txos.
As ISMINE_USED is not an actual isminetype that we attach to outputs and
was just passed into `CachedTxGetAvailableCredit` for convenience, we
pull the same determining logic from that function into `GetBalances` in
order to preserve existing behavior.
After adding a wallet descriptor (typically by import), mark all balance
caches dirty. This allows transactions that the wallet already knows
about that have outputs that are now ISMINE_SPENDABLE after the import
to actually be shown in balance calculations. Legacy wallet imports
would do this, but importdescriptors did not.
c3fe85e2d6dd4f251a62a99fd891b0fa370f9712 wallet, rpc, test: Remove deprecated getunconfirmedbalance (Ava Chow)
0ec255139be3745a135386e9db957fe81bc3d833 wallet, rpc: Remove deprecated balances from getwalletinfo (Ava Chow)
Pull request description:
`getwalletinfo` result fields `balance`, `immature_balance`, and `unconfirmed_balance`, and the `getunconfirmedbalance` RPC have all been deprecated since 0.19.0. It's been long enough that they should either be removed or undeprecated. The functionality provided by these RPCs is provided by `getbalances`.
ACKs for top commit:
davidgumberg:
ACK c3fe85e2d6
rkrux:
ACK c3fe85e2d6dd4f251a62a99fd891b0fa370f9712
BrandonOdiwuor:
ACK c3fe85e2d6dd4f251a62a99fd891b0fa370f9712 removing the deprecated `balance, unconfirmed_balance, immature_balance` fields from `getwalletinfo` and `getunconfirmedbalance` RPCs, as this infomation can be found on the `getbalances` RPC
w0xlt:
reACK c3fe85e2d6
Tree-SHA512: c7c4acfd9cabc7517ba813b95281a6c6a717a417312afd9346298669b4f7bd37724ad977148ce42db7fd47fc3d1f5a8482d8ff2e7b9cb74756b171a5b8b91ef2
47237cd1938058b29fdec242c3a37611e255fda0 wallet, rpc: Output wallet flags in getwalletinfo (Ava Chow)
bc2a26b296238cbead6012c071bc7741c40fbd02 wallet: Add GetWalletFlags (Ava Chow)
69f588a99a7a79d1d72300bc0f2c8475f95f6c6a wallet: Set upgraded descriptor cache flag for newly created wallets (Ava Chow)
Pull request description:
Newly created wallets will always have an upgraded descriptor cache, so set those.
Also, to verify this behavior, add a new `flags` field to `getwalletinfo` and check that in the functional tests.
Split from #32489
ACKs for top commit:
Sjors:
ACK 47237cd1938058b29fdec242c3a37611e255fda0
w0xlt:
ACK 47237cd193
rkrux:
ACK 47237cd1938058b29fdec242c3a37611e255fda0
Tree-SHA512: 97c7f85b858efe5ced9b8aafb6cd7c1a547de6f8013b82bfc75bc567cf73c9db5e168e3980355756541305520022fd776b8d4d240d3fb34ed86c27d2acaf4863
9eb2c82e7c911a066781d67e6846cf6bbbaba6e9 walletdb: Remove unused upgraded_txs (Ava Chow)
c66803370988f9806c0ded24c404edb58f60498f wallet: Remove unused fTimeReceivedIsTxTime (Ava Chow)
Pull request description:
`CWalletTx::fTimeReceivedIsTxTime` is no longer used and can be removed. This additionally allows the removal of the `upgraded_txs` loop in `LoadWallet`.
ACKs for top commit:
maflcko:
lgtm ACK 9eb2c82e7c911a066781d67e6846cf6bbbaba6e9
Eunovo:
ACK 9eb2c82e7c
davidgumberg:
ACK 9eb2c82e7c
PeterWrighten:
ACK 9eb2c82e7c
rkrux:
ACK 9eb2c82e7c911a066781d67e6846cf6bbbaba6e9
w0xlt:
ACK 9eb2c82e7c
Tree-SHA512: 05cf3a50f0d8ab6ef423ad1113c5ce6f45bfdc90e2c0dcf61c2dceced2465502e574b4b5b0091fcbb4bdd2182f8d69224f1e5516c7c505de07102b84a5f40e9c
272cd09b796a36596b325277bb43cb47b19c8e12 log: Use warning level while scanning wallet dir (MarcoFalke)
17776443675ddf804f92042883ad36ed040438c3 qa, wallet: Verify warning when failing to scan (Hodlinator)
893e51ffeb0543e1c8d33e83b20c56f02d2b793c wallet: Correct dir iteration error handling (Hodlinator)
Pull request description:
Make wallet DB properly detect and report failure to scan wallet directory. Seems to have been broken since moving from Boost to `std::filesystem`.
Found while reviewing: https://github.com/bitcoin/bitcoin/pull/31410#pullrequestreview-2604068753
ACKs for top commit:
achow101:
ACK 272cd09b796a36596b325277bb43cb47b19c8e12
maflcko:
re-ACK 272cd09b796a36596b325277bb43cb47b19c8e12 🍽
rkrux:
tACK 272cd09b796a36596b325277bb43cb47b19c8e12
Tree-SHA512: 969afde2e37f885ed0c823dc36d2dbeaa0378639849c6a26f8ac67b4f1997eea95bbcae6d58aef5b716807210f37eb166c0cda7ba1d6caffd34249970833af3a
9dfc61d95f0082672a9b90528386e6bcd7014a78 test: detect no external signer connected (Sjors Provoost)
0a4ee93529d68a31f3ba6c7c6009954be47bbbd6 wallet: use PSBTError::EXTERNAL_SIGNER_NOT_FOUND (Sjors Provoost)
8ba2f9b7c8a6c6a91cc718d256354f7a73083b68 refactor: use util::Result for GetExternalSigner() (Sjors Provoost)
Pull request description:
When attempting to sign a transaction involving an external signer, if the device isn't connected we throw an `std::runtime_error`. This prevents the (mainly GUI) code that's actually supposed to handle this case from running.
This PR returns a `PSBTError::EXTERNAL_SIGNER_NOT_FOUND` instead of throwing.
The first commit is a refactor to have `GetExternalSigner()` return a `util::Result<ExternalSigner>` so the caller can decide how to handle the error. There are two other places where call `GetExternalSigner()` which this PR doesn't change (which I think is fine there).
Before:
After (the translation already exist):
Fixes#32426
Additionally use `LogWarning` instead of `std::cerr` for both a missing signer and failure to sign.
ACKs for top commit:
achow101:
ACK 9dfc61d95f0082672a9b90528386e6bcd7014a78
brunoerg:
code review ACK 9dfc61d95f0082672a9b90528386e6bcd7014a78
Tree-SHA512: 22515f4f0b4f50cb0ef532b729e247f11a68be9c90e384942d4277087b2e76806a1cdaa57fb51d5883dacf0a428e5279674aab37cce8c0d3d7de0f96346b8233
ce90f0c99fded22dd24f08757d6f48b5c6b52990 rpc, wallet, refactor: Remove non-descriptor errors (pablomartin4btc)
573bcd75d7b65ff02aaeea40d6f870a9c0bc7490 wallet, refactor: Remove unused SetupGeneration (pablomartin4btc)
5431f2dc2159f55e0fbe89d07deb97fe2a73fb43 wallet, refactor: Remove Legacy warnings and errors (pablomartin4btc)
Pull request description:
Remove dead code due to legacy wallet support removal.
These changes have no impact on functionality. They are transparent to the end user, as legacy wallets can't be created or loaded anymore, so these checks are no longer reached. The legacy-to-descriptor wallet migration flow is not affected either, as these removals are not part of its process.
ACKs for top commit:
achow101:
ACK ce90f0c99fded22dd24f08757d6f48b5c6b52990
rkrux:
utACK ce90f0c99fded22dd24f08757d6f48b5c6b52990
Tree-SHA512: 9229ad9dda9ff1dece73b5b15a20d69c6ab1ff2c75b2ec430ddbbaeb3467f6a850f53df527bcb4a8114ccbf1aa9c794462d71a8d516aed6f9a9da74edae16feb
0def84d407facd319b52826d013cad0d5fc8dbf5 test: Verify parent_desc in RPCs (Ava Chow)
2554cee988fb2ddf65428b354a238f1a4efc1aca test: Enable default wallet for wallet_descriptor.py (Ava Chow)
3fc9d9f241a44ab64774aa9ddc3ded4bb589ed5a wallet, rpc: Push the normalized parent descriptor (Ava Chow)
Pull request description:
Instead of prividing the descriptor string as stored in the db, use the normalized descriptor as is done for getaddressinfo's parent_desc field.
Split from #32489
ACKs for top commit:
Sjors:
re-utACK 0def84d407
rkrux:
ACK 0def84d407facd319b52826d013cad0d5fc8dbf5
w0xlt:
reACK 0def84d407
Tree-SHA512: 575c5b545d6f0aa7e135696b7a55c004e754fca4dd35dd9cf71b0b45b49a2e86e7b20570e768534d587005953bb893645379ec1ba4f98cfd26811f9c2f17de2d
130a922980778b293b22169d5e5649afde3ba33b wallet, interfaces: Use BERKELEY_RO in isEncrypted (Ava Chow)
Pull request description:
The GUI uses `WalletLoader::isEncrypted()` to detect whether a wallet file is encrypted so that it knows whether to prompt for a passphrase when migrating a legacy wallet. However, legacy wallets need to be opened with `options.require_format = BERKELEY_RO`. Since this wasn't being provided, following #28710, encrypted legacy wallets could not be migrated.
This fixes the issue by detecting when a wallet file is for a legacy wallet, and re-attempting with `options.require_format = BERKELEY_RO` in that case.
Depends on #32449 for `DatabaseStatus::FAILED_LEGACY_DISABLED`
ACKs for top commit:
davidgumberg:
Tested ACK 130a922980
furszy:
utACK 130a922980778b293b22169d5e5649afde3ba33b
pablomartin4btc:
tACK 130a922980778b293b22169d5e5649afde3ba33b
w0xlt:
Code review ACK 130a922980
rkrux:
utACK 130a922980778b293b22169d5e5649afde3ba33b
Tree-SHA512: aa70defc3b5f41635333a4d83c46ecdb5cd3cb129d590b4c0fe7a5f16e8aeaba1592f932ead242ed5f84524b146d87319154f4a1820bb34d9e80f63d24fc6b20
Seems to have been broken since conversion from Boost in #20744. The std::filesystem iteration aborts upon failure while Boost might have allowed skipping over faulty entries.
fa9ca13f35be0a023aeed78775ad66f95717b28b refactor: Sort includes of touched source files (MarcoFalke)
facb152697b8d7b75a9e6108f8896f774b06b35f scripted-diff: Bump copyright headers after include changes (MarcoFalke)
fae71d30f7227594e2f59499cf7f7f9420284e04 clang-tidy: Apply modernize-deprecated-headers (MarcoFalke)
Pull request description:
Bitcoin Core is written in C++, so it is confusing to sometimes use the deprecated C headers (with the `.h` extension). For example, it is less clear whether `string.h` refers to the file in this repo or the cstring stdlib header (https://github.com/bitcoin/bitcoin/pull/31308#discussion_r2121492797).
The check is currently disabled for headers, to exclude subtree headers.
ACKs for top commit:
l0rinc:
ACK fa9ca13f35be0a023aeed78775ad66f95717b28b
achow101:
ACK fa9ca13f35be0a023aeed78775ad66f95717b28b
janb84:
ACK fa9ca13f35be0a023aeed78775ad66f95717b28b
stickies-v:
ACK fa9ca13f35be0a023aeed78775ad66f95717b28b
Tree-SHA512: 6639608308c598d612e24435aa519afe92d71b955874b87e527245291fb874b67f3ab95d3a0a5125c6adce5eb41c0d62f6ca488fbbfd60a94f2063d734173f4d
It is not possible to load a legacy/ non-descriptor wallet anymore
so no need to check for WALLET_FLAG_DESCRIPTORS in RPC calls, even when
passing -rpcwallet/ JSON `/wallet/<walletname>/` endpoint, that searches
for the wallets loaded already in the context.
SetupGeneration was supposed to be the function that all SPKMs used
to setup automatic generation, but it didn't work out that way and
ended up being legacy only. It should be deleted at this point.
This RPC lists all the descriptors present in the wallet, not only
the ones that were imported, but also the ones generated when a
new wallet is created.
It can be verified by creating a new wallet and calling the
`listdescriptors` RPC, which will contain 8 ranged descriptors that
are created for every new wallet.
Also, update the description to get rid of "descriptor-enabled"
because this is the only wallet type available now after removal of
legacy wallets.
Instead of throwing a runtime error, let the caller decide how to handle a missing signer.
GUI code was already in place to handle this, but it was unused until this commit.
Fixes#32426
Additionally use LogWarning instead of std::cerr.
f98e1aaf34e347088caa54403521e3b5cb55dd40 rpc: Note in fundrawtransaction doc, fee rate is for package (benthecarman)
Pull request description:
Accidentally made some transactions with a much higher fee rate than I wanted because I did not know this would do it for the package rather than the individual tx.
ACKs for top commit:
achow101:
ACK f98e1aaf34e347088caa54403521e3b5cb55dd40
rkrux:
re-ACK f98e1aaf34e347088caa54403521e3b5cb55dd40
danielabrozzoni:
ACK f98e1aaf34e347088caa54403521e3b5cb55dd40
Tree-SHA512: 9f961de1200803ec4d1c6901fd606bb6cf707ffd03942d9dc0d4b6554c827075f99d693b93e892f728679d67e63e12c71da4426dab091b3311d1605bc37251a2
Historically, the headers have been bumped some time after a file has
been touched. Do it now to avoid having to touch them again in the
future for that reason.
-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended 's;( 20[0-2][0-9])(-20[0-2][0-9])? The Bitcoin Core developers;\1-present The Bitcoin Core developers;g' $( git show --pretty="" --name-only HEAD~0 )
-END VERIFY SCRIPT-
This can be reproduced according to the developer notes with something
like
( cd ./src/ && ../contrib/devtools/run-clang-tidy.py -p ../bld-cmake -fix -j $(nproc) )
Also, the header related changes were done manually.
86e1111239cdb39dd32cfb5178653c608fa30515 test: verify node skips loading legacy wallets during startup (furszy)
9f94de5bb54ff683bd4d3a7723617b34a4706bb6 wallet: init, don't error out when loading legacy wallets (furszy)
Pull request description:
Instead of failing during initialization and shutting down the app when encountering a legacy wallet, skip loading the wallet and notify the user accordingly.
This allows users to access migration functionalities without needing to manually remove the wallet from settings.json or resort to using the bitcoin-wallet utility.
This means that GUI users will be able to use the migration button, and bitcoin-cli users will be able to call the migratewallet RPC directly after init.
ACKs for top commit:
achow101:
ACK 86e1111239cdb39dd32cfb5178653c608fa30515
w0xlt:
ACK 86e1111239
Tree-SHA512: 85d594a503ee7a833a23754b71b6ba4869ca34ed802c9ac0cd7b2fa56978f5fcad84ee4bd3acdcc61cf8e7f08f0789336febc5d76beae1eebf7bd51462512b78
ad9a13fc424e9deb262e2b1d54bcdc7370263ea0 walletdb: Log additional exception error messages for corrupted wallets (Ava Chow)
Pull request description:
Many exceptions thrown for corruption are `std::runtime_error`; we should catch those and log the message to help with debugging.
Split from #32489
ACKs for top commit:
davidgumberg:
ACK ad9a13fc42
furszy:
ACK ad9a13fc424e9deb262e2b1d54bcdc7370263ea0
rkrux:
ACK ad9a13fc424e9deb262e2b1d54bcdc7370263ea0
Sjors:
utACK ad9a13fc424e9deb262e2b1d54bcdc7370263ea0
Tree-SHA512: 107b938d67346804733ea27c44ed38822db0e020e5b1ac889ee35280d812ec56dcc9af7b3eab7a521d72cdd9cb4a8d6d35f3a3dfbcb2a6fd170a981f34fbdfc2
Currently, the migration process creates a brand-new descriptor wallet with no
connection to the user's legacy wallet when the legacy wallet lacks key material
and contains only watch-only scripts. This behavior is not aligned with user
expectations. If the legacy wallet contains only watch-only scripts, the migration
process should only generate a watch-only wallet instead.
