322 Commits

Author SHA1 Message Date
ismaelsadeeq
1c409004c8
test: remove wallet context from write_wallet_settings_concurrently 2024-09-05 20:32:20 +01:00
merge-script
d184fc3ba4
Merge bitcoin/bitcoin#30571: test: [refactor] Use m_rng directly
948238a683b6c99f4e91114aa75680c6c2d73714 test: Remove FastRandomContext global (Ryan Ofsky)
fa0fe08eca48064b2a42789571fea017e455d820 scripted-diff: [test] Use g_rng/m_rng directly (MarcoFalke)
fa54cab4734f02422f28fdffc0f11e6d3d51b8f0 test: refactor: Accept any RandomNumberGenerator in RandMoney (MarcoFalke)
68f77dd21e4aaf4f09d36d6e5ddd7d260824b94b test: refactor: Pass rng parameters to test functions (Ryan Ofsky)
fa19af555dff6d6c722caf36319b158699d2aa95 test: refactor: Move g_insecure_rand_ctx.Reseed out of the helper that calls MakeRandDeterministicDANGEROUS (MarcoFalke)
3dc527f4602297ffcec3a578eadc480a620d01ec test: refactor: Give unit test functions access to test state (Ryan Ofsky)
fab023e177d7eaef73902869ae1c95693f1e268b test: refactor: Make unsigned promotion explicit (MarcoFalke)
fa2cb654eca8dd6ed89101cd6d199ba1de0b81e0 test: Add m_rng alias for the global random context (MarcoFalke)
fae7e3791c9ed8053166773fcfb583ad19d006dd test: Correct the random seed log on a prevector test failure (MarcoFalke)

Pull request description:

  This is mostly a style-cleanup for the tests' random generation:

  1) `g_insecure_rand_ctx` in the tests is problematic, because the name is a leftover when the generator was indeed insecure. However, now the generator is *deterministic*, because the seed is either passed in or printed (c.f. RANDOM_CTX_SEED). Stating that deterministic randomness is insecure in the tests seems redundant at best. Fix it by just using `m_rng` for the name.

  2) The global random context has many one-line aliases, such as `InsecureRand32`. This is problematic, because the same line of code may use the context directly and through a wrapper at the same time. For example in net_tests (see below). This inconsistency is harmless, but confusing. Fix it by just removing the one-line aliases.

  ```
  src/test/net_tests.cpp:        auto msg_data_1 = g_insecure_rand_ctx.randbytes<uint8_t>(InsecureRandRange(100000));
  ````

  3) The wrapper for randmoney has the same problem that the same unit test uses the context directly and through a wrapper at the same time. Also, it has a single type of Rng hardcoded. Fix it by accepting any type.

ACKs for top commit:
  hodlinator:
    ACK 948238a683b6c99f4e91114aa75680c6c2d73714
  ryanofsky:
    Code review ACK 948238a683b6c99f4e91114aa75680c6c2d73714. Only changes since last review were changing a comments a little bit.
  marcofleon:
    Code review ACK 948238a683b6c99f4e91114aa75680c6c2d73714. Only changes since my last review are the improvements in `prevector_tests`.

Tree-SHA512: 69c6b46a42cb743138ee8c87ff26a588dbe083e3efb3dca49b8a133ba5d3b09e8bf01c590ec7e121a7d77cb1fd7dcacd927a9ca139ac65e1f7c6d1ec46f93b57
2024-08-28 16:56:32 +01:00
glozow
f93d5553d1
Merge bitcoin/bitcoin#22838: descriptors: Be able to specify change and receiving in a single descriptor string
a0abcbd3822bd17a1d73c42ccd5b040a150b0501 doc: Mention multipath specifier (Ava Chow)
0019f61fc546b4d5f42eb4086f42560863fe0efb tests: Test importing of multipath descriptors (Ava Chow)
f97d5c137d605ac48f1122a836c9aa5f834957ba wallet, rpc: Allow importdescriptors to import multipath descriptors (Ava Chow)
32dcbca3fb918bc899a0637f876db31c3419aafd rpc: Allow importmulti to import multipath descriptors correctly (Ava Chow)
64dfe3ce4bed9ac168d0b08def8af7485db94ef1 wallet: Move internal to be per key when importing (Ava Chow)
16922455253f47fae0466c4ec6c3adfadcfe9182 tests: Multipath descriptors for scantxoutset and deriveaddresses (Ava Chow)
cddc0ba9a9dca3ca5873d768b3b504cdb2ab947b rpc: Have deriveaddresses derive receiving and change (Ava Chow)
360456cd221501fde3efe11bdba5c6d999dbb323 tests: Multipath descriptors for getdescriptorinfo (Ava Chow)
a90eee444c965bbd7bcddf9656eca9cee14c3aec tests: Add unit tests for multipath descriptors (Ava Chow)
1bbf46e2dae4599d04c79aaacf7c5db00b2e707f descriptors: Change Parse to return vector of descriptors (Ava Chow)
0d640c6f02bc20e5c1be773443dd74d8806d953b descriptors: Have ParseKeypath handle multipath specifiers (Ava Chow)
a5f39b103461a98689fd5d382e8da29037f55bea descriptors: Change ParseScript to return vector of descriptors (Ava Chow)
0d55deae157f4f8226b2419d55e7dc0dfb6e4aec descriptors: Add DescriptorImpl::Clone (Ava Chow)
7e86541f723d62c7ec6768f7f592c09ba2047d9e descriptors: Add PubkeyProvider::Clone (Ava Chow)

Pull request description:

  It is convenient to have a descriptor which specifies both receiving and change addresses in a single string. However, as discussed in https://github.com/bitcoin/bitcoin/issues/17190#issuecomment-895515768, it is not feasible to use a generic multipath specification like BIP 88 due to combinatorial blow up and that it would result in unexpected descriptors.

  To resolve that problem, this PR proposes a targeted solution which allows only a single pair of 2 derivation indexes to be inserted in the place of a single derivation index. So instead of two descriptor `wpkh(xpub.../0/0/*)` and `wpkh(xpub.../0/1/*)` to represent receive and change addresses, this could be written as `wpkh(xpub.../0/<0;1>/*)`. The multipath specifier is of the form `<NUM;NUM>`. Each `NUM` can have its own hardened specifier, e.g. `<0;1h>` is valid. The multipath specifier can also only appear in one path index in the derivation path.

  This results in the parser returning two descriptors. The first descriptor uses the first `NUM` in all pairs present, and the second uses the second `NUM`. In our implementation, if a multipath descriptor is not provided, a pair is still returned, but the second element is just `nullptr`.

  The wallet will not output the multipath descriptors (yet). Furthermore, when a multipath descriptor is imported, it is expanded to the two descriptors and each imported on its own, with the second descriptor being implicitly for internal (change) addresses. There is no change to how the wallet stores or outputs descriptors (yet).

  Note that the path specifier is different from what was proposed. It uses angle brackets and the semicolon because these are unused characters available in the character set and I wanted to avoid conflicts with characters already in use in descriptors.

  Closes #17190

ACKs for top commit:
  darosior:
    re-ACK a0abcbd3822bd17a1d73c42ccd5b040a150b0501
  mjdietzx:
    reACK a0abcbd3822bd17a1d73c42ccd5b040a150b0501
  pythcoiner:
    reACK a0abcbd
  furszy:
    Code review ACK a0abcbd
  glozow:
    light code review ACK a0abcbd3822

Tree-SHA512: 84ea40b3fd1b762194acd021cae018c2f09b98e595f5e87de5c832c265cfe8a6d0bc4dae25785392fa90db0f6301ddf9aea787980a29c74f81d04b711ac446c2
2024-08-28 15:56:15 +01:00
ismaelsadeeq
1b41d45d46
wallet: bugfix: ensure atomicity in settings updates
- Settings updates were not thread-safe, as they were executed in
  three separate steps:

  1) Obtain settings value while acquiring the settings lock.
  2) Modify settings value.
  3) Overwrite settings value while acquiring the settings lock.

  This approach allowed concurrent threads to modify the same base value
  simultaneously, leading to data loss. When this occurred, the final
  settings state would only reflect the changes from the last thread
  that completed the operation, overwriting updates from other threads.

  Fix this by making the settings update operation atomic.

- Add test coverage for this behavior.

Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
2024-08-26 13:41:56 +01:00
MarcoFalke
fa0fe08eca
scripted-diff: [test] Use g_rng/m_rng directly
-BEGIN VERIFY SCRIPT-

 # Use m_rng in unit test files
 ren() { sed -i "s:\<$1\>:$2:g" $( git grep -l "$1" src/test/*.cpp src/wallet/test/*.cpp src/test/util/setup_common.cpp ) ; }
 ren InsecureRand32                m_rng.rand32
 ren InsecureRand256               m_rng.rand256
 ren InsecureRandBits              m_rng.randbits
 ren InsecureRandRange             m_rng.randrange
 ren InsecureRandBool              m_rng.randbool
 ren g_insecure_rand_ctx           m_rng
 ren g_insecure_rand_ctx_temp_path g_rng_temp_path

-END VERIFY SCRIPT-
2024-08-26 11:19:52 +02:00
furszy
8872b4a6ca
wallet: rename UnloadWallet to WaitForDeleteWallet
And update function's documentation.
2024-08-14 16:12:18 -03:00
Ava Chow
1bbf46e2da descriptors: Change Parse to return vector of descriptors
When given a descriptor which contins a multipath derivation specifier,
a vector of descriptors will be returned.
2024-08-08 12:47:22 -04:00
Ryan Ofsky
6300438a2e
Merge bitcoin/bitcoin#30115: rpc: avoid copying into UniValue
d7707d9843b03f20d2a8c5a45d7b3db58e169e6f rpc: avoid copying into UniValue (Cory Fields)

Pull request description:

  These are the simple (and hopefully obviously correct) copies that can be moves instead.

  This is a follow-up from https://github.com/bitcoin/bitcoin/pull/30094#issuecomment-2108751842

  As it turns out, there are hundreds of places where we copy UniValues needlessly. It should be the case that moves are always preferred over copies, so there should be no downside to these changes.

  willcl-ark, however, noticed that memory usage may increase in some cases. Logically this makes no sense to me. The only plausible explanation imo is that because the moves are faster, more ops/second occur in some cases.

  This list of moves was obtained by changing the function signatures of the UniValue functions to accept only rvalues, then compiling and fixing them up one by one. There still exist many places where copies are being made. These can/should be fixed up, but weren't done here for the sake of doing the easy ones first.

  I ran these changes through clang-tidy with `performance-move-const-arg` and `bugprone-use-after-move` and no bugs were detected (though that's obviously not to say it can be trusted 100%).

  As stated above, there are still lots of other less trivial fixups to do after these including:
  - Using non-const UniValues where possible so that moves can happen
  - Refactoring code in order to be able to move a UniValue without introducing a use-after-move
  - Refactoring functions to accept UniValues by value rather than by const reference

ACKs for top commit:
  achow101:
    ACK d7707d9843b03f20d2a8c5a45d7b3db58e169e6f
  ryanofsky:
    Code review ACK d7707d9843b03f20d2a8c5a45d7b3db58e169e6f. No changes since last review other than rebase. I agree benchmarks showing increased peak memory usage and RSS are surprising, but number of allocations is down as expected, and runtime is also decreased.
  willcl-ark:
    ACK d7707d9843b03f20d2a8c5a45d7b3db58e169e6f

Tree-SHA512: 7f511be73984553c278186286a7d161a34b2574c7f5f1a0edc87c2913b4c025a0af5241ef9af2df17547f2e4ef79710aa5bbb762fc9472435781c0488dba3435
2024-05-23 10:53:37 -04:00
Ava Chow
2289d45240 wallet, tests: Avoid stringop-overflow warning in PollutePubKey 2024-05-21 12:59:47 -04:00
Cory Fields
d7707d9843 rpc: avoid copying into UniValue
These are simple (and hopefully obviously correct) copies that can be moves
instead.
2024-05-20 16:48:19 +00:00
Ava Chow
c07935bcf5
Merge bitcoin/bitcoin#28960: kernel: Remove dependency on CScheduler
d5228efb5391b31a9a0673019e43e7fa2cd4ac07 kernel: Remove dependency on CScheduler (TheCharlatan)
06069b3913dda048f5d640a662b0852f86346ace scripted-diff: Rename MainSignals to ValidationSignals (TheCharlatan)
0d6d2b650d1017691f48c9109a6cd020ab46aa73 scripted-diff: Rename SingleThreadedSchedulerClient to SerialTaskRunner (TheCharlatan)
4abde2c4e3fd9b66394b79874583bdc2a9132c36 [refactor] Make MainSignals RAII styled (TheCharlatan)
84f5c135b8118cbe15b8bfb4db80d61237987f64 refactor: De-globalize g_signals (TheCharlatan)
473dd4b97ae40e43e1a1a97fdbeb40be4855e9bc [refactor] Prepare for g_signals de-globalization (TheCharlatan)
3fba3d5deec6d7bae33823b8da7682f9b03d9deb [refactor] Make signals optional in mempool and chainman (TheCharlatan)

Pull request description:

  By defining a virtual interface class for the scheduler client, users of the kernel can now define their own event consuming infrastructure, without having to spawn threads or rely on the scheduler design.

  Removing `CScheduler` also allows removing the thread and exception modules from the kernel library.

  To make the `CMainSignals` class easier to use from a kernel library perspective, remove its global instantiation and adopt RAII practices.

  Renames `CMainSignals` to `ValidationSignals`, which more accurately describes its purpose and scope.

  Also make the `ValidationSignals` in the `ChainstateManager` and CTxMemPool` optional. This could be useful in the future for using or testing these classes without having to instantiate any form of signal handling.

  ---

  This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587). It improves the kernel API and removes two modules from the kernel library.

ACKs for top commit:
  maflcko:
    re-ACK d5228efb5391b31a9a0673019e43e7fa2cd4ac07 🌄
  ryanofsky:
    Code review ACK d5228efb5391b31a9a0673019e43e7fa2cd4ac07. Just comment change since last review.
  vasild:
    ACK d5228efb5391b31a9a0673019e43e7fa2cd4ac07
  furszy:
    diff ACK d5228ef

Tree-SHA512: e93a5f10eb6182effb84bb981859a7ce750e466efd8171045d8d9e7fe46e4065631d9f6f533c5967c4d34c9bb7d7a67e9f4593bd4c5b30cd7b3bbad7be7b331b
2024-03-08 20:58:04 -05:00
TheCharlatan
84f5c135b8
refactor: De-globalize g_signals 2024-02-15 14:37:01 +01:00
furszy
33757814ce
wallet: bdb batch 'ErasePrefix', do not create txn internally
Transactions are intended to be started on upper layers rather than
internally by the bdb batch object. This enables us to consolidate
different write operations within a procedure in the same db txn,
improving consistency due to the atomic property of the transaction,
as well as its performance due to the reduction of disk write
operations.

Important Note:
This approach also ensures that the BerkeleyBatch::ErasePrefix
function behaves exactly as the SQLiteBatch::ErasePrefix function,
which does not create a db txn internally.

Furthermore, since the `BerkeleyBatch::ErasePrefix' implementation
erases records one by one (by traversing the db), this change
ensures that the function is always called within an active txn
context. Without this measure, there's a potential risk to consistency;
certain records may be removed while others could persist due to an
internal failure during the procedure.
2024-02-12 16:05:15 -03:00
Ava Chow
6ff0aa089c
Merge bitcoin/bitcoin#28987: wallet: simplify and batch zap wallet txes process
9a3c5c8697659e34d0476103af942a4615818f4e scripted-diff: rename ZapSelectTx to RemoveTxs (furszy)
83b762845f5804f23b63526d403b2f327fe99632 wallet: batch and simplify ZapSelectTx process (furszy)
595d50a1032ad7ffa9945464c86aa57f16665e93 wallet: migration, remove extra NotifyTransactionChanged call (furszy)
a2b071f9920c2f4893afcc43a152f593c03966bf wallet: ZapSelectTx, remove db rewrite code (furszy)

Pull request description:

  Work decoupled from #28574. Brother of #28894.

  Includes two different, yet interconnected, performance and code improvements to the zap wallet transactions process.

  1) As the goal of the `ZapSelectTx` function is to erase tx records that match any of the inputted hashes. There is no need to traverse the whole database record by record. We could just check if the tx exist, and remove it directly by calling `EraseTx()`.

  2) Instead of performing single write operations per removed tx record, this PR batches them all within a single atomic db txn.

  Moreover, these changes will enable us to consolidate all individual write operations that take place during the wallet migration process into a single db txn in the future.

ACKs for top commit:
  achow101:
    ACK 9a3c5c8697659e34d0476103af942a4615818f4e
  josibake:
    ACK 9a3c5c8697

Tree-SHA512: fb2ecc48224c400ab3b1fbb32e174b5b13bf03794717727f80f01f55fb183883b067a68c0a127b2de8885564da15425d021a96541953bf38a72becc2e9929ccf
2024-02-12 13:41:47 -05:00
furszy
9a3c5c8697
scripted-diff: rename ZapSelectTx to RemoveTxs
-BEGIN VERIFY SCRIPT-
sed -i 's/ZapSelectTx/RemoveTxs/g' $(git grep -l 'ZapSelectTx' ./src/wallet)
-END VERIFY SCRIPT-
2024-02-09 14:54:50 -03:00
furszy
83b762845f
wallet: batch and simplify ZapSelectTx process
The goal of the function is to erase the wallet transactions that
match the inputted hashes. There is no need to traverse the database,
reading record by record, to then perform single entry removals for
each of them.

To ensure consistency and improve performance, this change-set removes
all tx records within a single atomic db batch operation, as well as
it cleans up code, improves error handling and simplifies the
transactions removal process entirely.

This optimizes the removal of watch-only transactions during the wallet
migration process and the 'removeprunedfunds' RPC command.
2024-02-09 14:54:50 -03:00
Sebastian Falbesoner
fa1d49542e refactor: share and use GenerateRandomKey helper
Making the `GenerateRandomKey` helper available to other modules via
key.{h.cpp} allows us to create random private keys directly at
instantiation of CKey, in contrast to the two-step process of creating
the instance and then having to call `MakeNewKey(...)`.
2023-12-23 13:26:00 +01:00
furszy
37c75c5820
test: wallet, fix change position out of range error
Since #25273, the behavior of 'inserting change at a random
position' is instructed by passing std::nullopt instead of -1.

Also, added missing documentation about the meaning of
'change_pos=std::nullopt' inside 'CWallet::CreateTransaction()'
2023-12-12 15:20:38 -03:00
Andrew Chow
758501b713 wallet: use optional for change position as an optional in CreateTransaction
Instead of making -1 a magic number meaning no change or random change
position, use an optional to have that meaning.
2023-12-08 17:12:19 -05:00
MarcoFalke
fac39b56b7
refactor: SpanReader without nVersion
The field is unused, so remove it.

This is also required for future commits.
2023-11-28 12:42:07 +01:00
fanquake
c252a0fc0f
Merge bitcoin/bitcoin#28892: refactor: P2P transport without serialize version and type
fa79a881ce0537e1d74da296a7513730438d2a02 refactor: P2P transport without serialize version and type (MarcoFalke)
fa9b5f4fe32c0cfe2e477bb11912756f84a52cfe refactor: NetMsg::Make() without nVersion (MarcoFalke)
66669da4a5ca9edf2a40d20879d9a8aaf2b9e2ee Remove unused Make() overload in netmessagemaker.h (MarcoFalke)
fa0ed0794161d937d2d3385963c1aa5624b60d17 refactor: VectorWriter without nVersion (MarcoFalke)

Pull request description:

  Now that the serialize framework ignores the serialize version and serialize type, everything related to it can be removed from the code.

  This is the first step, removing dead code from the P2P stack. A different pull will remove it from the wallet and other parts.

ACKs for top commit:
  ajtowns:
    reACK fa79a881ce0537e1d74da296a7513730438d2a02

Tree-SHA512: 785b413580d980f51f0d4f70ea5e0a99ce14cd12cb065393de2f5254891be94a14f4266110c8b87bd2dbc37467676655bce13bdb295ab139749fcd8b61bd5110
2023-11-28 11:24:09 +00:00
dergoegge
9e58c5bcd9 Use Txid in COutpoint 2023-11-21 13:15:44 +00:00
MarcoFalke
fa0ed07941
refactor: VectorWriter without nVersion
The field is unused, so remove it.

This is also required for future commits.
2023-11-17 14:38:26 +01:00
MarcoFalke
faec889f93
refactor: Add LIFETIMEBOUND to all (w)txid getters
Then, use the compiler warnings to create copies only where needed.

Also, fix iwyu includes while touching the includes.
2023-10-27 13:01:42 +02:00
MarcoFalke
1111475b41
bugfix: Mark CNoDestination and PubKeyDestination constructor explicit
This should fix the bug reported in
https://github.com/bitcoin/bitcoin/pull/28246#discussion_r1371640502,
which caused the GUI to not detect the destination type of recipients,
thus picking the wrong change destination type.

Also, add missing lifetimebound attribute to a getter method.
2023-10-25 22:46:55 +02:00
MarcoFalke
fa05a726c2
tidy: modernize-use-emplace 2023-10-12 11:27:19 +02:00
fanquake
48b8910d12
Merge bitcoin/bitcoin#28508: refactor: Remove SER_GETHASH, hard-code client version in CKeyPool serialize
fac29a0ab19fda457b55d7a0a37c5cd3d9680f82 Remove SER_GETHASH, hard-code client version in CKeyPool serialize (MarcoFalke)
fa72f09d6ff8ee204f331a69d3f5e825223c9e11 Remove CHashWriter type (MarcoFalke)
fa4a9c0f4334678fb80358ead667807bf2a0a153 Remove unused GetType() from OverrideStream, CVectorWriter, SpanReader (MarcoFalke)

Pull request description:

  Removes a bunch of redundant, dead or duplicate code.

  Uses the idea from and finishes the idea https://github.com/bitcoin/bitcoin/pull/28428 by theuni

ACKs for top commit:
  ajtowns:
    ACK fac29a0ab19fda457b55d7a0a37c5cd3d9680f82
  kevkevinpal:
    added one nit but otherwise ACK [fac29a0](fac29a0ab1)

Tree-SHA512: cc805e2f38e73869a6691fdb5da09fa48524506b87fc93f05d32c336ad3033425a2d7608e317decd3141fde3f084403b8de280396c0c39132336fe0f7510af9e
2023-10-02 12:33:54 +02:00
MarcoFalke
fa4a9c0f43
Remove unused GetType() from OverrideStream, CVectorWriter, SpanReader
GetType() is never called, so it is completely unused and can be
removed.
2023-09-19 14:19:57 +00:00
Andrew Chow
ad0c469d98 wallet: Use CTxDestination in CRecipient rather than scriptPubKey 2023-09-12 12:14:31 -04:00
MarcoFalke
fa8fdbe229
Remove unused includes from blockfilter.h
This removes unused includes, primitives/block found manually, and the
others by iwyu:

blockfilter.h should remove these lines:
- #include <serialize.h>  // lines 16-16
- #include <undo.h>  // lines 18-18
2023-08-17 18:28:15 +02:00
Andrew Chow
91d924ede1 Rename script/standard.{cpp/h} to script/solver.{cpp/h}
Since script/standard only contains things that are used by the Solver
and its callers, rename the files to script/solver.
2023-08-14 17:39:49 -04:00
Andrew Chow
f3c9078b4c Clean up things that include script/standard.h
Remove standard.h from files that don't use anything in it, and include
it in files that do.
2023-08-14 17:38:27 -04:00
Andrew Chow
7a172c76d2 Move CTxDestination to its own file
CTxDestination is really our internal representation of an address and
doesn't really have anything to do with standard script types, so move
them to their own file.
2023-08-14 17:38:27 -04:00
jonatack
1b246fdd14 test: move remaining random test util code from setup_common to random
and drop the util/random dependency on util/setup_common.

This improves code separation and avoids creating a circular dependency if
setup_common needs to call the util/random functions.
2023-06-14 08:28:33 -06:00
Hennadii Stepanov
fade2adb5b
test: Avoid BOOST_ASSERT macro
The `BOOST_ASSERT` macro is defined in the `boost/assert.hpp` header.
This change allows to skip `#include <boost/assert.hpp>`.
2023-05-31 15:42:40 +01:00
Andrew Chow
c61d3f02f5 tests, bench: Consolidate {Test,Bench}Un/LoadWallet helper
The wallet tests and benchmarks both had helper functions for loading
and unloading the wallet for the test that were almost identical.
These functions are consolidated and reused.
2023-05-25 14:40:26 -04:00
furszy
69d43905b7
test: add coverage for wallet read write db deadlock 2023-05-15 12:29:38 -03:00
fanquake
d02df7db6b
Merge bitcoin/bitcoin#26715: Introduce MockableDatabase for wallet unit tests
33e2b82a4fc990253ff77655f437c7aed336bc55 wallet, bench: Remove unused database options from WalletBenchLoading (Andrew Chow)
80ace042d8fece9be50bfef1be64c6e5720e87e6 tests: Modify records directly in wallet ckey loading test (Andrew Chow)
b3bb17d5d07f51ac2e501e4a7a3bbcd17144070f tests: Update DuplicateMockDatabase for MockableDatabase (Andrew Chow)
f0eecf5e408238c64b77b0a4974ba2b9edb17487 scripted-diff: Replace CreateMockWalletDB with CreateMockableWalletDB (Andrew Chow)
075962bc25a90661612fe4613cd50ea1cae21f52 wallet, tests: Include wallet/test/util.h (Andrew Chow)
14aa4cb1e44f089a6022a2b14a98bca4a7dd9a01 wallet: Move DummyDatabase to salvage (Andrew Chow)
f67a385556c60b2e4788a378196a395fca0539f5 wallet, tests: Replace usage of dummy db with mockable db (Andrew Chow)
33c6245ac1ecdfe25b1ee4fd9e93c43393634ae3 Introduce MockableDatabase for wallet unit tests (Andrew Chow)

Pull request description:

  For the wallet's unit tests, we currently use either `DummyDatabase` or memory-only versions of either BDB or SQLite. The tests that use `DummyDatabase` just need a `WalletDatabase` so that the `CWallet` can be constructed, while the tests using the memory-only databases just need a backing data store. There is also a `FailDatabase` that is similar to `DummyDatabase` except it fails be default or can have a configured return value. Having all of these different database types can make it difficult to write tests, particularly tests that work when either BDB or SQLite is disabled.

  This PR unifies all of these different unit test database classes into a single `MockableDatabase`. Like `DummyDatabase`, most functions do nothing and just return true. Like `FailDatabase`, the return value of some functions can be configured on the fly to test various failure cases. Like the memory-only databases, records can actually be written to the `MockableDatabase` and be retrieved later, but all of this is still held in memory. Using `MockableDatabase` completely removes the need for having BDB or SQLite backed wallets in the unit tests for the tests that are not actually testing specific database behaviors.

  Because `MockableDatabase`s can be created by each unit test, we can also control what records are stored in the database. Records can be added and removed externally from the typical database modification functions. This will give us greater ability to test failure conditions, particularly those involving corrupted records.

  Possible alternative to #26644

ACKs for top commit:
  furszy:
    ACK 33e2b82
  TheCharlatan:
    ACK 33e2b82a4fc990253ff77655f437c7aed336bc55

Tree-SHA512: c2b09eff9728d063d2d4aea28a0f0e64e40b76483e75dc53f08667df23bd25834d52656cd4eafb02e552db0b9e619cfdb1b1c65b26b5436ee2c971d804768bcc
2023-05-15 11:39:43 +01:00
TheCharlatan
f0bb1021f0
refactor: Move functions to BlockManager methods
This is a commit in preparation for the next few commits. The functions
are moved to methods to avoid their re-declaration for the purpose of
passing in BlockManager options.

The functions that were now moved into the BlockManager should no longer
use the params as an argument, but instead use the member variable.

In the moved ReadBlockFromDisk and UndoReadFromDisk, change
the function signature to accept a reference to a CBlockIndex instead of
a raw pointer. The pointer is expected to be non-null, so reflect that
in the type.

To allow for the move of functions to BlockManager methods all call
sites require an instantiated BlockManager, or a callback to one.
2023-05-10 19:06:53 +02:00
Andrew Chow
f0eecf5e40 scripted-diff: Replace CreateMockWalletDB with CreateMockableWalletDB
Since we have a mockable wallet database, we don't really need to be
using BDB or SQLite's in-memory database capabilities. It doesn't really
help us to be using those as we aren't doing anything that requires one
type of db over the other, and will just prefer SQLite if it's
available.

MockableDatabase is suitable for these uses, so use
CreateMockableWalletDatabase to use that.

-BEGIN VERIFY SCRIPT-
sed -i "s/CreateMockWalletDatabase(options)/CreateMockableWalletDatabase()/" $(git grep -l "CreateMockWalletDatabase(options)" -- ":(exclude)src/wallet/walletdb.*")
sed -i "s/CreateMockWalletDatabase/CreateMockableWalletDatabase/" $(git grep -l "CreateMockWalletDatabase" -- ":(exclude)src/wallet/walletdb.*")
-END VERIFY SCRIPT-
2023-05-03 10:45:10 -04:00
Andrew Chow
f67a385556 wallet, tests: Replace usage of dummy db with mockable db 2023-05-03 10:45:10 -04:00
Andrew Chow
33c6245ac1 Introduce MockableDatabase for wallet unit tests
MockableDatabase is a WalletDatabase that allows us to interact with the
records to change them independently from the wallet, as well as
changing the return values from within the tests. This will give us
greater flexibility in testing the wallet.
2023-05-03 10:45:10 -04:00
Ryan Ofsky
a5986e82dd refactor: Remove CAddressBookData::destdata
This is cleanup that doesn't change external behavior.

- Removes awkward `StringMap` intermediate representation
- Simplifies CWallet code, deals with used address and received request
  serialization in walletdb.cpp
- Adds test coverage and documentation
- Reduces memory usage

This PR doesn't change externally observable behavior. Internally, only change
in behavior is that EraseDestData deletes directly from database because the
`StringMap` is gone. This is more direct and efficient because it uses a single
btree lookup and scan instead of multiple lookups

Motivation for this cleanup is making changes like #18550, #18192, #13756
easier to reason about and less likely to result in unintended behavior and
bugs

Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
2023-04-12 05:30:43 -04:00
Ryan Ofsky
5938ad0bdb wallet: Add DatabaseBatch::ErasePrefix method
This new function is not used yet this commit, but next commit adds usages and
test coverage for both BDB and sqlite.
2023-04-12 05:30:43 -04:00
furszy
dc1cc1c359
gui: bugfix, getAvailableBalance skips selected coins
The previous behavior for getAvailableBalance when coin control
has selected coins was to return the sum of them. Instead, we
are currently returning the wallet's available total balance minus
the selected coins total amount.

This turns into a GUI-only issue for the "use available balance"
button when the user manually select coins in the send screen.

Reason:
We missed to update the GetAvailableBalance function to include
the coin control selected coins on #25685.

Context:
Since #25685 we skip the selected coins inside `AvailableCoins`,
the reason is that there is no need to traverse the wallet's
txes map just to get coins that can directly be fetched by
their id.
2023-04-03 17:23:42 -03:00
furszy
6c9b342c30
refactor: wallet, remove global 'ArgsManager' access
we are not using it anymore
2023-02-15 15:49:45 -03:00
MarcoFalke
fa451d4b60
Fix clang-tidy readability-const-return-type violations 2023-02-01 11:33:35 +01:00
MarcoFalke
fa29e73cda
Use DataStream where possible 2023-01-26 10:44:05 +01:00
fanquake
a62231bca6
Merge bitcoin/bitcoin#26690: wallet: Refactor database cursor into its own object with proper return codes
4aebd832a405090c2608e4b60bb4f34501bcea61 db: Change DatabaseCursor::Next to return status enum (Andrew Chow)
d79e8dcf2981ef1964a2fde8c472b5de1ca1c963 wallet: Have cursor users use DatabaseCursor directly (Andrew Chow)
7a198bba0a1d0a0f0fd4ca947955cb52b84bdd4b wallet: Introduce DatabaseCursor RAII class for managing cursor (Andrew Chow)
69efbc011bb74fcd8dd9ed2a8a5d31bc9e323c10 Move SafeDbt out of BerkeleyBatch (Andrew Chow)

Pull request description:

  Instead of having database cursors be tied to a particular `DatabaseBatch` object and requiring its setup and teardown be separate functions in that batch, we can have cursors be separate RAII classes. This makes it easier to create and destroy cursors as well as having cursors that have slightly different behaviors.

  Additionally, since reading data from a cursor is a tri-state, this PR changes the return value of the `Next` function (formerly `ReadAtCursor`) to return an Enum rather than the current system of 2 booleans. This greatly simplifies and unifies the code that deals with cursors as now there is no confusion as to what the function returns when there are no records left to be read.

  Extracted from #24914

ACKs for top commit:
  furszy:
    diff ACK 4aebd83
  theStack:
    Code-review ACK 4aebd832a405090c2608e4b60bb4f34501bcea61

Tree-SHA512: 5d0be56a18de5b08c777dd5a73ba5a6ef1e696fdb07d1dca952a88ded07887b7c5c04342f9a76feb2f6fe24a45dc31f094f1f5d9500e6bdf4a44f4edb66dcaa1
2023-01-23 17:56:16 +00:00
MarcoFalke
3212d104f4
Merge bitcoin/bitcoin#23829: refactor: use braced init for integer literals instead of c style casts
f2fc03ec856d7d19a20c482514350cced38f9504 refactor: use braced init for integer constants instead of c style casts (Pasta)

Pull request description:

  See https://github.com/bitcoin/bitcoin/pull/23810 for more context. This is broken out from that PR, as it is less breaking, and should be trivial to review and merge.

  EDIT: Long term, the intention is to remove all C-style casts, as they can dangerously introduce reinterpret_casts. This is one step which removes a number of trivially removable C-style casts

ACKs for top commit:
  aureleoules:
    ACK f2fc03ec856d7d19a20c482514350cced38f9504

Tree-SHA512: 2fd11b92c9147e3f970ec3e130e3b3dce70e707ff02950a8c697d4b111ddcbbfa16915393db20cfc8f384bc76f13241c9b994a187987fcecd16a61f8cc0af14c
2023-01-05 17:30:52 +01:00