mirror of
https://github.com/bitcoin/bitcoin.git
synced 2026-02-20 04:19:01 +00:00
da6f041e39
b6f8c48946cbfceb066de660c485ae1bd2c27cc1 coins: increase default `dbbatchsize` to 32 MiB (Lőrinc) 8bbb7b8bf8e3b2b6465f318ec102cc5275e5bf8c refactor: Extract default batch size into kernel (Lőrinc) Pull request description: This change is part of [[IBD] - Tracking PR for speeding up Initial Block Download](https://github.com/bitcoin/bitcoin/pull/32043) ### Summary When the in-memory UTXO set is flushed to LevelDB (after IBD or AssumeUTXO load), it does so in batches to manage memory usage during the flush. A hidden `-dbbatchsize` config option exists to modify this value. This PR only changes the default from `16` MiB to `32` MiB. Using a larger default reduces the overhead of many small writes and improves I/O efficiency (especially on HDDs). It may also help LevelDB optimize writes more effectively (e.g., via internal ordering). The change is meant to speed up a critical part of IBD: dumping the accumulated work to disk. ### Context The UTXO set has grown significantly since [2017](https://github.com/bitcoin/bitcoin/pull/10148/files#diff-d102b6032635ce90158c1e6e614f03b50e4449aa46ce23370da5387a658342fdR26-R27), when the original fixed 16 MiB batch size was chosen. With the current multi-gigabyte UTXO set and the common practice of using larger `-dbcache` values, the fixed 16 MiB batch size leads to several inefficiencies: * Flushing the entire UTXO set often requires thousands of separate 16 MiB write operations. * Particularly on HDDs, the cumulative disk seek time and per-operation overhead from numerous small writes significantly slow down the flushing process. * Each `WriteBatch` call incurs internal LevelDB overhead (e.g., MemTable handling, compaction triggering logic). More frequent, smaller batches amplify this cumulative overhead. Flush times of 20-30 minutes are not uncommon, even on capable hardware. ### Considerations As [noted by sipa](https://github.com/bitcoin/bitcoin/pull/31645#issuecomment-2587500105), flushing involves a temporary memory usage increase as the batch is prepared. A larger batch size naturally leads to a larger peak during this phase. Crashing due to OOM during a flush is highly undesirable, but now that [#30611](https://github.com/bitcoin/bitcoin/pull/30611) is merged, the most we'd lose is the first hour of IBD. Increasing the LevelDB write batch size from 16 to 32 MiB raised the measured peaks by ~70 MiB in my tests during UTXO dump. The option remains hidden, and users can always override it. The increased peak memory usage (detailed below) is primarily attributed to LevelDB's `leveldb::Arena` (backing MemTables) and the temporary storage of serialized batch data (e.g., `std::string` in `CDBBatch::WriteImpl`). Performance gains are most pronounced on systems with slower I/O (HDDs), but some SSDs also show measurable improvements. ### Measurements: AssumeUTXO proxy, multiple runs with error bars (flushing time is faster that the measured loading + flushing): * Raspberry Pi, dbcache=500: ~30% faster with 32 MiB vs 16 MiB, peak +~75 MiB and still < 1 GiB. * i7 + HDD: results vary by dbcache, but 32 MiB usually beats 16 MiB and tracks close to 64 MiB without the larger peak. * i9 + fast NVMe: roughly flat across 16/32/64 MiB. The goal here is to avoid regressions, which holds. ### Reproducer: ```bash # Set up a clean demo environment rm -rfd demo && mkdir -p demo # Build Bitcoin Core cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j$(nproc) # Start bitcoind with minimal settings without mempool and internet connection build/bin/bitcoind -datadir=demo -stopatheight=1 build/bin/bitcoind -datadir=demo -blocksonly=1 -connect=0 -dbcache=3000 -daemon # Load the AssumeUTXO snapshot, making sure the path is correct # Expected output includes `"coins_loaded": 184821030` build/bin/bitcoin-cli -datadir=demo -rpcclienttimeout=0 loadtxoutset ~/utxo-880000.dat # Stop the daemon and verify snapshot flushes in the logs build/bin/bitcoin-cli -datadir=demo stop grep "FlushSnapshotToDisk: completed" demo/debug.log ``` --- This PR originally proposed 64 MiB, then a dynamic size, but both were dropped: 64 MiB increased peaks more than desired on low-RAM systems, and the dynamic variant underperformed across mixed hardware. 32 MiB is a simpler default that captures most of the gains with a modest peak increase. For more details see: https://github.com/bitcoin/bitcoin/pull/31645#issuecomment-3234329502 --- While the PR isn't about IBD in general, rather about a critical section of it, I have measured a reindex-chainstate until 900k blocks, showing a 1% overall speedup: <details> <summary>Details</summary> ```python COMMITS="e6bfd95d5012fa1d91f83bf4122cb292afd6277f af653f321b135a59e38794b537737ed2f4a0040b"; \ STOP=900000; DBCACHE=10000; \ CC=gcc; CXX=g++; \ BASE_DIR="/mnt/my_storage"; DATA_DIR="$BASE_DIR/BitcoinData"; LOG_DIR="$BASE_DIR/logs"; \ (echo ""; for c in $COMMITS; do git fetch -q origin $c && git log -1 --pretty='%h %s' $c || exit 1; done; echo "") && \ hyperfine \ --sort command \ --runs 1 \ --export-json "$BASE_DIR/rdx-$(sed -E 's/(\w{8})\w+ ?/\1-/g;s/-$//'<<<"$COMMITS")-$STOP-$DBCACHE-$CC.json" \ --parameter-list COMMIT ${COMMITS// /,} \ --prepare "killall bitcoind 2>/dev/null; rm -f $DATA_DIR/debug.log; git checkout {COMMIT}; git clean -fxd; git reset --hard && \ cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release && ninja -C build bitcoind && \ ./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=$STOP -dbcache=1000 -printtoconsole=0; sleep 10" \ --cleanup "cp $DATA_DIR/debug.log $LOG_DIR/debug-{COMMIT}-$(date +%s).log" \ "COMPILER=$CC ./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=$STOP -dbcache=$DBCACHE -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0" e6bfd95d50 Merge bitcoin-core/gui#881: Move `FreespaceChecker` class into its own module af653f321b coins: derive `batch_write_bytes` from `-dbcache` when unspecified Benchmark 1: COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=900000 -dbcache=10000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = e6bfd95d5012fa1d91f83bf4122cb292afd6277f) Time (abs ≡): 25016.346 s [User: 30333.911 s, System: 826.463 s] Benchmark 2: COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=900000 -dbcache=10000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = af653f321b135a59e38794b537737ed2f4a0040b) Time (abs ≡): 24801.283 s [User: 30328.665 s, System: 834.110 s] Relative speed comparison 1.01 COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=900000 -dbcache=10000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = e6bfd95d5012fa1d91f83bf4122cb292afd6277f) 1.00 COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=900000 -dbcache=10000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = af653f321b135a59e38794b537737ed2f4a0040b) ``` </details> ACKs for top commit: laanwj: Concept and code review ACK b6f8c48946cbfceb066de660c485ae1bd2c27cc1 TheCharlatan: ACK b6f8c48946cbfceb066de660c485ae1bd2c27cc1 andrewtoth: ACK b6f8c48946cbfceb066de660c485ae1bd2c27cc1 Tree-SHA512: a72008feca866e658f0cb4ebabbeee740f9fb13680e517b9d95eaa136e627a9dd5ee328456a2bf040401f4a1977ffa7446ad13f66b286b3419ff0c35095a3521