Andrew Chow d5e4f9a439 Merge bitcoin/bitcoin#25740: assumeutxo: background validation completion ... 2b373fe49d docs: update assumeutxo.md (James O'Beirne) 87a1108c81 test: add snapshot completion unittests (James O'Beirne) d70919a88f refactor: make MempoolMutex() public (James O'Beirne) 7300ced9de log: add LoadBlockIndex() message for assumedvalid blocks (James O'Beirne) d96c59cc5c validation: add ChainMan logic for completing UTXO snapshot validation (James O'Beirne) f2a4f3376f move-only-ish: init: factor out chainstate initialization (James O'Beirne) 637a90b973 add Chainstate::HasCoinsViews() (James O'Beirne) c29f26b47b validation: add CChainState::m_disabled and ChainMan::isUsable (James O'Beirne) 5ee22cdafd add ChainstateManager.GetSnapshot{BaseHeight,BaseBlock}() (James O'Beirne) Pull request description: This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: https://github.com/bitcoin/bitcoin/pull/15606) Part two of replacing https://github.com/bitcoin/bitcoin/pull/24232. --- When a user activates a snapshot, the serialized UTXO set data is used to create an "assumed-valid" chainstate, which becomes active in an attempt to get the node to network tip as quickly as possible. Simultaneously in the background, the already-existing chainstate continues "conventional" IBD to both accumulate full block data and serve as a belt-and-suspenders to validate the assumed-valid chainstate. Once the background chainstate's tip reaches the base block of the snapshot used, we set `m_stop_use` on that chainstate and immediately take the hash of its UTXO set; we verify that this matches the assumeutxo value in the source code. Note that while we ultimately want to remove this background chainstate, we don't do so until the following initialization process, when we again check the UTXO set hash of the background chainstate, and if it continues to match, we remove the (now unnecessary) background chainstate, and move the (previously) assumed-valid chainstate into its place. We then reinitialize the chainstate in the normal way. As noted in previous comments, we could do the filesystem operations "inline" immediately when the background validation completes, but that's basically just an optimization that saves disk space until the next restart. It didn't strike me as worth the risk of moving chainstate data around on disk during runtime of the node, though maybe my concerns are overblown. The final result of this completion process is a fully-validated chain, where the only evidence that the user synced using assumeutxo is the existence of a `base_blockhash` file in the `chainstate` directory. ACKs for top commit: achow101: ACK 2b373fe49d Tree-SHA512: a204e1d6e6932dd83c799af3606b01a9faf893f04e9ee1a36d63f2f1ccfa9118bdc1c107d86976aa0312814267e6a42074bf3e2bf1dead4b2513efc6d955e13d		2023-03-07 18:54:59 -05:00
..
bench	wallet: single output groups filtering and grouping process	2023-03-06 09:45:40 -03:00
common	Deduplicate bitcoind and bitcoin-qt init code	2023-02-28 12:04:47 -05:00
compat
config
consensus	Merge bitcoin/bitcoin#26345: refactor: modernize the implementation of uint256.*	2023-02-06 13:56:51 -05:00
crc32c
crypto	Merge bitcoin/bitcoin#26153: Reduce wasted pseudorandom bytes in ChaCha20 + various improvements	2023-02-15 14:58:47 +00:00
index	Add InitError(error, details) overload	2023-02-28 12:04:47 -05:00
init	refactor: Stop using gArgs global in system.cpp	2023-02-27 14:21:13 -05:00
interfaces	Remove reindex special case from the progress bar label	2023-02-07 11:02:01 +01:00
ipc
kernel	kernel: add missing include	2023-02-22 15:46:21 +00:00
leveldb
logging	Use steady clock for logging timer	2023-01-31 18:48:50 +01:00
minisketch
node	Merge bitcoin/bitcoin#25740: assumeutxo: background validation completion	2023-03-07 18:54:59 -05:00
policy	clang-tidy: Fix modernize-use-default-member-init in headers	2023-01-31 11:50:10 +00:00
primitives
qt	Merge bitcoin/bitcoin#27150: Deduplicate bitcoind and bitcoin-qt init code	2023-03-07 13:05:01 -05:00
rpc	Merge bitcoin/bitcoin#27155: doc: Expand scantxoutset help text to cover tr() and miniscript	2023-03-06 11:15:16 -05:00
script	Fixed a couple of typos in comments to make linter happy	2023-03-03 19:06:02 -03:00
secp256k1
support	util: Remove duplicate include	2023-02-23 17:58:40 -05:00
test	Merge bitcoin/bitcoin#25740: assumeutxo: background validation completion	2023-03-07 18:54:59 -05:00
univalue	Merge bitcoin/bitcoin#26612: refactor: RPC: pass named argument value as string_view	2023-03-03 15:23:43 +01:00
util	Merge bitcoin/bitcoin#27150: Deduplicate bitcoind and bitcoin-qt init code	2023-03-07 13:05:01 -05:00
wallet	Merge bitcoin/bitcoin#25806: wallet: group outputs only once, decouple it from Coin Selection	2023-03-06 18:51:34 -05:00
zmq	clang-tidy: Fix modernize-use-default-member-init in headers	2023-01-31 11:50:10 +00:00
.bear-tidy-config
.clang-format
.clang-tidy	Fix clang-tidy readability-const-return-type violations	2023-02-01 11:33:35 +01:00
addrdb.cpp	Merge bitcoin/bitcoin#26847: p2p: track AddrMan totals by network and table, improve precision of adding fixed seeds	2023-01-31 16:08:44 -05:00
addrdb.h
addrman.cpp	Merge bitcoin/bitcoin#25619: net: avoid overriding non-virtual ToString() in CService and use better naming	2023-02-17 13:34:40 -05:00
addrman.h	addrman: Use std::nullopt instead of {}	2023-02-01 10:18:08 -05:00
addrman_impl.h	addrman, refactor: combine two size functions	2023-01-26 18:11:13 -05:00
arith_uint256.cpp
arith_uint256.h	Fix clang-tidy readability-const-return-type violations	2023-02-01 11:33:35 +01:00
attributes.h
banman.cpp
banman.h
base58.cpp
base58.h
bech32.cpp
bech32.h
bitcoin-chainstate.cpp	refactor, validation: Add ChainstateManagerOpts db options	2023-02-10 04:39:11 -04:00
bitcoin-cli-res.rc
bitcoin-cli.cpp	refactor: Stop using gArgs global in system.cpp	2023-02-27 14:21:13 -05:00
bitcoin-tx-res.rc
bitcoin-tx.cpp	Merge bitcoin/bitcoin#26691: Update secp256k1 subtree to libsecp256k1 version 0.2.0	2023-01-13 09:40:57 +00:00
bitcoin-util-res.rc
bitcoin-util.cpp	Use DataStream where possible	2023-01-26 10:44:05 +01:00
bitcoin-wallet-res.rc
bitcoin-wallet.cpp	refactor: Stop using gArgs global in system.cpp	2023-02-27 14:21:13 -05:00
bitcoind-res.rc
bitcoind.cpp	Deduplicate bitcoind and bitcoin-qt init code	2023-02-28 12:04:47 -05:00
blockencodings.cpp	Use DataStream where possible	2023-01-26 10:44:05 +01:00
blockencodings.h	script: remove out-of-date snprintf TODO	2023-02-15 14:42:28 -08:00
blockfilter.cpp	refactor: use Hash helper for double-SHA256 calculations	2023-01-31 19:34:35 +01:00
blockfilter.h
chain.cpp
chain.h
chainparams.cpp
chainparams.h
chainparamsbase.cpp
chainparamsbase.h
chainparamsseeds.h
checkqueue.h
clientversion.cpp
clientversion.h
coins.cpp	Add deterministic mode to CCoinsViewCache	2023-02-02 09:00:15 -05:00
coins.h	Add deterministic mode to CCoinsViewCache	2023-02-02 09:00:15 -05:00
compressor.cpp
compressor.h
core_io.h	rpc: decode Miniscript descriptor when possible in decodescript	2023-02-03 18:15:42 +01:00
core_memusage.h
core_read.cpp	Use DataStream where possible	2023-01-26 10:44:05 +01:00
core_write.cpp	rpc: decode Miniscript descriptor when possible in decodescript	2023-02-03 18:15:42 +01:00
cuckoocache.h	clang-tidy: Fix modernize-use-default-member-init in headers	2023-01-31 11:50:10 +00:00
dbwrapper.cpp	refactor, dbwrapper: Add DBParams and DBOptions structs	2023-02-10 04:39:11 -04:00
dbwrapper.h	refactor, dbwrapper: Add DBParams and DBOptions structs	2023-02-10 04:39:11 -04:00
deploymentinfo.cpp
deploymentinfo.h
deploymentstatus.cpp
deploymentstatus.h
dummywallet.cpp
external_signer.cpp	Fix clang-tidy readability-const-return-type violations	2023-02-01 11:33:35 +01:00
external_signer.h	Fix clang-tidy readability-const-return-type violations	2023-02-01 11:33:35 +01:00
flatfile.cpp
flatfile.h
fs.cpp	Merge bitcoin/bitcoin#25898: util: remove WSL 1 workaround in fs	2023-02-16 12:33:26 +00:00
fs.h	clang-tidy: Fix performance-move-const-arg in headers	2023-01-18 15:47:06 +00:00
hash.cpp
hash.h	Merge bitcoin/bitcoin#26649: refactor: Use AutoFile and HashVerifier (without ser-type and ser-version) where possible	2023-01-30 15:57:12 +00:00
headerssync.cpp
headerssync.h
httprpc.cpp	Merge bitcoin/bitcoin#25619: net: avoid overriding non-virtual ToString() in CService and use better naming	2023-02-17 13:34:40 -05:00
httprpc.h
httpserver.cpp	Merge bitcoin/bitcoin#26742: http: Track active requests and wait for last to finish - 2nd attempt	2023-03-06 19:35:59 -05:00
httpserver.h
i2p.cpp	Merge bitcoin/bitcoin#26837: I2P network optimizations	2023-02-22 17:58:41 +00:00
i2p.h
indirectmap.h
init.cpp	refactor: Stop using gArgs global in system.cpp	2023-02-27 14:21:13 -05:00
init.h
key.cpp	refactor: use Hash helper for double-SHA256 calculations	2023-01-31 19:34:35 +01:00
key.h	clang-tidy: Fix modernize-use-default-member-init in headers	2023-01-31 11:50:10 +00:00
key_io.cpp
key_io.h
logging.cpp
logging.h
Makefile.am	Deduplicate bitcoind and bitcoin-qt init code	2023-02-28 12:04:47 -05:00
Makefile.bench.include
Makefile.crc32c.include
Makefile.leveldb.include
Makefile.minisketch.include
Makefile.qt.include
Makefile.qt_locale.include	qt: Periodic translation updates from Transifex	2023-02-27 13:53:29 +00:00
Makefile.qttest.include
Makefile.test.include	Merge bitcoin/bitcoin#27150: Deduplicate bitcoind and bitcoin-qt init code	2023-03-07 13:05:01 -05:00
Makefile.test_fuzz.include
Makefile.test_util.include	Merge bitcoin/bitcoin#26940: test: create random and coins utils, add amount helper, dedupe add_coin	2023-02-17 17:28:14 -05:00
Makefile.univalue.include
mapport.cpp	Merge bitcoin/bitcoin#25619: net: avoid overriding non-virtual ToString() in CService and use better naming	2023-02-17 13:34:40 -05:00
mapport.h	Remove configure-time setting of DEFAULT_UPNP	2023-01-28 15:24:13 +00:00
memusage.h
merkleblock.cpp
merkleblock.h
net.cpp	Merge bitcoin/bitcoin#26837: I2P network optimizations	2023-02-22 17:58:41 +00:00
net.h	Merge bitcoin/bitcoin#26837: I2P network optimizations	2023-02-22 17:58:41 +00:00
net_permissions.cpp
net_permissions.h
net_processing.cpp	Merge bitcoin/bitcoin#25619: net: avoid overriding non-virtual ToString() in CService and use better naming	2023-02-17 13:34:40 -05:00
net_processing.h
net_types.cpp
net_types.h
netaddress.cpp	Merge bitcoin/bitcoin#25619: net: avoid overriding non-virtual ToString() in CService and use better naming	2023-02-17 13:34:40 -05:00
netaddress.h	Merge bitcoin/bitcoin#25619: net: avoid overriding non-virtual ToString() in CService and use better naming	2023-02-17 13:34:40 -05:00
netbase.cpp	Merge bitcoin/bitcoin#25619: net: avoid overriding non-virtual ToString() in CService and use better naming	2023-02-17 13:34:40 -05:00
netbase.h
netgroup.cpp
netgroup.h
netmessagemaker.h
noui.cpp
noui.h
outputtype.cpp
outputtype.h
pow.cpp
pow.h
prevector.h
protocol.cpp
protocol.h
psbt.cpp	Merge bitcoin/bitcoin#24149: Signing support for Miniscript Descriptors	2023-02-16 10:01:33 +00:00
psbt.h	Merge bitcoin/bitcoin#26997: psbt: s/transcation/transaction/	2023-02-16 09:39:50 +00:00
pubkey.cpp	Merge bitcoin/bitcoin#26691: Update secp256k1 subtree to libsecp256k1 version 0.2.0	2023-01-13 09:40:57 +00:00
pubkey.h	Merge bitcoin/bitcoin#26691: Update secp256k1 subtree to libsecp256k1 version 0.2.0	2023-01-13 09:40:57 +00:00
random.cpp	random: consolidate WIN32 #ifdefs	2023-02-17 15:01:50 +00:00
random.h	Merge bitcoin/bitcoin#26153: Reduce wasted pseudorandom bytes in ChaCha20 + various improvements	2023-02-15 14:58:47 +00:00
randomenv.cpp	Merge bitcoin/bitcoin#26826: refactor: remove windows-only compat.h usage in randomenv	2023-01-23 16:36:27 +01:00
randomenv.h
rest.cpp	Use DataStream where possible	2023-01-26 10:44:05 +01:00
rest.h
reverse_iterator.h	doc: remove usages of C++11	2023-01-12 13:42:44 +00:00
scheduler.cpp
scheduler.h
serialize.h	clang-tidy: Fix modernize-use-default-member-init in headers	2023-01-31 11:50:10 +00:00
shutdown.cpp	Add InitError(error, details) overload	2023-02-28 12:04:47 -05:00
shutdown.h
signet.cpp
signet.h
span.h	clang-tidy: Fix modernize-use-default-member-init in headers	2023-01-31 11:50:10 +00:00
streams.h	Merge bitcoin/bitcoin#26992: refactor: Remove unused CDataStream SerializeMany constructor	2023-02-02 10:47:37 +00:00
sync.cpp
sync.h	doc: Export threadsafety.h from sync.h	2023-01-25 09:33:26 +01:00
threadsafety.h
timedata.cpp
timedata.h
tinyformat.h	clang-tidy: Fix modernize-use-default-member-init in headers	2023-01-31 11:50:10 +00:00
torcontrol.cpp	Merge bitcoin/bitcoin#25619: net: avoid overriding non-virtual ToString() in CService and use better naming	2023-02-17 13:34:40 -05:00
torcontrol.h	clang-tidy: Fix modernize-use-default-member-init in headers	2023-01-31 11:50:10 +00:00
txdb.cpp	refactor, txdb: Use DBParams struct in CBlockTreeDB	2023-02-10 04:39:11 -04:00
txdb.h	refactor, txdb: Use DBParams struct in CBlockTreeDB	2023-02-10 04:39:11 -04:00
txmempool.cpp	Fix clang-tidy readability-const-return-type violations	2023-02-01 11:33:35 +01:00
txmempool.h	Fix clang-tidy readability-const-return-type violations	2023-02-01 11:33:35 +01:00
txorphanage.cpp	Merge bitcoin/bitcoin#26551: p2p: Track orphans by who provided them	2023-01-26 10:36:18 +00:00
txorphanage.h	Merge bitcoin/bitcoin#26551: p2p: Track orphans by who provided them	2023-01-26 10:36:18 +00:00
txrequest.cpp
txrequest.h
uint256.cpp
uint256.h	Merge bitcoin/bitcoin#26345: refactor: modernize the implementation of uint256.*	2023-02-06 13:56:51 -05:00
undo.h
validation.cpp	Merge bitcoin/bitcoin#25740: assumeutxo: background validation completion	2023-03-07 18:54:59 -05:00
validation.h	Merge bitcoin/bitcoin#25740: assumeutxo: background validation completion	2023-03-07 18:54:59 -05:00
validationinterface.cpp	Fix clang-tidy readability-const-return-type violations	2023-02-01 11:33:35 +01:00
validationinterface.h
version.h
versionbits.cpp
versionbits.h
walletinitinterface.h
warnings.cpp
warnings.h