mirror of
https://github.com/bitcoin/bitcoin.git
synced 2025-02-08 10:31:50 -05:00
Merge bitcoin/bitcoin#29370: assumeutxo: Get rid of faked nTx and nChainTx values
9d9a7458a2
assumeutxo: Remove BLOCK_ASSUMED_VALID flag (Ryan Ofsky)ef174e9ed2
test: assumeutxo snapshot block CheckBlockIndex crash test (Ryan Ofsky)0391458d76
test: assumeutxo stale block CheckBlockIndex crash test (Ryan Ofsky)ef29c8b662
assumeutxo: Get rid of faked nTx and nChainTx values (Ryan Ofsky)9b97d5bbf9
doc: Improve comments describing setBlockIndexCandidates checks (Ryan Ofsky)0fd915ee6b
validation: Check GuessVerificationProgress is not called with disconnected block (Ryan Ofsky)63e8fc912c
ci: add getchaintxstats ubsan suppressions (Ryan Ofsky)f252e687ec
assumeutxo test: Add RPC test for fake nTx and nChainTx values (Ryan Ofsky) Pull request description: The `PopulateAndValidateSnapshot` function introduced inf6e2da5fb7
from #19806 has been setting fake `nTx` and `nChainTx` values that can show up in RPC results (https://github.com/bitcoin/bitcoin/issues/29328) and make `CBlockIndex` state hard to reason about, because it is difficult to know whether the values are real or fake. Revert to previous behavior of setting `nTx` and `nChainTx` to 0 when the values are unknown, instead of faking them. Also drop no-longer needed `BLOCK_ASSUMED_VALID` flag. Dropping the faked values also fixes assert failures in the `CheckBlockIndex` `(pindex->nChainTx == pindex->nTx + prev_chain_tx)` check that could happen previously if forked or out-of-order blocks before the snapshot got submitted while the snapshot was being validated. The PR includes two commits adding tests for these failures and describing them in detail. Compatibility note: This change could cause new `-checkblockindex` failures if a snapshot was loaded by a previous version of Bitcoin Core and not fully validated, because fake `nTx` values will have been saved to the block index. It would be pretty easy to avoid these failures by adding some compatibility code to `LoadBlockIndex` and changing `nTx` values from 1 to 0 when they are fake (when `(pindex->nStatus & BLOCK_VALID_MASK) < BLOCK_VALID_TRANSACTIONS`), but a little simpler not to worry about being compatible in this case. ACKs for top commit: Sjors: re-ACK9d9a7458a2
achow101: ACK9d9a7458a2
mzumsande: Tested ACK9d9a7458a2
maflcko: ACK9d9a7458a2
🎯 Tree-SHA512: b1e1e2731ec36be30d5f5914042517219378fc31486674030c29d9c7488ed83fb60ba7095600f469dc32f0d8ba79c49ff7706303006507654e1762f26ee416e0
This commit is contained in:
commit
b50554babd
8 changed files with 231 additions and 156 deletions
|
@ -51,18 +51,12 @@ The utility script
|
||||||
|
|
||||||
## Design notes
|
## Design notes
|
||||||
|
|
||||||
- A new block index `nStatus` flag is introduced, `BLOCK_ASSUMED_VALID`, to mark block
|
|
||||||
index entries that are required to be assumed-valid by a chainstate created
|
|
||||||
from a UTXO snapshot. This flag is used as a way to modify certain
|
|
||||||
CheckBlockIndex() logic to account for index entries that are pending validation by a
|
|
||||||
chainstate running asynchronously in the background.
|
|
||||||
|
|
||||||
- The concept of UTXO snapshots is treated as an implementation detail that lives
|
- The concept of UTXO snapshots is treated as an implementation detail that lives
|
||||||
behind the ChainstateManager interface. The external presentation of the changes
|
behind the ChainstateManager interface. The external presentation of the changes
|
||||||
required to facilitate the use of UTXO snapshots is the understanding that there are
|
required to facilitate the use of UTXO snapshots is the understanding that there are
|
||||||
now certain regions of the chain that can be temporarily assumed to be valid (using
|
now certain regions of the chain that can be temporarily assumed to be valid.
|
||||||
the nStatus flag mentioned above). In certain cases, e.g. wallet rescanning, this is
|
In certain cases, e.g. wallet rescanning, this is very similar to dealing with
|
||||||
very similar to dealing with a pruned chain.
|
a pruned chain.
|
||||||
|
|
||||||
Logic outside ChainstateManager should try not to know about snapshots, instead
|
Logic outside ChainstateManager should try not to know about snapshots, instead
|
||||||
preferring to work in terms of more general states like assumed-valid.
|
preferring to work in terms of more general states like assumed-valid.
|
||||||
|
|
67
src/chain.h
67
src/chain.h
|
@ -98,16 +98,20 @@ enum BlockStatus : uint32_t {
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Only first tx is coinbase, 2 <= coinbase input script length <= 100, transactions valid, no duplicate txids,
|
* Only first tx is coinbase, 2 <= coinbase input script length <= 100, transactions valid, no duplicate txids,
|
||||||
* sigops, size, merkle root. Implies all parents are at least TREE but not necessarily TRANSACTIONS. When all
|
* sigops, size, merkle root. Implies all parents are at least TREE but not necessarily TRANSACTIONS.
|
||||||
* parent blocks also have TRANSACTIONS, CBlockIndex::nChainTx will be set.
|
*
|
||||||
|
* If a block's validity is at least VALID_TRANSACTIONS, CBlockIndex::nTx will be set. If a block and all previous
|
||||||
|
* blocks back to the genesis block or an assumeutxo snapshot block are at least VALID_TRANSACTIONS,
|
||||||
|
* CBlockIndex::nChainTx will be set.
|
||||||
*/
|
*/
|
||||||
BLOCK_VALID_TRANSACTIONS = 3,
|
BLOCK_VALID_TRANSACTIONS = 3,
|
||||||
|
|
||||||
//! Outputs do not overspend inputs, no double spends, coinbase output ok, no immature coinbase spends, BIP30.
|
//! Outputs do not overspend inputs, no double spends, coinbase output ok, no immature coinbase spends, BIP30.
|
||||||
//! Implies all parents are either at least VALID_CHAIN, or are ASSUMED_VALID
|
//! Implies all previous blocks back to the genesis block or an assumeutxo snapshot block are at least VALID_CHAIN.
|
||||||
BLOCK_VALID_CHAIN = 4,
|
BLOCK_VALID_CHAIN = 4,
|
||||||
|
|
||||||
//! Scripts & signatures ok. Implies all parents are either at least VALID_SCRIPTS, or are ASSUMED_VALID.
|
//! Scripts & signatures ok. Implies all previous blocks back to the genesis block or an assumeutxo snapshot block
|
||||||
|
//! are at least VALID_SCRIPTS.
|
||||||
BLOCK_VALID_SCRIPTS = 5,
|
BLOCK_VALID_SCRIPTS = 5,
|
||||||
|
|
||||||
//! All validity bits.
|
//! All validity bits.
|
||||||
|
@ -124,21 +128,8 @@ enum BlockStatus : uint32_t {
|
||||||
|
|
||||||
BLOCK_OPT_WITNESS = 128, //!< block data in blk*.dat was received with a witness-enforcing client
|
BLOCK_OPT_WITNESS = 128, //!< block data in blk*.dat was received with a witness-enforcing client
|
||||||
|
|
||||||
/**
|
BLOCK_STATUS_RESERVED = 256, //!< Unused flag that was previously set on assumeutxo snapshot blocks and their
|
||||||
* If ASSUMED_VALID is set, it means that this block has not been validated
|
//!< ancestors before they were validated, and unset when they were validated.
|
||||||
* and has validity status less than VALID_SCRIPTS. Also that it may have
|
|
||||||
* descendant blocks with VALID_SCRIPTS set, because they can be validated
|
|
||||||
* based on an assumeutxo snapshot.
|
|
||||||
*
|
|
||||||
* When an assumeutxo snapshot is loaded, the ASSUMED_VALID flag is added to
|
|
||||||
* unvalidated blocks at the snapshot height and below. Then, as the background
|
|
||||||
* validation progresses, and these blocks are validated, the ASSUMED_VALID
|
|
||||||
* flags are removed. See `doc/design/assumeutxo.md` for details.
|
|
||||||
*
|
|
||||||
* This flag is only used to implement checks in CheckBlockIndex() and
|
|
||||||
* should not be used elsewhere.
|
|
||||||
*/
|
|
||||||
BLOCK_ASSUMED_VALID = 256,
|
|
||||||
};
|
};
|
||||||
|
|
||||||
/** The block chain is a tree shaped structure starting with the
|
/** The block chain is a tree shaped structure starting with the
|
||||||
|
@ -173,21 +164,16 @@ public:
|
||||||
//! (memory only) Total amount of work (expected number of hashes) in the chain up to and including this block
|
//! (memory only) Total amount of work (expected number of hashes) in the chain up to and including this block
|
||||||
arith_uint256 nChainWork{};
|
arith_uint256 nChainWork{};
|
||||||
|
|
||||||
//! Number of transactions in this block.
|
//! Number of transactions in this block. This will be nonzero if the block
|
||||||
|
//! reached the VALID_TRANSACTIONS level, and zero otherwise.
|
||||||
//! Note: in a potential headers-first mode, this number cannot be relied upon
|
//! Note: in a potential headers-first mode, this number cannot be relied upon
|
||||||
//! Note: this value is faked during UTXO snapshot load to ensure that
|
|
||||||
//! LoadBlockIndex() will load index entries for blocks that we lack data for.
|
|
||||||
//! @sa ActivateSnapshot
|
|
||||||
unsigned int nTx{0};
|
unsigned int nTx{0};
|
||||||
|
|
||||||
//! (memory only) Number of transactions in the chain up to and including this block.
|
//! (memory only) Number of transactions in the chain up to and including this block.
|
||||||
//! This value will be non-zero only if and only if transactions for this block and all its parents are available.
|
//! This value will be non-zero if this block and all previous blocks back
|
||||||
|
//! to the genesis block or an assumeutxo snapshot block have reached the
|
||||||
|
//! VALID_TRANSACTIONS level.
|
||||||
//! Change to 64-bit type before 2024 (assuming worst case of 60 byte transactions).
|
//! Change to 64-bit type before 2024 (assuming worst case of 60 byte transactions).
|
||||||
//!
|
|
||||||
//! Note: this value is faked during use of a UTXO snapshot because we don't
|
|
||||||
//! have the underlying block data available during snapshot load.
|
|
||||||
//! @sa AssumeutxoData
|
|
||||||
//! @sa ActivateSnapshot
|
|
||||||
unsigned int nChainTx{0};
|
unsigned int nChainTx{0};
|
||||||
|
|
||||||
//! Verification status of this block. See enum BlockStatus
|
//! Verification status of this block. See enum BlockStatus
|
||||||
|
@ -262,15 +248,14 @@ public:
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Check whether this block's and all previous blocks' transactions have been
|
* Check whether this block and all previous blocks back to the genesis block or an assumeutxo snapshot block have
|
||||||
* downloaded (and stored to disk) at some point.
|
* reached VALID_TRANSACTIONS and had transactions downloaded (and stored to disk) at some point.
|
||||||
*
|
*
|
||||||
* Does not imply the transactions are consensus-valid (ConnectTip might fail)
|
* Does not imply the transactions are consensus-valid (ConnectTip might fail)
|
||||||
* Does not imply the transactions are still stored on disk. (IsBlockPruned might return true)
|
* Does not imply the transactions are still stored on disk. (IsBlockPruned might return true)
|
||||||
*
|
*
|
||||||
* Note that this will be true for the snapshot base block, if one is loaded (and
|
* Note that this will be true for the snapshot base block, if one is loaded, since its nChainTx value will have
|
||||||
* all subsequent assumed-valid blocks) since its nChainTx value will have been set
|
* been set manually based on the related AssumeutxoData entry.
|
||||||
* manually based on the related AssumeutxoData entry.
|
|
||||||
*/
|
*/
|
||||||
bool HaveNumChainTxs() const { return nChainTx != 0; }
|
bool HaveNumChainTxs() const { return nChainTx != 0; }
|
||||||
|
|
||||||
|
@ -318,14 +303,6 @@ public:
|
||||||
return ((nStatus & BLOCK_VALID_MASK) >= nUpTo);
|
return ((nStatus & BLOCK_VALID_MASK) >= nUpTo);
|
||||||
}
|
}
|
||||||
|
|
||||||
//! @returns true if the block is assumed-valid; this means it is queued to be
|
|
||||||
//! validated by a background chainstate.
|
|
||||||
bool IsAssumedValid() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main)
|
|
||||||
{
|
|
||||||
AssertLockHeld(::cs_main);
|
|
||||||
return nStatus & BLOCK_ASSUMED_VALID;
|
|
||||||
}
|
|
||||||
|
|
||||||
//! Raise the validity level of this block index entry.
|
//! Raise the validity level of this block index entry.
|
||||||
//! Returns true if the validity was changed.
|
//! Returns true if the validity was changed.
|
||||||
bool RaiseValidity(enum BlockStatus nUpTo) EXCLUSIVE_LOCKS_REQUIRED(::cs_main)
|
bool RaiseValidity(enum BlockStatus nUpTo) EXCLUSIVE_LOCKS_REQUIRED(::cs_main)
|
||||||
|
@ -335,12 +312,6 @@ public:
|
||||||
if (nStatus & BLOCK_FAILED_MASK) return false;
|
if (nStatus & BLOCK_FAILED_MASK) return false;
|
||||||
|
|
||||||
if ((nStatus & BLOCK_VALID_MASK) < nUpTo) {
|
if ((nStatus & BLOCK_VALID_MASK) < nUpTo) {
|
||||||
// If this block had been marked assumed-valid and we're raising
|
|
||||||
// its validity to a certain point, there is no longer an assumption.
|
|
||||||
if (nStatus & BLOCK_ASSUMED_VALID && nUpTo >= BLOCK_VALID_SCRIPTS) {
|
|
||||||
nStatus &= ~BLOCK_ASSUMED_VALID;
|
|
||||||
}
|
|
||||||
|
|
||||||
nStatus = (nStatus & ~BLOCK_VALID_MASK) | nUpTo;
|
nStatus = (nStatus & ~BLOCK_VALID_MASK) | nUpTo;
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
|
@ -91,13 +91,16 @@ CreateAndActivateUTXOSnapshot(
|
||||||
// these blocks instead
|
// these blocks instead
|
||||||
CBlockIndex *pindex = orig_tip;
|
CBlockIndex *pindex = orig_tip;
|
||||||
while (pindex && pindex != chain.m_chain.Tip()) {
|
while (pindex && pindex != chain.m_chain.Tip()) {
|
||||||
pindex->nStatus &= ~BLOCK_HAVE_DATA;
|
// Remove all data and validity flags by just setting
|
||||||
pindex->nStatus &= ~BLOCK_HAVE_UNDO;
|
// BLOCK_VALID_TREE. Also reset transaction counts and sequence
|
||||||
// We have to set the ASSUMED_VALID flag, because otherwise it
|
// ids that are set when blocks are received, to make test setup
|
||||||
// would not be possible to have a block index entry without HAVE_DATA
|
// more realistic and satisfy consistency checks in
|
||||||
// and with nTx > 0 (since we aren't setting the pruned flag);
|
// CheckBlockIndex().
|
||||||
// see CheckBlockIndex().
|
assert(pindex->IsValid(BlockStatus::BLOCK_VALID_TREE));
|
||||||
pindex->nStatus |= BLOCK_ASSUMED_VALID;
|
pindex->nStatus = BlockStatus::BLOCK_VALID_TREE;
|
||||||
|
pindex->nTx = 0;
|
||||||
|
pindex->nChainTx = 0;
|
||||||
|
pindex->nSequenceId = 0;
|
||||||
pindex = pindex->pprev;
|
pindex = pindex->pprev;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -276,9 +276,6 @@ struct SnapshotTestSetup : TestChain100Setup {
|
||||||
BOOST_CHECK_EQUAL(
|
BOOST_CHECK_EQUAL(
|
||||||
*node::ReadSnapshotBaseBlockhash(found),
|
*node::ReadSnapshotBaseBlockhash(found),
|
||||||
*chainman.SnapshotBlockhash());
|
*chainman.SnapshotBlockhash());
|
||||||
|
|
||||||
// Ensure that the genesis block was not marked assumed-valid.
|
|
||||||
BOOST_CHECK(!chainman.ActiveChain().Genesis()->IsAssumedValid());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
const auto& au_data = ::Params().AssumeutxoForHeight(snapshot_height);
|
const auto& au_data = ::Params().AssumeutxoForHeight(snapshot_height);
|
||||||
|
@ -410,7 +407,7 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_activate_snapshot, SnapshotTestSetup)
|
||||||
//! - First, verify that setBlockIndexCandidates is as expected when using a single,
|
//! - First, verify that setBlockIndexCandidates is as expected when using a single,
|
||||||
//! fully-validating chainstate.
|
//! fully-validating chainstate.
|
||||||
//!
|
//!
|
||||||
//! - Then mark a region of the chain BLOCK_ASSUMED_VALID and introduce a second chainstate
|
//! - Then mark a region of the chain as missing data and introduce a second chainstate
|
||||||
//! that will tolerate assumed-valid blocks. Run LoadBlockIndex() and ensure that the first
|
//! that will tolerate assumed-valid blocks. Run LoadBlockIndex() and ensure that the first
|
||||||
//! chainstate only contains fully validated blocks and the other chainstate contains all blocks,
|
//! chainstate only contains fully validated blocks and the other chainstate contains all blocks,
|
||||||
//! except those marked assume-valid, because those entries don't HAVE_DATA.
|
//! except those marked assume-valid, because those entries don't HAVE_DATA.
|
||||||
|
@ -421,7 +418,6 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_loadblockindex, TestChain100Setup)
|
||||||
Chainstate& cs1 = chainman.ActiveChainstate();
|
Chainstate& cs1 = chainman.ActiveChainstate();
|
||||||
|
|
||||||
int num_indexes{0};
|
int num_indexes{0};
|
||||||
int num_assumed_valid{0};
|
|
||||||
// Blocks in range [assumed_valid_start_idx, last_assumed_valid_idx) will be
|
// Blocks in range [assumed_valid_start_idx, last_assumed_valid_idx) will be
|
||||||
// marked as assumed-valid and not having data.
|
// marked as assumed-valid and not having data.
|
||||||
const int expected_assumed_valid{20};
|
const int expected_assumed_valid{20};
|
||||||
|
@ -456,35 +452,30 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_loadblockindex, TestChain100Setup)
|
||||||
reload_all_block_indexes();
|
reload_all_block_indexes();
|
||||||
BOOST_CHECK_EQUAL(cs1.setBlockIndexCandidates.size(), 1);
|
BOOST_CHECK_EQUAL(cs1.setBlockIndexCandidates.size(), 1);
|
||||||
|
|
||||||
// Mark some region of the chain assumed-valid, and remove the HAVE_DATA flag.
|
// Reset some region of the chain's nStatus, removing the HAVE_DATA flag.
|
||||||
for (int i = 0; i <= cs1.m_chain.Height(); ++i) {
|
for (int i = 0; i <= cs1.m_chain.Height(); ++i) {
|
||||||
LOCK(::cs_main);
|
LOCK(::cs_main);
|
||||||
auto index = cs1.m_chain[i];
|
auto index = cs1.m_chain[i];
|
||||||
|
|
||||||
// Blocks with heights in range [91, 110] are marked ASSUMED_VALID
|
// Blocks with heights in range [91, 110] are marked as missing data.
|
||||||
if (i < last_assumed_valid_idx && i >= assumed_valid_start_idx) {
|
if (i < last_assumed_valid_idx && i >= assumed_valid_start_idx) {
|
||||||
index->nStatus = BlockStatus::BLOCK_VALID_TREE | BlockStatus::BLOCK_ASSUMED_VALID;
|
index->nStatus = BlockStatus::BLOCK_VALID_TREE;
|
||||||
|
index->nTx = 0;
|
||||||
|
index->nChainTx = 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
++num_indexes;
|
++num_indexes;
|
||||||
if (index->IsAssumedValid()) ++num_assumed_valid;
|
|
||||||
|
|
||||||
// Note the last fully-validated block as the expected validated tip.
|
// Note the last fully-validated block as the expected validated tip.
|
||||||
if (i == (assumed_valid_start_idx - 1)) {
|
if (i == (assumed_valid_start_idx - 1)) {
|
||||||
validated_tip = index;
|
validated_tip = index;
|
||||||
BOOST_CHECK(!index->IsAssumedValid());
|
|
||||||
}
|
}
|
||||||
// Note the last assumed valid block as the snapshot base
|
// Note the last assumed valid block as the snapshot base
|
||||||
if (i == last_assumed_valid_idx - 1) {
|
if (i == last_assumed_valid_idx - 1) {
|
||||||
assumed_base = index;
|
assumed_base = index;
|
||||||
BOOST_CHECK(index->IsAssumedValid());
|
|
||||||
} else if (i == last_assumed_valid_idx) {
|
|
||||||
BOOST_CHECK(!index->IsAssumedValid());
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
BOOST_CHECK_EQUAL(expected_assumed_valid, num_assumed_valid);
|
|
||||||
|
|
||||||
// Note: cs2's tip is not set when ActivateExistingSnapshot is called.
|
// Note: cs2's tip is not set when ActivateExistingSnapshot is called.
|
||||||
Chainstate& cs2 = WITH_LOCK(::cs_main,
|
Chainstate& cs2 = WITH_LOCK(::cs_main,
|
||||||
return chainman.ActivateExistingSnapshot(*assumed_base->phashBlock));
|
return chainman.ActivateExistingSnapshot(*assumed_base->phashBlock));
|
||||||
|
|
|
@ -3687,7 +3687,18 @@ void ChainstateManager::ReceivedBlockTransactions(const CBlock& block, CBlockInd
|
||||||
{
|
{
|
||||||
AssertLockHeld(cs_main);
|
AssertLockHeld(cs_main);
|
||||||
pindexNew->nTx = block.vtx.size();
|
pindexNew->nTx = block.vtx.size();
|
||||||
pindexNew->nChainTx = 0;
|
// Typically nChainTx will be 0 at this point, but it can be nonzero if this
|
||||||
|
// is a pruned block which is being downloaded again, or if this is an
|
||||||
|
// assumeutxo snapshot block which has a hardcoded nChainTx value from the
|
||||||
|
// snapshot metadata. If the pindex is not the snapshot block and the
|
||||||
|
// nChainTx value is not zero, assert that value is actually correct.
|
||||||
|
auto prev_tx_sum = [](CBlockIndex& block) { return block.nTx + (block.pprev ? block.pprev->nChainTx : 0); };
|
||||||
|
if (!Assume(pindexNew->nChainTx == 0 || pindexNew->nChainTx == prev_tx_sum(*pindexNew) ||
|
||||||
|
pindexNew == GetSnapshotBaseBlock())) {
|
||||||
|
LogWarning("Internal bug detected: block %d has unexpected nChainTx %i that should be %i (%s %s). Please report this issue here: %s\n",
|
||||||
|
pindexNew->nHeight, pindexNew->nChainTx, prev_tx_sum(*pindexNew), PACKAGE_NAME, FormatFullVersion(), PACKAGE_BUGREPORT);
|
||||||
|
pindexNew->nChainTx = 0;
|
||||||
|
}
|
||||||
pindexNew->nFile = pos.nFile;
|
pindexNew->nFile = pos.nFile;
|
||||||
pindexNew->nDataPos = pos.nPos;
|
pindexNew->nDataPos = pos.nPos;
|
||||||
pindexNew->nUndoPos = 0;
|
pindexNew->nUndoPos = 0;
|
||||||
|
@ -3707,7 +3718,15 @@ void ChainstateManager::ReceivedBlockTransactions(const CBlock& block, CBlockInd
|
||||||
while (!queue.empty()) {
|
while (!queue.empty()) {
|
||||||
CBlockIndex *pindex = queue.front();
|
CBlockIndex *pindex = queue.front();
|
||||||
queue.pop_front();
|
queue.pop_front();
|
||||||
pindex->nChainTx = (pindex->pprev ? pindex->pprev->nChainTx : 0) + pindex->nTx;
|
// Before setting nChainTx, assert that it is 0 or already set to
|
||||||
|
// the correct value. This assert will fail after receiving the
|
||||||
|
// assumeutxo snapshot block if assumeutxo snapshot metadata has an
|
||||||
|
// incorrect hardcoded AssumeutxoData::nChainTx value.
|
||||||
|
if (!Assume(pindex->nChainTx == 0 || pindex->nChainTx == prev_tx_sum(*pindex))) {
|
||||||
|
LogWarning("Internal bug detected: block %d has unexpected nChainTx %i that should be %i (%s %s). Please report this issue here: %s\n",
|
||||||
|
pindex->nHeight, pindex->nChainTx, prev_tx_sum(*pindex), PACKAGE_NAME, FormatFullVersion(), PACKAGE_BUGREPORT);
|
||||||
|
}
|
||||||
|
pindex->nChainTx = prev_tx_sum(*pindex);
|
||||||
pindex->nSequenceId = nBlockSequenceId++;
|
pindex->nSequenceId = nBlockSequenceId++;
|
||||||
for (Chainstate *c : GetAll()) {
|
for (Chainstate *c : GetAll()) {
|
||||||
c->TryAddBlockIndexCandidate(pindex);
|
c->TryAddBlockIndexCandidate(pindex);
|
||||||
|
@ -5050,16 +5069,31 @@ void ChainstateManager::CheckBlockIndex()
|
||||||
size_t nNodes = 0;
|
size_t nNodes = 0;
|
||||||
int nHeight = 0;
|
int nHeight = 0;
|
||||||
CBlockIndex* pindexFirstInvalid = nullptr; // Oldest ancestor of pindex which is invalid.
|
CBlockIndex* pindexFirstInvalid = nullptr; // Oldest ancestor of pindex which is invalid.
|
||||||
CBlockIndex* pindexFirstMissing = nullptr; // Oldest ancestor of pindex which does not have BLOCK_HAVE_DATA.
|
CBlockIndex* pindexFirstMissing = nullptr; // Oldest ancestor of pindex which does not have BLOCK_HAVE_DATA, since assumeutxo snapshot if used.
|
||||||
CBlockIndex* pindexFirstNeverProcessed = nullptr; // Oldest ancestor of pindex for which nTx == 0.
|
CBlockIndex* pindexFirstNeverProcessed = nullptr; // Oldest ancestor of pindex for which nTx == 0, since assumeutxo snapshot if used.
|
||||||
CBlockIndex* pindexFirstNotTreeValid = nullptr; // Oldest ancestor of pindex which does not have BLOCK_VALID_TREE (regardless of being valid or not).
|
CBlockIndex* pindexFirstNotTreeValid = nullptr; // Oldest ancestor of pindex which does not have BLOCK_VALID_TREE (regardless of being valid or not).
|
||||||
CBlockIndex* pindexFirstNotTransactionsValid = nullptr; // Oldest ancestor of pindex which does not have BLOCK_VALID_TRANSACTIONS (regardless of being valid or not).
|
CBlockIndex* pindexFirstNotTransactionsValid = nullptr; // Oldest ancestor of pindex which does not have BLOCK_VALID_TRANSACTIONS (regardless of being valid or not), since assumeutxo snapshot if used.
|
||||||
CBlockIndex* pindexFirstNotChainValid = nullptr; // Oldest ancestor of pindex which does not have BLOCK_VALID_CHAIN (regardless of being valid or not).
|
CBlockIndex* pindexFirstNotChainValid = nullptr; // Oldest ancestor of pindex which does not have BLOCK_VALID_CHAIN (regardless of being valid or not), since assumeutxo snapshot if used.
|
||||||
CBlockIndex* pindexFirstNotScriptsValid = nullptr; // Oldest ancestor of pindex which does not have BLOCK_VALID_SCRIPTS (regardless of being valid or not).
|
CBlockIndex* pindexFirstNotScriptsValid = nullptr; // Oldest ancestor of pindex which does not have BLOCK_VALID_SCRIPTS (regardless of being valid or not), since assumeutxo snapshot if used.
|
||||||
CBlockIndex* pindexFirstAssumeValid = nullptr; // Oldest ancestor of pindex which has BLOCK_ASSUMED_VALID
|
|
||||||
|
// After checking an assumeutxo snapshot block, reset pindexFirst pointers
|
||||||
|
// to earlier blocks that have not been downloaded or validated yet, so
|
||||||
|
// checks for later blocks can assume the earlier blocks were validated and
|
||||||
|
// be stricter, testing for more requirements.
|
||||||
|
const CBlockIndex* snap_base{GetSnapshotBaseBlock()};
|
||||||
|
CBlockIndex *snap_first_missing{}, *snap_first_notx{}, *snap_first_notv{}, *snap_first_nocv{}, *snap_first_nosv{};
|
||||||
|
auto snap_update_firsts = [&] {
|
||||||
|
if (pindex == snap_base) {
|
||||||
|
std::swap(snap_first_missing, pindexFirstMissing);
|
||||||
|
std::swap(snap_first_notx, pindexFirstNeverProcessed);
|
||||||
|
std::swap(snap_first_notv, pindexFirstNotTransactionsValid);
|
||||||
|
std::swap(snap_first_nocv, pindexFirstNotChainValid);
|
||||||
|
std::swap(snap_first_nosv, pindexFirstNotScriptsValid);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
while (pindex != nullptr) {
|
while (pindex != nullptr) {
|
||||||
nNodes++;
|
nNodes++;
|
||||||
if (pindexFirstAssumeValid == nullptr && pindex->nStatus & BLOCK_ASSUMED_VALID) pindexFirstAssumeValid = pindex;
|
|
||||||
if (pindexFirstInvalid == nullptr && pindex->nStatus & BLOCK_FAILED_VALID) pindexFirstInvalid = pindex;
|
if (pindexFirstInvalid == nullptr && pindex->nStatus & BLOCK_FAILED_VALID) pindexFirstInvalid = pindex;
|
||||||
if (pindexFirstMissing == nullptr && !(pindex->nStatus & BLOCK_HAVE_DATA)) {
|
if (pindexFirstMissing == nullptr && !(pindex->nStatus & BLOCK_HAVE_DATA)) {
|
||||||
pindexFirstMissing = pindex;
|
pindexFirstMissing = pindex;
|
||||||
|
@ -5067,10 +5101,7 @@ void ChainstateManager::CheckBlockIndex()
|
||||||
if (pindexFirstNeverProcessed == nullptr && pindex->nTx == 0) pindexFirstNeverProcessed = pindex;
|
if (pindexFirstNeverProcessed == nullptr && pindex->nTx == 0) pindexFirstNeverProcessed = pindex;
|
||||||
if (pindex->pprev != nullptr && pindexFirstNotTreeValid == nullptr && (pindex->nStatus & BLOCK_VALID_MASK) < BLOCK_VALID_TREE) pindexFirstNotTreeValid = pindex;
|
if (pindex->pprev != nullptr && pindexFirstNotTreeValid == nullptr && (pindex->nStatus & BLOCK_VALID_MASK) < BLOCK_VALID_TREE) pindexFirstNotTreeValid = pindex;
|
||||||
|
|
||||||
if (pindex->pprev != nullptr && !pindex->IsAssumedValid()) {
|
if (pindex->pprev != nullptr) {
|
||||||
// Skip validity flag checks for BLOCK_ASSUMED_VALID index entries, since these
|
|
||||||
// *_VALID_MASK flags will not be present for index entries we are temporarily assuming
|
|
||||||
// valid.
|
|
||||||
if (pindexFirstNotTransactionsValid == nullptr &&
|
if (pindexFirstNotTransactionsValid == nullptr &&
|
||||||
(pindex->nStatus & BLOCK_VALID_MASK) < BLOCK_VALID_TRANSACTIONS) {
|
(pindex->nStatus & BLOCK_VALID_MASK) < BLOCK_VALID_TRANSACTIONS) {
|
||||||
pindexFirstNotTransactionsValid = pindex;
|
pindexFirstNotTransactionsValid = pindex;
|
||||||
|
@ -5100,36 +5131,26 @@ void ChainstateManager::CheckBlockIndex()
|
||||||
if (!pindex->HaveNumChainTxs()) assert(pindex->nSequenceId <= 0); // nSequenceId can't be set positive for blocks that aren't linked (negative is used for preciousblock)
|
if (!pindex->HaveNumChainTxs()) assert(pindex->nSequenceId <= 0); // nSequenceId can't be set positive for blocks that aren't linked (negative is used for preciousblock)
|
||||||
// VALID_TRANSACTIONS is equivalent to nTx > 0 for all nodes (whether or not pruning has occurred).
|
// VALID_TRANSACTIONS is equivalent to nTx > 0 for all nodes (whether or not pruning has occurred).
|
||||||
// HAVE_DATA is only equivalent to nTx > 0 (or VALID_TRANSACTIONS) if no pruning has occurred.
|
// HAVE_DATA is only equivalent to nTx > 0 (or VALID_TRANSACTIONS) if no pruning has occurred.
|
||||||
// Unless these indexes are assumed valid and pending block download on a
|
if (!m_blockman.m_have_pruned) {
|
||||||
// background chainstate.
|
|
||||||
if (!m_blockman.m_have_pruned && !pindex->IsAssumedValid()) {
|
|
||||||
// If we've never pruned, then HAVE_DATA should be equivalent to nTx > 0
|
// If we've never pruned, then HAVE_DATA should be equivalent to nTx > 0
|
||||||
assert(!(pindex->nStatus & BLOCK_HAVE_DATA) == (pindex->nTx == 0));
|
assert(!(pindex->nStatus & BLOCK_HAVE_DATA) == (pindex->nTx == 0));
|
||||||
if (pindexFirstAssumeValid == nullptr) {
|
assert(pindexFirstMissing == pindexFirstNeverProcessed);
|
||||||
// If we've got some assume valid blocks, then we might have
|
|
||||||
// missing blocks (not HAVE_DATA) but still treat them as
|
|
||||||
// having been processed (with a fake nTx value). Otherwise, we
|
|
||||||
// can assert that these are the same.
|
|
||||||
assert(pindexFirstMissing == pindexFirstNeverProcessed);
|
|
||||||
}
|
|
||||||
} else {
|
} else {
|
||||||
// If we have pruned, then we can only say that HAVE_DATA implies nTx > 0
|
// If we have pruned, then we can only say that HAVE_DATA implies nTx > 0
|
||||||
if (pindex->nStatus & BLOCK_HAVE_DATA) assert(pindex->nTx > 0);
|
if (pindex->nStatus & BLOCK_HAVE_DATA) assert(pindex->nTx > 0);
|
||||||
}
|
}
|
||||||
if (pindex->nStatus & BLOCK_HAVE_UNDO) assert(pindex->nStatus & BLOCK_HAVE_DATA);
|
if (pindex->nStatus & BLOCK_HAVE_UNDO) assert(pindex->nStatus & BLOCK_HAVE_DATA);
|
||||||
if (pindex->IsAssumedValid()) {
|
if (snap_base && snap_base->GetAncestor(pindex->nHeight) == pindex) {
|
||||||
// Assumed-valid blocks should have some nTx value.
|
|
||||||
assert(pindex->nTx > 0);
|
|
||||||
// Assumed-valid blocks should connect to the main chain.
|
// Assumed-valid blocks should connect to the main chain.
|
||||||
assert((pindex->nStatus & BLOCK_VALID_MASK) >= BLOCK_VALID_TREE);
|
assert((pindex->nStatus & BLOCK_VALID_MASK) >= BLOCK_VALID_TREE);
|
||||||
} else {
|
|
||||||
// Otherwise there should only be an nTx value if we have
|
|
||||||
// actually seen a block's transactions.
|
|
||||||
assert(((pindex->nStatus & BLOCK_VALID_MASK) >= BLOCK_VALID_TRANSACTIONS) == (pindex->nTx > 0)); // This is pruning-independent.
|
|
||||||
}
|
}
|
||||||
|
// There should only be an nTx value if we have
|
||||||
|
// actually seen a block's transactions.
|
||||||
|
assert(((pindex->nStatus & BLOCK_VALID_MASK) >= BLOCK_VALID_TRANSACTIONS) == (pindex->nTx > 0)); // This is pruning-independent.
|
||||||
// All parents having had data (at some point) is equivalent to all parents being VALID_TRANSACTIONS, which is equivalent to HaveNumChainTxs().
|
// All parents having had data (at some point) is equivalent to all parents being VALID_TRANSACTIONS, which is equivalent to HaveNumChainTxs().
|
||||||
assert((pindexFirstNeverProcessed == nullptr) == pindex->HaveNumChainTxs());
|
// HaveNumChainTxs will also be set in the assumeutxo snapshot block from snapshot metadata.
|
||||||
assert((pindexFirstNotTransactionsValid == nullptr) == pindex->HaveNumChainTxs());
|
assert((pindexFirstNeverProcessed == nullptr || pindex == snap_base) == pindex->HaveNumChainTxs());
|
||||||
|
assert((pindexFirstNotTransactionsValid == nullptr || pindex == snap_base) == pindex->HaveNumChainTxs());
|
||||||
assert(pindex->nHeight == nHeight); // nHeight must be consistent.
|
assert(pindex->nHeight == nHeight); // nHeight must be consistent.
|
||||||
assert(pindex->pprev == nullptr || pindex->nChainWork >= pindex->pprev->nChainWork); // For every block except the genesis block, the chainwork must be larger than the parent's.
|
assert(pindex->pprev == nullptr || pindex->nChainWork >= pindex->pprev->nChainWork); // For every block except the genesis block, the chainwork must be larger than the parent's.
|
||||||
assert(nHeight < 2 || (pindex->pskip && (pindex->pskip->nHeight < nHeight))); // The pskip pointer must point back for all but the first 2 blocks.
|
assert(nHeight < 2 || (pindex->pskip && (pindex->pskip->nHeight < nHeight))); // The pskip pointer must point back for all but the first 2 blocks.
|
||||||
|
@ -5142,30 +5163,64 @@ void ChainstateManager::CheckBlockIndex()
|
||||||
assert((pindex->nStatus & BLOCK_FAILED_MASK) == 0); // The failed mask cannot be set for blocks without invalid parents.
|
assert((pindex->nStatus & BLOCK_FAILED_MASK) == 0); // The failed mask cannot be set for blocks without invalid parents.
|
||||||
}
|
}
|
||||||
// Make sure nChainTx sum is correctly computed.
|
// Make sure nChainTx sum is correctly computed.
|
||||||
unsigned int prev_chain_tx = pindex->pprev ? pindex->pprev->nChainTx : 0;
|
if (!pindex->pprev) {
|
||||||
assert((pindex->nChainTx == pindex->nTx + prev_chain_tx)
|
// If no previous block, nTx and nChainTx must be the same.
|
||||||
// Transaction may be completely unset - happens if only the header was accepted but the block hasn't been processed.
|
assert(pindex->nChainTx == pindex->nTx);
|
||||||
|| (pindex->nChainTx == 0 && pindex->nTx == 0)
|
} else if (pindex->pprev->nChainTx > 0 && pindex->nTx > 0) {
|
||||||
// nChainTx may be unset, but nTx set (if a block has been accepted, but one of its predecessors hasn't been processed yet)
|
// If previous nChainTx is set and number of transactions in block is known, sum must be set.
|
||||||
|| (pindex->nChainTx == 0 && prev_chain_tx == 0 && pindex->pprev)
|
assert(pindex->nChainTx == pindex->nTx + pindex->pprev->nChainTx);
|
||||||
// Transaction counts prior to snapshot are unknown.
|
} else {
|
||||||
|| pindex->IsAssumedValid());
|
// Otherwise nChainTx should only be set if this is a snapshot
|
||||||
|
// block, and must be set if it is.
|
||||||
|
assert((pindex->nChainTx != 0) == (pindex == snap_base));
|
||||||
|
}
|
||||||
|
|
||||||
// Chainstate-specific checks on setBlockIndexCandidates
|
// Chainstate-specific checks on setBlockIndexCandidates
|
||||||
for (auto c : GetAll()) {
|
for (auto c : GetAll()) {
|
||||||
if (c->m_chain.Tip() == nullptr) continue;
|
if (c->m_chain.Tip() == nullptr) continue;
|
||||||
if (!CBlockIndexWorkComparator()(pindex, c->m_chain.Tip()) && pindexFirstNeverProcessed == nullptr) {
|
// Two main factors determine whether pindex is a candidate in
|
||||||
|
// setBlockIndexCandidates:
|
||||||
|
//
|
||||||
|
// - If pindex has less work than the chain tip, it should not be a
|
||||||
|
// candidate, and this will be asserted below. Otherwise it is a
|
||||||
|
// potential candidate.
|
||||||
|
//
|
||||||
|
// - If pindex or one of its parent blocks back to the genesis block
|
||||||
|
// or an assumeutxo snapshot never downloaded transactions
|
||||||
|
// (pindexFirstNeverProcessed is non-null), it should not be a
|
||||||
|
// candidate, and this will be asserted below. The only exception
|
||||||
|
// is if pindex itself is an assumeutxo snapshot block. Then it is
|
||||||
|
// also a potential candidate.
|
||||||
|
if (!CBlockIndexWorkComparator()(pindex, c->m_chain.Tip()) && (pindexFirstNeverProcessed == nullptr || pindex == snap_base)) {
|
||||||
|
// If pindex was detected as invalid (pindexFirstInvalid is
|
||||||
|
// non-null), it is not required to be in
|
||||||
|
// setBlockIndexCandidates.
|
||||||
if (pindexFirstInvalid == nullptr) {
|
if (pindexFirstInvalid == nullptr) {
|
||||||
const bool is_active = c == &ActiveChainstate();
|
// If pindex and all its parents back to the genesis block
|
||||||
// If this block sorts at least as good as the current tip and
|
// or an assumeutxo snapshot block downloaded transactions,
|
||||||
// is valid and we have all data for its parents, it must be in
|
// and the transactions were not pruned (pindexFirstMissing
|
||||||
// setBlockIndexCandidates. m_chain.Tip() must also be there
|
// is null), it is a potential candidate. The check
|
||||||
// even if some data has been pruned.
|
// excludes pruned blocks, because if any blocks were
|
||||||
|
// pruned between pindex the current chain tip, pindex will
|
||||||
|
// only temporarily be added to setBlockIndexCandidates,
|
||||||
|
// before being moved to m_blocks_unlinked. This check
|
||||||
|
// could be improved to verify that if all blocks between
|
||||||
|
// the chain tip and pindex have data, pindex must be a
|
||||||
|
// candidate.
|
||||||
//
|
//
|
||||||
if ((pindexFirstMissing == nullptr || pindex == c->m_chain.Tip())) {
|
// If pindex is the chain tip, it also is a potential
|
||||||
// The active chainstate should always have this block
|
// candidate.
|
||||||
// as a candidate, but a background chainstate should
|
//
|
||||||
// only have it if it is an ancestor of the snapshot base.
|
// If the chainstate was loaded from a snapshot and pindex
|
||||||
if (is_active || GetSnapshotBaseBlock()->GetAncestor(pindex->nHeight) == pindex) {
|
// is the base of the snapshot, pindex is also a potential
|
||||||
|
// candidate.
|
||||||
|
if (pindexFirstMissing == nullptr || pindex == c->m_chain.Tip() || pindex == c->SnapshotBase()) {
|
||||||
|
// If this chainstate is the active chainstate, pindex
|
||||||
|
// must be in setBlockIndexCandidates. Otherwise, this
|
||||||
|
// chainstate is a background validation chainstate, and
|
||||||
|
// pindex only needs to be added if it is an ancestor of
|
||||||
|
// the snapshot that is being validated.
|
||||||
|
if (c == &ActiveChainstate() || snap_base->GetAncestor(pindex->nHeight) == pindex) {
|
||||||
assert(c->setBlockIndexCandidates.count(pindex));
|
assert(c->setBlockIndexCandidates.count(pindex));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@ -5196,7 +5251,7 @@ void ChainstateManager::CheckBlockIndex()
|
||||||
if (pindexFirstMissing == nullptr) assert(!foundInUnlinked); // We aren't missing data for any parent -- cannot be in m_blocks_unlinked.
|
if (pindexFirstMissing == nullptr) assert(!foundInUnlinked); // We aren't missing data for any parent -- cannot be in m_blocks_unlinked.
|
||||||
if (pindex->pprev && (pindex->nStatus & BLOCK_HAVE_DATA) && pindexFirstNeverProcessed == nullptr && pindexFirstMissing != nullptr) {
|
if (pindex->pprev && (pindex->nStatus & BLOCK_HAVE_DATA) && pindexFirstNeverProcessed == nullptr && pindexFirstMissing != nullptr) {
|
||||||
// We HAVE_DATA for this block, have received data for all parents at some point, but we're currently missing data for some parent.
|
// We HAVE_DATA for this block, have received data for all parents at some point, but we're currently missing data for some parent.
|
||||||
assert(m_blockman.m_have_pruned || pindexFirstAssumeValid != nullptr); // We must have pruned, or else we're using a snapshot (causing us to have faked the received data for some parent(s)).
|
assert(m_blockman.m_have_pruned);
|
||||||
// This block may have entered m_blocks_unlinked if:
|
// This block may have entered m_blocks_unlinked if:
|
||||||
// - it has a descendant that at some point had more work than the
|
// - it has a descendant that at some point had more work than the
|
||||||
// tip, and
|
// tip, and
|
||||||
|
@ -5209,7 +5264,7 @@ void ChainstateManager::CheckBlockIndex()
|
||||||
const bool is_active = c == &ActiveChainstate();
|
const bool is_active = c == &ActiveChainstate();
|
||||||
if (!CBlockIndexWorkComparator()(pindex, c->m_chain.Tip()) && c->setBlockIndexCandidates.count(pindex) == 0) {
|
if (!CBlockIndexWorkComparator()(pindex, c->m_chain.Tip()) && c->setBlockIndexCandidates.count(pindex) == 0) {
|
||||||
if (pindexFirstInvalid == nullptr) {
|
if (pindexFirstInvalid == nullptr) {
|
||||||
if (is_active || GetSnapshotBaseBlock()->GetAncestor(pindex->nHeight) == pindex) {
|
if (is_active || snap_base->GetAncestor(pindex->nHeight) == pindex) {
|
||||||
assert(foundInUnlinked);
|
assert(foundInUnlinked);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@ -5220,6 +5275,7 @@ void ChainstateManager::CheckBlockIndex()
|
||||||
// End: actual consistency checks.
|
// End: actual consistency checks.
|
||||||
|
|
||||||
// Try descending into the first subnode.
|
// Try descending into the first subnode.
|
||||||
|
snap_update_firsts();
|
||||||
std::pair<std::multimap<CBlockIndex*,CBlockIndex*>::iterator,std::multimap<CBlockIndex*,CBlockIndex*>::iterator> range = forward.equal_range(pindex);
|
std::pair<std::multimap<CBlockIndex*,CBlockIndex*>::iterator,std::multimap<CBlockIndex*,CBlockIndex*>::iterator> range = forward.equal_range(pindex);
|
||||||
if (range.first != range.second) {
|
if (range.first != range.second) {
|
||||||
// A subnode was found.
|
// A subnode was found.
|
||||||
|
@ -5231,6 +5287,7 @@ void ChainstateManager::CheckBlockIndex()
|
||||||
// Move upwards until we reach a node of which we have not yet visited the last child.
|
// Move upwards until we reach a node of which we have not yet visited the last child.
|
||||||
while (pindex) {
|
while (pindex) {
|
||||||
// We are going to either move to a parent or a sibling of pindex.
|
// We are going to either move to a parent or a sibling of pindex.
|
||||||
|
snap_update_firsts();
|
||||||
// If pindex was the first with a certain property, unset the corresponding variable.
|
// If pindex was the first with a certain property, unset the corresponding variable.
|
||||||
if (pindex == pindexFirstInvalid) pindexFirstInvalid = nullptr;
|
if (pindex == pindexFirstInvalid) pindexFirstInvalid = nullptr;
|
||||||
if (pindex == pindexFirstMissing) pindexFirstMissing = nullptr;
|
if (pindex == pindexFirstMissing) pindexFirstMissing = nullptr;
|
||||||
|
@ -5239,7 +5296,6 @@ void ChainstateManager::CheckBlockIndex()
|
||||||
if (pindex == pindexFirstNotTransactionsValid) pindexFirstNotTransactionsValid = nullptr;
|
if (pindex == pindexFirstNotTransactionsValid) pindexFirstNotTransactionsValid = nullptr;
|
||||||
if (pindex == pindexFirstNotChainValid) pindexFirstNotChainValid = nullptr;
|
if (pindex == pindexFirstNotChainValid) pindexFirstNotChainValid = nullptr;
|
||||||
if (pindex == pindexFirstNotScriptsValid) pindexFirstNotScriptsValid = nullptr;
|
if (pindex == pindexFirstNotScriptsValid) pindexFirstNotScriptsValid = nullptr;
|
||||||
if (pindex == pindexFirstAssumeValid) pindexFirstAssumeValid = nullptr;
|
|
||||||
// Find our parent.
|
// Find our parent.
|
||||||
CBlockIndex* pindexPar = pindex->pprev;
|
CBlockIndex* pindexPar = pindex->pprev;
|
||||||
// Find which child we just visited.
|
// Find which child we just visited.
|
||||||
|
@ -5313,6 +5369,12 @@ double GuessVerificationProgress(const ChainTxData& data, const CBlockIndex *pin
|
||||||
if (pindex == nullptr)
|
if (pindex == nullptr)
|
||||||
return 0.0;
|
return 0.0;
|
||||||
|
|
||||||
|
if (!Assume(pindex->nChainTx > 0)) {
|
||||||
|
LogWarning("Internal bug detected: block %d has unset nChainTx (%s %s). Please report this issue here: %s\n",
|
||||||
|
pindex->nHeight, PACKAGE_NAME, FormatFullVersion(), PACKAGE_BUGREPORT);
|
||||||
|
return 0.0;
|
||||||
|
}
|
||||||
|
|
||||||
int64_t nNow = time(nullptr);
|
int64_t nNow = time(nullptr);
|
||||||
|
|
||||||
double fTxTotal;
|
double fTxTotal;
|
||||||
|
@ -5719,30 +5781,14 @@ bool ChainstateManager::PopulateAndValidateSnapshot(
|
||||||
// Fake various pieces of CBlockIndex state:
|
// Fake various pieces of CBlockIndex state:
|
||||||
CBlockIndex* index = nullptr;
|
CBlockIndex* index = nullptr;
|
||||||
|
|
||||||
// Don't make any modifications to the genesis block.
|
// Don't make any modifications to the genesis block since it shouldn't be
|
||||||
// This is especially important because we don't want to erroneously
|
// neccessary, and since the genesis block doesn't have normal flags like
|
||||||
// apply BLOCK_ASSUMED_VALID to genesis, which would happen if we didn't skip
|
// BLOCK_VALID_SCRIPTS set.
|
||||||
// it here (since it apparently isn't BLOCK_VALID_SCRIPTS).
|
|
||||||
constexpr int AFTER_GENESIS_START{1};
|
constexpr int AFTER_GENESIS_START{1};
|
||||||
|
|
||||||
for (int i = AFTER_GENESIS_START; i <= snapshot_chainstate.m_chain.Height(); ++i) {
|
for (int i = AFTER_GENESIS_START; i <= snapshot_chainstate.m_chain.Height(); ++i) {
|
||||||
index = snapshot_chainstate.m_chain[i];
|
index = snapshot_chainstate.m_chain[i];
|
||||||
|
|
||||||
// Fake nTx so that LoadBlockIndex() loads assumed-valid CBlockIndex
|
|
||||||
// entries (among other things)
|
|
||||||
if (!index->nTx) {
|
|
||||||
index->nTx = 1;
|
|
||||||
}
|
|
||||||
// Fake nChainTx so that GuessVerificationProgress reports accurately
|
|
||||||
index->nChainTx = index->pprev->nChainTx + index->nTx;
|
|
||||||
|
|
||||||
// Mark unvalidated block index entries beneath the snapshot base block as assumed-valid.
|
|
||||||
if (!index->IsValid(BLOCK_VALID_SCRIPTS)) {
|
|
||||||
// This flag will be removed once the block is fully validated by a
|
|
||||||
// background chainstate.
|
|
||||||
index->nStatus |= BLOCK_ASSUMED_VALID;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Fake BLOCK_OPT_WITNESS so that Chainstate::NeedsRedownload()
|
// Fake BLOCK_OPT_WITNESS so that Chainstate::NeedsRedownload()
|
||||||
// won't ask to rewind the entire assumed-valid chain on startup.
|
// won't ask to rewind the entire assumed-valid chain on startup.
|
||||||
if (DeploymentActiveAt(*index, *this, Consensus::DEPLOYMENT_SEGWIT)) {
|
if (DeploymentActiveAt(*index, *this, Consensus::DEPLOYMENT_SEGWIT)) {
|
||||||
|
@ -5758,6 +5804,7 @@ bool ChainstateManager::PopulateAndValidateSnapshot(
|
||||||
}
|
}
|
||||||
|
|
||||||
assert(index);
|
assert(index);
|
||||||
|
assert(index == snapshot_start_block);
|
||||||
index->nChainTx = au_data.nChainTx;
|
index->nChainTx = au_data.nChainTx;
|
||||||
snapshot_chainstate.setBlockIndexCandidates.insert(snapshot_start_block);
|
snapshot_chainstate.setBlockIndexCandidates.insert(snapshot_start_block);
|
||||||
|
|
||||||
|
|
|
@ -585,9 +585,10 @@ public:
|
||||||
const CBlockIndex* SnapshotBase() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
|
const CBlockIndex* SnapshotBase() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The set of all CBlockIndex entries with either BLOCK_VALID_TRANSACTIONS (for
|
* The set of all CBlockIndex entries that have as much work as our current
|
||||||
* itself and all ancestors) *or* BLOCK_ASSUMED_VALID (if using background
|
* tip or more, and transaction data needed to be validated (with
|
||||||
* chainstates) and as good as our current tip or better. Entries may be failed,
|
* BLOCK_VALID_TRANSACTIONS for each block and its parents back to the
|
||||||
|
* genesis block or an assumeutxo snapshot block). Entries may be failed,
|
||||||
* though, and pruning nodes may be missing the data for the block.
|
* though, and pruning nodes may be missing the data for the block.
|
||||||
*/
|
*/
|
||||||
std::set<CBlockIndex*, node::CBlockIndexWorkComparator> setBlockIndexCandidates;
|
std::set<CBlockIndex*, node::CBlockIndexWorkComparator> setBlockIndexCandidates;
|
||||||
|
|
|
@ -34,6 +34,7 @@ Interesting starting states could be loading a snapshot when the current chain t
|
||||||
"""
|
"""
|
||||||
from shutil import rmtree
|
from shutil import rmtree
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
from test_framework.messages import tx_from_hex
|
from test_framework.messages import tx_from_hex
|
||||||
from test_framework.test_framework import BitcoinTestFramework
|
from test_framework.test_framework import BitcoinTestFramework
|
||||||
from test_framework.util import (
|
from test_framework.util import (
|
||||||
|
@ -174,10 +175,26 @@ class AssumeutxoTest(BitcoinTestFramework):
|
||||||
|
|
||||||
# Generate a series of blocks that `n0` will have in the snapshot,
|
# Generate a series of blocks that `n0` will have in the snapshot,
|
||||||
# but that n1 and n2 don't yet see.
|
# but that n1 and n2 don't yet see.
|
||||||
|
assert n0.getblockcount() == START_HEIGHT
|
||||||
|
blocks = {START_HEIGHT: Block(n0.getbestblockhash(), 1, START_HEIGHT + 1)}
|
||||||
for i in range(100):
|
for i in range(100):
|
||||||
|
block_tx = 1
|
||||||
if i % 3 == 0:
|
if i % 3 == 0:
|
||||||
self.mini_wallet.send_self_transfer(from_node=n0)
|
self.mini_wallet.send_self_transfer(from_node=n0)
|
||||||
|
block_tx += 1
|
||||||
self.generate(n0, nblocks=1, sync_fun=self.no_op)
|
self.generate(n0, nblocks=1, sync_fun=self.no_op)
|
||||||
|
height = n0.getblockcount()
|
||||||
|
hash = n0.getbestblockhash()
|
||||||
|
blocks[height] = Block(hash, block_tx, blocks[height-1].chain_tx + block_tx)
|
||||||
|
if i == 4:
|
||||||
|
# Create a stale block that forks off the main chain before the snapshot.
|
||||||
|
temp_invalid = n0.getbestblockhash()
|
||||||
|
n0.invalidateblock(temp_invalid)
|
||||||
|
stale_hash = self.generateblock(n0, output="raw(aaaa)", transactions=[], sync_fun=self.no_op)["hash"]
|
||||||
|
n0.invalidateblock(stale_hash)
|
||||||
|
n0.reconsiderblock(temp_invalid)
|
||||||
|
stale_block = n0.getblock(stale_hash, 0)
|
||||||
|
|
||||||
|
|
||||||
self.log.info("-- Testing assumeutxo + some indexes + pruning")
|
self.log.info("-- Testing assumeutxo + some indexes + pruning")
|
||||||
|
|
||||||
|
@ -207,7 +224,7 @@ class AssumeutxoTest(BitcoinTestFramework):
|
||||||
assert_equal(
|
assert_equal(
|
||||||
dump_output['txoutset_hash'],
|
dump_output['txoutset_hash'],
|
||||||
"a4bf3407ccb2cc0145c49ebba8fa91199f8a3903daf0883875941497d2493c27")
|
"a4bf3407ccb2cc0145c49ebba8fa91199f8a3903daf0883875941497d2493c27")
|
||||||
assert_equal(dump_output["nchaintx"], 334)
|
assert_equal(dump_output["nchaintx"], blocks[SNAPSHOT_BASE_HEIGHT].chain_tx)
|
||||||
assert_equal(n0.getblockchaininfo()["blocks"], SNAPSHOT_BASE_HEIGHT)
|
assert_equal(n0.getblockchaininfo()["blocks"], SNAPSHOT_BASE_HEIGHT)
|
||||||
|
|
||||||
# Mine more blocks on top of the snapshot that n1 hasn't yet seen. This
|
# Mine more blocks on top of the snapshot that n1 hasn't yet seen. This
|
||||||
|
@ -228,6 +245,29 @@ class AssumeutxoTest(BitcoinTestFramework):
|
||||||
assert_equal(loaded['coins_loaded'], SNAPSHOT_BASE_HEIGHT)
|
assert_equal(loaded['coins_loaded'], SNAPSHOT_BASE_HEIGHT)
|
||||||
assert_equal(loaded['base_height'], SNAPSHOT_BASE_HEIGHT)
|
assert_equal(loaded['base_height'], SNAPSHOT_BASE_HEIGHT)
|
||||||
|
|
||||||
|
def check_tx_counts(final: bool) -> None:
|
||||||
|
"""Check nTx and nChainTx intermediate values right after loading
|
||||||
|
the snapshot, and final values after the snapshot is validated."""
|
||||||
|
for height, block in blocks.items():
|
||||||
|
tx = n1.getblockheader(block.hash)["nTx"]
|
||||||
|
chain_tx = n1.getchaintxstats(nblocks=1, blockhash=block.hash)["txcount"]
|
||||||
|
|
||||||
|
# Intermediate nTx of the starting block should be set, but nTx of
|
||||||
|
# later blocks should be 0 before they are downloaded.
|
||||||
|
if final or height == START_HEIGHT:
|
||||||
|
assert_equal(tx, block.tx)
|
||||||
|
else:
|
||||||
|
assert_equal(tx, 0)
|
||||||
|
|
||||||
|
# Intermediate nChainTx of the starting block and snapshot block
|
||||||
|
# should be set, but others should be 0 until they are downloaded.
|
||||||
|
if final or height in (START_HEIGHT, SNAPSHOT_BASE_HEIGHT):
|
||||||
|
assert_equal(chain_tx, block.chain_tx)
|
||||||
|
else:
|
||||||
|
assert_equal(chain_tx, 0)
|
||||||
|
|
||||||
|
check_tx_counts(final=False)
|
||||||
|
|
||||||
normal, snapshot = n1.getchainstates()["chainstates"]
|
normal, snapshot = n1.getchainstates()["chainstates"]
|
||||||
assert_equal(normal['blocks'], START_HEIGHT)
|
assert_equal(normal['blocks'], START_HEIGHT)
|
||||||
assert_equal(normal.get('snapshot_blockhash'), None)
|
assert_equal(normal.get('snapshot_blockhash'), None)
|
||||||
|
@ -238,6 +278,15 @@ class AssumeutxoTest(BitcoinTestFramework):
|
||||||
|
|
||||||
assert_equal(n1.getblockchaininfo()["blocks"], SNAPSHOT_BASE_HEIGHT)
|
assert_equal(n1.getblockchaininfo()["blocks"], SNAPSHOT_BASE_HEIGHT)
|
||||||
|
|
||||||
|
self.log.info("Submit a stale block that forked off the chain before the snapshot")
|
||||||
|
# Normally a block like this would not be downloaded, but if it is
|
||||||
|
# submitted early before the background chain catches up to the fork
|
||||||
|
# point, it winds up in m_blocks_unlinked and triggers a corner case
|
||||||
|
# that previously crashed CheckBlockIndex.
|
||||||
|
n1.submitblock(stale_block)
|
||||||
|
n1.getchaintips()
|
||||||
|
n1.getblock(stale_hash)
|
||||||
|
|
||||||
self.log.info("Submit a spending transaction for a snapshot chainstate coin to the mempool")
|
self.log.info("Submit a spending transaction for a snapshot chainstate coin to the mempool")
|
||||||
# spend the coinbase output of the first block that is not available on node1
|
# spend the coinbase output of the first block that is not available on node1
|
||||||
spend_coin_blockhash = n1.getblockhash(START_HEIGHT + 1)
|
spend_coin_blockhash = n1.getblockhash(START_HEIGHT + 1)
|
||||||
|
@ -275,6 +324,16 @@ class AssumeutxoTest(BitcoinTestFramework):
|
||||||
|
|
||||||
self.log.info("Restarted node before snapshot validation completed, reloading...")
|
self.log.info("Restarted node before snapshot validation completed, reloading...")
|
||||||
self.restart_node(1, extra_args=self.extra_args[1])
|
self.restart_node(1, extra_args=self.extra_args[1])
|
||||||
|
|
||||||
|
# Send snapshot block to n1 out of order. This makes the test less
|
||||||
|
# realistic because normally the snapshot block is one of the last
|
||||||
|
# blocks downloaded, but its useful to test because it triggers more
|
||||||
|
# corner cases in ReceivedBlockTransactions() and CheckBlockIndex()
|
||||||
|
# setting and testing nChainTx values, and it exposed previous bugs.
|
||||||
|
snapshot_hash = n0.getblockhash(SNAPSHOT_BASE_HEIGHT)
|
||||||
|
snapshot_block = n0.getblock(snapshot_hash, 0)
|
||||||
|
n1.submitblock(snapshot_block)
|
||||||
|
|
||||||
self.connect_nodes(0, 1)
|
self.connect_nodes(0, 1)
|
||||||
|
|
||||||
self.log.info(f"Ensuring snapshot chain syncs to tip. ({FINAL_HEIGHT})")
|
self.log.info(f"Ensuring snapshot chain syncs to tip. ({FINAL_HEIGHT})")
|
||||||
|
@ -291,6 +350,8 @@ class AssumeutxoTest(BitcoinTestFramework):
|
||||||
}
|
}
|
||||||
self.wait_until(lambda: n1.getindexinfo() == completed_idx_state)
|
self.wait_until(lambda: n1.getindexinfo() == completed_idx_state)
|
||||||
|
|
||||||
|
self.log.info("Re-check nTx and nChainTx values")
|
||||||
|
check_tx_counts(final=True)
|
||||||
|
|
||||||
for i in (0, 1):
|
for i in (0, 1):
|
||||||
n = self.nodes[i]
|
n = self.nodes[i]
|
||||||
|
@ -365,6 +426,11 @@ class AssumeutxoTest(BitcoinTestFramework):
|
||||||
self.connect_nodes(0, 2)
|
self.connect_nodes(0, 2)
|
||||||
self.wait_until(lambda: n2.getblockcount() == FINAL_HEIGHT)
|
self.wait_until(lambda: n2.getblockcount() == FINAL_HEIGHT)
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Block:
|
||||||
|
hash: str
|
||||||
|
tx: int
|
||||||
|
chain_tx: int
|
||||||
|
|
||||||
if __name__ == '__main__':
|
if __name__ == '__main__':
|
||||||
AssumeutxoTest().main()
|
AssumeutxoTest().main()
|
||||||
|
|
|
@ -51,6 +51,7 @@ unsigned-integer-overflow:CCoinsViewCache::Uncache
|
||||||
unsigned-integer-overflow:CompressAmount
|
unsigned-integer-overflow:CompressAmount
|
||||||
unsigned-integer-overflow:DecompressAmount
|
unsigned-integer-overflow:DecompressAmount
|
||||||
unsigned-integer-overflow:crypto/
|
unsigned-integer-overflow:crypto/
|
||||||
|
unsigned-integer-overflow:getchaintxstats*
|
||||||
unsigned-integer-overflow:MurmurHash3
|
unsigned-integer-overflow:MurmurHash3
|
||||||
unsigned-integer-overflow:CBlockPolicyEstimator::processBlockTx
|
unsigned-integer-overflow:CBlockPolicyEstimator::processBlockTx
|
||||||
unsigned-integer-overflow:TxConfirmStats::EstimateMedianVal
|
unsigned-integer-overflow:TxConfirmStats::EstimateMedianVal
|
||||||
|
@ -61,6 +62,7 @@ implicit-integer-sign-change:CBlockPolicyEstimator::processBlockTx
|
||||||
implicit-integer-sign-change:SetStdinEcho
|
implicit-integer-sign-change:SetStdinEcho
|
||||||
implicit-integer-sign-change:compressor.h
|
implicit-integer-sign-change:compressor.h
|
||||||
implicit-integer-sign-change:crypto/
|
implicit-integer-sign-change:crypto/
|
||||||
|
implicit-integer-sign-change:getchaintxstats*
|
||||||
implicit-integer-sign-change:TxConfirmStats::removeTx
|
implicit-integer-sign-change:TxConfirmStats::removeTx
|
||||||
implicit-integer-sign-change:prevector.h
|
implicit-integer-sign-change:prevector.h
|
||||||
implicit-integer-sign-change:verify_flags
|
implicit-integer-sign-change:verify_flags
|
||||||
|
|
Loading…
Add table
Reference in a new issue