0
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-02-25 12:51:55 -05:00
Commit graph

200 commits

Author SHA1 Message Date
TheCharlatan
18e5ba7c80
refactor, blockstorage: Replace blocksdir arg
Add a blocks_dir field to the BlockManager options. Move functions
relying on the global gArgs to get the blocks_dir into the BlockManager
class.

This should eventually allow users of the BlockManager to not rely on
the global Args and instead pass in their own options.
2023-05-10 19:07:44 +02:00
TheCharlatan
ba8fc7d788
refactor: Replace string chain name constants with ChainTypes
This commit effectively moves the definition of these constants
out of the chainparamsbase to their own file.

Using the ChainType enums provides better type safety compared to
passing around strings.

The commit is part of an ongoing effort to decouple the libbitcoinkernel
library from the ArgsManager and other functionality that should not be
part of the kernel library.
2023-05-09 15:49:14 +02:00
fanquake
322ec63b01
Merge bitcoin/bitcoin#17860: fuzz: BIP 30, CVE-2018-17144
fa2d8b61f9 fuzz: BIP 42, BIP 30, CVE-2018-17144 (MarcoFalke)
faae7d5c00 Move LoadVerifyActivateChainstate to ChainTestingSetup (MarcoFalke)
fa26e3462a Avoid dereferencing interruption_point if it is nullptr (MarcoFalke)
fa846ee074 test: Add util to mine invalid blocks (MarcoFalke)

Pull request description:

  Add a validation fuzz test for BIP 30 and CVE-2018-17144

ACKs for top commit:
  dergoegge:
    Code review ACK fa2d8b61f9
  mzumsande:
    Tested ACK fa2d8b61f9

Tree-SHA512: 1f4620cc078709487abff24b304a6bb4eeab2e7628b392e2bc6de9cc0ce6745c413388ede6e93025d0c56eec905607ba9786633ef183e5779bf5183cc9ff92c0
2023-05-06 12:13:06 +01:00
MarcoFalke
faae7d5c00
Move LoadVerifyActivateChainstate to ChainTestingSetup 2023-05-05 13:19:09 +02:00
MarcoFalke
facdb8b331
Add BlockManagerOpts::chainparams reference
and use it in blockstorage.cpp
2023-05-04 19:26:43 +02:00
glozow
bdfe27c9d2
Merge bitcoin/bitcoin#26933: mempool: disallow txns under min relay fee, even in packages
bf77fc9cb4 [test] mempool full in package accept (glozow)
b51ebccc28 [validation] set PackageValidationState when mempool full (glozow)
563a2ee4f5 [policy] disallow transactions under min relay fee, even in packages (glozow)
c4554fe894 [test] package cpfp bumps parents <mempoolminfee but >=minrelaytxfee (glozow)
ac463e87df [test util] mock mempool minimum feerate (glozow)

Pull request description:

  Part of package relay, see #27463.

  Note that this still allows packages to bump transactions that are below the dynamic mempool minimum feerate, which means this still solves the "mempool is congested and my presigned 1sat/vB tx is screwed" problem for all transactions.

  On master, the package policy (only accessible through regtest-only RPC submitpackage) allows 0-fee (or otherwise below min relay feerate) transactions if they are bumped by a child. However, with default package limits, we don't yet have a DoS-resistant way of ensuring these transactions remain bumped throughout their time in the mempool. Primarily, the fee-bumping child may later be replaced by another transaction that doesn't bump the parent(s). The parent(s) could potentially stay bumped by other transactions, but not enough to ever be selected by the `BlockAssembler` (due to `blockmintxfee`).

  For example, (tested [here](https://github.com/glozow/bitcoin/commits/26933-motivation)):
  - The mempool accepts 24 below-minrelayfeerate transactions ("0-fee parents"), all bumped by a single high-fee transaction ("the fee-bumping child"). The fee-bumping child also spends a confirmed UTXO.
  - Two additional children are added to each 0-fee parent. These children each pay a feerate slightly above the minimum relay feerate (e.g. 1.9sat/vB) such that, for each 0-fee parent, the total fees of its two children divided by the total size of the children and parent is above the minimum relay feerate.
  - If a block template is built now, all transactions would be selected.
  - A transaction replaces the the fee-bumping child, spending only the confirmed UTXO and not any of the outputs from the 0-fee parents.
   - The 0-fee parents now each have 2 children. Their descendant feerates are above minrelayfeerate, which means that they remain in the mempool, even if the mempool evicts all below-minrelayfeerate packages.
   - If a block template is built now, none of the 0-fee parents or their children would be selected.
   - Even more low-feerate descendants can be added to these below-minrelayfeerate packages and they will not be evicted until they expire or the mempool reaches capacity.

  Unless we have a DoS-resistant way of ensuring package CPFP-bumped transactions are always bumped, allowing package CPFP to bump below-minrelayfeerate transactions can result in these problematic situations. See #27018 which proposes a partial solution with some limitations, and contains discussion about potential improvements to eviction strategy. While no adequate solution exists, for now, avoid these situations by requiring all transactions to meet min relay feerate.

ACKs for top commit:
  ajtowns:
    reACK bf77fc9cb4
  instagibbs:
    re-ACK bf77fc9cb4

Tree-SHA512: 28940f41493a9e280b010284316fb8caf1ed7b2090ba9a4ef8a3b2eafc5933601074b142f4f7d4e3c6c4cce99d3146f5c8e1393d9406c6f2070dd41c817985c9
2023-04-26 11:18:09 +01:00
TheCharlatan
be55f545d5
move-only: Extract common/args and common/config.cpp from util/system
This is an extraction of ArgsManager related functions from util/system
into their own common file.

Config file related functions are moved to common/config.cpp.

The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager. The ArgsManager belongs
into the common library, since the kernel library should not depend on
it. See doc/design/libraries.md for more information on this rationale.
2023-04-19 10:48:30 +02:00
glozow
ac463e87df [test util] mock mempool minimum feerate 2023-04-17 09:52:25 +01:00
MarcoFalke
fa721f1cab
Move ::nPruneTarget into BlockManager 2023-03-15 15:33:12 +01:00
Andrew Chow
832fa2d238
Merge bitcoin/bitcoin#25574: validation: Improve error handling when VerifyDB dosn't finish successfully
0af16e7134 doc: add release note for #25574 (Martin Zumsande)
57ef2a4812 validation: report if pruning prevents completion of verification (Martin Zumsande)
0c7785bb25 init, validation: Improve handling if VerifyDB() fails due to insufficient dbcache (Martin Zumsande)
d6f781f1cf validation: return VerifyDBResult::INTERRUPTED if verification was interrupted (Martin Zumsande)
6360b5302d validation: Change return value of VerifyDB to enum type (Martin Zumsande)

Pull request description:

  `VerifyDB()` can fail to complete due to insufficient dbcache at the level 3 checks. This PR improves the error handling in this case in the following ways:
  - The rpc `-verifychain` now returns false if the check can't be completed due to insufficient cache
  - During init, we only log a warning if the default values for `-checkblocks` and `-checklevel` are taken and the check doesn't complete. However, if the user actively specifies one of these args, we return with an InitError if we can't complete the check.

  This PR also changes `-verifychain` RPC to return `false` if the verification didn't finish due to missing block data (pruning) or due to being interrupted by the node being shutdown.

  Previously, this PR also included a fix for a possible assert during verification - this was done in #27009 (now merged).

ACKs for top commit:
  achow101:
    ACK 0af16e7134
  ryanofsky:
    Code review ACK 0af16e7134. Only small suggested changes since the last review, like renaming some of the enum values. I did leave more suggestions, but they are not very important and could be followups
  john-moffett:
    ACK 0af16e7134
  MarcoFalke:
    lgtm re-ACK 0af16e7134 🎚

Tree-SHA512: 84b4f767cf9bfbafef362312757c9bf765b41ae3977f4ece840e40c52a2266b1457832df0cdf70440be0aac2168d9b58fc817238630b0b6812f3836ca950bc0e
2023-02-22 14:19:44 -05:00
Martin Zumsande
0c7785bb25 init, validation: Improve handling if VerifyDB() fails due to insufficient dbcache
The rpc command verifychain now fails if the dbcache was not sufficient
to complete the verification at the specified level and depth.

In the same situation, the VerifyDB check during Init will now fail (and lead to
an early shutdown) if the user has explicitly specified -checkblocks or
-checklevel but the check couldn't be executed because of the limited
cache. If the user didn't change any of the two and is using the defaults, log a warning
but don't prevent the node from starting up.
2023-02-16 17:58:52 -05:00
Ryan Ofsky
aadd7c5b9b refactor, validation: Add ChainstateManagerOpts db options
Use ChainstateManagerOpts struct to remove ArgsManager uses from validation.cpp.

This commit does not change behavior.
2023-02-10 04:39:11 -04:00
Ryan Ofsky
0352258148 refactor, txdb: Use DBParams struct in CBlockTreeDB
Use DBParams struct to remove ArgsManager uses from txdb.

To reduce size of this commit, this moves references to gArgs variable out of
txdb.cpp to calling code in chainstate.cpp. But these moves are temporary. The
gArgs references in chainstate.cpp are moved out to calling code in init.cpp in
later commits.

This commit does not change behavior.
2023-02-10 04:39:11 -04:00
MarcoFalke
faf7b4f1fc
Add BlockManager::IsPruneMode() 2023-01-16 17:31:32 +01:00
Andrew Chow
2f6a8e5e02
Merge bitcoin/bitcoin#26695: bench: BlockAssembler on a mempool with packages
04528054fc [bench] BlockAssembler with mempool packages (glozow)
6ce265acf4 [test util] lock cs_main before pool.cs in PopulateMempool (glozow)
8791410662 [test util] randomize fee in PopulateMempool (glozow)
cba5934eb6 [miner] allow bypassing TestBlockValidity (glozow)
c058852308 [refactor] parameterize BlockAssembler::Options in PrepareBlock (glozow)
a2de971ba1 [refactor] add helper to apply ArgsManager to BlockAssembler::Options (glozow)

Pull request description:

  Performance of block template building matters as miners likely want to be able to start mining on a block with transactions asap after a block is found. We would want to know if a mempool PR accidentally caused, for example, a 100x slowdown. An `AssembleBlock()` bench exists, but it operates on a mempool with 101 transactions, each with 0 ancestors or descendants and with the same fee. Adding a bench with a more complex mempool is useful because (1) it's more realistic (2) updating packages can potentially cause the algorithm to take a long time.

ACKs for top commit:
  kevkevinpal:
    Tested ACK [0452805](04528054fc)
  achow101:
    ACK 04528054fc
  stickies-v:
    ACK 04528054f

Tree-SHA512: 38c138d6a75616651f9b1faf4e3a1cd833437a486f4e84308fbee958e8462bb570582c88f7ba7ab99d80191e97855ac2cf27c43cc21585d3e4b0e227effe2fb5
2023-01-11 18:11:11 -05:00
Hennadii Stepanov
9567bfeab9
clang-tidy: Add performance-no-automatic-move check
https://clang.llvm.org/extra/clang-tidy/checks/performance/no-automatic-move.html
2022-12-27 15:25:51 +00:00
Hennadii Stepanov
306ccd4927
scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-

Commits of previous years:
- 2021: f47dda2c58
- 2020: fa0074e2d8
- 2019: aaaaad6ac9
2022-12-24 23:49:50 +00:00
glozow
6ce265acf4
[test util] lock cs_main before pool.cs in PopulateMempool 2022-12-22 11:33:44 +00:00
glozow
8791410662
[test util] randomize fee in PopulateMempool
This makes the contents of the mempool more realistic and iterating by
ancestor feerate order more meaningful. If transactions have varying
feerates, it's also more likely that packages will need to be updated
during block template assembly.
2022-12-22 11:33:42 +00:00
Hennadii Stepanov
38941a703e
refactor: Move txmempool_entry.h --> kernel/mempool_entry.h 2022-11-30 10:37:57 +00:00
glozow
d0b1f613c2
Merge bitcoin/bitcoin#17786: refactor: Nuke policy/fees->mempool circular dependencies
c8dc0e3eaa refactor: Inline `CTxMemPoolEntry` class's functions (Hennadii Stepanov)
75bbe594e5 refactor: Move `CTxMemPoolEntry` class to its own module (Hennadii Stepanov)

Pull request description:

  This PR:
  - gets rid of the `policy/fees` -> `txmempool` -> `policy/fees` circular dependency
  - is an alternative to #13949, which nukes only one circular dependency

ACKs for top commit:
  ryanofsky:
    Code review ACK c8dc0e3eaa. Just include and whitespace changes since last review, and there's a moveonly commit now so it's very easy to review
  theStack:
    Code-review ACK c8dc0e3eaa
  glozow:
    utACK c8dc0e3eaa, agree these changes are an improvement.

Tree-SHA512: 36ece824e6ed3ab1a1e198b30a906c8ac12de24545f840eb046958a17315ac9260c7de26e11e2fbab7208adc3d74918db7a7e389444130f8810548ca2e81af41
2022-11-18 17:04:49 -08:00
Hennadii Stepanov
75bbe594e5
refactor: Move CTxMemPoolEntry class to its own module
This change nukes the policy/fees->mempool circular dependency.

Easy to review using `diff --color-moved=dimmed-zebra`.
2022-11-16 20:16:07 +00:00
fanquake
3a0b352c63
refactor: move url.h/cpp from lib util to lib common 2022-10-31 10:17:04 +00:00
MacroFake
a1fff275e7
Merge bitcoin/bitcoin#25704: refactor: Remove almost all validation option globals
aaaa7bd0ba iwyu: Add missing includes (MacroFake)
fa9ebec096 Remove g_parallel_script_checks (MacroFake)
fa7c834b9f Move ::fCheckBlockIndex into ChainstateManager (MacroFake)
fa43188d86 Move ::fCheckpointsEnabled into ChainstateManager (MacroFake)
cccca83099 Move ::nMinimumChainWork into ChainstateManager (MacroFake)
fa29d0b57c Move ::hashAssumeValid into ChainstateManager (MacroFake)
faf44876db Move ::nMaxTipAge into ChainstateManager (MacroFake)

Pull request description:

  It seems preferable to assign globals to a class (in this case `ChainstateManager`), than to leave them dangling. This should clarify scope for code-readers, as well as clarifying unit test behaviour.

ACKs for top commit:
  dergoegge:
    Code review ACK aaaa7bd0ba
  ryanofsky:
    Code review ACK aaaa7bd0ba. No changes since last review, other than rebase
  aureleoules:
    reACK aaaa7bd0ba

Tree-SHA512: 83ec3ba0fb4f1dad95810d4bd4e578454e0718dc1bdd3a794cc4e48aa819b6f5dad4ac4edab3719bdfd5f89cbe23c2740a50fd56c1ff81c99e521c5f6d4e898d
2022-10-26 11:41:57 +02:00
MacroFake
fa9ebec096
Remove g_parallel_script_checks 2022-10-18 14:12:42 +02:00
MacroFake
fa7c834b9f
Move ::fCheckBlockIndex into ChainstateManager
This changes the flag for the bitcoin-chainstate executable. Previously
it was false, now it is the chain's default value (still false for the
main chain).
2022-10-18 14:11:48 +02:00
MacroFake
fad7f2239c
test: Remove unused txmempool include from tests 2022-10-18 14:02:09 +02:00
James O'Beirne
cced4e7336 test: move-only-ish: factor out LoadVerifyActivateChainstate()
in TestingSetup(). This is used in the following commit to test
reinitializing chainstates after snapshot validation and cleanup.

Best reviewed with `git diff --color-moved=dimmed-zebra`.
2022-09-13 13:31:23 -04:00
James O'Beirne
51fc9241c0 test: allow on-disk coins and block tree dbs in tests
Used when testing cleanup of on-disk chainstate data for snapshot
testcases. Also necessary for simulating node restart in .cpp tests.
2022-09-13 13:31:23 -04:00
James O'Beirne
00eeb31c76 scripted-diff: rename CChainState -> Chainstate
-BEGIN VERIFY SCRIPT-
sed -i 's/CChainState/Chainstate/g' $(git grep -l CChainState ':(exclude)doc/release-notes*')
-END VERIFY SCRIPT-

Co-authored-by: MacroFake <falke.marco@gmail.com>
2022-09-09 11:47:27 -04:00
Andrew Chow
7281fac2e0
Merge bitcoin/bitcoin#25614: Severity-based logging, step 2
9580480570 Update debug logging section in the developer notes (Jon Atack)
1abaa31aa3 Update -debug and -debugexclude help docs for severity level logging (Jon Atack)
45f9282162 Create BCLog::Level::Trace log severity level (Jon Atack)
2a8712db4f Unit test coverage for -loglevel configuration option (klementtan)
eb7bee5f84 Create -loglevel configuration option (klementtan)
98a1f9c687 Unit test coverage for log severity levels (klementtan)
9c7507bf76 Create BCLog::Logger::LogLevelsString() helper function (klementtan)
8fe3457dbb Update LogAcceptCategory() and unit tests with log severity levels (klementtan)
c2797cfc60 Add BCLog::Logger::SetLogLevel()/SetCategoryLogLevel() for string inputs (klementtan)
f6c0cc0350 Add BCLog::Logger::m_category_log_levels data member and getter/setter (Jon Atack)
2978b387bf Add BCLog::Logger::m_log_level data member and getter/setter (Jon Atack)
f1379aeca9 Simplify BCLog::Level enum class and LogLevelToStr() function (Jon Atack)

Pull request description:

  This is an updated version of https://github.com/bitcoin/bitcoin/pull/25287 and the next steps in parent PR #25203 implementing, with Klement Tan, user-configurable, per-category severity log levels based on an idea by John Newbery and refined in GitHub discussions by Wladimir Van der Laan and Marco Falke.

  - simplify the `BCLog::Level` enum class and the `LogLevelToStr()` function and add documentation
  - update the logging logic to filter logs by log level both globally and per-category
  - add a hidden `-loglevel` help-debug config option to allow testing setting the global or per-category severity level on startup for logging categories enabled with the `-debug` configuration option or the logging RPC (Klement Tan)
  - add a `trace` log severity level selectable by the user; the plan is for the current debug messages to become trace, LogPrint ones to become debug, and LogPrintf ones to become info, warning, or error

  ```
  $ ./src/bitcoind -help-debug | grep -A10 loglevel
    -loglevel=<level>|<category>:<level>
         Set the global or per-category severity level for logging categories
         enabled with the -debug configuration option or the logging RPC:
         info, debug, trace (default=info); warning and error levels are
         always logged. If <category>:<level> is supplied, the setting
         will override the global one and may be specified multiple times
         to set multiple category-specific levels. <category> can be:
         addrman, bench, blockstorage, cmpctblock, coindb, estimatefee,
         http, i2p, ipc, leveldb, libevent, lock, mempool, mempoolrej,
         net, proxy, prune, qt, rand, reindex, rpc, selectcoins, tor,
         util, validation, walletdb, zmq.
  ```

  See the individual commit messages for details.

ACKs for top commit:
  jonatack:
    One final push per `git range-diff a5d5569 ce3c4c9 9580480` (should be trivial to re-ACK) to ensure this pull changes no default behavior in any way for users or the tests/CI in order to be completely v24 compatible, to update the unit test setup in general, and to update the debug logging section in the developer notes.
  klementtan:
    reACK 9580480570
  1440000bytes:
    reACK 9580480570
  vasild:
    ACK 9580480570
  dunxen:
    reACK 9580480
  brunoerg:
    reACK 9580480570

Tree-SHA512: 476a638e0581f40b5d058a9992691722e8b546471ec85e07cbc990798d1197fbffbd02e1b3d081b4978404e07a428378cdc8e159c0004b81f58be7fb01b7cba0
2022-09-01 15:57:56 -04:00
Suhas Daftuar
ed6cddd98e Require callers of AcceptBlockHeader() to perform anti-dos checks
In order to prevent memory DoS, we must ensure that we don't accept a new
header into memory until we've performed anti-DoS checks, such as verifying
that the header is part of a sufficiently high work chain. This commit adds a
new argument to AcceptBlockHeader() so that we can ensure that all call-sites
which might cause a new header to be accepted into memory have to grapple with
the question of whether the header is safe to accept, or needs further
validation.

This patch also fixes two places where low-difficulty-headers could have been
processed without such validation (processing an unrequested block from the
network, and processing a compact block).

Credit to Niklas Gögge for noticing this issue, and thanks to Sjors Provoost
for test code.
2022-08-29 08:10:35 -04:00
Jon Atack
45f9282162 Create BCLog::Level::Trace log severity level
for verbose log messages for development or debugging only, as bitcoind may run
more slowly, that are more granular/frequent than the Debug log level, i.e. for
very high-frequency, low-level messages to be logged distinctly from
higher-level, less-frequent debug logging that could still be usable in production.

An example would be to log higher-level peer events (connection, disconnection,
misbehavior, eviction) as Debug, versus Trace for low-level, high-volume p2p
messages in the BCLog::NET category. This will enable the user to log only the
former without the latter, in order to focus on high-level peer management events.

With respect to the name, "trace" is suggested as the most granular level
in resources like the following:
- https://sematext.com/blog/logging-levels
- https://howtodoinjava.com/log4j2/logging-levels

Update the test framework and add test coverage.
2022-08-20 11:55:17 +02:00
klementtan
eb7bee5f84 Create -loglevel configuration option
- add a -loglevel=<level>|<category:level> config option to allow users
  to set a global -loglevel and category-specific log levels. LogPrintLevel
  messages with a higher severity level than -loglevel will not be printed
  in the debug log.

- for now, this config option is debug-only during the migration to
  severity-based logging

- update unit and functional tests

Co-authored-by: "Jon Atack <jon@atack.com>"
2022-08-20 11:53:37 +02:00
Carl Dong
41c5201a90 validationcaches: Add and use ValidationCacheSizes
Also:

- Make DEFAULT_MAX_SIG_CACHE_SIZE into constexpr
  DEFAULT_MAX_SIG_CACHE_BYTES to utilize the compile-time integer
  arithmetic overflow checking available to constexpr.
- Fix comment (MiB instead of MB) for DEFAULT_MAX_SIG_CACHE_BYTES.
- Pass in max_size_bytes parameter to InitS*Cache(), modify log line to
  no longer allude to maxsigcachesize being split evenly between the two
  validation caches.
- Fix possible integer truncation and add a comment.

[META] I've kept the integer types as int64_t in order to not introduce
       unintended behaviour changes, in the next commit we will make
       them size_t.
2022-08-03 12:03:27 -04:00
Carl Dong
08dbc6ef72 cuckoocache: Return approximate memory size
Returning the approximate total size eliminates the need for
InitS*Cache() to do nElems*sizeof(uint256). The cuckoocache has a better
idea of this information.
2022-08-03 12:02:31 -04:00
MacroFake
ddddd6913b
sort after scripted-diff 2022-08-02 15:31:05 +02:00
MacroFake
fac812ca83
scripted-diff: Move mempool_args to src/node
It is part of the node library. Also, it won't be moved to the kernel
lib, as it will be pruned of ArgsManager.

-BEGIN VERIFY SCRIPT-
 # Move module
 git mv src/mempool_args.cpp src/node/
 git mv src/mempool_args.h   src/node/
 # Replacements
 sed -i 's:mempool_args\.h:node/mempool_args.h:g'     $(git grep -l mempool_args)
 sed -i 's:mempool_args\.cpp:node/mempool_args.cpp:g' $(git grep -l mempool_args)
 sed -i 's:MEMPOOL_ARGS_H:NODE_MEMPOOL_ARGS_H:g'      $(git grep -l MEMPOOL_ARGS_H)
-END VERIFY SCRIPT-
2022-08-02 15:31:01 +02:00
MacroFake
fa468bdfb6
Return optional error from ApplyArgsManOptions
Also pass in a (for now unused) reference to the params.

Both changes are needed for the next commit.
2022-08-02 15:21:50 +02:00
Russell Yanofsky
b3e7de7ee6 refactor: Reduce number of LoadChainstate return values 2022-07-19 15:54:52 -05:00
Russell Yanofsky
3b91d4b994 refactor: Reduce number of LoadChainstate parameters 2022-07-19 15:54:52 -05:00
Carl Dong
ce8b0f971b Use designated initializers for ChainstateManager::Options
This wasn't available at the time when ChainstateManager::Options was
introduced but is helpful to be explicit and ensure correctness.
2022-07-14 08:35:23 -04:00
Hennadii Stepanov
630c1711b4
refactor: Drop no longer needed util/designator.h 2022-07-07 20:01:01 +01:00
Carl Dong
d1684beabe fees: Pass in a filepath instead of referencing gArgs 2022-06-28 16:08:34 -04:00
Carl Dong
f1941e8bfd pool: Add and use MemPoolOptions, ApplyArgsManOptions
Reviewers: Note that CTxMemPool now requires a non-defaulted
CTxMemPool::Options for its constructor. Meaning that there's no need to
worry about a stray CTxMemPool constructor somewhere defaulting to
something incorrect. All instances of CTxMemPool construction are
addressed here in this commit.

We set options for CTxMemPool and construct it in many different ways. A
good example can be seen in how we determine CTxMemPool's check_ratio in
AppInitMain(...).

1. We first set the default based on chainparams's
   DefaultConsistencyChecks()
2. Then, we apply the ArgsManager option on top of that default
3. Finally, we clamp the result of that between 0 and 1 Million

With this patch, most CTxMemPool construction are along the lines of:

    MemPoolOptions mempool_opts{...default overrides...};
    ApplyArgsManOptions(argsman, mempool_opts);
    ...hard overrides...
    CTxMemPool pool{mempool_opts};

This "compositional" style of building options means that we can omit
unnecessary/irrelevant steps wherever we want but also maintain full
customizability.

For example:

- For users of libbitcoinkernel, where we eventually want to remove
  ArgsManager, they simply won't call (or even know about)
  ApplyArgsManOptions.

- See src/init.cpp to see how the check_ratio CTxMemPool option works
  after this change.

A MemPoolOptionsForTest helper was also added and used by tests/fuzz
tests where a local CTxMemPool needed to be created.

The change in src/test/fuzz/tx_pool.cpp seemingly changes behaviour by
applying ArgsManager options on top of the CTxMemPool::Options defaults.
However, in future commits where we introduce flags like -maxmempool,
the call to ApplyArgsManOptions is actually what preserves the existing
behaviour. Previously, although it wasn't obvious, our CTxMemPool would
consult gArgs for flags like -maxmempool when it needed it, so it
already relied on ArgsManager information. This patchset just laid bare
the obfuscatory perils of globals.

[META] As this patchset progresses, we will move more and more
       CTxMemPool-relevant options into MemPoolOptions and add their
       ArgsMan-related logic to ApplyArgsManOptions.
2022-06-28 15:30:05 -04:00
fanquake
a7a36590f5
Merge bitcoin/bitcoin#25223: [kernel 2e/n] miner: Make mempool optional, stop constructing temporary empty mempools
0f1a259657 miner: Make mempool optional for BlockAssembler (Carl Dong)
cc5739b27d miner: Make UpdatePackagesForAdded static (Carl Dong)
f024578b3a miner: Absorb SkipMapTxEntry into addPackageTxs (Carl Dong)

Pull request description:

  This is part of the libbitcoinkernel project: #24303, https://github.com/bitcoin/bitcoin/projects/18

  This is **_NOT_** dependent on, but is a "companion-PR" to #25215.

  ### Abstract

  This PR removes the need to construct `BlockAssembler` with temporary, empty mempools in cases where we don't want to source transactions from the mempool (e.g. in `TestChain100Setup::CreateBlock` and `generateblock`). After this PR, `BlockAssembler` will accept a `CTxMemPool` pointer and handle the `nullptr` case instead of requiring a `CTxMemPool` reference.

  An overview of the changes is best seen in the changes in the header file:

  ```diff
  diff --git a/src/node/miner.h b/src/node/miner.h
  index 7cf8e3fb9e..7e9f503602 100644
  --- a/src/node/miner.h
  +++ b/src/node/miner.h
  @@ -147,7 +147,7 @@ private:
       int64_t m_lock_time_cutoff;

       const CChainParams& chainparams;
  -    const CTxMemPool& m_mempool;
  +    const CTxMemPool* m_mempool;
       CChainState& m_chainstate;

   public:
  @@ -157,8 +157,8 @@ public:
           CFeeRate blockMinFeeRate;
       };

  -    explicit BlockAssembler(CChainState& chainstate, const CTxMemPool& mempool);
  -    explicit BlockAssembler(CChainState& chainstate, const CTxMemPool& mempool, const Options& options);
  +    explicit BlockAssembler(CChainState& chainstate, const CTxMemPool* mempool);
  +    explicit BlockAssembler(CChainState& chainstate, const CTxMemPool* mempool, const Options& options);

       /** Construct a new block template with coinbase to scriptPubKeyIn */
       std::unique_ptr<CBlockTemplate> CreateNewBlock(const CScript& scriptPubKeyIn);
  @@ -177,7 +177,7 @@ private:
       /** Add transactions based on feerate including unconfirmed ancestors
         * Increments nPackagesSelected / nDescendantsUpdated with corresponding
         * statistics from the package selection (for logging statistics). */
  -    void addPackageTxs(int& nPackagesSelected, int& nDescendantsUpdated) EXCLUSIVE_LOCKS_REQUIRED(m_mempool.cs);
  +    void addPackageTxs(const CTxMemPool& mempool, int& nPackagesSelected, int& nDescendantsUpdated) EXCLUSIVE_LOCKS_REQUIRED(mempool.cs);

       // helper functions for addPackageTxs()
       /** Remove confirmed (inBlock) entries from given set */
  @@ -189,15 +189,8 @@ private:
         * These checks should always succeed, and they're here
         * only as an extra check in case of suboptimal node configuration */
       bool TestPackageTransactions(const CTxMemPool::setEntries& package) const;
  -    /** Return true if given transaction from mapTx has already been evaluated,
  -      * or if the transaction's cached data in mapTx is incorrect. */
  -    bool SkipMapTxEntry(CTxMemPool::txiter it, indexed_modified_transaction_set& mapModifiedTx, CTxMemPool::setEntries& failedTx) EXCLUSIVE_LOCKS_REQUIRED(m_mempool.cs);
       /** Sort the package in an order that is valid to appear in a block */
       void SortForBlock(const CTxMemPool::setEntries& package, std::vector<CTxMemPool::txiter>& sortedEntries);
  -    /** Add descendants of given transactions to mapModifiedTx with ancestor
  -      * state updated assuming given transactions are inBlock. Returns number
  -      * of updated descendants. */
  -    int UpdatePackagesForAdded(const CTxMemPool::setEntries& alreadyAdded, indexed_modified_transaction_set& mapModifiedTx) EXCLUSIVE_LOCKS_REQUIRED(m_mempool.cs);
   };

   int64_t UpdateTime(CBlockHeader* pblock, const Consensus::Params& consensusParams, const CBlockIndex* pindexPrev);
  ```

  ### Alternatives

  Aside from approach in this current PR, we can also take the approach of moving the `CTxMemPool*` argument from the `BlockAssembler` constructor to `BlockAssembler::CreateNewBlock`, since that's where it's needed anyway. I did not push this approach because it requires quite a lot of call sites to be changed. However, I do have it coded up and can do that if people express a strong preference. This would look something like:

  ```
  BlockAssembler::BlockAssembler(CChainState& chainstate, const Options& options);
  BlockAssembler::CreateNewBlock(const CScript& scriptPubKeyIn, const CTxMemPool* maybe_mempool);
  ```

  ### Future work

  Although wholly out of scope for this PR, we could potentially refine the `BlockAssembler` interface further, so that we have:

  ```
  BlockAssembler::BlockAssembler(CChainState& chainstate, const Options& options);
  BlockAssembler::CreateNewBlock(const CScript& scriptPubKeyIn, std::vector<CTransaction>& txs);
  BlockAssembler::CreateNewBlock(const CScript& scriptPubKeyIn, const CTxMemPool& mempool);
  ```

  Whereby `TestChain100Setup::CreateBlock` and `generateblock` would call the `BlockAssembler::CreateNewBlock` that takes in `CTransaction`s and we can potentially remove `RegenerateCommitments` altogether. All other callers can use the `CTxMemPool` version.

ACKs for top commit:
  glozow:
    ACK 0f1a259657
  laanwj:
    Code review ACK 0f1a259657
  MarcoFalke:
    ACK 0f1a259657 🐊

Tree-SHA512: 2b4b1dbb43d85719f241ad1f19ceb7fc50cf764721da425a3d1ff71bd16328c4f86acff22e565bc9abee770d3ac8827a6676b66daa93dbf42dd817ad929e9448
2022-06-15 16:40:48 +01:00
Carl Dong
0f1a259657 miner: Make mempool optional for BlockAssembler
...also adjust callers

Changes:

- In BlockAssembler::CreateNewBlock, we now only lock m_mempool->cs and
  call addPackageTxs if m_mempool is not nullptr
- BlockAssembler::addPackageTxs now takes in a mempool reference, and is
  annotated to require that mempool's lock.
- In TestChain100Setup::CreateBlock and generateblock, don't construct
  an empty mempool, just pass in a nullptr for mempool
2022-06-06 15:38:09 -04:00
fanquake
aac9c259b0
Merge bitcoin/bitcoin#25065: [kernel 2c/n] Introduce kernel::Context, encapsulate global init/teardown
d87784ac87 kernel: SanityChecks: Return an error struct (Carl Dong)
265d6393bf Move init::SanityCheck to kernel::SanityCheck (Carl Dong)
fed085a1a4 init: Initialize globals with kernel::Context's life (Carl Dong)
7d03feef81 kernel: Introduce empty and unused kernel::Context (Carl Dong)
eeb4fc20c5 test: Use Set/UnsetGlobals in BasicTestingSetup (Carl Dong)

Pull request description:

  The full `init/common.cpp` is dependent on things like ArgsManager (which we wish to remove from libbitcoinkernel in the future) and sanity checks. These aren't necessary for libbitcoinkernel so we only extract the portion that is necessary (namely `init::{Set,Unset}Globals()`.

ACKs for top commit:
  theuni:
    ACK d87784ac87
  vasild:
    ACK d87784ac87

Tree-SHA512: cd6b4923ea1865001b5f0caed9a4ff99c198d22bf74154d935dc09a47fda22ebe585ec912398cea69f722454ed1dbb4898faab5a2d02fb4c5e719c5c8d71a3f9
2022-06-04 20:25:57 +01:00
Carl Dong
fed085a1a4 init: Initialize globals with kernel::Context's life
...instead of explicitly calling init::{Set,Unset}Globals.

Cool thing about this is that in both the testing and bitcoin-chainstate
codepaths, we no longer need to explicitly unset globals. The
kernel::Context goes out of scope and the globals are unset
"automatically".

Also construct kernel::Context outside of AppInitSanityChecks()
2022-06-02 11:40:03 -04:00
Carl Dong
eeb4fc20c5 test: Use Set/UnsetGlobals in BasicTestingSetup
...instead of calling initialization functions directly and having to
keep around a ECCVerifyHandle member variable.

This makes the initialization codepath of our tests more closely
resemble those of AppInitMain and potentially eases the review of
subsequent commit removing init::{Set,Unset}Globals.

[META] In a future commit, we will introduce a kernel::Context which
       calls init::{Set,Unset}Globals in its ctor and dtor. It will be
       owned by node::NodeContext, so in the end, this patchset won't
       have made the previously local ECCVerifyHandle global.
2022-05-31 14:18:31 -04:00