Add a blocks_dir field to the BlockManager options. Move functions
relying on the global gArgs to get the blocks_dir into the BlockManager
class.
This should eventually allow users of the BlockManager to not rely on
the global Args and instead pass in their own options.
Remove access to the global gArgs for the fastprune argument and
replace it by adding a field to the existing BlockManager Options
struct.
When running `clang-tidy-diff` on this commit, there is a diagnostic
error: `unknown type name 'uint64_t' [clang-diagnostic-error] uint64_t
prune_target{0};`, which is fixed by including cstdint.
This should eventually allow users of the BlockManager to not rely on
the global gArgs and instead pass in their own options.
This is a commit in preparation for the next few commits. The functions
are moved to methods to avoid their re-declaration for the purpose of
passing in BlockManager options.
The functions that were now moved into the BlockManager should no longer
use the params as an argument, but instead use the member variable.
In the moved ReadBlockFromDisk and UndoReadFromDisk, change
the function signature to accept a reference to a CBlockIndex instead of
a raw pointer. The pointer is expected to be non-null, so reflect that
in the type.
To allow for the move of functions to BlockManager methods all call
sites require an instantiated BlockManager, or a callback to one.
The lambda captures a reference to the chainman unique_ptr to retrieve
block data. An assert is added on the chainman to ensure that the lambda
is not used while the chainman is uninitialized.
This is done in preparation for the following commits where blockstorage
functions are made BlockManager methods.
fa266c4bbf Temporarily work around gcc-13 warning bug in interfaces_tests (MarcoFalke)
fa28850562 Fix clang-tidy performance-unnecessary-copy-initialization warnings (MarcoFalke)
faaa60a30e Remove unused find_value global function (MarcoFalke)
fa422aeec2 scripted-diff: Use UniValue::find_value method (MarcoFalke)
fa548ac872 Add UniValue::find_value method (MarcoFalke)
Pull request description:
The global function has issues:
* It causes gcc-13 warnings, see https://github.com/bitcoin/bitcoin/issues/26926
* There is no rationale for it being a global function, when it acts like a member function
* `performance-unnecessary-copy-initialization` clang-tidy isn't run on it
Fix all issues by making it a member function.
ACKs for top commit:
achow101:
ACK fa266c4bbf
hebasto:
re-ACK fa266c4bbf
Tree-SHA512: 6c4e25da3122cd3b91c376bef73ea94fb3beb7bf8ef5cb3853c5128d95bfbacbcbfb16cc843eb7b1a7ebd350c2b6311f8085eeacf9aeeab3366987037d209e44
Ensures better memory safety for this global. This came up during
discussion of the following commit, but is not strictly required for its
implementation.
If transactions are being added to the mempool at a rate faster than 7tx/s
(INVENTORY_BROADCAST_PER_SECOND) then peers' inventory_to_send queue can
become relatively large. If this happens, increase the number of txids
we include in an INV message (normally capped at 35) by 5 for each 1000
txids in the queue.
This will tend to clear a temporary excess out reasonably quickly; an
excess of 4000 invs to send will be cleared down to 1000 in about 30
minutes, while an excess of 20000 invs would be cleared down to 1000 in
about 60 minutes.
We use CompareDepthAndScore to choose an order of txs to inv. Rather
than sorting txs that have been evicted from the mempool at the end
of the list, sort them at the beginning so they are removed from
the queue immediately.
d168458d1f scripted-diff: Remove unused chainparamsbase includes (TheCharlatan)
e9ee8aaf3a Add missing definitions in prep for scripted diff (TheCharlatan)
ba8fc7d788 refactor: Replace string chain name constants with ChainTypes (TheCharlatan)
401453df41 refactor: Introduce ChainType getters for ArgsManager (TheCharlatan)
bfc21c31b2 refactor: Create chaintype files (TheCharlatan)
Pull request description:
This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". It is also a follow up to #26177.
It replaces pull request https://github.com/bitcoin/bitcoin/pull/27294, which just moved the constants to a new file, but did not re-declare them as enums.
The code move of the chain name constants out of the `chainparamsbase` to their own separate header allows the kernel `chainparams` to no longer include `chainparamsbase`. The `chainparamsbase` contain references to the `ArgsManager` and networking related options that should not belong to the kernel library. Besides this move, the constants are re-declared as enums with helper functions facilitating string conversions.
ACKs for top commit:
ryanofsky:
Code review ACK d168458d1f. Just suggested changes since last review.
Tree-SHA512: ac2fbe5cbbab4f52eae1e30af1f16700b6589eb4764c328a151a712adfc37f326cc94a65c385534c57d4bc92cc1a13bf1777d92bc924a20dbb30440e7380b316
fae1d9cded refactor: Remove unused GetTimeMillis (MarcoFalke)
Pull request description:
The function is unused, not type-safe, and does not denote the underlying clock type. So remove it.
ACKs for top commit:
willcl-ark:
tACK fae1d9cded
Tree-SHA512: 41ea7125d1964192b85a94265be974d02bf1e79b1feb61bff11486dc0ac811745156940ec5cad2ad1f94b653936f8ae563c959c1c4142203a55645fcb83203e8
This is a follow-up to previous commits moving the chain constants out
of chainparamsbase.
The script removes the chainparamsbase header in all files where it is
included, but not used. This is done by filtering against all defined
symbols of the header as well as its respective .cpp file.
The kernel chainparams now no longer relies on chainparamsbase.
-BEGIN VERIFY SCRIPT-
sed -i '/#include <chainparamsbase.h>/d' $( git grep -l 'chainparamsbase.h' | xargs grep -L 'CBaseChainParams\|CreateBaseChainParams\|SetupChainParamsBaseOptions\|BaseParams\|SelectBaseParams\|chainparamsbase.cpp' )
-END VERIFY SCRIPT-
This commit effectively moves the definition of these constants
out of the chainparamsbase to their own file.
Using the ChainType enums provides better type safety compared to
passing around strings.
The commit is part of an ongoing effort to decouple the libbitcoinkernel
library from the ArgsManager and other functionality that should not be
part of the kernel library.
These are introduced for the next commit where the usage of the
ChainType is adopted throughout the code.
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
This is the first of a number of commits with the goal of moving the
chain type definitions out of chainparamsbase to their own file and
implementing them as enums instead of constant strings. The goal is to
allow the kernel chainparams to no longer include chainparamsbase.
The commit is part of an ongoing effort to decouple the libbitcoinkernel
library from the ArgsManager and other functionality that should not be
part of the kernel library.
fe49f06c0e doc: clarify PR 26076 release note (Sjors Provoost)
bd13dc2f46 Switch hardened derivation marker to h in descriptors (Sjors Provoost)
Pull request description:
This makes it easier to handle descriptor strings manually, especially when importing from another Bitcoin Core wallet.
For example the `importdescriptors` RPC call is easiest to use `h` as the marker: `'["desc": ".../0h/..."]'`, avoiding the need for escape characters. With this change `listdescriptors` will use `h`, so you can copy-paste the result, without having to add escape characters or switch `'` to 'h' manually.
Both markers can still be parsed.
The `hdkeypath` field in `getaddressinfo` is also impacted by this change, except for legacy wallets. The latter is to prevent accidentally breaking ancient software that uses our legacy wallet.
See discussion in #15740
ACKs for top commit:
achow101:
ACK fe49f06c0e
darosior:
re-ACK fe49f06c0e
Tree-SHA512: f78bc873b24a6f7a2bf38f5dd58f2b723e35e6b10e4d65c36ec300e2d362d475eeca6e5afa04b3037ab4bee0bf8ebc93ea5fc18102a2111d3d88fc873c08dc89
fa83fb3161 wallet: Use steady clock to calculate number of derive iterations (MarcoFalke)
fa2c099cec wallet: Use steady clock to measure scanning duration (MarcoFalke)
fa97621804 qt: Use steady clock to throttle GUI notifications (MarcoFalke)
fa1d8044ab test: Use steady clock in index tests (MarcoFalke)
fa454dcb20 net: Use steady clock in InterruptibleRecv (MarcoFalke)
Pull request description:
`GetTimeMillis` has multiple issues:
* It doesn't denote the underlying clock type
* It isn't type-safe
* It is used incorrectly in places that should use a steady clock
Fix all issues here.
ACKs for top commit:
willcl-ark:
ACK fa83fb3161
martinus:
Code review ACK fa83fb3161, also ran all tests. All usages of the steady_clock are just for duration measurements, so the change to a different epoch is ok.
Tree-SHA512: 5ec4fede8c7f97e2e08863c011856e8304f16ba30a68fdeb42f96a50a04961092cbe46ccf9ea6ac99ff5203c09f9e0924eb483eb38d7df0759addc85116c8a9f
Avoid use of the expensive mod operator (%) when calculating the
buffer offset. No functional difference.
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
c4981e7f63 prune, import: fixes #23852 (mruddy)
Pull request description:
Fixes #23852
This allows pruning to work during the `-loadblock` import process.
An example use case is where you have a clean set of block files and you want to create a pruned node from them, but you don't want to alter the input set of block files.
#23852 noted that pruning was not working reliably during the loadblock import process. The reason why the loadblock process was not pruning regularly as it progressed is that the pruning process (`BlockManager::FindFilesToPrune`) checks the tip height of the active chainstate, and `CChainState::ActivateBestChain` was not called (which updates that tip height) in `ThreadImport` until after all the import files were processed.
An example bash command line that makes it easy to import a bunch of block files:
```
./src/qt/bitcoin-qt -debug -logthreadnames -datadir=/tmp/btc -prune=550 -loadblock=/readonly/btc/main/blk{00000..00043}.dat
```
One interesting side note is that `CChainState::ActivateBestChain` can be called while the import process is running (in the `loadblk` thread) by concurrent network message processing activity in the `msghand` thread. For example, one way to reproduce this easily is with the `getblockfrompeer` RPC (requesting a block with height greater than 100000) run from a node connected to an importing node. There are other ways too, but this is an easy way. I only mention this to explain how the `max_prune_height=225719` log message in the original issue came to occur.
ACKs for top commit:
achow101:
re-ACK c4981e7f63
Tree-SHA512: d287c7753952c22f598ba782914c47f45ad44ce60b0fbce9561354e701f1a2a98bafaaaa106c8428690b814e281305ca3622b177ed3cb2eb7559f07c958ab537
Also add flag to allow RPC methods that intendionally accept options and
parameters with the same name bypass the check.
Check and flag were suggested by ajtowns
https://github.com/bitcoin/bitcoin/pull/26485#issuecomment-1507916357
Co-authored-by: Anthony Towns <aj@erisian.com.au>
OBJ_NAMED_PARAMS type works the same as OBJ type except it registers the object
keys to be accepted as top-level named-only RPC parameters. Generated
documentation also lists the object keys seperately in a new "Named arguments"
section of help text.
Named-only RPC parameters have the same semantics as python keyword-only
arguments (https://peps.python.org/pep-3102/). They are always required to be
passed by name, so they don't affect interpretation of positional arguments,
and aren't affected when positional arguments are added or removed.
The new OBJ_NAMED_PARAMS type is used in the next commit to make it easier to
pass options values to various RPC methods.
Co-authored-by: Andrew Chow <github@achow101.com>
daba95700b refactor: Make ListSelected return vector (Sebastian Falbesoner)
94776621ba wallet: Move CoinCointrol definitions to .cpp (Aurèle Oulès)
1db23da6e1 wallet: Use std::optional for GetExternalOutput and fixups (Aurèle Oulès)
becc45b589 scripted-diff: Rename setSelected->m_selected_inputs (Aurèle Oulès)
Pull request description:
- Moves CoinControl function definitions from `coincontrol.h` to `coincontrol.cpp`
- Adds more documentation
- Renames class member for an improved comprehension
- Use `std::optional` for `GetExternalOutput`
ACKs for top commit:
achow101:
ACK daba95700b
Xekyo:
ACK daba95700b
Tree-SHA512: 3bf2dc834a3246c2f53f8c55154258e605fcb169431d3f7b156931f33c7e3b1ae28e03e16b37f9140a827890eb7798be485b2c36bfc23ff29bb01763f289a07c
In the wallet ckey loading test, we modify various ckey records to test
corruption handling. As the database is now a mockable database, we can
modify the records that the database will be initialized with. This
avoids having to use the verbose database reading and writing functions.
Since we have a mockable wallet database, we don't really need to be
using BDB or SQLite's in-memory database capabilities. It doesn't really
help us to be using those as we aren't doing anything that requires one
type of db over the other, and will just prefer SQLite if it's
available.
MockableDatabase is suitable for these uses, so use
CreateMockableWalletDatabase to use that.
-BEGIN VERIFY SCRIPT-
sed -i "s/CreateMockWalletDatabase(options)/CreateMockableWalletDatabase()/" $(git grep -l "CreateMockWalletDatabase(options)" -- ":(exclude)src/wallet/walletdb.*")
sed -i "s/CreateMockWalletDatabase/CreateMockableWalletDatabase/" $(git grep -l "CreateMockWalletDatabase" -- ":(exclude)src/wallet/walletdb.*")
-END VERIFY SCRIPT-
