0
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-02-19 12:02:37 -05:00
Commit graph

718 commits

Author SHA1 Message Date
Ryan Ofsky
b1ba1b178f
Merge bitcoin/bitcoin#30132: indexes: Don't wipe indexes again when continuing a prior reindex
f68cba29b3 blockman: Replace m_reindexing with m_blockfiles_indexed (Ryan Ofsky)
1b1c6dcca0 test: Add functional test for continuing a reindex (TheCharlatan)
201c1a9282 indexes: Don't wipe indexes again when already reindexing (TheCharlatan)
804f09dfa1 kernel: Add less confusing reindex options (Ryan Ofsky)
e172553223 validation: Remove needs_init from LoadBlockIndex (TheCharlatan)
533eab7d67 bugfix: Streamline setting reindex option (TheCharlatan)

Pull request description:

  When restarting `bitcoind` during an ongoing reindex without setting the `-reindex` flag again, the block and coins db is left intact, but any data from the optional indexes is discarded. While not a bug per se, wiping the data again is
  wasteful, both in terms of having to write it again,  as well as potentially leading to longer startup times. So keep the  index data instead when continuing a prior reindex.

  Also includes a bugfix and smaller code cleanups around the reindexing code. The bug was introduced in b47bd95920: "kernel: De-globalize fReindex".

ACKs for top commit:
  stickies-v:
    ACK f68cba29b3
  fjahr:
    Code review ACK f68cba29b3
  furszy:
    Code review ACK f68cba29b3
  ryanofsky:
    Code review ACK f68cba29b3. Only changes since last review were cherry-picking suggested commits that rename variables, improving comments, and making some tweaks to test code.

Tree-SHA512: b252228cc76e9f1eaac56d5bd9e4eac23408e0fc04aeffd97a85417f046229364673ee1ca7410b9b6e7b692b03f13ece17c42a10176da0d7e975a8915deb98ca
2024-06-10 10:12:30 -04:00
Ryan Ofsky
f68cba29b3
blockman: Replace m_reindexing with m_blockfiles_indexed
This is a just a mechanical change, renaming and inverting the meaning
of the indexing variable.

"m_blockfiles_indexed" is a more straightforward name for this variable
because this variable just indicates whether or not
<datadir>/blocks/blk?????.dat files have been indexed in the
<datadir>/blocks/index LevelDB database. The name "m_reindexing" was
more confusing, it could be true even if -reindex was not specified, and
false when it was specified. Also, the previous name unnecessarily
required thinking about the whole reindexing process just to understand
simple checks in validation code about whether blocks were indexed.

The motivation for this change is to follow up on previous commits,
moving away from having multiple variables called "reindex" internally,
and instead naming variables individually after what they do and
represent.
2024-06-07 19:18:46 +02:00
Ryan Ofsky
804f09dfa1
kernel: Add less confusing reindex options
Drop confusing kernel options:

  BlockManagerOpts::reindex
  ChainstateLoadOptions::reindex
  ChainstateLoadOptions::reindex_chainstate

Replacing them with more straightforward options:

  ChainstateLoadOptions::wipe_block_tree_db
  ChainstateLoadOptions::wipe_chainstate_db

Having two options called "reindex" which did slightly different things
was needlessly confusing (one option wiped the block tree database, and
the other caused block files to be rescanned). Also the previous set of
options did not allow rebuilding the block database without also
rebuilding the chainstate database, when it should be possible to do
those independently.
2024-06-07 19:17:11 +02:00
Fabian Jahr
6b6084850b
assumeutxo: Add network magic ctor param to SnapshotMetadata
This prevents SnapshotMetadata from using any globals implicitly.
2024-05-24 18:44:02 +02:00
Ava Chow
413844f1c2
Merge bitcoin/bitcoin#29612: rpc: Optimize serialization and enhance metadata of dumptxoutset output
542e13b293 rpc: Enhance metadata of the dumptxoutset output (Fabian Jahr)
4d8e5edbaa assumeutxo: Add documentation on dumptxoutset serialization format (Fabian Jahr)
c14ed7f384 assumeutxo: Add test for changed coin size value (Fabian Jahr)
de95953d87 rpc: Optimize serialization disk space of dumptxoutset (Fabian Jahr)

Pull request description:

  The second attempt at implementing the `dumptxoutset` space optimization as suggested in #25675. Closes #25675.

  This builds on the work done in #26045, addresses open feedback, adds some further improvements (most importantly usage of compact size), documentation, and an additional test.

  The [original snapshot at height 830,000](https://github.com/bitcoin/bitcoin/pull/29551) came in at 10.82 GB. With this change, the same snapshot is 8.94 GB, a reduction of 17.4%.

  This also enhances the metadata of the output file and adds the following data to allow for better error handling and make future upgrades easier:
  - A newly introduced utxo set magic
  - A version number
  - The network magic
  - The block height

ACKs for top commit:
  achow101:
    ACK 542e13b293
  TheCharlatan:
    Re-ACK 542e13b293
  theStack:
    ACK 542e13b293

Tree-SHA512: 0825d30e5c3c364062db3c6cbca4e3c680e6e6d3e259fa70c0c2b2a7020f24a47406a623582040988d5c7745b08649c31110df4c10656aa25f3f27eb35843d99
2024-05-23 12:31:23 -04:00
Fabian Jahr
542e13b293
rpc: Enhance metadata of the dumptxoutset output
The following data is added:
- A newly introduced utxo set magic
- A version number
- The network magic
- The block height
2024-05-21 13:57:09 +02:00
Ava Chow
058af75874
Merge bitcoin/bitcoin#29817: kernel: De-globalize fReindex
b47bd95920 kernel: De-globalize fReindex (TheCharlatan)

Pull request description:

  fReindex is one of the last remaining globals exposed by the kernel library, so move it into the blockstorage class to reduce the amount of global mutable state and make the kernel library a bit less awkward to use.

  ---

  This pull request is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).

ACKs for top commit:
  achow101:
    ACK b47bd95920
  ryanofsky:
    Code review ACK b47bd95920. I rereviewed the whole PR, but the only change since last review was reverting the bugfix https://github.com/bitcoin/bitcoin/pull/29817#discussion_r1578327024 and make the change a pure refactoring.
  mzumsande:
    Code Review ACK b47bd95920
  stickies-v:
    ACK b47bd95920

Tree-SHA512: f7399d01f93bc0c0c7428fe95d19b9d29b4ed00a4f1deabca78fb0c4fecb434ec971e890feecb105938b5247c926850b1b7b4a4a9caa333a061e40777d0c8463
2024-05-17 15:50:56 -04:00
Ryan Ofsky
2f53f2273d
Merge bitcoin/bitcoin#29975: blockstorage: Separate reindexing from saving new blocks
e41667b720 blockstorage: Don't move cursor backwards in UpdateBlockInfo (Ryan Ofsky)
17103637c6 blockstorage: Rename FindBlockPos and have it return a FlatFilePos (Martin Zumsande)
d9e477c4dc validation, blockstorage: Separate code paths for reindex and saving new blocks (Martin Zumsande)
064859bbad blockstorage: split up FindBlockPos function (Martin Zumsande)
fdae638e83 doc: Improve doc for functions involved in saving blocks to disk (Martin Zumsande)
0d114e3cb2 blockstorage: Add Assume for fKnown / snapshot chainstate (Martin Zumsande)

Pull request description:

  `SaveBlockToDisk` / `FindBlockPos` are used for two purposes, depending on whether they are called during reindexing (`dbp` set,  `fKnown = true`) or in the "normal" case when adding new blocks (`dbp == nullptr`,  `fKnown = false`).
  The actual tasks are quite different
  - In normal mode, preparations for saving a new block are made, which is then saved: find the correct position on disk (maybe skipping to a new blk file), check for available disk space, update the blockfile info db, save the block.
  - during reindex, most of this is not necessary (the block is already on disk after all), only the blockfile info needs to rebuilt because reindex wiped the leveldb it's saved in.

  Using one function with many conditional statements for this leads to code that is hard to read / understand and bug-prone:
  - many code paths in `FindBlockPos` are conditional on `fKnown` or `!fKnown`
  - It's not really clear what actually needs to be done during reindex (we don't need to "save a block to disk" or "find a block pos" as the function names suggest)
  - logic that should be applied to only one of the two modes is sometimes applied to both (see first commit, or #27039)

  #24858 and #27039 were recent bugs directly related to the differences between reindexing and normal mode, and in both cases the simple fix took a long time to be reviewed and merged.

  This PR proposes to clean this code up by splitting out the reindex logic into a separate function (`UpdateBlockInfo`) which will be called directly from validation. As a result, `SaveBlockToDisk` and `FindBlockPos` only need to cover the non-reindex logic.

ACKs for top commit:
  paplorinc:
    ACK e41667b720
  TheCharlatan:
    Re-ACK e41667b720
  ryanofsky:
    Code review ACK e41667b720. Just improvements to comments since last review.

Tree-SHA512: a14ff9a0facf6b1e3c1cd724a2d19a79a25d4b48de64398fdd172671532a472bc10a20cbb64ac3a3e55814dcc877d0597a3e1699cabc4f9d9a86b439b6eaba20
2024-05-16 11:16:08 -04:00
TheCharlatan
b47bd95920
kernel: De-globalize fReindex
fReindex is one of the last remaining globals exposed by the kernel
library, so move it into the blockstorage class to reduce the amount of
global mutable state and make the kernel library a bit less awkward to
use.
2024-05-16 11:28:46 +02:00
Ava Chow
f5fc3190fb
Merge bitcoin/bitcoin#29086: refactor: Simply include CTxMemPool::Options in CTxMemPool directly rather than duplicating definition
cc67d33fda refactor: Simply include CTxMemPool::Options in CTxMemPool directly rather than duplicating definition (Luke Dashjr)

Pull request description:

  Instead of duplicating mempool options two places, just include the Options struct directly on the CTxMemPool

ACKs for top commit:
  achow101:
    ACK cc67d33fda
  kristapsk:
    cr utACK cc67d33fda
  jonatack:
    ACK cc67d33fda

Tree-SHA512: 9deb5ea6f85eeb1c7e04536cded65303b0ec459936a97e4f257aff2c50b0984a4ddbf69a4651f48455b9c80200a1fd24e9c74926874fdd9be436bbbe406251ce
2024-05-14 20:00:34 -04:00
Ryan Ofsky
e41667b720 blockstorage: Don't move cursor backwards in UpdateBlockInfo
Previously, it was possible to move the cursor back to an older file
during reindex if blocks are enocuntered out of order during reindex.
This would mean that MaxBlockfileNum() would be incorrect, and
a wrong DB_LAST_BLOCK could be written to disk.

This improves the logic by only ever moving the cursor forward (if possible)
but not backwards.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2024-05-14 14:54:27 -04:00
Martin Zumsande
17103637c6 blockstorage: Rename FindBlockPos and have it return a FlatFilePos
The new name reflects that it is no longer called with existing blocks
for which the position is already known.

Returning a FlatFilePos directly simplifies the interface.
2024-05-14 14:54:27 -04:00
Martin Zumsande
d9e477c4dc validation, blockstorage: Separate code paths for reindex and saving new blocks
By calling SaveBlockToDisk only when we actually want to save a new
block to disk. In the reindex case, we now call UpdateBlockInfo
directly from validation.

This commit doesn't change behavior.
2024-05-14 14:54:27 -04:00
Martin Zumsande
064859bbad blockstorage: split up FindBlockPos function
FindBlockPos does different things depending on whether the block is known
or not, as shown by the fact that much of the existing code is conditional on fKnown set or not.

If the block position is known (during reindex) the function only updates the block info
statistics. It doesn't actually find a block position in this case.

This commit removes fKnown and splits up these two code paths by introducing a separate function
for the reindex case when the block position is known.
It doesn't change behavior.
2024-05-14 14:54:26 -04:00
Martin Zumsande
fdae638e83 doc: Improve doc for functions involved in saving blocks to disk
In particular, document the flat file positions expected and
returned by functions better.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-05-14 13:49:34 -04:00
Ryan Ofsky
dbb3113082
Merge bitcoin/bitcoin#30083: kernel: Remove batchpriority from kernel library
d4b17c7d46 kernel: Remove batchpriority from kernel library (TheCharlatan)

Pull request description:

  The current usage of ScheduleBatchPriority is not transparent. Once the thread scheduling is changed, it remains unchanged for the remainder of the thread's lifetime. So move the call from `ImportBlocks` to the init code where it is clearer that its effect lasts for the entire lifetime of the thread.

  Users of the kernel library might not expect `ImportBlocks` to have an influence on the thread it is called in. Particularly since it is only a compile time option and cannot be controlled at runtime. With this patch users of the kernel library can now freely choose their own scheduling policy.

  This PR is easier reviewed with `git diff --color-moved-ws=ignore-all-space --color-moved=dimmed-zebra`

  ---
  This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).

ACKs for top commit:
  maflcko:
    ACK d4b17c7d46 📭
  ryanofsky:
    Code review ACK d4b17c7d46, just added suggested comment since last review
  hebasto:
    ACK d4b17c7d46, I have reviewed the code and it looks OK.

Tree-SHA512: cafedecd9affad58ddd7f30f68bba71291ca951bb186ff4b2da04b7f21f0b26e5e3143846d032b9e391bd5ce6c7466b97aa3758d2a85ebd7353eb8b69139641a
2024-05-14 11:20:33 -04:00
TheCharlatan
d4b17c7d46
kernel: Remove batchpriority from kernel library
The current usage of ScheduleBatchPriority is not transparent. Once the
thread scheduling is changed, it remains unchanged for the remainder of
the thread's lifetime. So move the call from `ImportBlocks` to the init
code where it is clearer that its effect lasts for the entire lifetime
of the thread.

Users of the kernel library might not expect `ImportBlocks` to have an
influence on the thread it is called in. Particularly since it is only a
compile time option and cannot be controlled at runtime. With this patch
users of the kernel library can now choose their own scheduling policy.
2024-05-14 10:26:28 +02:00
TheCharlatan
41eba5bd71
kernel: Remove key module from kernel library
The key module's functionality is not used by the kernel library, but
currently kernel users are still required to initialize the key module's
`secp256k1_context_sign` global as part of the `kernel::Context` through
`ECC_Start`.
2024-05-09 15:56:08 +02:00
Martin Zumsande
0d114e3cb2 blockstorage: Add Assume for fKnown / snapshot chainstate
fKnown is true during reindex (and only then), which deletes
any existing snapshot chainstate. As a result, this function can never
be called wth fKnown set and a snapshot chainstate.
Add an Assume for this, and make the code initializing a blockfile cursor
for the snapshot conditional on !fKnown.

This is a preparation for splitting the reindex logic out of
FindBlockPos in the following commits.
2024-05-08 18:19:47 -04:00
Ava Chow
8efd03ad04
Merge bitcoin/bitcoin#29494: build: Assume HAVE_CONFIG_H, Add IWYU pragma keep to bitcoin-config.h includes
fa09451f8e Add lint check for bitcoin-config.h include IWYU pragma (MarcoFalke)
dddd40ba82 scripted-diff: Add IWYU pragma keep to bitcoin-config.h includes (MarcoFalke)

Pull request description:

  The `bitcoin-config.h` includes have issues:

  * The header is incompatible with iwyu, because symbols may be defined or not defined. So the `IWYU pragma: keep` is needed to keep the include when a symbol is not defined on a platform. Compare the previous discussion in https://github.com/bitcoin/bitcoin/pull/29408#issuecomment-1948959711
  * Guarding the includes by `HAVE_CONFIG_H` is verbose and brittle. Now that all build config dependencies have been removed from low level headers, the benefits are questionable, and the guard can be removed. The linter could also be tricked by guarding the include by `#if defined(HAVE_C0NFIG_H)` (`O` replaced by `0`). Compare the previous discussion in https://github.com/bitcoin/bitcoin/pull/29404#discussion_r1483189853 .

ACKs for top commit:
  achow101:
    ACK fa09451f8e
  TheCharlatan:
    ACK fa09451f8e
  hebasto:
    re-ACK fa09451f8e, only rebased since my recent [review](https://github.com/bitcoin/bitcoin/pull/29494#pullrequestreview-2028864535) (`timedata.cpp` removed in https://github.com/bitcoin/bitcoin/pull/29623).

Tree-SHA512: 47cb973f7f24bc625acc4e78683371863675d186780236d55d886cf4130e05a78bb04f1d731aae7088313b8e963a9677cc77cf518187dbd99d776f6421ca9b52
2024-05-07 14:14:03 -04:00
Luke Dashjr
cc67d33fda refactor: Simply include CTxMemPool::Options in CTxMemPool directly rather than duplicating definition 2024-05-06 20:34:10 +00:00
stickies-v
42fb5311b1
rpc: return warnings as an array instead of just a single one
The RPC documentation for `getblockchaininfo`, `getmininginfo` and
`getnetworkinfo` states that "warnings" returns "any network and
blockchain warnings". In practice, only a single warning is returned.

Fix that by returning all warnings as an array.

As a side benefit, cleans up the GetWarnings() logic.
2024-05-01 14:44:57 +01:00
MarcoFalke
dddd40ba82
scripted-diff: Add IWYU pragma keep to bitcoin-config.h includes
-BEGIN VERIFY SCRIPT-
 perl -0777 -pi -e 's/#if defined\(HAVE_CONFIG_H\)\n#include <config\/bitcoin-config.h>.*\n#endif.*\n/#include <config\/bitcoin-config.h> \/\/ IWYU pragma: keep\n/g' $( git grep -l '#include <config/bitcoin-config.h>' )
-END VERIFY SCRIPT-
2024-05-01 08:33:04 +02:00
Ava Chow
0c3a3c9394
Merge bitcoin/bitcoin#29623: Simplify network-adjusted time warning logic
c6be144c4b Remove timedata (stickies-v)
92e72b5d0d [net processing] Move IgnoresIncomingTxs to PeerManagerInfo (dergoegge)
7d9c3ec622 [net processing] Introduce PeerManagerInfo (dergoegge)
ee178dfcc1 Add TimeOffsets helper class (stickies-v)
55361a15d1 [net processing] Use std::chrono for type-safe time offsets (stickies-v)
038fd979ef [net processing] Move nTimeOffset to net_processing (dergoegge)

Pull request description:

  [An earlier approach](1d226ae1f9/) in #28956 involved simplifying and refactoring the network-adjusted time calculation logic, but this was eventually [left out](https://github.com/bitcoin/bitcoin/pull/28956#issuecomment-1904214370) of the PR to make it easier for reviewers to focus on consensus logic changes.

  Since network-adjusted time is now only used for warning/informational purposes, cleaning up the logic (building on @dergoegge's approach in #28956) should be quite straightforward and uncontroversial. The main changes are:

  - Previously, we would only calculate the time offset from the first 199 outbound peers that we connected to. This limitation is now removed, and we have a proper rolling calculation. I've reduced the set to 50 outbound peers, which seems plenty.
  - Previously, we would automatically use the network-adjusted time if the difference was < 70 mins, and warn the user if the difference was larger than that. Since there is no longer any automated time adjustment, I've changed the warning threshold to ~~20~~ 10 minutes (which is an arbitrary number).
  - Previously, a warning would only be raised once, and then never again until node restart. This behaviour is now updated to  1) warn to log for every new outbound peer for as long as we appear out of sync, 2) have the RPC warning toggled on/off whenever we go in/out of sync, and 3) have the GUI warn whenever we are out of sync (again), but limited to 1 messagebox per 60 minutes
  - no more globals
  - remove the `-maxtimeadjustment` startup arg

  Closes #4521

ACKs for top commit:
  sr-gi:
    Re-ACK [c6be144](c6be144c4b)
  achow101:
    reACK c6be144c4b
  dergoegge:
    utACK c6be144c4b

Tree-SHA512: 1063d639542e882186cdcea67d225ad1f97847f44253621a8c4b36c4d777e8f5cb0efe86bc279f01e819d33056ae4364c3300cc7400c087fb16c3f39b3e16b96
2024-04-30 18:49:34 -04:00
stickies-v
ee178dfcc1
Add TimeOffsets helper class
This helper class is an alternative to CMedianFilter, but without a
lot of the special logic and exceptions that we needed while it was
still used for consensus.
2024-04-10 17:01:27 +02:00
dergoegge
78407b99ed [clang-tidy] Enable the misc-no-recursion check
Co-authored-by: stickies-v <stickies-v@protonmail.com>
Co-authored-by: Gloria Zhao <gloriajzhao@gmail.com>
2024-04-07 14:04:45 +01:00
TheCharlatan
ddc7872c08
node: Make translations of fatal errors consistent
The extra `bilingual_str` argument of the fatal error notifications and
`node::AbortNode()` is often unused and when used usually contains the
same string as the message argument. It also seems to be confusing,
since it is not consistently used for errors requiring user action. For
example some assumeutxo fatal errors require the user to do something,
but are not translated.

So simplify the fatal error and abort node interfaces by only passing a
translated string. This slightly changes the fatal errors displayed to
the user.

Also de-duplicate the abort error log since it is repeated in noui.cpp.
2024-03-21 16:40:22 +01:00
Ava Chow
69ddee6f39
Merge bitcoin/bitcoin#27039: blockstorage: do not flush block to disk if it is already there
dfcef536d0 blockstorage: do not flush block to disk if it is already there (Matthew Zipkin)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/2039

  When reindexing from flat-file block storage there is no need to write anything back to disk, since the block data is already there. This PR skips flushing to disk those blocks that already have a known position in the datastore. Skipping this means that users can write-protect the `blk` files on disk which may be useful for security or even safely sharing that data between multiple bitcoind instances.

  `FindBlockPos()` may also flush the undo data file, but again this is skipped if the corresponding block position is known, like during the initial stage of a reindex when block data is being indexed. Once the block index is complete the validation mechanism will call `ConnectBlock()` which will save undo data at that time.

  The call stack looks like this:

  ```
  init()
  ThreadImport() <-- process fReindex flag
  LoadExternalBlockFile()
  AcceptBlock()
  SaveBlockToDisk()
  FindBlockPos()
  FlushBlockFile() <-- unnecessary if block is already on disk
  ```

  A larger refactor of this part of the code was started by mzumsande here:  https://github.com/mzumsande/bitcoin/tree/202207_refactor_findblockpos including this fix, reviewers can let me know if the changes should be combined.

ACKs for top commit:
  sipa:
    utACK dfcef536d0
  mzumsande:
    re-ACK dfcef536d0
  achow101:
    ACK dfcef536d0
  furszy:
    Rebase diff ACK dfcef53.

Tree-SHA512: 385c5ac1288b325135398d0ddd3ab788fa98cc0ca19bd2474c74039f2ce70d5088c1d1c9d4dd10aefcbd4c757767ec5805d07ba8cee9289a66f96e6f9eaa5279
2024-03-20 12:41:33 -04:00
glozow
5d045c31a5
Merge bitcoin/bitcoin#28950: RPC: Add maxfeerate and maxburnamount args to submitpackage
38f70ba6ac RPC: Add maxfeerate and maxburnamount args to submitpackage (Greg Sanders)

Pull request description:

  Resolves https://github.com/bitcoin/bitcoin/issues/28949

  I couldn't manage to do it very cleanly outside of (sub)package evaluation itself, since it would change the current interface very heavily. Instead I threaded through the max fee argument and used that directly via ATMPArgs. From that perspective, this is somewhat a reversion from https://github.com/bitcoin/bitcoin/pull/19339. In a post-cluster mempool world, these checks could be consolidated to right after the given (ancestor) package is linearized/chunked, by just checking the feerate of the top chunk and rejecting the submission entirely if the top chunk is too high.

  The implication here is that subpackages can be submitted to the mempool prior to hitting this new fee-based error condition.

ACKs for top commit:
  ismaelsadeeq:
    Re-ACK 38f70ba6ac 👍🏾
  glozow:
    ACK 38f70ba6ac with some non-blocking nits
  murchandamus:
    LGTM, code review ACK 38f70ba6ac

Tree-SHA512: 38212aa9de25730944cee58b0806a3d37097e42719af8dd7de91ce86bb5d9770b6f7c37354bf418bd8ba571c52947da1dcdbb968bf429dd1dbdf8715315af18f
2024-03-18 18:24:06 +00:00
Greg Sanders
38f70ba6ac RPC: Add maxfeerate and maxburnamount args to submitpackage
And thread the feerate value through ProcessNewPackage to
reject individual transactions that exceed the given
feerate. This allows subpackage processing, and is
compatible with future package RBF work.
2024-03-13 09:45:43 -04:00
Andrew Toth
da338aada7
blockstorage: check nPos in ReadRawBlockFromDisk before seeking back
ReadRawBlockFromDisk assumes a non-null pos that has an nPos >= 8.
This simple check makes the function safer to call in the future,
so callers don't need to worry about causing UB if the pos is null.
2024-03-12 12:46:07 -04:00
Matthew Zipkin
dfcef536d0
blockstorage: do not flush block to disk if it is already there
test: ensure we can reindex from read-only block files now
2024-03-12 10:09:53 -04:00
MarcoFalke
fad0335517
scripted-diff: Replace error() with LogError()
This fixes the log output when -logsourcelocations is used.

Also, instead of 'ERROR:', the log will now say '[error]', like other
errors logged with LogError.

-BEGIN VERIFY SCRIPT-
 sed -i --regexp-extended 's!  error\("([^"]+)"!  LogError("\1\\n"!g' $( git grep -l '  error(' ./src/ )
-END VERIFY SCRIPT-
2024-03-11 13:49:37 +01:00
MarcoFalke
fa808fb749
refactor: Make error() return type void
This is needed for the next commit to compile.
2024-03-11 13:49:35 +01:00
MarcoFalke
fa1d624348
scripted-diff: return error(...); ==> error(...); return false;
This is needed for the next commit.

-BEGIN VERIFY SCRIPT-
 # Separate sed invocations to replace one-line, and two-line error(...) calls
 sed -i             --regexp-extended 's!( +)return (error\(.*\);)!\1\2\n\1return false;!g'             $( git grep -l 'return error(' )
 sed -i --null-data --regexp-extended 's!( +)return (error\([^\n]*\n[^\n]*\);)!\1\2\n\1return false;!g' $( git grep -l 'return error(' )
-END VERIFY SCRIPT-
2024-03-11 13:49:25 +01:00
TheCharlatan
06069b3913
scripted-diff: Rename MainSignals to ValidationSignals
-BEGIN VERIFY SCRIPT-
s() { git grep -l "$1" src | xargs sed -i "s/$1/$2/g"; }

s 'CMainSignals'    'ValidationSignals'
s 'MainSignalsImpl' 'ValidationSignalsImpl'
-END VERIFY SCRIPT-
2024-02-15 14:45:51 +01:00
TheCharlatan
84f5c135b8
refactor: De-globalize g_signals 2024-02-15 14:37:01 +01:00
MarcoFalke
fad0fafd5a
refactor: Fix timedata includes 2024-02-01 13:52:05 +01:00
Ava Chow
3c13f5d612
Merge bitcoin/bitcoin#28956: Nuke adjusted time from validation (attempt 2)
ff9039f6ea Remove GetAdjustedTime (dergoegge)

Pull request description:

  This picks up parts of #25908.

  The use of adjusted time is removed from validation code while the warning to users if their clock is out of sync with the rest of the network remains.

ACKs for top commit:
  naumenkogs:
    ACK ff9039f6ea
  achow101:
    ACK ff9039f6ea
  maflcko:
    lgtm ACK ff9039f6ea 🤽
  stickies-v:
    ACK ff9039f6ea

Tree-SHA512: d1f6b9445c236915503fd2ea828f0d3b92285a5dbc677b168453276115e349972edbad37194d8becd9136d8e7219b576af64ec51c72bdb1923e57e405c0483fc
2024-01-31 15:58:47 -05:00
glozow
cad2df24b3
Merge bitcoin/bitcoin#29308: doc: update BroadcastTransaction comment
31cce4a1bd doc: update `BroadcastTransaction` comment (ismaelsadeeq)

Pull request description:

  `BroadcastTransaction` is also called by `submitpackage` RPC.

  All transactions that are accepted into the mempool post package processing are broadcasted to peers individually here
  ea4ddd8652/src/rpc/mempool.cpp (L926)

  It's not maintainable to list all the callers of a function.

ACKs for top commit:
  stickies-v:
    ACK 31cce4a1bd
  kristapsk:
    ACK 31cce4a1bd
  naumenkogs:
    ACK 31cce4a1bd

Tree-SHA512: 8aea92c53c1911a0ac36fe9e3a24d37d83e7d9b40a16f0832bfa7a719328697621e3f94a5dc80d1840e7ae705e0c3aab7a3df7064986e1e53a4a4114adf078a8
2024-01-30 12:09:52 +00:00
ismaelsadeeq
31cce4a1bd doc: update BroadcastTransaction comment
BroadcastTransaction is also called by submitpackage RPC.

It's not maintainable to list all the callers of a function.
2024-01-29 13:07:47 +01:00
Ava Chow
36720994a4
Merge bitcoin/bitcoin#20827: During IBD, prune as much as possible until we get close to where we will eventually keep blocks
d298ff8b62 During IBD, prune as much as possible until we get close to where we will eventually keep blocks (Luke Dashjr)

Pull request description:

  This should reduce pruning flushes even more, speeding up IBD with pruning on systems that have a sufficient dbcache.

  Assumes 1 MB per block between tip and best header chain. Simply adds this to the buffer pruning is trying to leave available, which results in pruning almost everything up until we get close to where we need to be keeping blocks.

ACKs for top commit:
  andrewtoth:
    ACK d298ff8b62
  fjahr:
    utACK d298ff8b62
  achow101:
    ACK d298ff8b62

Tree-SHA512: 2a482376bfb177e2ba7c2f0bb0b58b02efdb38b34755a18d1fc3e869df5959c85b6f1009e1386fa8b89c4f90d520383e36bd3e21dec221042315134efb1a455b
2024-01-25 15:20:17 -05:00
dergoegge
ff9039f6ea Remove GetAdjustedTime 2024-01-05 17:16:38 +00:00
fanquake
143ace65db
Merge bitcoin/bitcoin#28890: rpc: Remove deprecated -rpcserialversion
fa46cc22bc Remove deprecated -rpcserialversion (MarcoFalke)

Pull request description:

  The flag is problematic for many reasons:

  * It is deprecated
  * It is a global flag, requiring a restart to change, as opposed to a flag that can be set on each RPC invocation
  * It may be hidden in config files by accident, hard to debug, causing LND crashes and bugs, see https://github.com/bitcoin/bitcoin/issues/28730#issuecomment-1780940868
  * It makes performance improvements harder to implement: https://github.com/bitcoin/bitcoin/pull/17529#issuecomment-556082818

  Fix all issues by removing it.

  If there is a use-case, likely a per-RPC flag can be added, if needed.

ACKs for top commit:
  ajtowns:
    crACK fa46cc22bc
  TheCharlatan:
    lgtm ACK fa46cc22bc

Tree-SHA512: 96ba1c60356ce93954fe5c2a59045771c6d1516ad0d9dc436ef1800a1f1b0153f0d5fb78ca99d53ad54ba25fbce36962bdf1d4325aceedfc8154a61347a6a915
2024-01-05 10:42:10 +00:00
Luke Dashjr
d298ff8b62 During IBD, prune as much as possible until we get close to where we will eventually keep blocks 2023-12-27 02:57:30 +00:00
ismaelsadeeq
8dec9c560b wallet, mempool: propagete checkChainLimits error message to wallet
Update CheckPackageLimits to use util::Result to pass the error message
instead of out parameter.

Also update test to reflect the error message from `CTxMempool`
`CheckPackageLimits` output.
2023-12-17 21:13:44 +01:00
Ava Chow
4ad5c71adb
Merge bitcoin/bitcoin#28051: Get rid of shutdown.cpp/shutdown.h, use SignalInterrupt directly
6db04be102 Get rid of shutdown.cpp/shutdown.h, use SignalInterrupt directly (Ryan Ofsky)
213542b625 refactor: Add InitContext function to initialize NodeContext with global pointers (Ryan Ofsky)
feeb7b816a refactor: Remove calls to StartShutdown from KernelNotifications (Ryan Ofsky)
6824eecaf1 refactor: Remove call to StartShutdown from stop RPC (Ryan Ofsky)
1d92d89edb util: Get rid of uncaught exceptions thrown by SignalInterrupt class (Ryan Ofsky)
ba93966368 refactor: Remove call to ShutdownRequested from IndexWaitSynced (Ryan Ofsky)
42e5829d97 refactor: Remove call to ShutdownRequested from HTTPRequest (Ryan Ofsky)
73133c36aa refactor: Add NodeContext::shutdown member (Ryan Ofsky)
f4a8bd6e2f refactor: Remove call to StartShutdown from qt (Ryan Ofsky)
f0c73c1336 refactor: Remove call to ShutdownRequested from rpc/mining (Ryan Ofsky)
263b23f008 refactor: Remove call to ShutdownRequested from chainstate init (Ryan Ofsky)

Pull request description:

  This change drops `shutdown.h` and `shutdown.cpp` files, replacing them with a `NodeContext::shutdown` member which is used to trigger shutdowns directly. This gets rid of an unnecessary layer of indirection, and allows getting rid of the `kernel::g_context` global.

  Additionally, this PR tries to improve error handling of `SignalInterrupt` code by marking relevant methods `[[nodiscard]]` to avoid the possibility of uncaught exceptions mentioned https://github.com/bitcoin/bitcoin/pull/27861#discussion_r1255496707.

  Behavior is changing In a few cases which are noted in individual commit messages. Particularly: GUI code more consistently interrupts RPCs when it is shutting down, shutdown state no longer persists between unit tests, the stop RPC now returns an RPC error if requesting shutdown fails instead of aborting, and other failed shutdown calls now log errors instead of aborting.

  This PR is a net reduction in lines of code, but in some cases the explicit error handling and lack of global shutdown functions do make it more verbose. The verbosity can be seen as good thing if it discourages more code from directly triggering shutdowns, and instead encourages code to return errors or send notifications that could be translated into shutdowns. Probably a number of existing shutdown calls could just be replaced by better error handling.

ACKs for top commit:
  achow101:
    ACK 6db04be102
  TheCharlatan:
    Re-ACK 6db04be102
  maflcko:
    ACK 6db04be102 👗
  stickies-v:
    re-ACK 6db04be102

Tree-SHA512: 7a34cb69085f37e813c43bdaded1a0cbf6c53bd95fdde96f0cb45346127fc934604c43bccd3328231ca2f1faf712a7418d047ceabd22ef2dca3c32ebb659e634
2023-12-14 15:14:00 -05:00
MarcoFalke
fa46cc22bc
Remove deprecated -rpcserialversion 2023-12-11 18:22:13 +01:00
MarcoFalke
fa604eb6cf
refactor: Use reference instead of pointer in IsBlockPruned
This makes it harder to pass nullptr and cause issues such as
dde7ac5c70
2023-12-07 12:02:54 +01:00
Ryan Ofsky
6db04be102 Get rid of shutdown.cpp/shutdown.h, use SignalInterrupt directly
This change is mostly a refectoring that removes some code and gets rid of an
unnecessary layer of indirection after #27861

But it is not a pure refactoring since StartShutdown, AbortShutdown, and
WaitForShutdown functions used to abort on failure, and the replacement code
logs or returns errors instead.
2023-12-04 15:39:15 -04:00