0
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-02-09 10:43:19 -05:00
Commit graph

24530 commits

Author SHA1 Message Date
glozow
3ea71feb11 [validation] don't LimitMempoolSize in any subpackage submissions
Don't do any mempool evictions until package validation is done,
preventing the mempool minimum feerate from changing. Whether we submit
transactions separately or as a package depends on whether they meet the
mempool minimum feerate threshold, so it's best that the value not
change while we are evaluating a package.
This avoids a situation where we have a CPFP package in which
the parents meet the mempool minimum feerate and are submitted by
themselves, but they are evicted before we have submitted the child.
2023-09-13 16:14:18 +01:00
glozow
d227b7234c [validation] return correct result when already-in-mempool tx gets evicted
Bug fix: a transaction may be in the mempool when package evaluation
begins (so it is added to results_final with MEMPOOL_ENTRY or
DIFFERENT_WITNESS), but get evicted due to another transaction
submission.
2023-09-13 16:14:17 +01:00
glozow
9698b81828 [refactor] back-fill results in AcceptPackage
Instead of populating the last PackageMempoolAcceptResult with stuff
from results_final and individual_results_nonfinal, fill results_final
and create a PackageMempoolAcceptResult using that one.

A future commit will add LimitMempoolSize() which may change the status
of each of these transactions from "already in mempool" or "submitted to
mempool" to "no longer in mempool". We will change those transactions'
results here.

A future commit also gets rid of the last AcceptSubPackage outside of
the loop. It makes more sense to use results_final as the place where
all results end up.
2023-09-13 16:14:17 +01:00
glozow
8ad7ad3392 [validation] make PackageMempoolAcceptResult members mutable
After the PackageMempoolAcceptResult is returned from
AcceptMultipleTransactions, leave room for results to change due to
LimitMempool() eviction.
2023-09-13 16:14:17 +01:00
glozow
03b87c11ca [validation] add AcceptSubPackage to delegate Accept* calls and clean up m_view
(1) Call AcceptSingleTransaction when there is only 1 transaction in the
  subpackage. This avoids calling PackageMempoolChecks() which enforces
rules that don't need to be applied for a single transaction, i.e.
disabling CPFP carve out.
There is a slight change in the error type returned, as shown in the
txpackage_tests change.  When a transaction is the last one left in the
package and its fee is too low, this returns a PCKG_TX instead of
PCKG_POLICY. This interface is clearer; "package-fee-too-low" for 1
transaction would be a bit misleading.

(2) Clean up m_view and m_viewmempool so that coins created in this
sub-package evaluation are not available for other sub-package
evaluations. The contents of the mempool may change, so coins that are
available now might not be later.
2023-09-13 16:14:17 +01:00
glozow
3f01a3dab1 [CCoinsViewMemPool] track non-base coins and allow Reset
Temporary coins should not be available in separate subpackage submissions.
Any mempool coins that are cached in m_view should be removed whenever
mempool contents change, as they may be spent or no longer exist.
2023-09-13 16:14:17 +01:00
glozow
7d7f7a1189 [policy] check for duplicate txids in package
Duplicates of normal transactions would be found by looking for
conflicting inputs, but this doesn't catch identical empty transactions.
These wouldn't be valid but exiting early is good and AcceptPackage's
result sanity checks assume non-duplicate transactions.
2023-09-13 16:14:17 +01:00
fanquake
b2ec0326fd
Merge bitcoin/bitcoin#28008: BIP324 ciphersuite
1c7582ead6 tests: add decryption test to bip324_tests (Pieter Wuille)
990f0f8da9 Add BIP324Cipher, encapsulating key agreement, derivation, and stream/AEAD ciphers (Pieter Wuille)
c91cedf281 crypto: support split plaintext in ChaCha20Poly1305 Encrypt/Decrypt (Pieter Wuille)
af2b44c76e bench: add benchmark for FSChaCha20Poly1305 (Pieter Wuille)
aa8cee9334 crypto: add FSChaCha20Poly1305, rekeying wrapper around ChaCha20Poly1305 (Pieter Wuille)
0fee267792 crypto: add FSChaCha20, a rekeying wrapper around ChaCha20 (Pieter Wuille)
9ff0768bdc crypto: add the ChaCha20Poly1305 AEAD as specified in RFC8439 (Pieter Wuille)
9fd085a1a4 crypto: remove outdated variant of ChaCha20Poly1305 AEAD (Pieter Wuille)

Pull request description:

  Depends on #27985 and #27993, based on and partially replaces #25361, part of #27634. Draft while dependencies are not merged.

  This adds implementations of:
  * The ChaCha20Poly1305 AEAD from [RFC8439 section 2.8](https://datatracker.ietf.org/doc/html/rfc8439#section-2.8), including test vectors.
  * The FSChaCha20 stream cipher as specified in [BIP324](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#rekeying-wrappers-fschacha20poly1305-and-fschacha20), a rekeying wrapper around ChaCha20.
  * The FSChaCha20Poly1305 AEAD as specified in [BIP324](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#rekeying-wrappers-fschacha20poly1305-and-fschacha20), a rekeying wrapper around ChaCha20Poly1305.
  * A BIP324Cipher class that encapsulates key agreement, key derivation, and stream ciphers and AEADs for [BIP324 packet encoding](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#overall-packet-encryption-and-decryption-pseudocode).

  The ChaCha20Poly1305 and FSChaCha20Poly1305 implementations are new, taking advance of the improvements in #27993.

ACKs for top commit:
  jamesob:
    reACK 1c7582e
  theStack:
    ACK 1c7582ead6
  stratospher:
    tested ACK 1c7582e.

Tree-SHA512: 06728b4b95b21c5b732ed08faf40e94d0583f9d86ff4db3b92dd519dcd9fbfa0f310bc66ef1e59c9e49dd844ba8c5ac06e2001762a804fb5aa97027816045a46
2023-08-10 11:58:59 +02:00
glozow
0d9a13ddd8
Merge bitcoin/bitcoin#28149: net processing: clamp PeerManager::Options user input
547fa52443 net processing: clamp -blockreconstructionextratxn to uint32_t bounds (stickies-v)
e451d1e3c6 net processing: clamp -maxorphantx to uint32_t bounds (stickies-v)
aa89e04e07 doc: document PeerManager::Options members (stickies-v)

Pull request description:

  Avoid out-of-bounds user input for `PeerManager::Options` by safely clamping `-maxorphantx` and `-blockreconstructionextratxn`, and avoid platform-specific behaviour by changing `PeerManager::Options::max_extra_txs` from `size_t` to a `uint32_t`. Addresses https://github.com/bitcoin/bitcoin/pull/27499#pullrequestreview-1544114932.

  Also documents all `PeerManager::Options` members, addressing https://github.com/bitcoin/bitcoin/pull/27499#discussion_r1272302469.

ACKs for top commit:
  dergoegge:
    Code review ACK 547fa52443
  glozow:
    reACK 547fa52443

Tree-SHA512: 042d47b35bb8a7b29ef3dadd4c0c5d26f13a8f174f33687855d603c19f8de0fcbbda94418453331e149885412d4edd5f402d640d938f6d94b4dcf54e2fdbbcc9
2023-08-09 14:26:03 +02:00
fanquake
b565485c24
Merge bitcoin/bitcoin#28186: kernel: Prune leveldb headers
d8f1222ac5 refactor: Correct dbwrapper key naming (TheCharlatan)
be8f159ac5 build: Remove leveldb from BITCOIN_INCLUDES (TheCharlatan)
c95b37d641 refactor: Move CDBWrapper leveldb members to their own context struct (TheCharlatan)
c534a615e9 refactor: Split dbwrapper CDBWrapper::EstimateSize implementation (TheCharlatan)
586448888b refactor: Move HandleError to dbwrapper implementation (TheCharlatan)
dede0eef7a refactor: Split dbwrapper CDBWrapper::Exists implementation (TheCharlatan)
a5c2eb5748 refactor: Fix logging.h includes (TheCharlatan)
84058e0eed refactor: Split dbwrapper CDBWrapper::Read implementation (TheCharlatan)
e4af2408f2 refactor: Pimpl leveldb::Iterator for CDBIterator (TheCharlatan)
ef941ff128 refactor: Split dbwrapper CDBIterator::GetValue implementation (TheCharlatan)
b7a1ab5cb4 refactor: Split dbwrapper CDBIterator::GetKey implementation (TheCharlatan)
d7437908cd refactor: Split dbwrapper CDBIterator::Seek implementation (TheCharlatan)
ea8135de7e refactor: Pimpl leveldb::batch for CDBBatch (TheCharlatan)
b9870c920d refactor: Split dbwrapper CDBatch::Erase implementation (TheCharlatan)
532ee812a4 refactor: Split dbwrapper CDBBatch::Write implementation (TheCharlatan)
afc534df9a refactor: Wrap DestroyDB in dbwrapper helper (TheCharlatan)

Pull request description:

  Leveldb headers are currently included in the `dbwrapper.h` file and thus available to many of Bitcoin Core's source files. However, leveldb-specific functionality should be abstracted by the `dbwrapper` and does not need to be available to the rest of the code. Having leveldb included in a widely-used header such as `dbwrapper.h` bloats the entire project's header tree.

  The `dbwrapper` is a key component of the libbitcoinkernel library. Future users of this library would not want to contend with  having the leveldb headers exposed and potentially polluting their project's namespace.

  For these reasons, the leveldb headers are removed from the `dbwrapper` by moving leveldb-specific code to the implementation file and creating a [pimpl](https://en.cppreference.com/w/cpp/language/pimpl) where leveldb member variables are indispensable. As a final step, the leveldb include flags are removed from the `BITCOIN_INCLUDES` and moved to places where the dbwrapper is compiled.

  ---

  This pull request is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587), and more specifically its stage 1 step 3 "Decouple most non-consensus headers from libbitcoinkernel".

ACKs for top commit:
  stickies-v:
    re-ACK d8f1222ac5
  MarcoFalke:
    ACK d8f1222ac5  🔠

Tree-SHA512: 0f58309be165af0162e648233451cd80fda88726fc10c0da7bfe4ec2ffa9afe63fbf7ffae9493698d3f39653b4ad870c372eee652ecc90ab1c29d86c387070f3
2023-08-07 22:31:46 +02:00
fanquake
624333455a
Merge bitcoin/bitcoin#26296: ci: Integrate bitcoin-tidy clang-tidy plugin
1c976c691c tidy: Integrate bicoin-tidy clang-tidy plugin (fanquake)
7de23cceb8 refactor: fix unterminated LogPrintf()s (fanquake)
0a1029aa29 lint: remove  /* Continued */ markers from codebase (fanquake)
910007995d lint: remove lint-logs.py (fanquake)
d86a83d6b8 lint: drop DIR_IWYU global (fanquake)

Pull request description:

  Demo of integrating the [bitcoin-tidy](https://github.com/theuni/bitcoin-tidy-plugin), [clang-tidy plugin](https://clang.llvm.org/extra/clang-tidy/) written by theuni into our tidy CI job.

  The plugin currently has a single check, `bitcoin-unterminated-logprintf`. This would replace our current Python driven, `git-grep`-based, `.cpp` file only, lint-logs linter.

ACKs for top commit:
  TheCharlatan:
    ACK 1c976c691c
  theuni:
    ACK 1c976c691c
  MarcoFalke:
    re-ACK 1c976c691c  👠

Tree-SHA512: 725b45c70e431d48e6f276671e05c694e10b6047cae1a31906ac3ee9093bc8105fb226b36a5bac6709557526ca6007222112d66aecec05a574434edc4897e4b8
2023-08-07 17:14:07 +02:00
fanquake
97ba72117c
Merge bitcoin/bitcoin#27401: tracepoints: Disables -Wgnu-zero-variadic-macro-arguments to compile without warnings
5197660e94 tracepoints: Disables `-Wgnu-zero-variadic-macro-arguments` to compile without warnings (Martin Leitner-Ankerl)

Pull request description:

  Fixes #26916 by disabling the warning `-Wgnu-zero-variadic-macro-arguments` when clang is used as the compiler.

  Also see the comments
  * Proposed changes in the bug  https://github.com/bitcoin/bitcoin/issues/26916#issuecomment-1480997053
  * Proposed changes when moving to a variadic maro: https://github.com/bitcoin/bitcoin/pull/26593#discussion_r1155488768

ACKs for top commit:
  hebasto:
    ACK 5197660e94, I've reconsidered my [comment](https://github.com/bitcoin/bitcoin/pull/27401#issuecomment-1507142439) and I think the current localized approach is optimal.
  fanquake:
    ACK 5197660e94 - checked that this fixes the warnings under Clang.

Tree-SHA512: c3dda3bcbb2540af6283ffff65885a9937bfdaaef3b00dc7d60b9f9740031d5c36ac9cb3d3d8756dbadce4812201a9754f5b8770df0d5e0d5ee690ba8a7135d2
2023-08-07 16:03:55 +02:00
fanquake
be44332803
Merge bitcoin/bitcoin#28191: refactor: Remove unused MessageStartChars parameters from BlockManager methods
fa69e3a95c Remove unused MessageStartChars parameters from BlockManager methods (MarcoFalke)

Pull request description:

  Seems odd to expose these for mocking, when it is not needed.

  Fix this by removing the the unused parameters and use the already existing member field instead.

ACKs for top commit:
  Empact:
    utACK fa69e3a95c
  dergoegge:
    utACK fa69e3a95c

Tree-SHA512: 7814e9560abba8d9c0926bcffc70f92e502d22f543af43671248f6fcd1433f35238553c0f05123fde6d8e0f80261af0ab0500927548115153bd68d57fe2da746
2023-08-07 10:57:39 +02:00
fanquake
b7138252ac
Merge bitcoin/bitcoin#27213: p2p: Diversify automatic outbound connections with respect to networks
1b52d16d07 p2p: network-specific management of outbound connections (Martin Zumsande)
65cff00cee test: Add test for outbound protection by network (Martin Zumsande)
034f61f83b p2p: Protect extra full outbound peers by network (Martin Zumsande)
654d9bc276 p2p: Introduce data struct to track connection counts by network (Amiti Uttarwar)

Pull request description:

  This is joint work with mzumsande.

  This is a proposal to diversify outbound connections with respect to reachable networks. The existing logic evaluates peers for connection based purely on the frequency of available addresses in `AddrMan`. This PR adds logic to automatically connect to alternate reachable networks and adds eviction logic that protects one existing connection to each network.

  For instance, if `AddrMan` is populated primarily with IPv4 and IPv6 addresses and only a handful of onion addresses, it is likely that we won't establish any automatic outbound connections to Tor, even if we're capable of doing so. For smaller networks like CJDNS, this is even more of an issue and often requires adding manual peers to ensure regularly being connected to the network.

  Connecting to multiple networks improves resistance to eclipse attacks for individual nodes. It also benefits the entire p2p network by increasing partition resistance and privacy in general.

  The automatic connections to alternate networks is done defensively, by first filling all outbound slots with random addresses (as in the status quo) and then adding additional peers from reachable networks the node is currently not connected to. This approach ensures that outbound slots are not left unfilled while attempting to connect to a network that may be unavailable due to a technical issue or misconfiguration that bitcoind cannot detect.

  Once an additional peer is added and we have one more outbound connection than we want, outbound eviction ensures that peers are protected if they are the only ones for their network.

  Manual connections are also taken into account: If a user already establishes manual connections to a trusted peer from a network, there is no longer a need to make extra efforts to ensure we also have an automatic connection to it (although this may of course happen by random selection).

ACKs for top commit:
  naumenkogs:
    ACK 1b52d16d07
  vasild:
    ACK 1b52d16d07

Tree-SHA512: 5616c038a5fbb868d4c46c5963cfd53e4599feee25db04b0e18da426d77d22e0994dc4e1da0b810f5b457f424ebbed3db1704f371aa6cad002b3565b20170ec0
2023-08-06 18:44:42 +02:00
TheCharlatan
d8f1222ac5
refactor: Correct dbwrapper key naming
The ss- prefix should connotate a DataStream variable. Now that these
variables are byte spans, drop the prefix.
2023-08-05 10:45:19 +02:00
TheCharlatan
be8f159ac5
build: Remove leveldb from BITCOIN_INCLUDES
Since leveldb is no longer in our header tree, move its include flags to
whereever dbwrapper.cpp is built.
2023-08-05 10:45:17 +02:00
TheCharlatan
c95b37d641
refactor: Move CDBWrapper leveldb members to their own context struct
The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:45:12 +02:00
TheCharlatan
c534a615e9
refactor: Split dbwrapper CDBWrapper::EstimateSize implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

Since CharCast is no longer needed in the header, move it to the
implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:43:01 +02:00
TheCharlatan
586448888b
refactor: Move HandleError to dbwrapper implementation
Make it a static function in dbwrapper.cpp, since it is not used
elsewhere and when left in the header, would expose a leveldb type.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:59 +02:00
TheCharlatan
dede0eef7a
refactor: Split dbwrapper CDBWrapper::Exists implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:58 +02:00
TheCharlatan
a5c2eb5748
refactor: Fix logging.h includes
These were uncovered as missing by the next commit.
2023-08-05 10:42:56 +02:00
TheCharlatan
84058e0eed
refactor: Split dbwrapper CDBWrapper::Read implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:55 +02:00
TheCharlatan
e4af2408f2
refactor: Pimpl leveldb::Iterator for CDBIterator
Hide the leveldb::Iterator member variable with a pimpl in order not to
expose it directly in the header.

Also, move CDBWrapper::NewIterator to the dbwrapper implementation to
use the pimpl for CDBIterator initialziation.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:53 +02:00
TheCharlatan
ef941ff128
refactor: Split dbwrapper CDBIterator::GetValue implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:51 +02:00
TheCharlatan
b7a1ab5cb4
refactor: Split dbwrapper CDBIterator::GetKey implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:48 +02:00
TheCharlatan
d7437908cd
refactor: Split dbwrapper CDBIterator::Seek implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:45 +02:00
TheCharlatan
ea8135de7e
refactor: Pimpl leveldb::batch for CDBBatch
Hide the leveldb::WriteBatch member variable with a pimpl in order not
to expose it directly in the header.

Also move CDBBatch::Clear to the dbwrapper implementation to use the new
impl_batch.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:38 +02:00
TheCharlatan
b9870c920d
refactor: Split dbwrapper CDBatch::Erase implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:27:53 +02:00
TheCharlatan
532ee812a4
refactor: Split dbwrapper CDBBatch::Write implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:27:47 +02:00
fanquake
f138422d37
Merge bitcoin/bitcoin#28203: refactor: serialization simplifications
f054bd072a refactor: use "if constexpr" in std::vector's Unserialize() (Martin Leitner-Ankerl)
088caa68fb refactor: use "if constexpr" in std::vector's Serialize() (Martin Leitner-Ankerl)
0fafaca4d3 refactor: use "if constexpr" in prevector's Unserialize() (Martin Leitner-Ankerl)
c8839ec5cd refactor: use "if constexpr" in prevector's Serialize() (Martin Leitner-Ankerl)
1403d181c1 refactor: use fold expressions instead of recursive calls in UnserializeMany() (Martin Leitner-Ankerl)
bd08a008b4 refactor: use fold expressions instead of recursive calls in SerializeMany() (Martin Leitner-Ankerl)

Pull request description:

  This simplifies the serialization code a bit and should also make it a bit faster.

  * use fold expressions instead of recursive calls. This simplifies the code, makes it most likely faster because it reduces the number of function calls, and compiles faster because there are fewer template instantiations.

  * use `if constexpr` instead of unnecessarily creating a temporary object only to call the right overload. This is used for `std::vector` and `prevector` serialization.

ACKs for top commit:
  MarcoFalke:
    only change is to add a missing `&`. lgtm, re-ACK f054bd072a 📦
  jonatack:
    ACK f054bd072a
  sipa:
    utACK f054bd072a
  john-moffett:
    ACK f054bd072a

Tree-SHA512: 0417bf2d6be486c581732297945449211fc3481bac82964e27628b38ef55a47dfa58d730148aeaf1b19fa8eb1076489cc646ceebb178162a9afa59034601501d
2023-08-04 14:50:49 +02:00
Martin Zumsande
1b52d16d07 p2p: network-specific management of outbound connections
Diversify outbound connections with respect to
networks: Every ~5 minutes, try to add an extra connection
to a reachable network which we currently don't have a connection to.
This is done defensively - only try management with respect to networks
after all existing outbound slots are filled.
The resulting situation with an extra outbound peer will be handled
by the extra outbound eviction logic, which protects peers from
eviction if they are the only ones for their network.

Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2023-08-03 19:27:23 -06:00
Martin Zumsande
65cff00cee test: Add test for outbound protection by network
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2023-08-03 19:27:23 -06:00
Martin Zumsande
034f61f83b p2p: Protect extra full outbound peers by network
If a peer is the only one of its network, protect it from eviction.
This improves the diversity of outbound connections with respect to
reachable networks.

Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2023-08-03 19:27:23 -06:00
Amiti Uttarwar
654d9bc276 p2p: Introduce data struct to track connection counts by network
Connman uses this new map to keep a count of active OUTBOUND_FULL_RELAY and
MANUAL connections. Unused until next commit.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-08-03 12:46:24 -06:00
fanquake
1c976c691c
tidy: Integrate bicoin-tidy clang-tidy plugin
Enable `bitcoin-unterminated-logprintf`.

Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
2023-08-03 17:52:24 +01:00
fanquake
7de23cceb8
refactor: fix unterminated LogPrintf()s 2023-08-03 17:52:24 +01:00
fanquake
0a1029aa29
lint: remove /* Continued */ markers from codebase 2023-08-03 17:52:24 +01:00
fanquake
da3816e4e8
Merge bitcoin/bitcoin#27832: doc: Clarify -datacarriersize, add -datacarriersize=2 tests
faafc35a77 doc: Clarify that -datacarriersize applies to the full raw scriptPubKey, not the data push (MarcoFalke)
55550e7fe7 test: Add -datacarriersize=2 tests (MarcoFalke)

Pull request description:

  Clarify with a test that `-datacarriersize` applies to the serialized size of the scriptPubKey, not the size of the pushed data. So for example,

  * `-datacarriersize=2` will reject a `raw(6a01aa)`, even though only one byte is pushed
  * `-datacarriersize=0` (or `-datacarrier=0`) will reject a `raw(6a)`, even though no byte is pushed
  * `-datacarriersize=0` (or `-datacarrier=0`) will reject a `raw(6a00)`, even though zero bytes are pushed

ACKs for top commit:
  ajtowns:
    ACK faafc35a77
  instagibbs:
    ACK faafc35a77

Tree-SHA512: f01ace02798f596ac2a02461e9f2a6ef91b3b37c976ea0b3bc860e2d3efb0ace0fd8b779dd18249cee7f84ebbe5fd21d8506afd3a15edadc00b843ff3b4aacc7
2023-08-03 17:46:43 +01:00
fanquake
61849f0464
Merge bitcoin/bitcoin#27918: fuzz: addrman, avoid ConsumeDeserializable when possible
025fda0a76 fuzz: addrman, avoid `ConsumeDeserializable` when possible (brunoerg)

Pull request description:

  Using specific functions like `ConsumeService`, `ConsumeAddress` and `ConsumeNetAddr` may be more effective than using `ConsumeDeserializable`. They always return some value while `ConsumeDeserializable` may return `std::nullopt`.

  E.g.: In this part of the code, if `op_net_addr` is `std::nullopt`,  we basically generated the addresses (if so) unnecessarily, because we won't be able to use them:
  ```cpp
  std::vector<CAddress> addresses;
  LIMITED_WHILE(fuzzed_data_provider.ConsumeBool(), 10000) {
      const std::optional<CAddress> opt_address = ConsumeDeserializable<CAddress>(fuzzed_data_provider);
      if (!opt_address) {
          break;
      }
      addresses.push_back(*opt_address);
  }
  const std::optional<CNetAddr> opt_net_addr = ConsumeDeserializable<CNetAddr>(fuzzed_data_provider);
  if (opt_net_addr) {
      addr_man.Add(addresses, *opt_net_addr, std::chrono::seconds{ConsumeTime(fuzzed_data_provider, 0, 100000000)});
  }
  ```

  Also, if we are not calling `Add` effectively, it would also be affect other functions that may "depend" on it.

ACKs for top commit:
  dergoegge:
    Code review ACK 025fda0a76

Tree-SHA512: 02450bec0b084c15ba0cd1cbdfbac067c8fea4ccf27be0c86d54e020f029a6c749a16d8e0558f9d6d35a7ca9db8916f180c872f09474702b5591129e9be0d192
2023-08-03 17:32:46 +01:00
glozow
7c66a4b610
Merge bitcoin/bitcoin#28059: refactor: Make more transaction size variables signed
92de74ef18 refactor: Make more transaction size variables signed (Hennadii Stepanov)

Pull request description:

  This PR is a continuation of https://github.com/bitcoin/bitcoin/pull/23962 and it:
  - gets rid of two static casts,
  - addresses https://github.com/bitcoin/bitcoin/pull/23962#issuecomment-1593289706,
  - is useful for https://github.com/bitcoin/bitcoin/pull/25972, see the failed ARM and multiprocess CI jobs.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 92de74ef18  🥔
  glozow:
    ACK 92de74ef18

Tree-SHA512: 84225961af8e08439664e75661b98fe86560217e891e5633a28316bf248d88df317a0c6b5a5f6b03feb2b0e0fd40a1f91dd4a85a0610d567470805bf47a84487
2023-08-03 11:45:51 +01:00
Martin Leitner-Ankerl
f054bd072a refactor: use "if constexpr" in std::vector's Unserialize()
This gets rid of unnecessarily creating a temporary object T() to call
the right function.
2023-08-03 10:35:22 +02:00
Martin Leitner-Ankerl
088caa68fb refactor: use "if constexpr" in std::vector's Serialize()
This gets rid of unnecessarily creating a temporary object T() to call
the right function.
2023-08-03 10:34:42 +02:00
Martin Leitner-Ankerl
0fafaca4d3 refactor: use "if constexpr" in prevector's Unserialize()
This gets rid of unnecessarily creating a temporary object T() to call
the right function.
2023-08-03 10:34:09 +02:00
Martin Leitner-Ankerl
c8839ec5cd refactor: use "if constexpr" in prevector's Serialize()
This gets rid of unnecessarily creating a temporary object T() to call
the right function.
2023-08-03 10:32:25 +02:00
Martin Leitner-Ankerl
1403d181c1 refactor: use fold expressions instead of recursive calls in UnserializeMany()
Instead of recursively calling `UnserializeMany` and peeling off one
argument at a time, use a fold expression. This simplifies the code,
makes it most likely faster because it reduces the number of function
calls, and compiles faster because there are fewer template
instantiations.
2023-08-03 10:31:33 +02:00
Martin Leitner-Ankerl
bd08a008b4 refactor: use fold expressions instead of recursive calls in SerializeMany()
Instead of recursively calling `SerializeMany` and peeling off one
argument at a time, use a fold expression. This simplifies the code,
makes it most likely faster because it reduces the number of function
calls, and compiles faster because there are fewer template
instantiations.
2023-08-03 10:30:02 +02:00
TheCharlatan
afc534df9a
refactor: Wrap DestroyDB in dbwrapper helper
Wrap leveldb::DestroyDB in a helper function without exposing
leveldb-specifics.

Also, add missing optional include.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-01 22:14:15 +02:00
fanquake
ceda819886
Merge bitcoin/bitcoin#28060: util: Teach AutoFile how to XOR
fa633aa690 streams: Teach AutoFile how to XOR (MarcoFalke)
000019e158 Add AutoFile::detail_fread member function (MarcoFalke)
fa7724bc9d refactor: Modernize AutoFile (MarcoFalke)
fa8d227d58 doc: Remove comments that just repeat what the code does (MarcoFalke)
fafe2ca0ce refactor: Remove redundant file check from AutoFile shift operators (MarcoFalke)
9999a49b32 Extract util::Xor, Add key_offset option, Add bench (MarcoFalke)

Pull request description:

  This allows `AutoFile` to roll an XOR pattern while reading or writing to the underlying file.

  This is needed for https://github.com/bitcoin/bitcoin/pull/28052, but can also be used in any other place.

  Also, there are tests, so I've split this up from the larger pull to make review easier, hopefully.

ACKs for top commit:
  Crypt-iQ:
    crACK fa633aa
  willcl-ark:
    Lightly tested ACK fa633aa690
  jamesob:
    reACK fa633aa690 ([`jamesob/ackr/28060.4.MarcoFalke.util_add_xorfile`](https://github.com/jamesob/bitcoin/tree/ackr/28060.4.MarcoFalke.util_add_xorfile))

Tree-SHA512: 6d66cad0a089a096d3f95e4f2b28bce80b349d4b76f53d09dc9a0bea4fc1b7c0652724469c37971ba27728c7d46398a4c1d289c252af4c0f83bb2fcbc6f8e90b
2023-08-01 16:38:06 +01:00
fanquake
e5a9f2fb62
Merge bitcoin/bitcoin#28194: test: python E721 and flake8 updates
bee2d57a65 script: update flake8 to 6.1.0 (Jon Atack)
38c3fd846b test: python E721 updates (Jon Atack)

Pull request description:

  Update our functional tests per [E721](https://www.flake8rules.com/rules/E721.html) enforced by [flake8 6.1.0](https://flake8.pycqa.org/en/latest/release-notes/6.1.0.html), and update our CI lint task to use that release.  This makes the following linter output on current master with flake8 6.1.0 green.

  ```
  $ ./test/lint/lint-python.py ; ./test/lint/lint-spelling.py
  test/functional/p2p_invalid_locator.py:35:16: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
  test/functional/test_framework/siphash.py:34:12: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
  test/functional/test_framework/siphash.py:64:12: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
  src/test/fuzz/descriptor_parse.cpp:88: occurences ==> occurrences
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/spelling.ignore-words.txt
  ```

ACKs for top commit:
  MarcoFalke:
    lgtm ACK bee2d57a65

Tree-SHA512: f3788a543ca98e44eeeba1d06c32f1b11eec95d4aef068aa1b6b5c401261adfa3fb6c6d6c769f3fe6839d78e74a310d5c926867e7c367d6513a53d580fd376f3
2023-08-01 09:42:07 +01:00
Ryan Ofsky
f4f1d6d230
Merge bitcoin/bitcoin#27746: Rework validation logic for assumeutxo
a733dd79e2 Remove unused function `reliesOnAssumedValid` (Suhas Daftuar)
d4a11abb19 Cache block index entry corresponding to assumeutxo snapshot base blockhash (Suhas Daftuar)
3556b85022 Move CheckBlockIndex() from Chainstate to ChainstateManager (Suhas Daftuar)
0ce805b632 Documentation improvements for assumeutxo (Ryan Ofsky)
768690b7ce Fix initialization of setBlockIndexCandidates when working with multiple chainstates (Suhas Daftuar)
d43a1f1a2f Tighten requirements for adding elements to setBlockIndexCandidates (Suhas Daftuar)
d0d40ea9a6 Move block-storage-related logic to ChainstateManager (Suhas Daftuar)
3cfc75366e test: Clear block index flags when testing snapshots (Suhas Daftuar)
272fbc370c Update CheckBlockIndex invariants for chains based on an assumeutxo snapshot (Suhas Daftuar)
10c05710ce Add wrapper for adding entries to a chainstate's block index candidates (Suhas Daftuar)
471da5f6e7 Move block-arrival information / preciousblock counters to ChainstateManager (Suhas Daftuar)
1cfc887d00 Remove CChain dependency in node/blockstorage (Suhas Daftuar)
fe86a7cd48 Explicitly track maximum block height stored in undo files (Suhas Daftuar)

Pull request description:

  This PR proposes a clean up of the relationship between block storage and the chainstate objects, by moving the decision of whether to store a block on disk to something that is not chainstate-specific.  Philosophically, the decision of whether to store a block on disk is related to validation rules that do not require any UTXO state; for anti-DoS reasons we were using some chainstate-specific heuristics, and those have been reworked here to achieve the proposed separation.

  This PR also fixes a bug in how a chainstate's `setBlockIndexCandidates` was being initialized; it should always have all the HAVE_DATA block index entries that have more work than the chain tip.  During startup, we were not fully populating `setBlockIndexCandidates` in some scenarios involving multiple chainstates.

  Further, this PR establishes a concept that whenever we have 2 chainstates, that we always know the snapshotted chain's base block and the base block's hash must be an element of our block index. Given that, we can establish a new invariant that the background validation chainstate only needs to consider blocks leading to that snapshotted block entry as potential candidates for its tip. As a followup I would imagine that when writing net_processing logic to download blocks for the background chainstate, that we would use this concept to only download blocks towards the snapshotted entry as well.

ACKs for top commit:
  achow101:
    ACK a733dd79e2
  jamesob:
    reACK a733dd79e2 ([`jamesob/ackr/27746.5.sdaftuar.rework_validation_logic`](https://github.com/jamesob/bitcoin/tree/ackr/27746.5.sdaftuar.rework_validation_logic))
  Sjors:
    Code review ACK a733dd79e2.
  ryanofsky:
    Code review ACK a733dd79e2. Just suggested changes since the last review. There are various small things that could be followed up on, but I think this is ready for merge.

Tree-SHA512: 9ec17746f22b9c27082743ee581b8adceb2bd322fceafa507b428bdcc3ffb8b4c6601fc61cc7bb1161f890c3d38503e8b49474da7b5ab1b1f38bda7aa8668675
2023-07-31 16:18:20 -04:00