0
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-02-11 11:16:09 -05:00
Commit graph

4890 commits

Author SHA1 Message Date
fanquake
29c2c90362
Merge bitcoin/bitcoin#28721: multiprocess compatibility updates
3b70f7b615 doc: fix broken doc/design/multiprocess.md links after #24352 (Ryan Ofsky)
6d43aad742 span: Make Span template deduction guides work in SFINAE context (Ryan Ofsky)
8062c3bdb9 util: Add ArgsManager SetConfigFilePath method (Ryan Ofsky)
441d00c60f interfaces: Rename CalculateBumpFees methods to be compatible with capn'proto (Ryan Ofsky)
156f49d682 interfaces: Change getUnspentOutput return type to avoid multiprocess segfault (Ryan Ofsky)
4978754c00 interfaces: Add schedulerMockForward method so mockscheduler RPC can work across processes (Ryan Ofsky)
924327eaf3 interfaces: Fix const virtual method that breaks multiprocess support (Ryan Ofsky)
82a379eca8 streams: Add SpanReader ignore method (Russell Yanofsky)

Pull request description:

  This is a collection of small changes to interfaces and code which were needed as part of multiprocess PR #10102, but have been moved here to make that PR smaller.

  All of these changes are refactoring changes which do not affect behavior of current code

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).

ACKs for top commit:
  achow101:
    ACK 3b70f7b615
  naumenkogs:
    ACK 3b70f7b615
  maflcko:
    re-ACK 3b70f7b615  🎆

Tree-SHA512: 2368772b887056ad8a9f84c299cfde76ba45943770e3b5353130580900afa9611302195b899ced7b6e303b11f053ff204cae7c28ff4e12c55562fcc81119ba4c
2023-11-13 12:32:55 +00:00
fanquake
e862bceb17
Merge bitcoin/bitcoin#27935: fuzz: call lookup functions before calling Ban
fca0a8938e ci: remove "--exclude banman" for fuzzing in mac (brunoerg)
f9b286353f fuzz: call lookup functions before calling `Ban` (brunoerg)

Pull request description:

  Fixes #27924

  To not have any discrepancy, it's required to call lookup functions before calling `Ban`. If we don't do it, the assertion `assert(banmap == banmap_read);` may fail because `BanMapFromJson` will call `LookupSubNet` and cause the discrepancy between the banned and the loaded one. It happens especially in MacOS (#27924).

  Also, calling lookup functions before banning is what RPC `setban` does.

ACKs for top commit:
  maflcko:
    lgtm ACK fca0a8938e
  dergoegge:
    ACK fca0a8938e

Tree-SHA512: a3d635088a556df4507e65542157f10b41d4f87dce42927b58c3b812f262f4544b6b57f3384eef1097ffdd7c32b8dd1556aae201254960cbfbf48d45551200f7
2023-11-13 10:57:01 +00:00
fanquake
dd5f5713bc
Merge bitcoin/bitcoin#28391: refactor: Simplify CTxMempool/BlockAssembler fields, remove some external mapTx access
4dd94ca18f [refactor] remove access to mapTx in validation_block_tests (TheCharlatan)
d0cd2e804e [refactor] rewrite BlockAssembler inBlock and failedTx as sets of txids (glozow)
55b0939cab scripted-diff: rename vTxHashes to txns_randomized (TheCharlatan)
a03aef9cec [refactor] rewrite vTxHashes as a vector of CTransactionRef (glozow)
938643c3b2 [refactor] remove access to mapTx in validation.cpp (glozow)
333367a940 [txmempool] make CTxMemPoolEntry::lockPoints mutable (glozow)
1bf4855016 [refactor] use CheckPackageLimits for checkChainLimits (glozow)
dbc5bdbf59 [refactor] remove access to mapTx.find in mempool_tests.cpp (glozow)
f80909e7a3 [refactor] remove access to mapTx in blockencodings_tests.cpp (glozow)
8892d6b744 [refactor] remove access to mapTx from rpc/mempool.cpp (glozow)
fad61aa561 [refactor] get wtxid from entry instead of vTxHashes (glozow)
9cd8cafb77 [refactor] use exists() instead of mapTx.find() (glozow)
14804699e5 [refactor] remove access to mapTx from policy/rbf.cpp (glozow)
1c6a73abbd [refactor] Add helper for retrieving mempool entry (TheCharlatan)
453b4813eb [refactor] Add helper for iterating through mempool entries (stickies-v)

Pull request description:

  Motivation
  * It seems preferable to use stdlib data structures instead of boost if they can achieve close to the same thing.
  * Code external to mempool should ideally use its public helper methods instead of accessing `mapTx` or its iterators directly.
  * Reduce the number of complex boost multi index type interactions
  * Also see #28335 for further context/motivation. This PR together with #28385 simplifies that one.

  Overview of things done in this PR:
  * Make `vTxHashes` a vector of transaction references instead of a pair of transaction hash and iterator. The trade off here is that the data is retrieved on the fly with `GetEntry` instead of being cached in `vTxHashes`.
  * Introduce `GetEntry` helper method to replace the more involved `GetIter` where applicable
  * Replace `mapTx` access with `CTxMemPool` helper methods
  * Simplify `checkChainLimits` call in `node/interfaces.cpp`
  * Make `CTxMemPoolEntry`s `lockPoints`mutable such that they can be changed with a const iterator directly instead of going through `mapTx`
  * Make `BlockAssembler`'s `inBlock` and `failedTx` sets of transaction hashes.

ACKs for top commit:
  glozow:
    reACK 4dd94ca
  maflcko:
    re-ACK 4dd94ca18f 👝
  stickies-v:
    re-ACK 4dd94ca18f

Tree-SHA512: c4d043f2186e4fde337591883fac66cade3058173987b49502bd65cecf69207a3df1077f6626809652ab63230013167b7f39a2b39f1c5166959e5495df57065f
2023-11-13 10:51:41 +00:00
TheCharlatan
4dd94ca18f
[refactor] remove access to mapTx in validation_block_tests
Use the helper function instead of reaching into the mapTx member
object.
2023-11-10 16:44:47 +01:00
TheCharlatan
55b0939cab
scripted-diff: rename vTxHashes to txns_randomized
-BEGIN VERIFY SCRIPT-
git grep -l "vTxHashesIdx" src | xargs sed -i "s/vTxHashesIdx/idx_randomized/g"
git grep -l "vTxHashes" src | xargs sed -i "s/vTxHashes/txns_randomized/g"
-END VERIFY SCRIPT-
2023-11-10 16:44:44 +01:00
glozow
a03aef9cec
[refactor] rewrite vTxHashes as a vector of CTransactionRef
vTxHashes exposes a complex mapTx iterator type that its external users
don't need. Directly populate it with CTransactionRef instead.
2023-11-10 16:44:42 +01:00
glozow
dbc5bdbf59
[refactor] remove access to mapTx.find in mempool_tests.cpp 2023-11-10 16:44:35 +01:00
glozow
f80909e7a3
[refactor] remove access to mapTx in blockencodings_tests.cpp 2023-11-10 16:44:33 +01:00
TheCharlatan
1c6a73abbd
[refactor] Add helper for retrieving mempool entry
In places where the iterator is only needed for accessing the actual
entry, it should not be required to first retrieve the iterator.
2023-11-10 16:44:25 +01:00
Greg Sanders
6a917918b7 fuzz: allow fake and duplicate inputs in tx_package_eval target 2023-11-09 09:07:03 -05:00
Greg Sanders
a0626ccdad fuzz: allow reaching MempoolAcceptResult::ResultType::DIFFERENT_WITNESS in tx_package_eval target 2023-11-09 09:07:03 -05:00
brunoerg
f9b286353f fuzz: call lookup functions before calling Ban
Also, compare banmaps only if there are no invalid
entries.
2023-11-09 10:11:51 -03:00
glozow
d60ebea597
Merge bitcoin/bitcoin#28808: refactor: Miniminer package linearization followups
b4b01d3fb4 [refactor] updating miniminer comments to be more accurate (kevkevin)
83933eff00 [refactor] Miniminer var cached_descendants to descendants (kevkevin)
43423fd834 [refactor] Change MiniMinerMempoolEntry order (kevkevin)

Pull request description:

  ### Motivation
  In https://github.com/bitcoin/bitcoin/pull/28762 there were some post merge comments which are being addressed in this PR with the following commits

  ### [8d4c46f](8d4c46f54d) Reorganizing `MiniMinerMempoolEntry` to match the order we have elsewhere
  * https://github.com/bitcoin/bitcoin/pull/28762#discussion_r1381775670

  ### [7505ec2](7505ec2054) Renaming `cached_descendants` to `descendants` for simpler variable naming
  * https://github.com/bitcoin/bitcoin/pull/28762#discussion_r1381819567

  ### [b21f2f2](b21f2f2f55) Code comment modifications to be more accurate to what is actually happening
  * https://github.com/bitcoin/bitcoin/pull/28762#discussion_r1381902909 and
  * https://github.com/bitcoin/bitcoin/pull/28762#discussion_r1382002278 and
  * https://github.com/bitcoin/bitcoin/pull/28762#discussion_r1383041819

ACKs for top commit:
  murchandamus:
    reACK b4b01d3fb4
  theStack:
    LGTM ACK b4b01d3fb4

Tree-SHA512: 54f044a578fb203d8a3c1aa0bcd1fc4bcdff0bc9b024351925a4caf0ccece7c7736b0694ad1168c3cbb447bdb58a91f4cac365f46114da29a889fbc8ea595b82
2023-11-09 09:30:58 +00:00
kevkevin
b4b01d3fb4
[refactor] updating miniminer comments to be more accurate 2023-11-08 14:45:18 -06:00
fanquake
f1f3f2d9cc
Merge bitcoin/bitcoin#28815: fuzz: Avoid timeout and bloat in fuzz targets
fabb5046a7 fuzz: Avoid timeout and bloat in fuzz targets (MarcoFalke)

Pull request description:

  If the fuzz input contains invalid data *in a loop*, abort early. This will teach the fuzz engine to look for useful data and avoids bloating the fuzz input folder with useless (repeated) data.

ACKs for top commit:
  dergoegge:
    utACK fabb5046a7
  brunoerg:
    crACK fabb5046a7

Tree-SHA512: 26da100d7558ae6fdd5292fb146d8858b2af8f78c546ca2509b9d27b33a33e9462ecb6035de142f9f36dd5de32f8cbad099d6c7a697902d23e1bb621cd27dc88
2023-11-08 14:19:35 +00:00
glozow
9ad19fc7c7
Merge bitcoin/bitcoin#28155: net: improves addnode / m_added_nodes logic
0420f99f42 Create net_peer_connection unit tests (Jon Atack)
4b834f6499 Allow unit tests to access additional CConnman members (Jon Atack)
34b9ef443b net/rpc: Makes CConnman::GetAddedNodeInfo able to return only non-connected address on request (Sergi Delgado Segura)
94e8882d82 rpc: Prevents adding the same ip more than once when formatted differently (Sergi Delgado Segura)
2574b7e177 net/rpc: Check all resolved addresses in ConnectNode rather than just one (Sergi Delgado Segura)

Pull request description:

  ## Rationale

  Currently, `addnode` has a couple of corner cases that allow it to either connect to the same peer more than once, hence wasting outbound connection slots, or add redundant information to `m_added_nodes`, hence making Bitcoin iterate through useless data on a regular basis.

  ### Connecting to the same node more than once

  In general, connecting to the same node more than once is something we should try to prevent. Currently, this is possible via `addnode` in two different ways:

  1. Calling `addnode` more than once in a short time period, using two equivalent but distinct addresses
  2. Calling `addnode add` using an IP, and `addnode onetry` after with an address that resolved to the same IP

  For the former, the issue boils down to `CConnman::ThreadOpenAddedConnections` calling `CConnman::GetAddedNodeInfo` once, and iterating over the result to open connections (`CConman::OpenNetworkConnection`) on the same loop for all addresses.`CConnman::ConnectNode` only checks a single address, at random, when resolving from a hostname, and uses it to check whether we are already connected to it.

  An example to test this would be calling:

  ```
  bitcoin-cli addnode "127.0.0.1:port" add
  bitcoin-cli addnode "localhost:port" add
  ```

  And check how it allows us to perform both connections some times, and some times it fails.

  The latter boils down to the same issue, but takes advantage of `onetry` bypassing the `CConnman::ThreadOpenAddedConnections` logic and calling `CConnman::OpenNetworkConnection` straightaway. A way to test this would be:

  ```
  bitcoin-cli addnode "127.0.0.1:port" add
  bitcoin-cli addnode "localhost:port" onetry
  ```

  ### Adding the same peer with two different, yet equivalent, addresses

  The current implementation of `addnode` is pretty naive when checking what data is added to `m_added_nodes`. Given the collection stores strings, the checks at `CConnman::AddNode()` basically check wether the exact provided string is already in the collection. If so, the data is rejected, otherwise, it is accepted. However, ips can be formatted in several ways that would bypass those checks.

  Two examples would be `127.0.0.1` being equal to `127.1` and `[::1]` being equal to `[0:0:0:0:0:0:0:1]`. Adding any pair of these will be allowed by the rpc command, and both will be reported as connected by `getaddednodeinfo`, given they map to the same `CService`.

  This is less severe than the previous issue, since even tough both nodes are reported as connected by `getaddednodeinfo`, there is only a single connection to them (as properly reported by `getpeerinfo`). However, this adds redundant data to `m_added_nodes`, which is undesirable.

  ### Parametrize `CConnman::GetAddedNodeInfo`
  Finally, this PR also parametrizes `CConnman::GetAddedNodeInfo` so it returns either all added nodes info, or only info about the nodes we are **not** connected to. This method is used both for `rpc`, in `getaddednodeinfo`, in which we are reporting all data to the user, so the former applies, and to check what nodes we are not connected to, in `CConnman::ThreadOpenAddedConnections`, in which we are currently returning more data than needed and then actively filtering using `CService.fConnected()`

ACKs for top commit:
  jonatack:
    re-ACK 0420f99f42
  kashifs:
    > > tACK [0420f9](0420f99f42)
  sr-gi:
    > > > tACK [0420f9](0420f99f42)
  mzumsande:
    Tested ACK 0420f99f42

Tree-SHA512: a3a10e748c12d98d439dfb193c75bc8d9486717cda5f41560f5c0ace1baef523d001d5e7eabac9fa466a9159a30bb925cc1327c2d6c4efb89dcaf54e176d1752
2023-11-08 11:31:36 +00:00
fanquake
d690f89b57
Merge bitcoin/bitcoin#28785: validation: return more helpful results for reconsiderable fee failures and skipped transactions
1147e00e59 [validation] change package-fee-too-low, return wtxid(s) and effective feerate (glozow)
10dd9f2441 [test] use CheckPackageMempoolAcceptResult in previous tests (glozow)
3979f1afcb [validation] add TxValidationResult::TX_RECONSIDERABLE, TX_UNKNOWN (glozow)
5c786a026a [refactor] use Wtxid for m_wtxids_fee_calculations (glozow)

Pull request description:

  Split off from #26711 (suggested in https://github.com/bitcoin/bitcoin/pull/26711#issuecomment-1786392253). This is part of #27463.

  - Add 2 new TxValidationResults
    - `TX_RECONSIDERABLE` helps us encode transactions who have failed fee checks that can be bypassed using package validation. This is distinguished from `TX_MEMPOOL_POLICY` so that we re-validate a transaction if and only if it is eligible for package CPFP. In the future, we will have a separate cache for reconsiderable rejects so these transactions don't go in `m_recent_rejects`.
    - `TX_UNKNOWN` helps us communicate that we aborted package validation and didn't finish looking at this transaction: it's not valid but it's also not invalid (i.e. don't cache it as a rejected tx)
  - Return effective feerate and the wtxids of transactions used to calculate that effective feerate when the error is `TX_SINGLE_FAILURE`. Previously, we would only provide this information if the transaction passed. Now that we have package validation, it's much more helpful to the caller to know how the failing feerate was calculated. This can also be used to improve our submitpackage RPC result (which is currently a bit unhelpful when things fail).
  - Use the newly added `CheckPackageMempoolAcceptResult` for existing package validation tests. This increases test coverage and helps test the changes made in this PR.

ACKs for top commit:
  instagibbs:
    reACK 1147e00e59
  achow101:
    ACK 1147e00e59
  murchandamus:
    reACK 1147e00e59
  ismaelsadeeq:
    ACK 1147e00e59

Tree-SHA512: ac1cd73c2b487a1b99d329875d39d8107c91345a5b0b241d54a6a4de67faf11be69a2721cc732c503024a9cca381dac33d61e187957279e3c82653bea118ba91
2023-11-08 10:17:05 +00:00
MarcoFalke
fabb5046a7
fuzz: Avoid timeout and bloat in fuzz targets
Also, fix iwyu
2023-11-08 09:51:54 +01:00
Andrew Chow
82ea4e787c
Merge bitcoin/bitcoin#28464: net: improve max-connection limits code
df69b22f2e doc: improve documentation around connection limit maximums (Amiti Uttarwar)
adc171edf4 scripted-diff: Rename connection limit variables (Amiti Uttarwar)
e9fd9c0225 net: add m_max_inbound to connman (Amiti Uttarwar)
c25e0e0555 net, refactor: move calculations for connection type limits into connman (Amiti Uttarwar)

Pull request description:

  This is joint work with amitiuttarwar.

  This has the first few commits of #28463. It is not strictly a prerequisite for that, but has changes that in our opinion make sense on their own.
  It improves the handling of maximum numbers for different connection types (that are set during init and don’t change after) by:
  * moving all calculations into one place, `CConnMan::Init()`. Before, they were dispersed between `Init`, `CConnman::Init` and other parts of `CConnman`, resulting in some duplicated test code.
  * removing the possibility of having a negative maximum of inbound connections, which is hard to argue about
  * renaming of variables and doc improvements

ACKs for top commit:
  amitiuttarwar:
    co-author review ACK df69b22f2e
  naumenkogs:
    ACK df69b22f2e
  achow101:
    ACK df69b22f2e

Tree-SHA512: 913d56136bc1df739978de50db67302f88bac2a9d34748ae96763288d97093e998fc0f94f9b6eff12867712d7e86225af6128f4170bf2b5b8ab76f024870a22c
2023-11-07 17:01:02 -05:00
Andrew Chow
c981771bc3
Merge bitcoin/bitcoin#28224: shutdown: Destroy kernel last, make test shutdown order consistent
c1144f0076 tests: Reset node context members on ~BasicTestingSetup (TheCharlatan)
9759af17ff shutdown: Destroy kernel last (TheCharlatan)

Pull request description:

  The destruction/resetting of node context members in the tests should roughly follow the behavior of the `Shutdown` function in `init.cpp`.

  This was originally requested by MarcoFalke in this [comment](https://github.com/bitcoin/bitcoin/pull/25065#discussion_r890161249) in response to the [original pull request](https://github.com/bitcoin/bitcoin/pull/25065) introducing the `kernel::Context`.

ACKs for top commit:
  maflcko:
    ACK c1144f0076 🗣
  achow101:
    ACK c1144f0076
  ryanofsky:
    Code review ACK c1144f0076. No code changes since last review, just updated commits and descriptions

Tree-SHA512: 819bb85ff82a5c6c60e429674d5684f3692fe9062500d00a87b361cc59e6bda145be21b5a4466dee6791faed910cbde4d26baab325bf6daa1813af13a63588ff
2023-11-07 16:17:29 -05:00
Andrew Chow
0528cfd307
Merge bitcoin/bitcoin#28649: Do the SOCKS5 handshake reliably
af0fca530e netbase: use reliable send() during SOCKS5 handshake (Vasil Dimov)
1b19d1117c sock: change Sock::SendComplete() to take Span (Vasil Dimov)

Pull request description:

  The `Socks5()` function which does the SOCKS5 handshake with the SOCKS5 proxy sends bytes to the socket without retrying partial writes.

  `send(2)` may write only part of the provided data and return. In this case the caller is responsible for retrying the operation with the remaining data. Change `Socks5()` to do that. There is already a method `Sock::SendComplete()` which does exactly that, so use it in `Socks5()`.

  A minor complication for this PR is that `Sock::SendComplete()` takes `std::string` argument whereas `Socks5()` has `std::vector<uint8_t>`. Thus the necessity for the first commit. It is possible to do also in other ways - convert the data in `Socks5()` to `std::string` or have just one `Sock::SendComplete()` that takes `void*` and change the callers to pass `str.data(), str.size()` or `vec.data(), vec.size()`.

  This came up while testing https://github.com/bitcoin/bitcoin/pull/27375.

ACKs for top commit:
  achow101:
    ACK af0fca530e
  jonatack:
    ACK af0fca530e
  pinheadmz:
    ACK af0fca530e

Tree-SHA512: 1d4a53d0628f7607378038ac56dc3b8624ce9322b034c9547a0c3ce052eafb4b18213f258aa3b57bcb4d990a5e0548a37ec70af2bd55f6e8e6399936f1ce047a
2023-11-07 14:11:58 -05:00
kevkevin
43423fd834
[refactor] Change MiniMinerMempoolEntry order
Changes MiniMinerMempoolEntry order to match the order of the params
elsewhere in the codebase
2023-11-07 08:56:36 -06:00
glozow
1147e00e59 [validation] change package-fee-too-low, return wtxid(s) and effective feerate
With subpackage evaluation and de-duplication, it's not always the
entire package that is used in CheckFeerate. To be more helpful to the
caller, specify which transactions were included in the evaluation and
what the feerate was.

Instead of PCKG_POLICY (which is supposed to be for package-wide
errors), use PCKG_TX.
2023-11-07 11:26:17 +00:00
glozow
10dd9f2441 [test] use CheckPackageMempoolAcceptResult in previous tests
Increases test coverage (check every result field) and makes it easier
to test the changes in the next commit.
2023-11-07 11:23:07 +00:00
fanquake
2b3f43b96e
Merge bitcoin/bitcoin#28789: fuzz: Avoid utxo_total_supply timeout (take 2)
fa7ba92630 fuzz: Avoid utxo_total_supply timeout (MarcoFalke)

Pull request description:

  Looks like this still may take a long time to run large fuzz inputs. Thus, reduce it further, but still allow it to catch the regression, if re-introduced:

  ```diff
  diff --git a/src/consensus/tx_check.cpp b/src/consensus/tx_check.cpp
  index f949655909..4bdd15c5ee 100644
  --- a/src/consensus/tx_check.cpp
  +++ b/src/consensus/tx_check.cpp
  @@ -40,7 +40,7 @@ bool CheckTransaction(const CTransaction& tx, TxValidationState& state)
       std::set<COutPoint> vInOutPoints;
       for (const auto& txin : tx.vin) {
           if (!vInOutPoints.insert(txin.prevout).second)
  -            return state.Invalid(TxValidationResult::TX_CONSENSUS, "bad-txns-inputs-duplicate");
  +            {}//return state.Invalid(TxValidationResult::TX_CONSENSUS, "bad-txns-inputs-duplicate");
       }

       if (tx.IsCoinBase())
  ```

  This is the second take, see https://github.com/bitcoin/bitcoin/pull/27780. If in the future it still times out, I think the fuzz test can just be removed.

  Example input:

  ```
  JREROy5pcnAgQyw7IC4ODg4ODg4ODg4O0dEODg4ODg4ZDg4ODg4ODg4ODg7RDg4ODg4ODg4O0dEODg4ODg4ODg4ODg7R0Q4ODg4ODg4ODtHRDg4ODtHR0dEODg4O0dEODg7R0Q4ODg4ODg4ODtHRDg4ODg4ODg4ODg4O0dEODg4ODg4ODg7R0Q4ODg7R0Q4O0dEODg4ODg4ODg4ODg7R0Q4ODg4ODtHRDg4ODtHR

ACKs for top commit:
  dergoegge:
    ACK fa7ba92630
  brunoerg:
    utACK fa7ba92630

Tree-SHA512: 154a4895834babede6ce7b775562a7026637af1097e53e55676e92f6cf966ae0c092300ebf7e51a397eebd11f7b41d020586663e781f70d084efda1c0fe851b4
2023-11-07 11:17:00 +00:00
Sjors Provoost
43de4d3630
doc: fix typos
As found by lint-spelling.py using codespell 2.2.6.
2023-11-07 10:21:51 +09:00
glozow
5c786a026a [refactor] use Wtxid for m_wtxids_fee_calculations 2023-11-06 14:33:32 +00:00
MarcoFalke
fa7ba92630
fuzz: Avoid utxo_total_supply timeout 2023-11-03 21:16:12 +01:00
Greg Sanders
5380f05513 test: bugfix CheckPackageMempoolAcceptResult return all error strings 2023-11-03 16:05:55 -04:00
Andrew Chow
d9007f51a7
Merge bitcoin/bitcoin#28762: MiniMiner changes for package linearization
d9cc99d04e [test] MiniMiner::Linearize and manual construction (glozow)
dfd6a3788c [refactor] unify fee amounts in miniminer_tests (glozow)
f4b1b24a3b [MiniMiner] track inclusion order and add Linearize() function (glozow)
004075963f [test] add case for MiniMiner working with negative fee txns (glozow)
fe6332c0ba [MiniMiner] make target_feerate optional (glozow)
5a83f55c96 [MiniMiner] allow manual construction with non-mempool txns (glozow)
e3b2e630b2 [refactor] change MiniMinerMempoolEntry ctor to take values, update includes (glozow)
4aa98b79b2 [lint] update expected boost includes (glozow)

Pull request description:

  This is part of #27463. It splits off the `MiniMiner`-specific changes from #26711 for ease of review, as suggested in https://github.com/bitcoin/bitcoin/pull/26711#issuecomment-1786392253.

  - Allow using `MiniMiner` on transactions that aren't in the mempool.
  - Make `target_feerate` param of `BuildMockTemplate` optional, meaning "don't stop building the template until all the transactions have been selected."
    - Add clarification for how this is different from `target_feerate=0` (https://github.com/bitcoin/bitcoin/pull/26711#discussion_r1377019133)
  - Track the order in which transactions are included in the template to get the "linearization order" of the transactions.
  - Tests

  Reviewers can take a look at #26711 to see how these functions are used to linearize the `AncestorPackage` there.

ACKs for top commit:
  TheCharlatan:
    ACK d9cc99d04e
  kevkevinpal:
    reACK [d9cc99d](d9cc99d04e)
  achow101:
    re-ACK d9cc99d04e

Tree-SHA512: 32b80064b6679536ac573d674825c5ca0cd6245e49c2fd5eaf260dc535335a57683c74ddd7ce1f249b5b12b2683de4362a7b0f1fc0814c3b3b9f14c682665583
2023-11-03 10:50:50 -04:00
fanquake
5d9f45082b
Merge bitcoin/bitcoin#28758: refactors for subpackage evaluation
b5a60abe87 MOVEONLY: CleanupTemporaryCoins into its own function (glozow)
10c0a8678c [test util] CreateValidTransaction multi-in/out, configurable feerate, signal BIP125 (glozow)
6ff647a7e0 scripted-diff: rename CheckPackage to IsWellFormedPackage (glozow)
da9aceba21 [refactor] move package checks into helper functions (glozow)

Pull request description:

  This is part of #27463. It splits off the more trivial changes from #26711 for ease of review, as requested in https://github.com/bitcoin/bitcoin/pull/26711#issuecomment-1786392253.

  - Split package sanitization in policy/packages.h into helper functions
    - Add some tests for its quirks (https://github.com/bitcoin/bitcoin/pull/26711#discussion_r1340521597)
  - Rename `CheckPackage` to `IsPackageWellFormed`
  - Improve the `CreateValidTransaction` unit test utility to:
    - Configure the target feerate and return the fee paid
    - Signal BIP125 on transactions to enable RBF tests
    - Allow the specification of multiple inputs and outputs
  - Move `CleanupTemporaryCoins` into its own function to be reused later without duplication

ACKs for top commit:
  dergoegge:
    Code review ACK b5a60abe87
  instagibbs:
    ACK b5a60abe87

Tree-SHA512: 39d67a5f0041e381f0d0f802a98ccffbff11e44daa3a49611189d6306b03f18613d5ff16c618898d490c97a216753e99e0db231ff14d327f92c17ae4d269cfec
2023-11-03 14:41:17 +00:00
glozow
f23ac10ca5
Merge bitcoin/bitcoin#28764: Fuzz: Check individual and package transaction invariants
fcb3069fa3 Use CheckPackageMempoolAcceptResult for package evaluation fuzzing (Greg Sanders)
34088d6c9e [test util] CheckPackageMempoolAcceptResult for sanity-checking results (glozow)
651fa404e4 fuzz: tx_pool checks ATMP result invariants (Greg Sanders)

Pull request description:

  Poached from https://github.com/bitcoin/bitcoin/pull/26711 since that PR is being split apart, and modified to match current behavior.

ACKs for top commit:
  glozow:
    reACK fcb3069fa3, only whitespace changes
  dergoegge:
    ACK fcb3069fa3

Tree-SHA512: abd687e526d8dfc8d65b3a873ece8ca35fdcbd6b0f7b93da6a723ef4e47cf85612de819e6f2b8631bdf897e1aba27cdd86f89b7bd85fc3356e74be275dcdf8cc
2023-11-03 13:51:12 +00:00
glozow
d9cc99d04e [test] MiniMiner::Linearize and manual construction 2023-11-03 10:39:29 +00:00
glozow
dfd6a3788c [refactor] unify fee amounts in miniminer_tests
Name {low,med,high}_fee constants for reuse across file.
2023-11-03 10:17:41 +00:00
glozow
004075963f [test] add case for MiniMiner working with negative fee txns 2023-11-03 10:17:41 +00:00
Greg Sanders
fcb3069fa3 Use CheckPackageMempoolAcceptResult for package evaluation fuzzing 2023-11-02 09:33:47 -04:00
glozow
34088d6c9e [test util] CheckPackageMempoolAcceptResult for sanity-checking results 2023-11-02 09:33:47 -04:00
glozow
023418a140
Merge bitcoin/bitcoin#28530: tests, bug fix: DisconnectedBlockTransactions rewrite followups
9b3da70bd0 [test] DisconnectedBlockTransactions::DynamicMemoryUsage (glozow)
b2d0447964 bugfix: correct DisconnectedBlockTransactions memory usage (stickies-v)
f4254e2098 assume duplicate transactions are not added to `iters_by_txid` (ismaelsadeeq)
29eb219c12 move only: move implementation code to disconnected_transactions.cpp (ismaelsadeeq)
81dfeddea7 refactor: update `MAX_DISCONNECTED_TX_POOL` from kb to bytes (ismaelsadeeq)

Pull request description:

  This PR is a follow-up to fix review comments and a bugfix from #28385

  The PR

  - Updated `DisconnectedBlockTransactions`'s `MAX_DISCONNECTED_TX_POOL` from kb to bytes.
  - Moved `DisconnectedBlockTransactions` implementation code to `kernel/disconnected_transactions.cpp`.
  - `AddTransactionsFromBlock` now assume duplicate transactions are not passed by asserting after inserting each transaction to `iters_by_txid`.
  - Included a Bug fix: In the current master we are underestimating the memory usage of `DisconnectedBlockTransactions`.

      * When adding and subtracting `cachedInnerUsage` we call `RecursiveDynamicUsage` with `CTransaction` which invokes this [`RecursiveDynamicUsage(const CTransaction& tx)`](6e721c923c/src/core_memusage.h (L32)) version of `RecursiveDynamicUsage`, the output of that call only account for the memory usage of the inputs and outputs of the `CTransaction`, this omits the memory usage of the `CTransaction` object and the control block.
      * This PR fixes this bug by calling `RecursiveDynamicUsage` with `CTransactionRef` when adding and subtracting `cachedInnerUsage` which invokes [`RecursiveDynamicUsage(const std::shared_ptr<X>& p)`](6e721c923c/src/core_memusage.h (L67)) version of `RecursiveDynamicUsage` the output of the calculation accounts for the` CTransaction` object, the control blocks, inputs and outputs memory usage.
      * see  [comment ](https://github.com/bitcoin/bitcoin/pull/28385#discussion_r1322948452)
  - Added test for DisconnectedBlockTransactions memory limit.

ACKs for top commit:
  stickies-v:
    ACK 9b3da70bd0 - nice work!
  BrandonOdiwuor:
    re ACK 9b3da70bd0
  glozow:
    ACK 9b3da70bd0

Tree-SHA512: 69b9595d09f4d0209038f97081d790cea92ccf63efb94e9e372749979fcbe527f7f17a8e454720cedd12021be0c8e11cf99874625d3dafd9ec602b12dbeb4098
2023-11-02 11:12:17 +00:00
glozow
10c0a8678c [test util] CreateValidTransaction multi-in/out, configurable feerate, signal BIP125
Support the creation of a transaction with multiple specified inputs or
outputs. Also accept a target feerate and return the fee paid.

Also, signal BIP125 by default - a subsequent commit needs to RBF
something.

Co-authored-by: Andrew Chow <achow101@gmail.com>
2023-11-01 17:21:54 +00:00
glozow
6ff647a7e0 scripted-diff: rename CheckPackage to IsWellFormedPackage
-BEGIN VERIFY SCRIPT-
sed -i 's/CheckPackage(/IsWellFormedPackage(/g' $(git grep -l CheckPackage)
-END VERIFY SCRIPT-
2023-11-01 17:21:54 +00:00
glozow
da9aceba21 [refactor] move package checks into helper functions
This allows IsSorted() and IsConsistent() to be used by themselves.
IsSorted() with a precomputed set is used so that we don't create this
set multiple times.
2023-11-01 17:21:54 +00:00
Greg Sanders
651fa404e4 fuzz: tx_pool checks ATMP result invariants 2023-10-31 14:52:45 -04:00
Jon Atack
0420f99f42 Create net_peer_connection unit tests
for initial partial unit test coverage of these CConnman class methods:

- AddNode()
- ConnectNode()
- GetAddedNodeInfo()
- AlreadyConnectedToAddress()
- ThreadOpenAddedConnections()

and of the GetAddedNodeInfo() call in RPC addnode.
2023-10-31 13:37:12 -04:00
Vasil Dimov
af0fca530e
netbase: use reliable send() during SOCKS5 handshake
`send(2)` can be interrupted or for another reason it may not fully
complete sending all the bytes. We should be ready to retry the send
with the remaining bytes. This is what `Sock::SendComplete()` does,
thus use it in `Socks5()`.

Since `Sock::SendComplete()` takes a `CThreadInterrupt` argument,
change also the recv part of `Socks5()` to use `CThreadInterrupt`
instead of a boolean.

Easier reviewed with `git show -b` (ignore white-space changes).
2023-10-31 18:19:37 +01:00
fanquake
d51fb9caa6
Merge bitcoin/bitcoin#28503: refactor: Remove WithParams serialization helper, use SER_PARAMS_OPFUNC
99990194ce Remove WithParams serialization helper (MarcoFalke)
ffffb4af83 scripted-diff: Use ser params operator (MarcoFalke)
fae9054793 test: Use SER_PARAMS_OPFUNC in serialize_tests.cpp (MarcoFalke)

Pull request description:

  Every serialization parameter struct already has the `SER_PARAMS_OPFUNC`, except for one in the tests.

  For consistency, and to remove verbose code, convert the test to `SER_PARAMS_OPFUNC`, and use it everywhere, then remove the `WithParams` helper.

ACKs for top commit:
  ajtowns:
    reACK 99990194ce
  TheCharlatan:
    Re-ACK 99990194ce

Tree-SHA512: be9cae4225a502486fe8d552aaf4b2cd2904a9f73cce9d931c6b7c757594ff1982fcc2c30d00d012cd12b0a9531fd609f8bcd7c94b811e965ac087eb8a3589d3
2023-10-31 11:11:25 +00:00
Jon Atack
4b834f6499 Allow unit tests to access additional CConnman members
that are otherwise private:
- CConnman::m_nodes
- CConnman::ConnectNodes()
- CConnman::AlreadyConnectedToAddress()

and update the #include headers per iwyu.
2023-10-30 11:44:59 -04:00
Sergi Delgado Segura
34b9ef443b net/rpc: Makes CConnman::GetAddedNodeInfo able to return only non-connected address on request
`CConnman::GetAddedNodeInfo` is used both to get a list of addresses to manually connect to
in `CConnman::ThreadOpenAddedConnections`, and to report about manually added connections in
`getaddednodeinfo`. In both cases, all addresses added to `m_added_nodes` are returned, however
the nodes we are already connected to are only relevant to the latter, in the former they are
actively discarded.

Parametrizes `CConnman::GetAddedNodeInfo` so we can ask for only addresses we are not connected to,
to avoid passing useless information around.
2023-10-30 11:39:21 -04:00
fanquake
ec5116ae14
Merge bitcoin/bitcoin#28695: net: Sanity check private keys received from SAM proxy
5cf4d266d9 [test] Test i2p private key constraints (Vasil Dimov)
cf70a8d565 [net] Check i2p private key constraints (dergoegge)

Pull request description:

  Not sanity checking can lead to crashes or worse:

  ```
  ==1715589==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6140000055c2 at pc 0x5622ed66e7ad bp 0x7ffee547a2c0 sp 0x7ffee547a2b8
  READ of size 2 at 0x6140000055c2 thread T0 (b-test)
      #0 0x5622ed66e7ac in memcpy include/bits/string_fortified.h:29:10
      #1 0x5622ed66e7ac in i2p::sam::Session::MyDestination() const src/i2p.cpp:362:5
      #2 0x5622ed662e46 in i2p::sam::Session::CreateIfNotCreatedAlready() src/i2p.cpp:414:40
      #3 0x5622ed6619f2 in i2p::sam::Session::Listen(i2p::Connection&) src/i2p.cpp:143:9
  ```

ACKs for top commit:
  maflcko:
    code lgtm ACK 5cf4d266d9
  stickies-v:
    re-ACK 5cf4d266d9
  vasild:
    ACK 5cf4d266d9

Tree-SHA512: 3de3bd396538fa619de67957b9c8a58011ab911f0f51097c387e730c13908278b7322aa3357051fb245a20b15bef34b0e9fadcb1eff8ad751139d2aa634c78ad
2023-10-30 14:44:40 +01:00
Vasil Dimov
5cf4d266d9 [test] Test i2p private key constraints 2023-10-30 11:41:11 +00:00
MarcoFalke
faec889f93
refactor: Add LIFETIMEBOUND to all (w)txid getters
Then, use the compiler warnings to create copies only where needed.

Also, fix iwyu includes while touching the includes.
2023-10-27 13:01:42 +02:00