0
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-02-14 11:26:09 -05:00
Commit graph

1275 commits

Author SHA1 Message Date
glozow
0f8c95dccd
Merge bitcoin/bitcoin#27021: Implement Mini version of BlockAssembler to calculate mining scores
6b605b91c1 [fuzz] Add MiniMiner target + diff fuzz against BlockAssembler (glozow)
3f3f2d59ea [unit test] GatherClusters and MiniMiner unit tests (glozow)
59afcc8354 Implement Mini version of BlockAssembler to calculate mining scores (glozow)
56484f0fdc [mempool] find connected mempool entries with GatherClusters(…) (glozow)

Pull request description:

  Implement Mini version of BlockAssembler to calculate mining scores

  Run the mining algorithm on a subset of the mempool, only disturbing the
  mempool to copy out fee information for relevant entries. Intended to be
  used by wallet to calculate amounts needed for fee-bumping unconfirmed
  transactions.

  From comments of sipa and glozow below:

  > > In what way does the code added here differ from the real block assembly code?
  >
  >    * Only operates on the relevant transactions rather than full mempool
  >    * Has the ability to remove transactions that will be replaced so they don't impact their ancestors
  >    * Does not hold mempool lock outside of the constructor, makes copies of the entries it needs instead (though I'm not sure if this has an effect in practice)
  >    * Doesn't do the sanity checks like keeping weight within max block weight and `IsFinalTx()`
  >    * After the block template is built, additionally calculates fees to bump remaining ancestor packages to target feerate

ACKs for top commit:
  achow101:
    ACK 6b605b91c1
  Xekyo:
    > ACK [6b605b9](6b605b91c1) modulo `miniminer_overlap` test.
  furszy:
    ACK 6b605b91 modulo `miniminer_overlap` test.
  theStack:
    Code-review ACK 6b605b91c1

Tree-SHA512: f86a8b4ae0506858a7b15d90f417ebceea5038b395c05c825e3796123ad3b6cb8a98ebb948521316802a4c6d60ebd7041093356b1e2c2922a06b3b96b3b8acb6
2023-05-19 10:26:19 -04:00
Andrew Chow
3ff67f7783
Merge bitcoin/bitcoin#19690: util: improve FindByte() performance
72efc26439 util: improve streams.h:FindByte() performance (Larry Ruane)
604df63f6c [bench] add streams findbyte (gzhao408)

Pull request description:

  This PR is strictly a performance improvement; there is no functional change. The `CBufferedFile::FindByte()` method searches for the next occurrence of the given byte in the file. Currently, this is done by explicitly inspecting each byte in turn. This PR takes advantage of `std::find()` to do the same more efficiently, improving its CPU runtime by a factor of about 25 in typical use.

ACKs for top commit:
  achow101:
    re-ACK 72efc26439
  stickies-v:
    re-ACK 72efc26439

Tree-SHA512: ddf0bff335cc8aa34f911aa4e0558fa77ce35d963d602e4ab1c63090b4a386faf074548daf06ee829c7f2c760d06eed0125cf4c34e981c6129cea1804eb3b719
2023-05-10 17:50:42 -04:00
glozow
99f8046829 [rpc] add getprioritisedtransactions
This allows the user to see prioritisation for not-in-mempool
transactions.
2023-05-10 21:10:44 +01:00
MarcoFalke
fa422aeec2
scripted-diff: Use UniValue::find_value method
-BEGIN VERIFY SCRIPT-
 sed --regexp-extended -i 's/find_value\(([^ ,]+), /\1.find_value(/g' $(git grep -l find_value)
-END VERIFY SCRIPT-
2023-05-09 18:47:14 +02:00
fanquake
fc06881f13
Merge bitcoin/bitcoin#27491: refactor: Move chain constants to the util library
d168458d1f scripted-diff: Remove unused chainparamsbase includes (TheCharlatan)
e9ee8aaf3a Add missing definitions in prep for scripted diff (TheCharlatan)
ba8fc7d788 refactor: Replace string chain name constants with ChainTypes (TheCharlatan)
401453df41 refactor: Introduce ChainType getters for ArgsManager (TheCharlatan)
bfc21c31b2 refactor: Create chaintype files (TheCharlatan)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". It is also a follow up to #26177.

  It replaces pull request https://github.com/bitcoin/bitcoin/pull/27294, which just moved the constants to a new file, but did not re-declare them as enums.

  The code move of the chain name constants out of the `chainparamsbase` to their own separate header allows the kernel `chainparams` to no longer include `chainparamsbase`. The `chainparamsbase` contain references to the `ArgsManager` and networking related options that should not belong to the kernel library. Besides this move, the constants are re-declared as enums with helper functions facilitating string conversions.

ACKs for top commit:
  ryanofsky:
    Code review ACK d168458d1f. Just suggested changes since last review.

Tree-SHA512: ac2fbe5cbbab4f52eae1e30af1f16700b6589eb4764c328a151a712adfc37f326cc94a65c385534c57d4bc92cc1a13bf1777d92bc924a20dbb30440e7380b316
2023-05-09 15:42:21 +01:00
TheCharlatan
d168458d1f
scripted-diff: Remove unused chainparamsbase includes
This is a follow-up to previous commits moving the chain constants out
of chainparamsbase.

The script removes the chainparamsbase header in all files where it is
included, but not used. This is done by filtering against all defined
symbols of the header as well as its respective .cpp file.

The kernel chainparams now no longer relies on chainparamsbase.

-BEGIN VERIFY SCRIPT-
sed -i '/#include <chainparamsbase.h>/d' $( git grep -l 'chainparamsbase.h' | xargs grep -L 'CBaseChainParams\|CreateBaseChainParams\|SetupChainParamsBaseOptions\|BaseParams\|SelectBaseParams\|chainparamsbase.cpp' )
-END VERIFY SCRIPT-
2023-05-09 15:49:19 +02:00
TheCharlatan
ba8fc7d788
refactor: Replace string chain name constants with ChainTypes
This commit effectively moves the definition of these constants
out of the chainparamsbase to their own file.

Using the ChainType enums provides better type safety compared to
passing around strings.

The commit is part of an ongoing effort to decouple the libbitcoinkernel
library from the ArgsManager and other functionality that should not be
part of the kernel library.
2023-05-09 15:49:14 +02:00
Andrew Chow
fa53611cf1
Merge bitcoin/bitcoin#26076: Switch hardened derivation marker to h
fe49f06c0e doc: clarify PR 26076 release note (Sjors Provoost)
bd13dc2f46 Switch hardened derivation marker to h in descriptors (Sjors Provoost)

Pull request description:

  This makes it easier to handle descriptor strings manually, especially when importing from another Bitcoin Core wallet.

  For example the `importdescriptors` RPC call is easiest to use `h` as the marker: `'["desc": ".../0h/..."]'`, avoiding the need for escape characters. With this change `listdescriptors` will use `h`, so you can copy-paste the result, without having to add escape characters or switch `'` to 'h' manually.

  Both markers can still be parsed.

  The `hdkeypath` field in `getaddressinfo` is also impacted by this change, except for legacy wallets. The latter is to prevent accidentally breaking ancient software that uses our legacy wallet.

  See discussion in #15740

ACKs for top commit:
  achow101:
    ACK fe49f06c0e
  darosior:
    re-ACK fe49f06c0e

Tree-SHA512: f78bc873b24a6f7a2bf38f5dd58f2b723e35e6b10e4d65c36ec300e2d362d475eeca6e5afa04b3037ab4bee0bf8ebc93ea5fc18102a2111d3d88fc873c08dc89
2023-05-08 13:31:28 -04:00
fanquake
322ec63b01
Merge bitcoin/bitcoin#17860: fuzz: BIP 30, CVE-2018-17144
fa2d8b61f9 fuzz: BIP 42, BIP 30, CVE-2018-17144 (MarcoFalke)
faae7d5c00 Move LoadVerifyActivateChainstate to ChainTestingSetup (MarcoFalke)
fa26e3462a Avoid dereferencing interruption_point if it is nullptr (MarcoFalke)
fa846ee074 test: Add util to mine invalid blocks (MarcoFalke)

Pull request description:

  Add a validation fuzz test for BIP 30 and CVE-2018-17144

ACKs for top commit:
  dergoegge:
    Code review ACK fa2d8b61f9
  mzumsande:
    Tested ACK fa2d8b61f9

Tree-SHA512: 1f4620cc078709487abff24b304a6bb4eeab2e7628b392e2bc6de9cc0ce6745c413388ede6e93025d0c56eec905607ba9786633ef183e5779bf5183cc9ff92c0
2023-05-06 12:13:06 +01:00
fanquake
e460c0a24a
Merge bitcoin/bitcoin#27405: util: Use steady clock instead of system clock to measure durations
fa83fb3161 wallet: Use steady clock to calculate number of derive iterations (MarcoFalke)
fa2c099cec wallet: Use steady clock to measure scanning duration (MarcoFalke)
fa97621804 qt: Use steady clock to throttle GUI notifications (MarcoFalke)
fa1d8044ab test: Use steady clock in index tests (MarcoFalke)
fa454dcb20 net: Use steady clock in InterruptibleRecv (MarcoFalke)

Pull request description:

  `GetTimeMillis` has multiple issues:

  * It doesn't denote the underlying clock type
  * It isn't type-safe
  * It is used incorrectly in places that should use a steady clock

  Fix all issues here.

ACKs for top commit:
  willcl-ark:
    ACK fa83fb3161
  martinus:
    Code review ACK fa83fb3161, also ran all tests. All usages of the steady_clock are just for duration measurements, so the change to a different epoch is ok.

Tree-SHA512: 5ec4fede8c7f97e2e08863c011856e8304f16ba30a68fdeb42f96a50a04961092cbe46ccf9ea6ac99ff5203c09f9e0924eb483eb38d7df0759addc85116c8a9f
2023-05-06 12:03:50 +01:00
Larry Ruane
72efc26439 util: improve streams.h:FindByte() performance
Avoid use of the expensive mod operator (%) when calculating the
buffer offset. No functional difference.

Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2023-05-05 06:03:17 -06:00
MarcoFalke
fa2d8b61f9
fuzz: BIP 42, BIP 30, CVE-2018-17144 2023-05-05 13:31:01 +02:00
ishaanam
fb2a3a70e8 rpc: add descriptorprocesspsbt rpc
This RPC can be the Updater, Signer, and optionally the Input
Finalizer for a psbt, and has no interaction with the Bitcoin
Core wallet.
2023-05-04 13:06:39 -04:00
MarcoFalke
fa846ee074
test: Add util to mine invalid blocks
With the current utils it is only possible to mine valid blocks. This
commit adds new util methods to mine invalid blocks.
2023-05-02 17:17:06 +02:00
brunoerg
35a2175ad8 fuzz: addrman, add coverage for network field in Select(), Size() and GetAddr()
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-05-02 11:05:36 -03:00
fanquake
c63c8a1590
Merge bitcoin/bitcoin#27464: fuzz: re-enable prioritisetransaction & analyzepsbt RPC
faa7144d3c fuzz: re-enable prioritisetransaction & analyzepsbt RPC (MarcoFalke)

Pull request description:

  The linked issue seems fixed, so it should be fine to re-enable

ACKs for top commit:
  dergoegge:
    utACK faa7144d3c

Tree-SHA512: a681c726fceacc27ab5a03d455c7808d33f3cb11fe7d253d455526568af840b29f0c3c1d97c54785ef9277e7891a3aa742ac73ccd3cf115b7606eba50864aaa9
2023-04-21 11:53:23 +01:00
fanquake
669af32632
Merge bitcoin/bitcoin#27419: move-only: Extract common/args from util/system
be55f545d5 move-only: Extract common/args and common/config.cpp from util/system (TheCharlatan)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". It is part of a series of patches splitting up the `util/system` files. Its preceding pull request is https://github.com/bitcoin/bitcoin/pull/27254.

  The pull request contains an extraction of ArgsManager related functions from util/system into their own common/ file.

  The background of this commit is an ongoing effort to decouple the libbitcoinkernel library from the ArgsManager. The ArgsManager belongs into the common library, since the kernel library should not depend on it. See [doc/design/libraries.md](https://github.com/bitcoin/bitcoin/blob/master/doc/design/libraries.md) for more information on this rationale.

ACKs for top commit:
  MarcoFalke:
    re-ACK be55f545d5  🚲
  ryanofsky:
    Code review ACK be55f545d5. Just small cleanups since the last review.
  hebasto:
    ACK be55f545d5, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 90eb03334af0155b823030b4f2ecf286d35058d700ee2ddbbaa445be19e31eb0fe982656f35bd14ecee3ad2c3d0db3746855cb8f3777eff7253713e42873e111
2023-04-21 11:19:08 +01:00
Andrew Chow
5aa0c82ccd
Merge bitcoin/bitcoin#25325: Add pool based memory resource
9f947fc3d4 Use PoolAllocator for CCoinsMap (Martin Leitner-Ankerl)
5e4ac5abf5 Call ReallocateCache() on each Flush() (Martin Leitner-Ankerl)
1afca6b663 Add PoolResource fuzzer (Martin Leitner-Ankerl)
e19943f049 Calculate memory usage correctly for unordered_maps that use PoolAllocator (Martin Leitner-Ankerl)
b8401c3281 Add pool based memory resource & allocator (Martin Leitner-Ankerl)

Pull request description:

  A memory resource similar to `std::pmr::unsynchronized_pool_resource`, but optimized for node-based containers. The goal is to be able to cache more coins with the same memory usage, and allocate/deallocate faster.

  This is a reimplementation of #22702. The goal was to implement it in a way that is simpler to review & test

  * There is now a generic `PoolResource` for allocating/deallocating memory. This has practically the same API as `std::pmr::memory_resource`. (Unfortunately I cannot use std::pmr because libc++ simply doesn't implement that API).
  * Thanks to sipa there is now a fuzzer for PoolResource! On a fast machine I ran it for ~770 million executions without finding any issue.

  * The estimation of the correct node size is now gone, PoolResource now has multiple pools and just needs to be created large enough to have space for the unordered_map nodes.

  I ran benchmarks with #22702, mergebase, and this PR. Frequency locked Intel i7-8700, clang++ 13.0.1 to reindex up to block 690000.

  ```sh
  bitcoind -dbcache=5000 -assumevalid=00000000000000000002a23d6df20eecec15b21d32c75833cce28f113de888b7 -reindex-chainstate -printtoconsole=0 -stopatheight=690000
  ```

  The performance is practically identical with #22702, just 0.4% slower. It's ~21% faster than master:

  ![Progress in Million Transactions over Time(2)](https://user-images.githubusercontent.com/14386/173288685-91952ade-f304-4825-8bfb-0725a71ca17b.png)

  ![Size of Cache in MiB over Time](https://user-images.githubusercontent.com/14386/173291421-e6b410be-ac77-479b-ad24-5fafcebf81eb.png)
  Note that on cache drops mergebase's memory doesnt go so far down because it does not free the `CCoinsMap` bucket array.

  ![Size of Cache in Million tx over Time(1)](https://user-images.githubusercontent.com/14386/173288703-a80c9c9e-93c8-4a16-9df8-610c89c61cc4.png)

ACKs for top commit:
  LarryRuane:
    ACK 9f947fc3d4
  achow101:
    re-ACK 9f947fc3d4
  john-moffett:
    ACK 9f947fc3d4
  jonatack:
    re-ACK 9f947fc3d4

Tree-SHA512: 48caf57d1775875a612b54388ef64c53952cd48741cacfe20d89049f2fb35301b5c28e69264b7d659a3ca33d4c714d47bafad6fd547c4075f08b45acc87c0f45
2023-04-20 16:20:15 -04:00
TheCharlatan
be55f545d5
move-only: Extract common/args and common/config.cpp from util/system
This is an extraction of ArgsManager related functions from util/system
into their own common file.

Config file related functions are moved to common/config.cpp.

The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager. The ArgsManager belongs
into the common library, since the kernel library should not depend on
it. See doc/design/libraries.md for more information on this rationale.
2023-04-19 10:48:30 +02:00
MarcoFalke
faa7144d3c
fuzz: re-enable prioritisetransaction & analyzepsbt RPC 2023-04-14 17:34:22 +02:00
fanquake
c17d4d3b6b
Merge bitcoin/bitcoin#26662: fuzz: Add HeadersSyncState target
3153e7d779 [fuzz] Add HeadersSyncState target (dergoegge)
53552affca [headerssync] Make m_commit_offset protected (dergoegge)

Pull request description:

  This adds a fuzz target for the `HeadersSyncState` class.

  I am unsure how well this is able to cover the logic since it is just processing unserialized CBlockHeaders straight from the fuzz input (headers are sometimes made continuous). However, it does manage to get to the redownload phase so i thought it is better then not having fuzzing at all.

  It would also be nice to fuzz the p2p logic that is using `HeadersSyncState` (e.g. `TryLowWorkHeadersSync`, `IsContinuationOfLowWorkHeadersSync`) but that likely requires some more work (refactoring👻).

ACKs for top commit:
  mzumsande:
    ACK 3153e7d779

Tree-SHA512: 8a4630ceeeb30e4eeabaa8eb5491d98f0bf900efe7cda07384eaac9f2afaccfbcaa979cc1cc7f0b6ca297a8f5c17a7759f94809dd87eb87d35348d847c83e8ab
2023-04-11 16:17:04 +01:00
glozow
6b605b91c1
[fuzz] Add MiniMiner target + diff fuzz against BlockAssembler
Co-authored-by: dergoegge <n.goeggi@gmail.com>
Co-authored-by: mzumsande <mzumsande@gmail.com>
Co-authored-by: Murch <murch@murch.one>
2023-04-06 15:15:39 -04:00
fanquake
a56c96507a
ci: use clang-16 in tidy task 2023-04-05 11:43:42 +01:00
Sjors Provoost
bd13dc2f46
Switch hardened derivation marker to h in descriptors
This makes it easier to handle descriptor strings manually. E.g. an RPC call that takes an array of descriptors can now use '["desc": ".../0h/..."]'.

Both markers can still be parsed. The default for new descriptors is changed to h. In normalized form h is also used. For private keys the chosen marker is preserved in a round trip.

The hdkeypath field in getaddressinfo is also impacted by this change.
2023-04-04 18:33:08 +02:00
MarcoFalke
fa454dcb20
net: Use steady clock in InterruptibleRecv 2023-04-04 12:33:49 +02:00
fanquake
369d4c03b7
Merge bitcoin/bitcoin#27254: refactor: Extract util/fs from util/system
00e9b97f37 refactor: Move fs.* to util/fs.* (TheCharlatan)
106b46d9d2 Add missing fs.h includes (TheCharlatan)
b202b3dd63 Add missing cstddef include in assumptions.h (TheCharlatan)
18fb36367a refactor: Extract util/fs_helpers from util/system (Ben Woosley)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". This commit was originally authored by empact and is taken from its parent PR #25152.

  #### Context

  There is an ongoing effort to decouple the `ArgsManager` used for command line parsing user-provided arguments from the libbitcoinkernel library (https://github.com/bitcoin/bitcoin/pull/25290, https://github.com/bitcoin/bitcoin/pull/25487, https://github.com/bitcoin/bitcoin/pull/25527, https://github.com/bitcoin/bitcoin/pull/25862, https://github.com/bitcoin/bitcoin/pull/26177, and https://github.com/bitcoin/bitcoin/pull/27125). The `ArgsManager` is defined in `system.h`. A similar pull request extracting functionality from `system.h` has been merged in https://github.com/bitcoin/bitcoin/pull/27238.

  #### Changes

  Next to providing better code organization, this PR removes some reliance of the tree of libbitcoinkernel header includes on `system.h` (and thus the `ArgsManager` definition) by moving filesystem related functions out of the `system.*` files.

  There is already a pair of `fs.h` / `fs.cpp` in the top-level `src/` directory. They were not combined with the files introduced here, to keep the patch cleaner and more importantly because they are often included without the utility functions. The new files are therefore named `fs_helpers` and the existing `fs` files are moved into the util directory.

  Further commits splitting more functionality out of `system.h` are still in #25152 and will be submitted in separate PRs once this PR has been processed.

ACKs for top commit:
  hebasto:
    ACK 00e9b97f37

Tree-SHA512: 31422f148d14ba3c843b99b1550a6fd77c77f350905ca324f93d4f97b652246bc58fa9696c64d1201979cf88733e40be02d262739bb7d417cf22bf506fdb7666
2023-04-03 14:41:22 +01:00
fanquake
8d31d769b7
Merge bitcoin/bitcoin#27344: fuzz: Remove legacy int parse fuzz tests
faf8dc496e fuzz: Remove legacy int parse fuzz tests (MarcoFalke)

Pull request description:

  The fuzz tests checked that the result of the new function was equal to the legacy function. (Side note: The checks were incomplete, as evident by the follow-up fix in commit b5c9bb5cb9).

  Given that they haven't found any issues in years (beside missing the above issue, that they couldn't catch), it seems time to remove them.

  They may come in handy in the rare case that someone would want to modify `LocaleIndependentAtoi()` or `Parse*Int*()`, however that seems unlikely. Also, appropriate checks can be added then.

ACKs for top commit:
  fanquake:
    ACK faf8dc496e
  dergoegge:
    ACK faf8dc496e

Tree-SHA512: 4ec88b9fa8ba49a923b0604016f0f471b3c9b9e0ba6c5c3dc4e20503c6994789921e7221d9ec467a2a37a73f21a70ba51ba3370ed5ad311dee989e218290b29a
2023-03-28 12:03:39 +01:00
fanquake
d254f942a5
Merge bitcoin/bitcoin#27324: net: #27257 follow-ups
cd0c8eeb09 [net] Pass nRecvFloodSize to CNode (dergoegge)
860402ef2e [net] Remove trivial GetConnectionType() getter (dergoegge)
b5a85b365a [net] Delete CNetMessage copy constructor/assignment op (dergoegge)

Pull request description:

  Follow-up PR for #27257

  * Deletes the copy constructor/assignment operator of `CNetMessage`
  * Removes trivial getter for the connection type
  * Avoids passing `nRecvFloodSize` to CNode methods by passing it to `CNode` on creation

ACKs for top commit:
  jnewbery:
    utACK cd0c8eeb09
  theStack:
    ACK cd0c8eeb09

Tree-SHA512: 673a758668617f69fba77e61f0eaa1538da27a4849c82c98742436692baa2d7f001129af3e7a66b160e599d12109dac08137a146f10ff9b9ebdc5c2237311d41
2023-03-28 11:48:02 +01:00
MarcoFalke
faf8dc496e
fuzz: Remove legacy int parse fuzz tests 2023-03-27 16:37:31 +02:00
dergoegge
cd0c8eeb09 [net] Pass nRecvFloodSize to CNode 2023-03-27 16:00:02 +02:00
Hennadii Stepanov
7e975e6cf8
clang-tidy: Add performance-inefficient-vector-operation check
https://clang.llvm.org/extra/clang-tidy/checks/performance/inefficient-vector-operation.html
2023-03-26 20:17:55 +01:00
Martin Leitner-Ankerl
9f947fc3d4 Use PoolAllocator for CCoinsMap
In my benchmarks, using this pool allocator for CCoinsMap gives about
20% faster `-reindex-chainstate` with -dbcache=5000 with practically the
same memory usage. The change in max RSS changed was 0.3%.

The `validation_flush_tests` tests need to be updated because
memory allocation is now done in large pools instead of one node at a
time, so the limits need to be updated accordingly.
2023-03-23 19:38:38 +01:00
Martin Leitner-Ankerl
1afca6b663 Add PoolResource fuzzer
Fuzzes PoolResource with random allocations/deallocations, and multiple
asserts.

Co-Authored-By: Pieter Wuille <pieter@wuille.net>
2023-03-23 19:38:38 +01:00
stickies-v
6c8bde6d54
test: move coverage on ParseNonRFCJSONValue() to UniValue::read()
Preparation to deprecate ParseNonRFCJSONValue() but keep test coverage
on the underlying UniValue::read() unaffected. The test coverage on
AmountFromValue is no longer included, since that is already tested
in the rpc_parse_monetary_values test case.

Fuzzing coverage on ParseNonRFCJSONValue() was duplicated between string.cpp
and parse_univalue.cpp, only the one in parse_univalue.cpp is kept.
2023-03-23 18:18:46 +00:00
TheCharlatan
00e9b97f37
refactor: Move fs.* to util/fs.*
The fs.* files are already part of the libbitcoin_util library. With the
introduction of the fs_helpers.* it makes sense to move fs.* into the
util/ directory as well.
2023-03-23 12:55:18 +01:00
fanquake
a70911492f
Merge bitcoin/bitcoin#26749: refactor: Use move semantics instead of custom swap functions
95ad70ab65 test: Default initialize `should_freeze` to `true` (Hennadii Stepanov)
cea50521fe refactor: Drop no longer used `swap` member functions (Hennadii Stepanov)
a87fb6bee5 clang-tidy: Fix modernize-use-default-member-init in `CScriptCheck` (Hennadii Stepanov)
b4bed5c1f9 refactor: Drop no longer used `CScriptCheck()` default constructor (Hennadii Stepanov)
d8427cc28e refactor: Use move semantics in `CCheckQueue::Loop` (Hennadii Stepanov)
9a0b524139 clang-tidy, test: Fix bugprone-use-after-move in `Correct_Queue_range()` (Hennadii Stepanov)
04831fee6d refactor: Make move semantics explicit for callers (Hennadii Stepanov)
6c2d5972f3 refactor: Use move semantics in `CCheckQueue::Add` (Hennadii Stepanov)
0682003214 test, refactor: Avoid `CScriptCheck::swap` in `transaction_tests` (Hennadii Stepanov)
15209d97c6 consensus, refactor: Avoid `CScriptCheck::swap` in `CheckInputScripts` (Hennadii Stepanov)

Pull request description:

  This PR makes code more succinct and readable by using move semantics.

ACKs for top commit:
  martinus:
    re-ACK 95ad70ab65
  achow101:
    ACK 95ad70ab65
  TheCharlatan:
    re-ACK 95ad70ab65
  MarcoFalke:
    re-ACK 95ad70ab65 🚥

Tree-SHA512: adda760891b12d252dc9b823fe7c41eed660364b6fb1a69f17607d7a31eb0bbb82a80d154a7acfaa241b5de37d42a293c2b6e059f26a8e92d88d3a87c99768fb
2023-03-22 11:16:56 +00:00
Hennadii Stepanov
d8427cc28e
refactor: Use move semantics in CCheckQueue::Loop
Co-authored-by: Martin Leitner-Ankerl <martin.ankerl@gmail.com>
2023-03-21 13:04:21 +00:00
Hennadii Stepanov
04831fee6d
refactor: Make move semantics explicit for callers 2023-03-21 13:04:01 +00:00
Hennadii Stepanov
6c2d5972f3
refactor: Use move semantics in CCheckQueue::Add
Co-authored-by: Martin Leitner-Ankerl <martin.ankerl@gmail.com>
2023-03-21 13:03:41 +00:00
MarcoFalke
fa67b8181c
Refactor: Remove unused FlatFilePos::SetNull 2023-03-21 13:54:11 +01:00
hernanmarino
987f1bb41c Fixed a couple of typos in comments to make linter happy 2023-03-03 19:06:02 -03:00
Pieter Wuille
56e37e71a2 Make miniscript fuzzers avoid script size limit
Use the same technique as is using in the FromString miniscript parser to
predict the final script size of the miniscript being generated in the
miniscript_stable and miniscript_smart fuzzers (by counting every unexplored
sub node as 1 script byte, which is possible because every leaf node always
adds at least 1 byte). This allows bailing out early if the script being
generated would exceed the maximum allowed size (before actually constructing
the miniscript, as that may happen only significantly later potentially).

Also add a self-check to make sure this predicted script size matches that
of generated scripts.
2023-02-28 09:42:33 -05:00
Pieter Wuille
bcec5ab4ff Make miniscript fuzzers avoid ops limit
Keep track of the total number of ops the constructed script will have
during miniscript_stable and miniscript_smart fuzzers' GenNode, so it
can abort early if the 201 ops limit would be exceeded.

Also add a self-check that the final constructed node has the predicted
ops size limit, so we know the fuzzer's logic for keeping track of this
is correct.
2023-02-28 09:22:42 -05:00
Pieter Wuille
213fffa513 Enforce type consistency in miniscript_stable fuzz test
Add a self-check to the fuzzer that the constructed types match the expected
types in the miniscript_stable fuzzer too.
2023-02-28 09:22:42 -05:00
Pieter Wuille
e1f30414c6 Simplify miniscript fuzzer NodeInfo struct
Since we now keep track of all expected child node types (even if rudimentary)
in both miniscript_stable and miniscript_smart fuzzers, there is no need anymore
for the former shortcut NodeInfo constructors without sub types.
2023-02-28 09:22:42 -05:00
Pieter Wuille
5abb0f5ac3 Do base type propagation in miniscript_stable fuzzer
Keep track of which base type (B, K, V, or W) is desired in the miniscript_stable
ConsumeStableNode function. This allows aborting early if the constructed node
won't have the right type.

Note that this does not change the fuzzer format; the meaning of inputs in
ConsumeStableNode is unmodified. The only change is that often the fuzzer will
abort early.

The direct motivation is preventing recursing v: wrappers, which are the only
fragment type that does not otherwise increase the overall minimum possible script
size. In a later commit this will be exploited to prevent overly-large scripts from
being constructed.
2023-02-28 09:22:42 -05:00
fanquake
8b4dc94734
Merge bitcoin/bitcoin#27117: fuzz: avoid redundant dup key checks when creating Miniscript nodes
c1b7bd047f fuzz: avoid redundant dup key checks when creating Miniscript nodes (Antoine Poinsot)

Pull request description:

  I thought i had done that already in #24149, but it must have slipped through the rebase. It's a 2x speed improvement against the existing corpora and will probably be much more as we extend them with larger nodes.

ACKs for top commit:
  sipa:
    ACK c1b7bd047f

Tree-SHA512: 9e6ceb6254183964b6c5538e21ba6321df95a68acb343a15a6ecfef5c51a1980d2627df5aeef9aef1db41656e18cc4f3bc96e6f24314d12fa60368b04a350001
2023-02-22 09:37:07 +00:00
Andrew Chow
35fbc97208
Merge bitcoin/bitcoin#25619: net: avoid overriding non-virtual ToString() in CService and use better naming
c9d548c91f net: remove CService::ToStringPort() (Vasil Dimov)
fd4f0f41e9 gui: simplify OptionsDialog::updateDefaultProxyNets() (Vasil Dimov)
96c791dd20 net: remove CService::ToString() use ToStringAddrPort() instead (Vasil Dimov)
944a9de08a net: remove CNetAddr::ToString() and use ToStringAddr() instead (Vasil Dimov)
043b9de59a scripted-diff: rename ToStringIP[Port]() to ToStringAddr[Port]() (Vasil Dimov)

Pull request description:

  Before this PR we had the somewhat confusing combination of methods:

  `CNetAddr::ToStringIP()`
  `CNetAddr::ToString()` (duplicate of the above)
  `CService::ToStringIPPort()`
  `CService::ToString()` (duplicate of the above, overrides a non-virtual method from `CNetAddr`)
  `CService::ToStringPort()`

  Avoid [overriding non-virtual methods](https://github.com/bitcoin/bitcoin/pull/25349/#issuecomment-1185226396).

  "IP" stands for "Internet Protocol" and while sometimes "IP addresses" are called just "IPs", it is incorrect to call Tor or I2P addresses "IPs". Thus use "Addr" instead of "IP".

  Change the above to:

  `CNetAddr::ToStringAddr()`
  `CService::ToStringAddrPort()`

  The changes touch a lot of files, but are mostly mechanical.

ACKs for top commit:
  sipa:
    utACK c9d548c91f
  achow101:
    ACK c9d548c91f
  jonatack:
    re-ACK c9d548c91f only change since my previous reviews is rebase, but as a sanity check rebased to current master and at each commit quickly re-reviewed and re-verified clean build and green unit tests
  LarryRuane:
    ACK c9d548c91f

Tree-SHA512: 633fb044bdecf9f551b5e3314c385bf10e2b78e8027dc51ec324b66b018da35e5b01f3fbe6295bbc455ea1bcd1a3629de1918d28de510693afaf6a52693f2157
2023-02-17 13:34:40 -05:00
Antoine Poinsot
c1b7bd047f
fuzz: avoid redundant dup key checks when creating Miniscript nodes
Check it only once on the top level node.

Running libfuzzer with -runs=0 against the qa-assets corpus (1b9ddc96586769d92b1b62775f397b7f1a63f142).
Without this patch:
	miniscript_stable: Done 6616 runs in 118 second(s)
	miniscript_smart: Done 13182 runs in 253 second(s)
With this patch:
	miniscript_stable: Done 6616 runs in 57 second(s)
	miniscript_smart: Done 13182 runs in 124 second(s)
2023-02-17 12:41:04 +01:00
fanquake
fb82d91a9c
Merge bitcoin/bitcoin#24149: Signing support for Miniscript Descriptors
6c7a17a8e0 psbt: support externally provided preimages for Miniscript satisfaction (Antoine Poinsot)
840a396029 qa: add a "smart" Miniscript fuzz target (Antoine Poinsot)
17e3547241 qa: add a fuzz target generating random nodes from a binary encoding (Antoine Poinsot)
611e12502a qa: functional test Miniscript signing with key and timelocks (Antoine Poinsot)
d57b7f2021 refactor: make descriptors in Miniscript functional test more readable (Antoine Poinsot)
0a8fc9e200 wallet: check solvability using descriptor in AvailableCoins (Antoine Poinsot)
560e62b1e2 script/sign: signing support for Miniscripts with hash preimage challenges (Antoine Poinsot)
a2f81b6a8f script/sign: signing support for Miniscript with timelocks (Antoine Poinsot)
61c6d1a844 script/sign: basic signing support for Miniscript descriptors (Antoine Poinsot)
4242c1c521 Align 'e' property of or_d and andor with website spec (Pieter Wuille)
f5deb41780 Various additional explanations of the satisfaction logic from Pieter (Pieter Wuille)
22c5b00345 miniscript: satisfaction support (Antoine Poinsot)

Pull request description:

  This makes the Miniscript descriptors solvable.

  Note this introduces signing support for much more complex scripts than the wallet was previously able to solve, and the whole tooling isn't provided for a complete Miniscript integration in the wallet. Particularly, the PSBT<->Miniscript integration isn't entirely covered in this PR.

ACKs for top commit:
  achow101:
    ACK 6c7a17a8e0
  sipa:
    utACK 6c7a17a8e0 (to the extent that it's not my own code).

Tree-SHA512: a71ec002aaf66bd429012caa338fc58384067bcd2f453a46e21d381ed1bacc8e57afb9db57c0fb4bf40de43b30808815e9ebc0ae1fbd9e61df0e7b91a17771cc
2023-02-16 10:01:33 +00:00