0
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-02-24 12:41:41 -05:00
Commit graph

5476 commits

Author SHA1 Message Date
Pieter Wuille
2965fbf203 clusterlin: track upper bound potential set for work items (optimization)
In each work item, keep track of a conservative overestimate of the best
possible feerate that can be reached from it, and then use these to avoid
exploring hopeless work items.
2024-09-12 15:15:36 -04:00
Pieter Wuille
b80e6dfe78 clusterlin: add reordering support for DepGraph
Add a DepGraph(depgraph, reordering) function that constructs a new DepGraph
corresponding to an old one, but with its transactions is a modified order
(given as a vector from old to new positions).

Also use this reordering feature inside DepGraphFormatter::Unser, which needs
a small modification so that its reordering mapping is old-to-new (rather than
the new-to-old it used before).
2024-09-12 15:15:36 -04:00
Pieter Wuille
85a285a306 clusterlin: separate initial search entries per component (optimization)
Before this commit, the worst case for linearization involves clusters which
break apart in several smaller components after the first candidate is
included in the output linearization.

Address this by never considering work items that span multiple components
of what remains of the cluster.
2024-09-12 15:15:36 -04:00
Ryan Ofsky
e46bebb444
Merge bitcoin/bitcoin#30546: util: Use consteval checked format string in FatalErrorf, LogConnectFailure
fa5bc450d5 util: Use compile-time check for LogConnectFailure (MarcoFalke)
fa7087b896 util: Use compile-time check for FatalErrorf (MarcoFalke)
faa62c0112 util: Add ConstevalFormatString (MarcoFalke)
fae7b83eb5 lint: Remove forbidden functions from lint-format-strings.py (MarcoFalke)

Pull request description:

  The `test/lint/lint-format-strings.py` was designed to count the number of format specifiers and assert that they are equal to the number of parameters passed to the format function. The goal seems reasonable, but the implementation has many problems:

  * It is written in Python, meaning that C++ code can not be parsed correctly. Currently it relies on brittle regex and string parsing.
  * Apart from the parsing errors, there are also many logic errors. For example, `count_format_specifiers` allows a mix of positional specifiers and non-positional specifiers, which can lead to runtime format bugs. Also, `count_format_specifiers` silently skipped over "special" format specifiers, which are valid in tinyformat, which again can lead to runtime format bugs being undetected.
  * The brittle logic has a history of breaking in pull requests that are otherwise fine. This causes the CI to fail and the pull request being blocked from progress until the bug in the linter is fixed, or the code is rewritten to work around the bug.
  * It is only run in the CI, or when the developer invokes the script. It would be better if the developer got the error message at compile-time, directly when writing the code.

  Fix all issues by using a `consteval` checked format string in `FatalErrorf` and `LogConnectFailure`.

  This is the first step toward https://github.com/bitcoin/bitcoin/issues/30530 and a follow-up will apply the approach to the other places.

ACKs for top commit:
  stickies-v:
    re-ACK fa5bc450d5
  l0rinc:
    ACK fa5bc450d5
  hodlinator:
    ACK fa5bc450d5
  ryanofsky:
    Code review ACK fa5bc450d5

Tree-SHA512: d6189096b16083143687ed1b1559cf4f92f97dd87bc5d00673e44f4fb9fce7bb7b215cfdfc39b6e6a24f0b75a79a03ededce966639e554f7172e1fc22cf015ae
2024-09-12 13:21:53 -04:00
Ryan Ofsky
be768dbd18
Merge bitcoin/bitcoin#30618: test: support std::optional in BOOST_CHECK_* and increase FromUserHex fuzz feature coverage
1eac96a503 Compare FromUserHex result against other hex validators and parsers (Lőrinc)
19947863e1 Use BOOST_CHECK_EQUAL for optional, arith_uint256, uint256, uint160 (Lőrinc)
743ac30e34 Add std::optional support to Boost's equality check (Lőrinc)

Pull request description:

  Enhanced `FromUserHex` coverage by:

  * Added `std::optional` support to `BOOST_CHECK_EQUAL`, allowing direct comparisons of `std::optional<T>` with other `T` expected values.
  * Increased fuzz testing for hex parsing to validate against other hex validators and parsers.

  ----

  * Use BOOST_CHECK_EQUAL for https://github.com/bitcoin/bitcoin/pull/30569#discussion_r1706637780 arith_uint256, uint256, uint160

  Example error before:
  > unknown location:0: fatal error: in "validation_chainstatemanager_tests/chainstatemanager_args": std::bad_optional_access: bad_optional_access
  test/validation_chainstatemanager_tests.cpp:781: last checkpoint

  after:
  > test/validation_chainstatemanager_tests.cpp:801: error: in "validation_chainstatemanager_tests/chainstatemanager_args": check set_opts({"-assumevalid=0"}).assumed_valid_block == uint256::ZERO has failed [std::nullopt != 0000000000000000000000000000000000000000000000000000000000000000]

ACKs for top commit:
  stickies-v:
    re-ACK 1eac96a503
  ryanofsky:
    Code review ACK 1eac96a503. Only changes since last review were auto type and fuzz test tweaks.
  hodlinator:
    ACK 1eac96a503

Tree-SHA512: f1d2c65f0ee4e97830700be5b330189207b11ed0c89a8cebf0f97d43308402a6b3732e10130c79a0c044f7d2eeabfb5359990825aadf02c4ec19428dcd982b00
2024-09-12 12:36:37 -04:00
MarcoFalke
faa62c0112
util: Add ConstevalFormatString
The type is used to wrap a format string once it has been compile-time
checked to contain the right number of format specifiers.
2024-09-12 15:00:53 +02:00
merge-script
85833cf05f
Merge bitcoin/bitcoin#30847: test: Drop no longer needed workarounds
5c80192ff6 test: Drop no longer needed workarounds (Hennadii Stepanov)

Pull request description:

  This PR deletes the workarounds introduced in https://github.com/bitcoin/bitcoin/pull/16564 and https://github.com/bitcoin/bitcoin/pull/15382, as `ctest` skips these cases gracefully: 5c80192ff6/src/test/CMakeLists.txt (L201-L203)

ACKs for top commit:
  kevkevinpal:
    ACK [5c80192](5c80192ff6)
  fanquake:
    ACK 5c80192ff6. Looks correct:

Tree-SHA512: c47c606ecf7d64016b3c6353c3d4898350edc2caeac494dfd44484417f500a73f0c88c39f0f24651f3a02ef31ed9ca5c70d938bb9a8ca1eea54927e4d6a8fcd2
2024-09-12 11:28:27 +01:00
merge-script
db8350b0e3
Merge bitcoin/bitcoin#30803: build: Minor build system fixes and amendments
1cc93fe7b4 build: Delete dead code that implements `IF_CHECK_FAILED` option (Hennadii Stepanov)
341ad23809 build: Delete MSVC special case for `BUILD_FOR_FUZZING` option (Hennadii Stepanov)
fdad128b52 build: Stop enabling CMake's CMP0141 policy (Hennadii Stepanov)
b2a6f545b4 doc: Drop `ctest` command from Windows cross-compiling instructions (Hennadii Stepanov)
73b618582d build: Print `CMAKE_CXX_COMPILER_ARG1` in summary (Hennadii Stepanov)
f03c942095 build, test: Add missed log options (Hennadii Stepanov)
6f2cb0eafd doc: Amend comment about ZeroMQ config files (Hennadii Stepanov)

Pull request description:

  This PR addresses the following comments:
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1742342524
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1728692369
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1736110362
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1742931121
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1747723657
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1742328675
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1723106474

ACKs for top commit:
  sipsorcery:
    tACK 1cc93fe7b4 (win11 msvc).
  maflcko:
    re-ACK 1cc93fe7b4

Tree-SHA512: a390797bb4d3b7eb9163653b6c9c324e7a01090f6cdda74df7349a24a5c4a2084e5912878747f56561315afc70cae9adb1c363f47ceb0af96004ea591d25171b
2024-09-12 10:30:06 +01:00
Lőrinc
5e190cd11f Replace CScript _hex_v_u8 appends with _hex
This will skip vector conversion before serializing to the prevector in CScript.
2024-09-11 17:41:27 +02:00
Lőrinc
1eac96a503 Compare FromUserHex result against other hex validators and parsers 2024-09-11 15:41:15 +02:00
Lőrinc
19947863e1 Use BOOST_CHECK_EQUAL for optional, arith_uint256, uint256, uint160
Example error before:
> unknown location:0: fatal error: in "validation_chainstatemanager_tests/chainstatemanager_args": std::bad_optional_access: bad_optional_access
test/validation_chainstatemanager_tests.cpp:781: last checkpoint

after:
> test/validation_chainstatemanager_tests.cpp:801: error: in "validation_chainstatemanager_tests/chainstatemanager_args": check set_opts({"-assumevalid=0"}).assumed_valid_block == uint256::ZERO has failed [std::nullopt != 0000000000000000000000000000000000000000000000000000000000000000]

Also added extra minimum_chainwork test to make it symmetric with assumevalid

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
2024-09-11 15:41:15 +02:00
Ryan Ofsky
c66c68345e
Merge bitcoin/bitcoin#30773: Remove unsafe uint256S() and test-only uint160S()
43cd83b0c7 test: move uint256_tests/operator_with_self to arith_uint256_tests (stickies-v)
c6c994cb2b test: remove test-only uint160S (stickies-v)
62cc4656e2 test: remove test-only uint256S (stickies-v)
adc00ad728 test: remove test-only arith_uint256S (stickies-v)
f51b237723 refactor: rpc: use uint256::FromHex for ParseHashV (stickies-v)

Pull request description:

  _Continuation of #30569._

  Since fad2991ba0, `uint256S()` has been [deprecated](fad2991ba0 (diff-800776e2dda39116e889839f69409571a5d397de048a141da7e4003bc099e3e2R138)) because it is less robust than the `base_blob::FromHex()` introduced in https://github.com/bitcoin/bitcoin/pull/30482. Specifically, it tries to recover from length-mismatches, recover from untrimmed whitespace, 0x-prefix and garbage at the end, instead of simply requiring exactly 64 hex-only characters. (see also https://github.com/bitcoin/bitcoin/pull/30532)

  This PR removes `uint256S()` (and `uint160S()`) completely, with no non-test behaviour change.

  Specifically, the main changes in this PR are:
  - the (minimal) last non-test usage of `uint256S()` in `ParseHashV()` is removed without behaviour change, which can partially be verified by cherry-picking and/or modifying [this test commit](1f2b0fa86d)).
  - the test usage of `uint{160,256}S()` is removed, largely replacing it with `uint{160,256}::FromHex()` where applicable, potentially modifying the test by removing non-hex characters or dropping the test entirely if removing non-hex characters makes it redundant
  - the now unused `uint{160,256}S()` functions are removed completely.
  - unit test coverage on converting `uint256` <-> `arith_uint256` through `UintToArith256()` and `ArithToUint256()` is beefed up, and `arith_uint256` tests are moved to `arith_uint256_tests.cpp`, removing the `uint256_tests.cpp` dependency on `uint256h`, mirroring how the code is structured.

  _Note:  `uint256::FromUserHex()` exists to more leniently construct uint256 from user input, allowing "0x" prefixes and too-short-input, as safer alternative to `uint256S()` where necessary._

ACKs for top commit:
  l0rinc:
    reACK 43cd83b0c7
  hodlinator:
    re-ACK 43cd83b0c7
  ryanofsky:
    Code review ACK 43cd83b0c7. Only code change is a small refactoring which looks good. The rest of the PR is all test changes, which I only lightly reviewed, but seem to be positive and do what's described

Tree-SHA512: 48147a4c6af671597df0f72c1b477ae4631cd2cae4645ec54d0e327611ff302c9899e344518c81242cdde82930f6ad23a3a7e6e0b80671816e9f457b9de90a5c
2024-09-10 15:41:35 -04:00
brunoerg
829becd990 addrman: change Select to support multiple networks 2024-09-10 12:58:54 -03:00
marcofleon
a97f43d63a fuzz: Add harness for p2p headers sync 2024-09-10 11:56:07 +01:00
Ava Chow
df3f63ccfa
Merge bitcoin/bitcoin#30509: multiprocess: Add -ipcbind option to bitcoin-node
30073e6b3a multiprocess: Add -ipcbind option to bitcoin-node (Russell Yanofsky)
73fe7d7230 multiprocess: Add unit tests for connect, serve, and listen functions (Ryan Ofsky)
955d4077aa multiprocess: Add IPC connectAddress and listenAddress methods (Russell Yanofsky)
4da20434d4 depends: Update libmultiprocess library for CustomMessage function and ThreadContext bugfix (Ryan Ofsky)

Pull request description:

  Add `-ipcbind` option to `bitcoin-node` to make it listen on a unix socket and accept connections from other processes. The default socket path is `<datadir>/node.sock`, but this can be customized.

  This option lets potential wallet, gui, index, and mining processes connect to the node and control it. See examples in #19460, #19461, and #30437.

  Motivation for this PR, in combination with #30510, is be able to release a bitcoin core node binary that can generate block templates for a separate Stratum v2 mining service, like the one being implemented in https://github.com/Sjors/bitcoin/pull/48, that connects over IPC.

  Other things to know about this PR:

  - While the `-ipcbind` option lets other processes to connect to the `bitcoin-node` process, the only thing they can actually do after connecting is call methods on the [`Init`](https://github.com/bitcoin/bitcoin/blob/master/src/ipc/capnp/init.capnp#L17-L20) interface which is currently very limited and doesn't do much. But PRs [#30510](https://github.com/bitcoin/bitcoin/pull/30510), [#29409](https://github.com/bitcoin/bitcoin/pull/29409), and [#10102](https://github.com/bitcoin/bitcoin/pull/10102) expand the `Init` interface to expose mining, wallet, and gui functionality respectively.

  - This PR is not needed for [#10102](https://github.com/bitcoin/bitcoin/pull/10102), which runs GUI, node, and wallet code in different processes, because [#10102](https://github.com/bitcoin/bitcoin/pull/10102) does not use unix sockets or allow outside processes to connect to existing processes. [#10102](https://github.com/bitcoin/bitcoin/pull/10102) lets parent and child processes communicate over internal socketpairs, not externally accessible sockets.

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).

ACKs for top commit:
  achow101:
    ACK 30073e6b3a
  TheCharlatan:
    Re-ACK 30073e6b3a
  itornaza:
    Code review ACK 30073e6b3a

Tree-SHA512: 2b766e60535f57352e8afda9c3748a32acb5a57b2827371b48ba865fa9aa1df00f340732654f2e300c6823dbc6f3e14377fca87e4e959e613fe85a6d2312d9c8
2024-09-09 17:14:15 -04:00
Lőrinc
743ac30e34 Add std::optional support to Boost's equality check
Also moved the operators to the bottom of the file since they're less important and to group them together.

Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: stickies-v <stickies-v@protonmail.com>
2024-09-09 21:29:44 +02:00
Hennadii Stepanov
5c80192ff6
test: Drop no longer needed workarounds
`ctest` skips "no test cases matching filter" tests gracefully.
2024-09-08 09:05:39 +01:00
Hennadii Stepanov
f03c942095
build, test: Add missed log options 2024-09-06 21:59:51 +01:00
stickies-v
43cd83b0c7
test: move uint256_tests/operator_with_self to arith_uint256_tests
move/formatting-only change.

These tests do not cover uint256, so move them to the appropriate
test suite. Additionally, apply clang-format suggestions.
2024-09-06 17:36:28 +02:00
stickies-v
c6c994cb2b
test: remove test-only uint160S
uint160S is a test-only function, and testing input that
is not allowed in uint160::FromHex() is superfluous.

Tests that can't use uint160::FromHex() because they use input
with non-hex digit characters are
a) modified by dropping the non-hex digit characters if that
provides useful test coverage.
b) dropped if the test without non-hex digit characters does
not provide useful test coverage, e.g. because it is now
duplicated.
2024-09-06 17:36:27 +02:00
stickies-v
62cc4656e2
test: remove test-only uint256S
uint256S was previously deprecated for being unsafe. All non-test
usage has already been removed in earlier commits.

1. Tests now use uint256::FromHex() or other constructors wherever
possible without further modification.
2. Tests that can't use uint256::FromHex() because they use input
with non-hex digit characters are
  a) modified by dropping the non-hex digit characters if that
     provides useful test coverage.
  b) dropped if the test without non-hex digit characters does
     not provide useful test coverage, e.g. because it is now
     duplicated.

Additionally, use BOOST_CHECK_EQUAL where relevant on touched lines
to make error messages more readable.
2024-09-06 17:36:18 +02:00
stickies-v
adc00ad728
test: remove test-only arith_uint256S
Tests that are solely testing constructing from a hex string
are dropped, others are modified to use a uint256 constructor
or the arith_uint256 uint64_t constructor.

Since an arith_uint256 can not be constructed from a string
directly, we need to ensure that test coverage on
UintToArith256(uint256::FromHex()) is not reduced.
uint256::FromHex() already has good test coverage, but
the test coverage on UintToArith256() and ArithToUint256()
is increased in this commit by upgrading the `conversion`
test case.

Moreover, since `uint256.h` does not have any dependencies
on `arith_uint256.h`, the conversion tests are moved to
`arith_uint256_tests.cpp` so the dependency can be cleaned
up entirely in a future commit.
2024-09-06 17:36:18 +02:00
MarcoFalke
fa9d7d5d20
test: Work around boost compilation error 2024-09-06 15:57:33 +02:00
Russell Yanofsky
30073e6b3a multiprocess: Add -ipcbind option to bitcoin-node
Add `-ipcbind` option to `bitcoin-node` to listen on an IPC socket and accept
connections from other processes. In the future, there will be an `-ipcconnect`
option added to `bitcoin-wallet` and `bitcoin-node` to allow wallet and gui
processes to connect to the node and access it.

Example usage:

    src/bitcoin-node -regtest -debug -ipcbind=unix
    src/bitcoin-wallet -regtest -ipcconnect=unix info
    src/bitcoin-gui -regtest -ipcconnect=unix
    src/bitcoin-mine -regtest -ipcconnect=unix
2024-09-06 09:08:10 -04:00
Ryan Ofsky
73fe7d7230 multiprocess: Add unit tests for connect, serve, and listen functions 2024-09-06 09:08:10 -04:00
merge-script
118b55c462
Merge bitcoin/bitcoin#30790: bench: Remove redundant logging benchmarks
fadbcd51fc bench: Remove redundant logging benchmarks (MarcoFalke)
fa8dd952e2 bench: Use LogInfo instead of the deprecated alias LogPrintf (MarcoFalke)

Pull request description:

  `LogPrint*ThreadNames` is redundant with `LogWith(out)ThreadNames`,
  because they all measure toggling the thread names (and check that it
  has no effect on performance).

  Fix it by removing the redundant ones. This also allows to drop a deprecated logging alias.

ACKs for top commit:
  stickies-v:
    ACK fadbcd51fc

Tree-SHA512: 4fe137f374aa4ee1aa0e1da4a1f9839c0e52c23dbb93198ecafee98de39d311cc47304bba4191f3807aa00c51b1eae543e3f270f03d341c84910e5e341a1d475
2024-09-06 09:50:19 +01:00
merge-script
c0cbe26a86
Merge bitcoin/bitcoin#30748: test: Pin and document TEST_DIR_PATH_ELEMENT, SeedRand::FIXED_SEED
fa84f9decd test: Pin and document TEST_DIR_PATH_ELEMENT (MarcoFalke)
2222f7a874 test: Rename SeedRand::SEED to FIXED_SEED for clarity (MarcoFalke)

Pull request description:

  Two small test changes:

  * A refactor to update the name and documentation around `SeedRand::FIXED_SEED`.
  * A change to extract and document `TEST_DIR_PATH_ELEMENT`, and to change its value to better match the `TMPDIR_PREFIX` in functional tests. The value previously included `PACKAGE_NAME`, which is cute, but doesn't explain why it was used (to include a space). So just use `test_common bitcoin` to achieve the same with less effort.

ACKs for top commit:
  hodlinator:
    ACK fa84f9decd
  ryanofsky:
    Code review ACK fa84f9decd

Tree-SHA512: eb35d6598bb08f9b996e3a4762d8f26b2441c0ca00780798e473015af735dfc9997120895a922b94d4b6ada45adadba4a686e9cf9c285ddf688848e764c64840
2024-09-06 09:42:02 +01:00
brunoerg
f482d0e366 fuzz: reduce number of iterations in crypto_aeadchacha20poly1305 target
By reducing the number of iterations we improve the performance of
this target and may increase coverage.
2024-09-05 17:56:54 -03:00
merge-script
6852d1d487
Merge bitcoin/bitcoin#30796: test: Use std::span and std::string_view for raw data
faecca9a85 test: Use span for raw data (MarcoFalke)
fac973647d test: Use string_view for json_tests (MarcoFalke)

Pull request description:

  The build system converts raw data into a C++ header file for tests.

  This change modernizes the code to use the convenience wrappers `std::span` and `std::string_view`, so that redundant copies can be avoided.

ACKs for top commit:
  fjahr:
    re-ACK faecca9a85
  TheCharlatan:
    ACK faecca9a85
  stickies-v:
    ACK faecca9a85
  hebasto:
    ACK faecca9a85, I have reviewed the code and the generated headers.

Tree-SHA512: 1f4951c54aff11ba27c41fb70f2821bdb79e06ca0abae734b970bd0d64dda9d8cced824a891fd51b3e9d4e5715ee9eb49ed5d369010a45eca7c3bec9f8641235
2024-09-05 13:46:22 +01:00
MarcoFalke
faecca9a85
test: Use span for raw data
This change allows to drop brittle sizeof calls in favor of the
std::span::size method.

Other improvements include:

* Use of a namespace to mark test and bench data
* Use of the modern std::byte
* Drop of a no longer used std::vector copy and the bench/data module
2024-09-05 10:57:19 +02:00
MarcoFalke
fadbcd51fc
bench: Remove redundant logging benchmarks
LogPrint*ThreadNames is redundant with LogWith(out)ThreadNames, because
they all measure toggling the thread names (and check that it has no
effect on performance).

This also allows to remove unused and deprecated macros.
2024-09-05 07:17:22 +02:00
Ava Chow
5373aa30e2
Merge bitcoin/bitcoin#30788: test: fixing failing system_tests/run_command under some Locales
ae48a22a3d test: fixing failing system_tests/run_command under some Locales (Jadi)

Pull request description:

  the run_command test under system_tests fails if the locale is anything
  other than English ones because results such as "No such file or directory"
  will be different under Non-English locales.

  On the old version, a `ls nonexistingfile` was used to generate the error
  output which is not ideal. In the current version we are using a Python one-liner
  to generate a non 0 zero return value and "err" on stderr and check the
  expected value against this.

  fixes https://github.com/bitcoin/bitcoin/issues/30608

ACKs for top commit:
  maflcko:
    review ACK ae48a22a3d
  achow101:
    ACK ae48a22a3d
  hebasto:
    ACK ae48a22a3d, tested on Ubuntu 24.04 by switching locale.

Tree-SHA512: af7522ddcd786fa4a6832c8336ca89d8ff05f49ff963cbe1a96653b0edf29e0f950a032f23d742b16b3895e90cf5117b5f6a95464268dec67039df166d7d8639
2024-09-04 15:28:41 -04:00
Sebastian Falbesoner
58499b00d0 refactor: move SignSignature helpers to test utils
These helpers haven't been used in production code since segwit was
merged more than eight years ago (see commit 605e8473, PR #8149),
so it seems appropriate to move them to the test utils module.

Can be reviewed via `--color-moved=dimmed-zebra`.
2024-09-04 21:06:09 +02:00
Ava Chow
3210d87dfc
Merge bitcoin/bitcoin#29043: fuzz: make FuzzedDataProvider usage deterministic
01960c53c7 fuzz: make FuzzedDataProvider usage deterministic (Martin Leitner-Ankerl)

Pull request description:

  There exist many usages of `fuzzed_data_provider` where it is evaluated directly in the function call.
  Unfortunately, [the order of evaluation of function arguments is unspecified](https://en.cppreference.com/w/cpp/language/eval_order), and a simple example shows that it can differ e.g. between clang++ and g++: https://godbolt.org/z/jooMezWWY

  When the evaluation order is not consistent, the same fuzzing/random input will produce different output, which is bad for coverage/reproducibility. This PR fixes all these cases I have found where unspecified evaluation order could be a problem.

  Finding these has been manual work; I grepped the sourcecode for these patterns, and looked at each usage individually. So there is a chance I missed some.

  * `fuzzed_data_provider`
  * `.Consume`
  * `>Consume`
  * `.rand`

  I first discovered this in https://github.com/bitcoin/bitcoin/pull/29013#discussion_r1420236394. Note that there is a possibility that due to this fix the evaluation order is now different in many cases than when the fuzzing corpus has been created. If that is the case, the fuzzing corpus will have worse coverage than before.

  Update: In list-initialization the order of evaluation is well defined, so e.g. usages in `initializer_list` or constructors that use `{...}` is ok.

ACKs for top commit:
  achow101:
    ACK 01960c53c7
  vasild:
    ACK 01960c53c7
  ismaelsadeeq:
    ACK 01960c53c7

Tree-SHA512: e56d087f6f4bf79c90b972a5f0c6908d1784b3cfbb8130b6b450d5ca7d116c5a791df506b869a23bce930b2a6977558e1fb5115bb4e061969cc40f568077a1ad
2024-09-04 15:04:53 -04:00
Ava Chow
81276540d3
Merge bitcoin/bitcoin#30148: cli: restrict multiple exclusive argument usage in bitcoin-cli
c8e6771af0 test: restrict multiple CLI arguments (naiyoma)
8838c4f171 common/args.h: automate check for multiple cli commands (naiyoma)

Pull request description:

  This PR is a continuation of the validation suggested [here](https://github.com/bitcoin/bitcoin/pull/27815) to ensure that only one Request Handler can be specified at a time.

ACKs for top commit:
  stratospher:
    reACK c8e6771.
  achow101:
    ACK c8e6771af0
  tdb3:
    cr re ACK c8e6771af0

Tree-SHA512: f4fe036fee342a54f1a7ac702ac35c40bf3d420fde6ab16313a75327292d5ee5c8ece1be9f852a13fcf73da1148b143b37b4894e294052abdde6eefb2e8c6f3f
2024-09-04 14:47:38 -04:00
glozow
f66011e88f
Merge bitcoin/bitcoin#30784: test: add check that too large txs aren't put into orphanage
66d13c8702 test: add check that large txs aren't put into orphanage (Sebastian Falbesoner)
ed7d224666 test: add `BulkTransaction` helper to unit test transaction utils (Sebastian Falbesoner)

Pull request description:

  This PR adds test coverage for the following check in `TxOrphanage::AddTx`, where large orphan txs are ignored in order to avoid memory exhaustion attacks:
  5abb9b1af4/src/txorphanage.cpp (L22-L34)
  Note that this code-path isn't reachable under normal circumstances, as txs larger than `MAX_STANDARD_TX_WEIGHT` are already rejected earlier in the course of doing the mempool standardness checks (see `MemPoolAccept::PreChecks` -> `IsStandardTx` -> `reason = "tx-size";`), so this is only relevant if tx standardness rules are disabled via `-acceptnonstdtxns=1`. The ignore path is checked ~~by asserting the debug log, which is ugly, but as far as I know there is currently no way to access the orphanage entries from the outside~~ via unit test that checks the return value of `AddTx`. As an alternative to adding test coverage, one might consider removing this check altogether (or replacing it with an `Assume`), as it's redundant as explained above.

ACKs for top commit:
  maflcko:
    review ACK 66d13c8702
  glozow:
    ACK 66d13c8702
  tdb3:
    re-ACK 66d13c8702

Tree-SHA512: 88e8254ab5fca70c387a5992649ea6a704a65162999be972cc86bd74fc26c5f0f1e13e04856708d07ad5524cb77c0918e19663db92b3593e842469dfe04af6a1
2024-09-04 10:20:33 -04:00
merge-script
ab317ad2ef
Merge bitcoin/bitcoin#30804: fuzz: Rename fuzz_seed_corpus to fuzz_corpora
8888beea8d scripted-diff: fuzz: Rename fuzz_seed_corpus to fuzz_corpora (MarcoFalke)

Pull request description:

  Now that cmake was a breaking change for all fuzz scripts, it seems fine to bundle it with another breaking change to rename the fuzz corpora directory, as discussed and approved in https://github.com/bitcoin-core/qa-assets/issues/200:

  * The word "seed" in the old name doesn't really apply. In reality it is a collection of fuzz input seeds, as well as fuzz inputs.
  * The rename will also allow in the future (when there is a need and desire) to provide a minimal set of possibly hand-crafted or otherwise non-fuzz-generated fuzz seed inputs to some fuzz targets (and possibly store them in a separate folder and validate that their format is still accurate and matches the fuzz target code).
  * Finally, "corpus" is renamed to corpora, to clarify that the folder holds the fuzz inputs for several fuzz targets.

ACKs for top commit:
  brunoerg:
    ACK 8888beea8d
  marcofleon:
    ACK 8888beea8d

Tree-SHA512: abc693ca5d946850f04b6349e2a98f8fbc2ba9991be5a025bc0f357e341cbe7510f2f5f0e47b997d07136736d818df361270f372b8fb70860995a0605ca81e4d
2024-09-04 14:04:27 +01:00
Jadi
ae48a22a3d test: fixing failing system_tests/run_command under some Locales
the run_command test under system_tests fails if the locale is anything
other than English ones because results such as "No such file or directory"
will be different under Non-English locales.

On the old version, a `ls nonexistingfile` was used to generate the error
output which is not ideal. In the current version we are using a Python one-liner
to generate a non 0 zero return value and "err" on stderr and check the
expected value against this.

fixes #30608
2024-09-04 09:30:05 +03:30
Sebastian Falbesoner
66d13c8702 test: add check that large txs aren't put into orphanage 2024-09-03 22:23:48 +02:00
Sebastian Falbesoner
ed7d224666 test: add BulkTransaction helper to unit test transaction utils
The padding method used matches the one used in MiniWallet,
`MiniWallet._bulk_tx`.
2024-09-03 22:20:01 +02:00
Ava Chow
d4b5553849
Merge bitcoin/bitcoin#30742: kernel: Use spans instead of vectors for passing block headers to validation functions
a2955f0979 validation: Use span for ImportBlocks paths (TheCharlatan)
20515ea3f5 validation: Use span for CalculateClaimedHeadersWork (TheCharlatan)
52575e96e7 validation: Use span for ProcessNewBlockHeaders (TheCharlatan)

Pull request description:

  Makes it friendlier for potential future users of the kernel library if they do not store the headers in a std::vector, but can guarantee contiguous memory.

  Take this opportunity to also change the argument of ImportBlocks previously taking a `std::vector` to a `std::span`.

ACKs for top commit:
  stickies-v:
    re-ACK a2955f0979 - no changes except further walking the ~file~ path of modernizing variable names.
  maflcko:
    ACK a2955f0979 🕑
  achow101:
    ACK a2955f0979
  danielabrozzoni:
    ACK a2955f0979

Tree-SHA512: 8b07f4ad26e270b65600d1968cd78847b85caca5bfbb83fd9860389f26656b1d9a40b85e0990339f50403d18cedcd2456990054f3b8b0bedce943e50222d2709
2024-09-03 15:40:40 -04:00
MarcoFalke
8888beea8d
scripted-diff: fuzz: Rename fuzz_seed_corpus to fuzz_corpora
-BEGIN VERIFY SCRIPT-
 ren() { sed -i "s:\<$1\>:$2:g" $( git grep -l "$1" ) ; }
 ren fuzz_seed_corpus     fuzz_corpora
 ren FUZZ_SEED_CORPUS_DIR FUZZ_CORPORA_DIR
-END VERIFY SCRIPT-
2024-09-03 20:40:35 +02:00
MarcoFalke
fac973647d
test: Use string_view for json_tests
This avoids a static constructor of the global std::string, and rules
out possibly expensive and implicit copies of the string completely.
2024-09-03 16:06:20 +02:00
Hennadii Stepanov
26c460aa8b
build: Fix linking for fuzz target when building with MSan 2024-09-02 23:18:16 +01:00
marcofleon
a0eaa4749f Add FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION in PoW check
To avoid PoW being a blocker for fuzz tests,
`FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is used in fuzz builds to
bypass the actual PoW validation in `CheckProofOfWork`. It's
replaced with a check on the last byte of the hash, which allows the
fuzzer to quickly generate (in)valid blocks by checking a single bit,
rather than performing the full PoW computation.

If PoW is the target of a fuzz test, then it should call
`CheckProofOfWorkImpl`.
2024-09-02 15:43:33 +01:00
merge-script
d4cc0c6845
Merge bitcoin/bitcoin#30750: scripted-diff: LogPrint -> LogDebug
fa09cb41f5 refactor: Remove unused LogPrint (MarcoFalke)
3333415890 scripted-diff: LogPrint -> LogDebug (MarcoFalke)

Pull request description:

  `LogPrint` has many issues:

  * It seems to indicate that something is being "printed", however config options such as `-printtoconsole` actually control what and where something is logged.
  * It does not mention the log severity (debug).
  * It is a deprecated alias for `LogDebug`, according to the dev notes.
  * It wastes review cycles, because reviewers sometimes point out that it is deprecated.
  * It makes the code inconsistent, when both are used, possibly even in lines right next to each other (like in `InitHTTPServer`)

  Fix all issues by removing the deprecated alias.

  I checked all conflicting pull requests and at the time of writing there are no conflicts, except in pull requests that are marked as draft, are yet unreviewed, or are blocked on feedback for other reasons. So I think it is fine to do now.

ACKs for top commit:
  stickies-v:
    ACK fa09cb41f5
  danielabrozzoni:
    utACK fa09cb41f5
  TheCharlatan:
    ACK fa09cb41f5

Tree-SHA512: 14270f4cfa3906025a0b994cbb5b2e3c8c2427c0beb19c717a505a2ccbfb1fd1ecf2fd03f6c52d22cde69a8d057e50d2207119fab2c2bc8228db3f10d4288d0f
2024-09-02 11:59:56 +01:00
merge-script
ef6f49ecaf
Merge bitcoin/bitcoin#30664: build: Remove Autotools-based build system
faa382ae76 ci, doc: Drop reference to `src/.bear-tidy-config` (Hennadii Stepanov)
d71ac76842 build: Remove Autotools-based build system (Hennadii Stepanov)
e268b48419 doc: Adjust `doc/design/libraries.md` (Hennadii Stepanov)
d209e4f156 doc: Drop mentions of `share/genbuild.sh` (Hennadii Stepanov)

Pull request description:

  This PR deletes the Autotools-based build system.

  The MSVC build system is deleted in https://github.com/bitcoin/bitcoin/pull/30731.

ACKs for top commit:
  maflcko:
    re-ACK faa382ae76 🍦
  TheCharlatan:
    ACK faa382ae76
  fanquake:
    ACK faa382ae76

Tree-SHA512: 53df977b5b199a1c38f7f61a042a62b24831c559ba65a461b4ac1c96a1a56e2dfd676df79f1358fd1cc1749ff27e7b548086157f337d4f596c1054cb3d2d5739
2024-09-02 11:39:56 +01:00
MarcoFalke
fa84f9decd
test: Pin and document TEST_DIR_PATH_ELEMENT 2024-09-02 09:28:52 +02:00
Ryan Ofsky
b52d547361
Merge bitcoin/bitcoin#30377: refactor: Replace ParseHex with consteval ""_hex literals
8756ccd712 scripted-diff: Replace ParseHex[<std::byte>]("str") -> "str"_hex[_u8] (Hodlinator)
9cb687351f refactor: Prepare for ParseHex -> ""_hex scripted-diff (Hodlinator)
50bc017040 refactor: Hand-replace some ParseHex -> ""_hex (Hodlinator)
5b74a849cf util: Add consteval ""_hex[_v][_u8] literals (l0rinc)
dc5f6f6812 test refactor: util_tests - parse_hex clean up (Hodlinator)
2b5e6eff36 refactor: Make XOnlyPubKey tolerate constexpr std::arrays (Hodlinator)
403d86f1cc refactor: vector -> span in CCrypter (Hodlinator)
bd0830bbd4 refactor: de-Hungarianize CCrypter (Hodlinator)
d99c816971 refactor: Improve CCrypter related lines (Hodlinator)
7e1d9a8468 refactor: Enforce lowercase hex digits for consteval uint256 (Hodlinator)

Pull request description:

  Motivation:
  * Validates and converts the hex string into bytes at compile time instead of at runtime like `ParseHex()`.
  * Eliminates runtime dependencies: https://github.com/bitcoin/bitcoin/pull/30377#issuecomment-2214432177, https://github.com/bitcoin/bitcoin/pull/30048#discussion_r1592108480
  * Has stricter requirements than `ParseHex()` (disallows whitespace and uppercase hex digits) and replaces it in a bunch of places.
  * Makes it possible to derive other compile time constants.
  * Minor: should shave off a few runtime CPU cycles.

  `""_hex` produces `std::array<std::byte>` as the momentum in the codebase is to use `std::byte` over `uint8_t`.

  Also makes `uint256` hex string constructor disallow uppercase hex digits. Discussed: https://github.com/bitcoin/bitcoin/pull/30560#discussion_r1701323070

  Surprisingly does not change the size of the Guix **bitcoind** binary (on x86_64-linux-gnu) by 1 single byte.

  Spawned already merged PRs: #30436, #30482, #30532, #30560.

ACKs for top commit:
  l0rinc:
    ACK 8756ccd712
  stickies-v:
    Rebase re-ACK 8756ccd712, no changes since a096215c9c71a2ec03e76f1fd0bcdda0727996e0
  ryanofsky:
    Code review ACK 8756ccd712, just rebasing since last review and taking advantage of CScript constructors in #29369, also tweaking a code comment

Tree-SHA512: 9b2011b7c37e0ef004c669f8601270a214b388916316458370f5902c79c2856790b1b2c7c123efa65decad04886ab5eff95644301e0d84358bb265cf1f8ec195
2024-08-31 10:18:00 -04:00
Hennadii Stepanov
d71ac76842
build: Remove Autotools-based build system 2024-08-30 21:31:39 +01:00