0
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-02-11 11:16:09 -05:00
Commit graph

5107 commits

Author SHA1 Message Date
Ryan Ofsky
e6794e475c serialization: Accept multiple parameters in ParamsStream constructor
Before this change it was possible but awkward to create ParamStream streams
with multiple parameter objects. After this change it is straightforward.

The change to support multiple parameters is implemented by letting
ParamsStream contain substream instances, instead of just references to
external substreams. So a side-effect of this change is that ParamStream can
now accept rvalue stream arguments and be easier to use in some other cases. A
test for rvalues is added in this commit, and some simplifications to non-test
code are made in the next commit.
2024-02-21 07:07:50 -05:00
fanquake
45b2a91897
Merge bitcoin/bitcoin#29404: refactor: bitcoin-config.h includes cleanup
9d1dbbd4ce scripted-diff: Fix bitcoin_config_h includes (TheCharlatan)

Pull request description:

  As mentioned in https://github.com/bitcoin/bitcoin/pull/26924#issuecomment-1403449932 and https://github.com/bitcoin/bitcoin/pull/29263#issuecomment-1922334399, it is currently not safe to remove `bitcoin-config.h` includes from headers because some unrelated file might be depending on it.

  See also #26972 for discussion.

  Solve this by including the file directly everywhere it's required, regardless of whether or not it's already included by another header.

  There should be no functional change here, but it will allow us to safely remove includes from headers in the future.

  ~I'm afraid it's a bit tedious to reproduce these commits, but it's reasonably straightforward:~
  Edit: See note below

  ```bash
  # All commands executed from the src/ subdir.

  # Collect all tokens from bitcoin-config.h.in
  # Isolate the tokens and remove blank lines
  # Replace newlines with | and remove the last trailing one
  # Collect all files which use these tokens
  # Filter out subprojects (proper forwarding can be verified from Makefiles)
  # Filter out .rc files
  # Save to a text file
  git grep -E -l `grep undef config/bitcoin-config.h.in | cut -d" " -f2 | grep -v '^$' | tr '\n' '|' | sed 's/|$//'` | grep -v -e "^leveldb/" -e "^secp256k1/" -e "^crc32c/" -e "^minisketch/" -e "^Makefile" -e "\.rc$" > files-with-config-include.txt

  # Find all files from the above list which don't include bitcoin-config.h
  git grep -L -E "config/bitcoin-config.h" -- `cat files-with-config-include.txt`

  # Include them manually with the exception of some files in crypto:
  # crypto/sha256_arm_shani.cpp crypto/sha256_avx2.cpp crypto/sha256_sse41.cpp crypto/sha256_x86_shani.cpp
  # These are exceptions which don't use bitcoin-config.h, rather the Makefile.am adds these cppflags manually.

  # Commit changes. This should match the first commit of this PR.

  # Use the same search as above to find all files which DON'T use any config tokens
  git grep -E -L `grep undef config/bitcoin-config.h.in | cut -d" " -f2 | grep -v '^$' | tr '\n' '|' | sed 's/|$//'` | grep -v -e "^leveldb/" -e "^secp256k1/" -e "^crc32c/" -e "^minisketch/" -e "^Makefile" -e "\.rc$" > files-without-config-include.txt

  # Manually remove the includes and commit changes. This should match the second commit of this PR.
  ```

  Edit: I'll keep this old description for posterity, but the manual approach has been replaced with a scripted diff from TheCharlatan

ACKs for top commit:
  maflcko:
    ACK 9d1dbbd4ce 🚪
  TheCharlatan:
    ACK 9d1dbbd4ce
  hebasto:
    ACK 9d1dbbd4ce, I have reviewed the code and it looks OK.
  fanquake:
    ACK 9d1dbbd4ce

Tree-SHA512: f11ddc4ae6a887f96b954a6b77f310558ddb271088a3fda3edc833669c4251b7f392515224bbb8e5f67eb2c799b4ffed3b07d96454e82ec635c686d0df545872
2024-02-20 13:07:48 +00:00
Pieter Wuille
6e873df347 serfloat: improve/simplify tests 2024-02-20 11:33:32 +00:00
Pieter Wuille
b45f1f5658 serfloat: do not test encode(bits)=bits anymore 2024-02-20 10:25:41 +00:00
TheCharlatan
d5228efb53
kernel: Remove dependency on CScheduler
By defining a virtual interface class for the scheduler client, users of
the kernel can now define their own event consuming infrastructure,
without having to spawn threads or rely on the scheduler design.

Removing CScheduler also allows removing the thread and
exception modules from the kernel library.
2024-02-16 17:12:52 +01:00
TheCharlatan
06069b3913
scripted-diff: Rename MainSignals to ValidationSignals
-BEGIN VERIFY SCRIPT-
s() { git grep -l "$1" src | xargs sed -i "s/$1/$2/g"; }

s 'CMainSignals'    'ValidationSignals'
s 'MainSignalsImpl' 'ValidationSignalsImpl'
-END VERIFY SCRIPT-
2024-02-15 14:45:51 +01:00
TheCharlatan
0d6d2b650d
scripted-diff: Rename SingleThreadedSchedulerClient to SerialTaskRunner
-BEGIN VERIFY SCRIPT-
s() { git grep -l "$1" src | (grep -v "$3" || cat;) | xargs sed -i "s/$1/$2/g"; }

s 'SingleThreadedSchedulerClient'   'SerialTaskRunner'  ''
s 'SinglethreadedSchedulerClient'   'SerialTaskRunner'  ''
s 'm_schedulerClient'               'm_task_runner'     ''
s 'AddToProcessQueue'               'insert'            ''
s 'EmptyQueue'                      'flush'             ''
s 'CallbacksPending'                'size'              'validation'
sed -i '109s/CallbacksPending/size/' src/validationinterface.cpp
-END VERIFY SCRIPT-

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2024-02-15 14:43:14 +01:00
TheCharlatan
4abde2c4e3
[refactor] Make MainSignals RAII styled 2024-02-15 14:43:12 +01:00
TheCharlatan
84f5c135b8
refactor: De-globalize g_signals 2024-02-15 14:37:01 +01:00
TheCharlatan
3fba3d5dee
[refactor] Make signals optional in mempool and chainman
This is done in preparation for the next two commits, where the
CMainSignals are de-globalized.

This avoids adding new constructor arguments to the ChainstateManager
and CTxMemPool classes over the next two commits.

This could also allow future tests that are only interested in the
internal behaviour of the classes to forgo instantiating the signals.
2024-02-15 13:28:45 +01:00
TheCharlatan
9d1dbbd4ce scripted-diff: Fix bitcoin_config_h includes
-BEGIN VERIFY SCRIPT-

regex_string='^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|CHAR_EQUALS_INT8|CLIENT_VERSION_BUILD|CLIENT_VERSION_IS_RELEASE|CLIENT_VERSION_MAJOR|CLIENT_VERSION_MINOR|COPYRIGHT_HOLDERS|COPYRIGHT_HOLDERS_FINAL|COPYRIGHT_HOLDERS_SUBSTITUTION|COPYRIGHT_YEAR|ENABLE_ARM_SHANI|ENABLE_AVX2|ENABLE_EXTERNAL_SIGNER|ENABLE_SSE41|ENABLE_TRACING|ENABLE_WALLET|ENABLE_X86_SHANI|ENABLE_ZMQ|HAVE_BOOST|HAVE_BUILTIN_CLZL|HAVE_BUILTIN_CLZLL|HAVE_BYTESWAP_H|HAVE_CLMUL|HAVE_CONSENSUS_LIB|HAVE_CXX20|HAVE_DECL_BE16TOH|HAVE_DECL_BE32TOH|HAVE_DECL_BE64TOH|HAVE_DECL_BSWAP_16|HAVE_DECL_BSWAP_32|HAVE_DECL_BSWAP_64|HAVE_DECL_FORK|HAVE_DECL_FREEIFADDRS|HAVE_DECL_GETIFADDRS|HAVE_DECL_HTOBE16|HAVE_DECL_HTOBE32|HAVE_DECL_HTOBE64|HAVE_DECL_HTOLE16|HAVE_DECL_HTOLE32|HAVE_DECL_HTOLE64|HAVE_DECL_LE16TOH|HAVE_DECL_LE32TOH|HAVE_DECL_LE64TOH|HAVE_DECL_PIPE2|HAVE_DECL_SETSID|HAVE_DECL_STRERROR_R|HAVE_DEFAULT_VISIBILITY_ATTRIBUTE|HAVE_DLFCN_H|HAVE_DLLEXPORT_ATTRIBUTE|HAVE_ENDIAN_H|HAVE_EVHTTP_CONNECTION_GET_PEER_CONST_CHAR|HAVE_FDATASYNC|HAVE_GETENTROPY_RAND|HAVE_GETRANDOM|HAVE_GMTIME_R|HAVE_INTTYPES_H|HAVE_LIBADVAPI32|HAVE_LIBCOMCTL32|HAVE_LIBCOMDLG32|HAVE_LIBGDI32|HAVE_LIBIPHLPAPI|HAVE_LIBKERNEL32|HAVE_LIBOLE32|HAVE_LIBOLEAUT32|HAVE_LIBSHELL32|HAVE_LIBSHLWAPI|HAVE_LIBUSER32|HAVE_LIBUUID|HAVE_LIBWINMM|HAVE_LIBWS2_32|HAVE_MALLOC_INFO|HAVE_MALLOPT_ARENA_MAX|HAVE_MINIUPNPC_MINIUPNPC_H|HAVE_MINIUPNPC_UPNPCOMMANDS_H|HAVE_MINIUPNPC_UPNPERRORS_H|HAVE_NATPMP_H|HAVE_O_CLOEXEC|HAVE_POSIX_FALLOCATE|HAVE_PTHREAD|HAVE_PTHREAD_PRIO_INHERIT|HAVE_STDINT_H|HAVE_STDIO_H|HAVE_STDLIB_H|HAVE_STRERROR_R|HAVE_STRINGS_H|HAVE_STRING_H|HAVE_STRONG_GETAUXVAL|HAVE_SYSCTL|HAVE_SYSCTL_ARND|HAVE_SYSTEM|HAVE_SYS_ENDIAN_H|HAVE_SYS_PRCTL_H|HAVE_SYS_RESOURCES_H|HAVE_SYS_SELECT_H|HAVE_SYS_STAT_H|HAVE_SYS_SYSCTL_H|HAVE_SYS_TYPES_H|HAVE_SYS_VMMETER_H|HAVE_THREAD_LOCAL|HAVE_TIMINGSAFE_BCMP|HAVE_UNISTD_H|HAVE_VM_VM_PARAM_H|LT_OBJDIR|PACKAGE_BUGREPORT|PACKAGE_NAME|PACKAGE_STRING|PACKAGE_TARNAME|PACKAGE_URL|PACKAGE_VERSION|PTHREAD_CREATE_JOINABLE|QT_QPA_PLATFORM_ANDROID|QT_QPA_PLATFORM_COCOA|QT_QPA_PLATFORM_MINIMAL|QT_QPA_PLATFORM_WINDOWS|QT_QPA_PLATFORM_XCB|QT_STATICPLUGIN|STDC_HEADERS|STRERROR_R_CHAR_P|USE_ASM|USE_BDB|USE_DBUS|USE_NATPMP|USE_QRCODE|USE_SQLITE|USE_UPNP|_FILE_OFFSET_BITS|_LARGE_FILES)'

exclusion_files=":(exclude)src/minisketch :(exclude)src/crc32c :(exclude)src/secp256k1 :(exclude)src/crypto/sha256_arm_shani.cpp :(exclude)src/crypto/sha256_avx2.cpp :(exclude)src/crypto/sha256_sse41.cpp :(exclude)src/crypto/sha256_x86_shani.cpp"

git grep --perl-regexp --files-with-matches "$regex_string" -- '*.cpp' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do line_number=$(awk -v my_file="$file" '/\/\/ file COPYING or https?:\/\/www.opensource.org\/licenses\/mit-license.php\./ {line = NR} /^\/\// && NR == line + 1 {while(getline && /^\/\//) line = NR} END {print line+1}' "$file"); sed -i "${line_number}i\\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done;

git grep --perl-regexp --files-with-matches "$regex_string" -- '*.h' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do sed -i "/#define.*_H/a \\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done;

for file in $(git grep --files-with-matches 'bitcoin-config.h' -- '*.cpp' '*.h' $exclusion_files); do if ! grep -q --perl-regexp "$regex_string" $file; then sed -i '/HAVE_CONFIG_H/{N;N;N;d;}' $file; fi; done;

-END VERIFY SCRIPT-

The first command creates a regular expression for matching all bitcoin-config.h symbols in the following form: ^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|...|_LARGE_FILES). It was generated with:
./autogen.sh && printf '^(?!//).*(%s)' $(awk '/^#undef/ {print $2}' src/config/bitcoin-config.h.in | paste -sd "|" -)

The second command holds a list of files and directories that should not be processed. These include subtree directories as well as some crypto files that already get their symbols through the makefile.

The third command checks for missing bitcoin-config headers in .cpp files and adds the header if it is missing.

The fourth command checks for missing bitcoin-config headers in .h files and adds the header if it is missing.

The fifth command checks for unneeded bitcoin-config headers in sources files and removes the header if it is unneeded.
2024-02-13 20:10:44 +00:00
fanquake
d7dabdbfcd
Merge bitcoin/bitcoin#29413: fuzz: increase length of string used for NetWhitelist{bind}Permissions::TryParse
864e2e9097 fuzz: increase length of string used for `NetWhitelist{bind}Permissions::TryParse` (brunoerg)

Pull request description:

  The string `s` represents the value from `-whitelist`/`-whitebind` (e.g. "bloom,forcerelay,noban@1.2.3.4:32") and it is used in `NetWhitelistPermissions::TryParse` and `NetWhitebindPermissions::TryParse`. However, a max length of 32 is not enough to cover a lot of cases. Even disconsidering the permissions, 32 would not be enough to cover a lot of addresses. This PR fixes it.

ACKs for top commit:
  maflcko:
    lgtm ACK 864e2e9097
  epiccurious:
    utACK 864e2e9097.
  vasild:
    ACK 864e2e9097

Tree-SHA512: 2b89031b9f2ea92d636f05fd167b1e5ac726742a7e7c1af8ddaeaf90236e659731aaa6b7c23f65ec16ce52ac1b9e68e7b16e23c59e355312d057e001976d172a
2024-02-13 11:47:10 -03:00
glozow
a9346421db [test] PackageV3Checks with inheritance violation in mempool ancestor 2024-02-12 14:47:12 +00:00
glozow
63b62e123e [doc] fix docs and comments from v3 2024-02-12 14:27:25 +00:00
Ava Chow
7143d43884
Merge bitcoin/bitcoin#28948: v3 transaction policy for anti-pinning
29029df5c7 [doc] v3 signaling in mempool-replacements.md (glozow)
e643ea795e [fuzz] v3 transactions and sigop-adjusted vsize (glozow)
1fd16b5c62 [functional test] v3 transaction submission (glozow)
27c8786ba9 test framework: Add and use option for tx-version in MiniWallet methods (MarcoFalke)
9a1fea55b2 [policy/validation] allow v3 transactions with certain restrictions (glozow)
eb8d5a2e7d [policy] add v3 policy rules (glozow)
9a29d470fb [rpc] return full string for package_msg and package-error (glozow)
158623b8e0 [refactor] change Workspace::m_conflicts and adjacent funcs/structs to use Txid (glozow)

Pull request description:

  See #27463 for overall package relay tracking.

  Delving Bitcoin discussion thread: https://delvingbitcoin.org/t/v3-transaction-policy-for-anti-pinning/340
  Delving Bitcoin discussion for LN usage: https://delvingbitcoin.org/t/lightning-transactions-with-v3-and-ephemeral-anchors/418

  Rationale:
  - There are various pinning problems with RBF and our general ancestor/descendant limits. These policies help mitigate many pinning attacks and make package RBF feasible (see #28984 which implements package RBF on top of this). I would focus the most here on Rule 3 pinning. [1][2]
  - Switching to a cluster-based mempool (see #27677 and #28676) requires the removal of CPFP carve out, which applications depend on. V3 + package RBF + ephemeral anchors + 1-parent-1-child package relay provides an intermediate solution.

  V3 policy is for "Priority Transactions." [3][4] It allows users to opt in to more restrictive topological limits for shared transactions, in exchange for the more robust fee-bumping abilities that offers. Even though we don't have cluster limits, we are able to treat these transactions as having as having a maximum cluster size of 2.

  Immediate benefits:

  - You can presign a transaction with 0 fees (not just 1sat/vB!) and add a fee-bump later.
  - Rule 3 pinning is reduced by a significant amount, since the attacker can only attach a maximum of 1000vB to your shared transaction.

  This also enables some other cool things (again see #27463 for overall roadmap):
  - Ephemeral Anchors
  - Package RBF for these 1-parent-1-child packages. That means e.g. a commitment tx + child can replace another commitment tx using the child's fees.
  - We can transition to a "single anchor" universe without worrying about package limit pinning. So current users of CPFP carve out would have something else to use.
  - We can switch to a cluster-based mempool [5] (#27677 #28676), which removes CPFP carve out [6].

  [1]: Original mailing list post and discussion about RBF pinning problems https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff, https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html
  [2]: A FAQ is "we need this for cluster mempool, but is this still necessary afterwards?" There are some pinning issues that are fixed here and not fully fixed in cluster mempool, so we will still want this or something similar afterward.
  [3]: Mailing list post for v3 https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html
  [4]: Original PR #25038 also contains a lot of the discussion
  [5]: https://delvingbitcoin.org/t/an-overview-of-the-cluster-mempool-proposal/393/7
  [6]: https://delvingbitcoin.org/t/an-overview-of-the-cluster-mempool-proposal/393#the-cpfp-carveout-rule-can-no-longer-be-supported-12

ACKs for top commit:
  sdaftuar:
    ACK 29029df5c7
  achow101:
    ACK 29029df5c7
  instagibbs:
    ACK 29029df5c7 modulo that

Tree-SHA512: 9664b078890cfdca2a146439f8835c9d9ab483f43b30af8c7cd6962f09aa557fb1ce7689d5e130a2ec142235dbc8f21213881baa75241c5881660f9008d68450
2024-02-09 23:37:57 -05:00
glozow
e643ea795e [fuzz] v3 transactions and sigop-adjusted vsize
Ensure we are checking sigop-adjusted virtual size by creating setups
and packages where sigop cost is larger than bip141 vsize.

Co-authored-by: Gregory Sanders <gsanders87@gmail.com>
2024-02-08 21:50:55 +00:00
glozow
9a1fea55b2 [policy/validation] allow v3 transactions with certain restrictions
Co-authored-by: Suhas Daftuar <sdaftuar@gmail.com>
2024-02-08 21:50:55 +00:00
glozow
eb8d5a2e7d [policy] add v3 policy rules
Co-authored-by: Suhas Daftuar <sdaftuar@gmail.com>
2024-02-08 21:50:55 +00:00
brunoerg
864e2e9097 fuzz: increase length of string used for NetWhitelist{bind}Permissions::TryParse 2024-02-08 15:09:03 -03:00
TheCharlatan
5ca9b24da1
test: Add makefile target for running unit tests
make check runs a bunch of other subtree tests that exercise code that
is hardly ever changed and have a comparatively long runtime. There
seems to be no target for running just the unit tests, so add one.
2024-02-03 17:59:43 +01:00
MarcoFalke
fad0fafd5a
refactor: Fix timedata includes 2024-02-01 13:52:05 +01:00
Ava Chow
aa9231fafe
Merge bitcoin/bitcoin#26859: fuzz: extend ConsumeNetAddr() to return I2P and CJDNS addresses
b851c5385d fuzz: extend ConsumeNetAddr() to return I2P and CJDNS addresses (Vasil Dimov)

Pull request description:

  In the process of doing so, refactor `ConsumeNetAddr()` to generate the addresses from IPv4, IPv6, Tor, I2P and CJDNS networks in the same way - by preparing some random stream and deserializing from it. Similar code was already found in `RandAddr()`.

ACKs for top commit:
  achow101:
    ACK b851c5385d
  mzumsande:
    ACK b851c5385d
  brunoerg:
    utACK b851c5385d

Tree-SHA512: 9905acff0e996f30ddac0c14e5ee9e1db926c7751472c06d6441111304242b563f7c942b162b209d80e8fb65a97249792eef9ae0a96100419565bf7f59f59676
2024-01-31 16:45:00 -05:00
Ava Chow
3c13f5d612
Merge bitcoin/bitcoin#28956: Nuke adjusted time from validation (attempt 2)
ff9039f6ea Remove GetAdjustedTime (dergoegge)

Pull request description:

  This picks up parts of #25908.

  The use of adjusted time is removed from validation code while the warning to users if their clock is out of sync with the rest of the network remains.

ACKs for top commit:
  naumenkogs:
    ACK ff9039f6ea
  achow101:
    ACK ff9039f6ea
  maflcko:
    lgtm ACK ff9039f6ea 🤽
  stickies-v:
    ACK ff9039f6ea

Tree-SHA512: d1f6b9445c236915503fd2ea828f0d3b92285a5dbc677b168453276115e349972edbad37194d8becd9136d8e7219b576af64ec51c72bdb1923e57e405c0483fc
2024-01-31 15:58:47 -05:00
Ava Chow
0b768746ef
Merge bitcoin/bitcoin#28170: p2p: adaptive connections services flags
27f260aa6e net: remove now unused global 'g_initial_block_download_completed' (furszy)
aff7d92b15 test: add coverage for peerman adaptive connections service flags (furszy)
6ed53602ac net: peer manager, dynamically adjust desirable services flag (furszy)
9f36e591c5 net: move state dependent peer services flags (furszy)
f9ac96b8d6 net: decouple state independent service flags from desirable ones (furszy)
97df4e3887 net: store best block tip time inside PeerManager (furszy)

Pull request description:

  Derived from #28120 discussion.

  By relocating the peer desirable services flags into the peer manager, we
  allow the connections acceptance process to handle post-IBD potential
  stalling scenarios.

  The peer manager will be able to dynamically adjust the services flags
  based on the node's proximity to the tip (back and forth). Allowing the node
  to recover from the following post-IBD scenario:
  Suppose the node has successfully synced the chain, but later experienced
  dropped connections and remained inactive for a duration longer than the limited
  peers threshold (the timeframe within which limited peers can provide blocks). In
  such cases, upon reconnecting to the network, the node might only establish
  connections with limited peers, filling up all available outbound slots. Resulting
  in an inability to synchronize the chain (because limited peers will not provide
  blocks older than the `NODE_NETWORK_LIMITED_MIN_BLOCKS` threshold).

ACKs for top commit:
  achow101:
    ACK 27f260aa6e
  vasild:
    ACK 27f260aa6e
  naumenkogs:
    ACK 27f260aa6e
  mzumsande:
    Light Code Review ACK 27f260aa6e
  andrewtoth:
    ACK 27f260aa6e

Tree-SHA512: 07befb9bcd0b60a4e7c45e4429c02e7b6c66244f0910f4b2ad97c9b98258b6f46c914660a717b5ed4ef4814d0dbfae6e18e6559fe9bec7d0fbc2034109200953
2024-01-31 11:44:41 -05:00
Ava Chow
2f218c664b
Merge bitcoin/bitcoin#28921: multiprocess: Add basic type conversion hooks
6acec6b9ff multiprocess: Add type conversion code for UniValue types (Ryan Ofsky)
0cc74fce72 multiprocess: Add type conversion code for serializable types (Ryan Ofsky)
4aaee23921 test: add ipc test to test multiprocess type conversion code (Ryan Ofsky)

Pull request description:

  Add type conversion hooks to allow `UniValue` objects, and objects that have `CDataStream` `Serialize` and `Unserialize` methods to be used as arguments and return values in Cap'nProto interface methods. Also add unit test to verify the hooks are working and data can be round-tripped correctly.

  The non-test code in this PR was previously part of #10102 and has been split off for easier review, but the test code is new.

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).

ACKs for top commit:
  achow101:
    ACK 6acec6b9ff
  dergoegge:
    reACK 6acec6b9ff

Tree-SHA512: 5d2cbc5215d488b876d34420adf91205dabf09b736183dcc85aa86255e3804c2bac5bab6792dacd585ef99a1d92cf29c8afb3eb65e4d953abc7ffe41994340c6
2024-01-23 16:22:29 -05:00
Ava Chow
874c8bdb9e
Merge bitcoin/bitcoin#29144: init: handle empty settings file gracefully
e9014042a6 settings: add auto-generated warning msg for editing the file manually (furszy)
966f5de99a init: improve corrupted/empty settings file error msg (furszy)

Pull request description:

  Small and simple issue reported [here](https://community.umbrel.com/t/bitcoin-docker-container-keeps-restarting/2144).

  Improving a confusing situation reported by users who did not understand why a
  settings parsing error occurred when the file was empty and did not know how to solve it.

  Empty setting file could be due (1) corruption or (2) an user manually cleaning up the file content.
  In both scenarios, the 'Unable to parse settings file' error does not help the user move forward.

ACKs for top commit:
  achow101:
    ACK e9014042a6
  hebasto:
    re-ACK e9014042a6.
  ryanofsky:
    Code review ACK e9014042a6. Just whitespace formatting changes and shortening a test string literal since last review
  shaavan:
    Code review ACK e9014042a6

Tree-SHA512: 2910654c6b9e9112de391eedb8e46980280f822fa3059724dd278db7436804dd27fae628d2003f2c6ac1599b07ac5c589af016be693486e949f558515e662bec
2024-01-23 15:14:03 -05:00
fanquake
8c9dceb962
Merge bitcoin/bitcoin#29291: Add test for negative transaction version w/ CSV to tx_valid.json
97181decf5 Add test for negative transaction version w/ CSV to tx_valid.json (Chris Stewart)

Pull request description:

  This PR adds a static test vector corresponding to the bug found in various implementations of the bitcoin protocol discovered by dergoegge

  For more information see:

  https://delvingbitcoin.org/t/disclosure-btcd-consensus-bugs-due-to-usage-of-signed-transaction-version/455

ACKs for top commit:
  darosior:
    ACK 97181decf5
  dergoegge:
    ACK 97181decf5

Tree-SHA512: 92bbcd3cd10a569757b4de91e1b2bcfebc2b75ddb0160be36d8e512a6fa4623cced1aba93bd1cc044962cd2b10e1d184ef109ccdfe3cfcf85cf4b9585d80d115
2024-01-23 16:53:37 +00:00
furszy
aff7d92b15
test: add coverage for peerman adaptive connections service flags 2024-01-23 10:25:15 -03:00
Vasil Dimov
b851c5385d
fuzz: extend ConsumeNetAddr() to return I2P and CJDNS addresses
In the process of doing so, refactor `ConsumeNetAddr()` to generate the
addresses from IPv4, IPv6, Tor, I2P and CJDNS networks in the same way -
by preparing some random stream and deserializing from it. Similar code
was already found in `RandAddr()`.
2024-01-23 11:49:32 +01:00
furszy
966f5de99a
init: improve corrupted/empty settings file error msg
The preceding "Unable to parse settings file" message lacked
the necessary detail and guidance for users on what steps to
take next in order to resolve the startup error.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-01-22 10:50:03 -03:00
Sebastian Falbesoner
28287cfbe1 test: add script compression coverage for not-on-curve P2PK outputs 2024-01-22 13:38:48 +01:00
stickies-v
282b12ddb0
refactor: remove CTxMemPool::queryHashes()
Its only usage can easily be replaced with CTxMemPool::entryAll()
2024-01-18 21:54:56 +00:00
Ava Chow
5711da6588
Merge bitcoin/bitcoin#29213: doc, test: test and explain service flag handling
74ebd4d135 doc, test: Test and explain service flag handling (Martin Zumsande)

Pull request description:

  Service flags received from the peer-to-peer network are handled differently, depending on how we receive them.
  If received directly from an outbound peer the flags belong to, they replace existing flags.
  If received via gossip relay (so that anyone could send them), new flags are added, but existing ones but cannot be overwritten.

  Document that and add test coverage for it.

ACKs for top commit:
  achow101:
    ACK 74ebd4d135
  furszy:
    ACK 74ebd4d135
  brunoerg:
    utACK 74ebd4d135

Tree-SHA512: 604adc3304b8e3cb1a10dfd017025c10b029bebd3ef533f96bcb5856fee5d4396a9aed4949908b8e7ef267ad21320d1814dd80f88426330c5c9c2c529c497591
2024-01-16 13:35:45 -05:00
glozow
158623b8e0 [refactor] change Workspace::m_conflicts and adjacent funcs/structs to use Txid
It's preferable to use type-safe transaction identifiers to avoid
confusing txid and wtxid. The next commit will add a reference to this
set; we use this opportunity to change it to Txid ahead of time instead
of adding new uses of uint256.
2024-01-16 14:20:33 +00:00
Martin Zumsande
74ebd4d135 doc, test: Test and explain service flag handling
Service flags are handled differently, depending on whether
validated (if received from the peer) or unvalidated (received
via gossip relay).
2024-01-15 16:19:53 -05:00
furszy
9f36e591c5
net: move state dependent peer services flags
No behavior change. Just an intermediate refactoring.

By relocating the peer desirable services flags into the peer
manager, we allow the connections acceptance process to handle
post-IBD potential stalling scenarios.

In the follow-up commit(s), the desirable service flags will be
dynamically adjusted to detect post-IBD stalling scenarios (such
as a +48-hour inactive node that must prefer full node connections
instead of limited peer connections because they cannot provide
historical blocks). Additionally, this encapsulation enable us
to customize the connections decision-making process based on
new user's configurations in the future.
2024-01-15 10:28:20 -03:00
fanquake
8c5e4f42d5
Merge bitcoin/bitcoin#29208: build: Bump clang minimum supported version to 14
aaaace2fd1 fuzz: Assume presence of __builtin_*_overflow, without checks (MarcoFalke)
fa223ba5eb Revert "build: Fix undefined reference to __mulodi4" (MarcoFalke)
fa7c751bd9 build: Bump clang minimum supported version to 14 (MarcoFalke)

Pull request description:

  Most supported operating systems ship with clang-14 (or later), so bump the minimum to that and allow new code to drop workarounds for previous clang bugs.

  For reference:
  * https://packages.debian.org/bookworm/clang (`clang-14`)
  * https://packages.ubuntu.com/jammy/clang (`clang-14`)
  * CentOS-like 8/9 Stream: All Clang versions from 15 to 17
  * FreeBSD 12/13: All Clang versions from 15 to 16
  * OpenSuse Tumbleweed ships with https://software.opensuse.org/package/clang (`clang17`); No idea about OpenSuse Leap

  On operating systems where the clang version is not shipped by default, the user would have to use GCC, or install clang in a different way. For example:

  * https://packages.debian.org/bullseye/g++ (g++-10)
  * https://packages.ubuntu.com/focal/g++-10
  * https://apt.llvm.org/, or nix, or guix, or compile clang from source, ...

ACKs for top commit:
  fanquake:
    ACK aaaace2fd1

Tree-SHA512: 81d066b14cc568d27312f1cc814b09540b038a10a0a8e9d71fc9745b024fb6c32a959af673e6819b817ea7cef98da4abfa63dff16cffb7821b40083016b0291f
2024-01-12 10:03:22 +00:00
Chris Stewart
97181decf5 Add test for negative transaction version w/ CSV to tx_valid.json 2024-01-11 15:05:01 -06:00
Ryan Ofsky
f3a2b52376 serialization: Support for multiple parameters
This commit makes a minimal change to the ParamsStream class to let it retrieve
multiple parameters. Followup commits after this commit clean up code using
ParamsStream and make it easier to set multiple parameters.

Currently it is only possible to attach one serialization parameter to a stream
at a time. For example, it is not possible to set a parameter controlling the
transaction format and a parameter controlling the address format at the same
time because one parameter will override the other.

This limitation is inconvenient for multiprocess code since it is not possible
to create just one type of stream and serialize any object to it. Instead it is
necessary to create different streams for different object types, which
requires extra boilerplate and makes using the new parameter fields a lot more
awkward than the older version and type fields.

Fix this problem by allowing an unlimited number of serialization stream
parameters to be set, and allowing them to be requested by type. Later
parameters will still override earlier parameters, but only if they have the
same type.

This change requires replacing the stream.GetParams() method with a
stream.GetParams<T>() method in order for serialization code to retrieve the
desired parameters. This change is more verbose, but probably a good thing for
readability because previously it could be difficult to know what type the
GetParams() method would return, and now it is more obvious.
2024-01-11 06:51:57 -05:00
fanquake
4ae5171d42
Merge bitcoin/bitcoin#29219: fuzz: Improve fuzzing stability for ellswift_roundtrip harness
154fcce55c [fuzz] Improve fuzzing stability for ellswift_roundtrip harness (dergoegge)

Pull request description:

  See #29018

ACKs for top commit:
  sipa:
    utACK 154fcce55c
  brunoerg:
    crACK 154fcce55c

Tree-SHA512: 1e1ee47467a4a0d3a4e79f672018b440d8b3ccafba7428d37b9d0b8d3afd07e3f64f53ee668ed8a6a9ad1919422b5970814eaf857890acae7546951d8cb141d6
2024-01-11 11:51:57 +00:00
Ava Chow
507dbe4ca2
Merge bitcoin/bitcoin#29211: fuzz: fix connman initialization
e84dc36733 fuzz: fix `connman` initialization (brunoerg)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/pull/29172#issuecomment-1883547121

ACKs for top commit:
  achow101:
    ACK e84dc36733

Tree-SHA512: e5f3c378cfe367cc4c387fa1b13663a74d8b667a5d130d62919e21455861cfb9383b63ef4ebe56daab7b2c09e3b5031acc463065455f71607c5fb9e3c370d3ad
2024-01-10 14:20:57 -05:00
Ava Chow
7ff8e6b240
Merge bitcoin/bitcoin#28318: logging: Simplify API for level based logging
e60fc7d5d3 logging: Replace uses of LogPrintfCategory (Anthony Towns)
f7ce5ac08c logging: add LogError, LogWarning, LogInfo, LogDebug, LogTrace (Anthony Towns)
fbd7642c8e logging: add -loglevelalways=1 option (Anthony Towns)
782bb6a056 logging: treat BCLog::ALL like BCLog::NONE (Anthony Towns)
667ce3e329 logging: Drop BCLog::Level::None (Anthony Towns)
ab34dc6012 logging: Log Info messages unconditionally (Anthony Towns)
dfe98b6874 logging: make [cat:debug] and [info] implicit (Anthony Towns)
c5c76dc615 logging: refactor: pull prefix code out (Anthony Towns)

Pull request description:

  Replace `LogPrint*` functions with severity based logging functions:

   * `LogInfo(...)`, `LogWarning(...)`, `LogError(...)` for unconditional (uncategorised) logging (replaces `LogPrintf`)
   * `LogDebug(CATEGORY, ...)` and `LogTrace(CATEGORY, ...)` for conditional logging (replaces `LogPrint`)
   * `LogPrintLevel(CATEGORY, LEVEL, ...)` for when the level isn't known in advance, or a category needs to be added for an info/warning/error log message (mostly unchanged, but rarely needed)

  Logs look roughly as they do now with `LogInfo` not having an `[info]` prefix, and `LogDebug` having a `[cat]` prefix, rather than a `[cat:debug]` prefix. This removes `BCLog::Level::None` entirely -- for `LogFlags::NONE` just use `Level::Info`, for any actual category, use `Level::Debug`.

  Adds docs to developer-notes about when to use which level.

  Adds `-loglevelalways=1` option so that you get `[net:debug]`, `[all:info]`, `[all:warning]` etc, which might be helpful for automated parsing, or just if you like everything to be consistent. Defaults to off to reduce noise in the default config, and to avoid unnecessary changes on upgrades.

  Changes the behaviour of `LogPrintLevel(CATEGORY, BCLog::Level::Info, ...)` to be logged unconditionally, rather than only being an additional optional logging level in addition to trace and debug. Does not change the behaviour of `LogPrintLevel(NONE, Debug, ...)` and `LogPrintLevel(NONE, Trace, ...)` being no-ops.

ACKs for top commit:
  maflcko:
    re-ACK e60fc7d5d3 🌚
  achow101:
    ACK e60fc7d5d3
  stickies-v:
    ACK e60fc7d5d3
  jamesob:
    ACK e60fc7d5d3 ([`jamesob/ackr/28318.1.ajtowns.logging_simplify_api_for`](https://github.com/jamesob/bitcoin/tree/ackr/28318.1.ajtowns.logging_simplify_api_for))

Tree-SHA512: e7a4588779b148242495b7b6f64198a00c314cd57100affab11c43e9d39c9bbf85118ee2002792087fdcffdea08c84576e20844b3079f27083e26ddd7ca15d7f
2024-01-10 14:11:32 -05:00
dergoegge
154fcce55c [fuzz] Improve fuzzing stability for ellswift_roundtrip harness
`CPubKey::VerifyPubKey` uses rng internally which leads to instability
in the fuzz test.

We fix this by avoiding `VerifyPubKey` in the test and verifying the
decoded public key with a fuzzer chosen message instead.
2024-01-10 16:21:16 +00:00
brunoerg
e84dc36733 fuzz: fix connman initialization 2024-01-09 15:15:36 -03:00
MarcoFalke
aaaace2fd1
fuzz: Assume presence of __builtin_*_overflow, without checks 2024-01-09 16:46:58 +01:00
MarcoFalke
fa223ba5eb
Revert "build: Fix undefined reference to __mulodi4"
This reverts commit e4c8bb62e4.
2024-01-09 15:38:57 +01:00
fanquake
f921d949a0
Merge bitcoin/bitcoin#29172: fuzz: set nMaxOutboundLimit in connman target
e5b9ee0221 fuzz: set `nMaxOutboundLimit` in connman target (brunoerg)

Pull request description:

  Setting `nMaxOutboundLimit` (`-maxuploadtarget`) will make fuzz to reach more coverage in connman target. This value is used in `GetMaxOutboundTimeLeftInCycle`, `OutboundTargetReached` and `GetOutboundTargetBytesLeft`.

ACKs for top commit:
  dergoegge:
    utACK e5b9ee0221
  jonatack:
    ACK e5b9ee0221

Tree-SHA512: d19c83602b0a487e6da0e3be539aa2abc95b8bbf36cf9a3e391a4af53b959f68ca38548a96d27d56742e3b772f648da04e2bf8973dfc0ab1cdabf4f2e8d44de6
2024-01-09 09:43:13 +00:00
dergoegge
ff9039f6ea Remove GetAdjustedTime 2024-01-05 17:16:38 +00:00
brunoerg
e5b9ee0221 fuzz: set nMaxOutboundLimit in connman target 2024-01-05 12:38:35 -03:00
fanquake
7c248b972b
Merge bitcoin/bitcoin#29042: doc: Clarify C++20 comments
fa87f8feb7 doc: Clarify C++20 comments (MarcoFalke)

Pull request description:

  Turns out "class template argument deduction for aggregates" is one of the few things implemented only in recent compilers, see https://en.cppreference.com/w/cpp/compiler_support/20

  So clarify the comments.

ACKs for top commit:
  hebasto:
    ACK fa87f8feb7, I verified the code with clang-{16,17}.

Tree-SHA512: f6d20f946cb6f8e34db224e074ed8f9dfa598377c066d1b58a8feb9e64d007444f1e2c0399e91a3e282fd5d59f90e0d7df90aa3956824d96bc78070ee12f603c
2024-01-05 15:37:06 +00:00