0
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-02-04 10:07:27 -05:00
Commit graph

4696 commits

Author SHA1 Message Date
Antoine Poinsot
687a0b0fa5
miniscript: introduce a multi_a fragment
It is the equivalent of multi() but for Tapscript, using CHECKSIGADD
instead of CHECKMULTISIG.

It shares the same properties as multi() but for 'n', since a threshold
multi_a() may have an empty vector as the top element of its
satisfaction. It could also have the 'o' property when it only has a
single key, but in this case a 'pk()' is always preferable anyways.
2023-10-08 02:43:15 +02:00
Antoine Poinsot
91b4db8590
miniscript: store the script context within the Node structure
Some checks will be different depending on the script context (for
instance the maximum script size).
2023-10-08 02:43:14 +02:00
Antoine Poinsot
c3738d0344
miniscript: introduce a MsContext() helper to contexts
We are going to introduce Tapscript support in Miniscript, for which
some of Miniscript rules and properties change (new or modified
fragments, different typing rules, different resources consumption, ..).
2023-10-08 02:43:14 +02:00
Fabian Jahr
a482f86779
chain: Rename HaveTxsDownloaded to HaveNumChainTxs
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2023-10-06 19:43:32 +02:00
Fabian Jahr
73700fb554
validation, test: Improve and document nChainTx check for testability
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-10-06 19:43:31 +02:00
Andrew Chow
54bdb6e074
Merge bitcoin/bitcoin#27609: rpc: allow submitpackage to be called outside of regtest
5b878be742 [doc] add release note for submitpackage (glozow)
7a9bb2a2a5 [rpc] allow submitpackage to be called outside of regtest (glozow)
5b9087a9a7 [rpc] require package to be a tree in submitpackage (glozow)
e32ba1599c [txpackages] IsChildWithParentsTree() (glozow)
b4f28cc345 [doc] parent pay for child in aggregate CheckFeeRate (glozow)

Pull request description:

  Permit (restricted topology) submitpackage RPC outside of regtest. Suggested in https://github.com/bitcoin/bitcoin/pull/26933#issuecomment-1510851570

  This RPC should be safe but still experimental - interface may change, not all features (e.g. package RBF) are implemented, etc. If a miner wants to expose this to people, they can effectively use "package relay" before the p2p changes are implemented. However, please note **this is not package relay**; transactions submitted this way will not relay to other nodes if the feerates are below their mempool min fee. Users should put this behind some kind of rate limit or permissions.

ACKs for top commit:
  instagibbs:
    ACK 5b878be742
  achow101:
    ACK 5b878be742
  dergoegge:
    Code review ACK 5b878be742
  ajtowns:
    ACK 5b878be742
  ariard:
    Code Review ACK  5b878be742. Though didn’t manually test the PR.

Tree-SHA512: 610365c0b2ffcccd55dedd1151879c82de1027e3319712bcb11d54f2467afaae4d05dca5f4b25f03354c80845fef538d3938b958174dda8b14c10670537a6524
2023-10-05 19:08:19 -04:00
Andrew Chow
6e5cf8e953
Merge bitcoin/bitcoin#28587: descriptors: disallow hybrid public keys
c1e6c542af descriptors: disallow hybrid public keys (Pieter Wuille)

Pull request description:

  Fixes #28511

  The descriptor documentation (`doc/descriptors.md`) and [BIP380](https://github.com/bitcoin/bips/blob/master/bip-0380.mediawiki) explicitly require that hex-encoded public keys start with 02 or 03 (compressed) or 04 (uncompressed). However, the current parsing/inference code permit 06 and 07 (hybrid) encoding as well. Fix this.

ACKs for top commit:
  darosior:
    ACK c1e6c542af
  achow101:
    ACK c1e6c542af

Tree-SHA512: 23b674fb420619b2536d12da10008bb87cf7bc0333ec59e618c0d02c3574b468cc71248475ece37f76658d743ef51e68566948e903bca79fda5f7d75416fea4d
2023-10-05 11:58:07 -04:00
Vasil Dimov
6e308651c4
net: move IsReachable() code to netbase and encapsulate it
`vfLimited`, `IsReachable()`, `SetReachable()` need not be in the `net`
module. Move them to `netbase` because they will be needed in
`LookupSubNet()` to possibly flip the result to CJDNS (if that network
is reachable).

In the process, encapsulate them in a class.

`NET_UNROUTABLE` and `NET_INTERNAL` are no longer ignored when adding
or removing reachable networks. This was unnecessary.
2023-10-05 15:10:34 +02:00
Vasil Dimov
c42ded3d9b
fuzz: ConsumeNetAddr(): avoid IPv6 addresses that look like CJDNS
The fuzz testing framework runs as if `-cjdnsreachable` is set and in
this case addresses like `{net=IPv6, addr=fc...}` are not possible.
2023-10-05 15:10:33 +02:00
fanquake
52c6904c78
Merge bitcoin/bitcoin#28558: Make PeerManager own a FastRandomContext
4cafe9f176 [test] Make PeerManager's rng deterministic in tests (dergoegge)
fecec3e1c6 [net processing] FeeFilterRounder doesn't own a FastRandomContext (dergoegge)
47520ed209 [net processing] Make fee filter rounder non-global (dergoegge)
77506f4ac6 [net processing] Addr shuffle uses PeerManager's rng (dergoegge)
a648dd79e5 [net processing] PushAddress uses PeerManager's rng (dergoegge)
87c706713e [net processing] PeerManager holds a FastRandomContext (dergoegge)

Pull request description:

  This lets us avoid some non-determinism in tests (also see #28537).

ACKs for top commit:
  MarcoFalke:
    re-ACK 4cafe9f176  🕗
  glozow:
    concept && light code review ACK 4cafe9f176

Tree-SHA512: 3c18700773d0bc547ccb6442c41567e6f26b0b50fab5b79620da417ec91b9c0ae1395d15258da3aa4a91447b8ce560145dd135e39fbbd0610749e528e665b111
2023-10-05 14:06:39 +01:00
Vasil Dimov
5c8e15c451
i2p: destroy the session if we get an unexpected error from the I2P router
From https://geti2p.net/en/docs/api/samv3:

  If SILENT=false was passed, which is the default value, the SAM bridge
  sends the client a ASCII line containing the base64 public destination
  key of the requesting peer

So, `Accept()` is supposed to receive a Base64 encoded destination of
the connecting peer, but if it receives something like this instead:

  STREAM STATUS RESULT=I2P_ERROR MESSAGE="Session was closed"

then destroy the session.
2023-10-05 14:11:13 +02:00
Pieter Wuille
c1e6c542af descriptors: disallow hybrid public keys
The descriptor documentation (doc/descriptors.md) and BIP380 explicitly
require that hex-encoded public keys start with 02 or 03 (compressed) or
04 (uncompressed). However, the current parsing/inference code permit 06
and 07 (hybrid) encoding as well. Fix this.
2023-10-04 11:28:13 -04:00
dergoegge
4cafe9f176 [test] Make PeerManager's rng deterministic in tests 2023-10-04 13:16:53 +01:00
dergoegge
fecec3e1c6 [net processing] FeeFilterRounder doesn't own a FastRandomContext 2023-10-04 13:16:52 +01:00
Andrew Chow
01bd9d7b99
Merge bitcoin/bitcoin#28523: rpc: add hidden getrawaddrman RPC to list addrman table entries
352d5eb2a9 test: getrawaddrman RPC (0xb10c)
da384a286b rpc: getrawaddrman for addrman entries (0xb10c)

Pull request description:

  Inspired by `getaddrmaninfo` (#27511), this adds a hidden/test-only `getrawaddrman` RPC. The RPC returns information on all addresses in the address manager new and tried tables. Addrman table contents can be used in tests and during development.

  The RPC result encodes the `bucket` and `position`, the internal location of addresses in the tables, in the address object's string key. This allows users to choose to consume or to ignore the location information. If the internals of the address manager implementation change, the location encoding might change too.

  ```
  getrawaddrman

  EXPERIMENTAL warning: this call may be changed in future releases.

  Returns information on all address manager entries for the new and tried tables.

  Result:
  {                                  (json object)
    "table" : {                      (json object) buckets with addresses in the address manager table ( new, tried )
      "bucket/position" : {          (json object) the location in the address manager table (<bucket>/<position>)
        "address" : "str",           (string) The address of the node
        "port" : n,                  (numeric) The port number of the node
        "network" : "str",           (string) The network (ipv4, ipv6, onion, i2p, cjdns) of the address
        "services" : n,              (numeric) The services offered by the node
        "time" : xxx,                (numeric) The UNIX epoch time when the node was last seen
        "source" : "str",            (string) The address that relayed the address to us
        "source_network" : "str"     (string) The network (ipv4, ipv6, onion, i2p, cjdns) of the source address
      },
      ...
    },
    ...
  }

  Examples:
  > bitcoin-cli getrawaddrman
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "getrawaddrman", "params": []}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  ```

ACKs for top commit:
  willcl-ark:
    reACK 352d5eb2a9
  amitiuttarwar:
    reACK 352d5eb2a9
  stratospher:
    reACK 352d5eb.
  achow101:
    ACK 352d5eb2a9

Tree-SHA512: cc462666b5c709617c66b0e3e9a17c4c81e9e295f91bdd9572492d1cb6466fc9b6d48ee805ebe82f9f16010798370effe5c8f4db15065b8c7c0d8637675d615e
2023-10-03 11:38:20 -04:00
Ryan Ofsky
d0b928b29d
Merge bitcoin/bitcoin#26312: Remove Sock::Get() and Sock::Sock()
7df4508369 test: improve sock_tests/move_assignment (Vasil Dimov)
5086a99b84 net: remove Sock default constructor, it's not necessary (Vasil Dimov)
7829272f78 net: remove now unnecessary Sock::Get() (Vasil Dimov)
944b21b70a net: don't check if the socket is valid in ConnectSocketDirectly() (Vasil Dimov)
aeac68d036 net: don't check if the socket is valid in GetBindAddress() (Vasil Dimov)
5ac1a51ee5 i2p: avoid using Sock::Get() for checking for a valid socket (Vasil Dimov)

Pull request description:

  _This is a piece of #21878, chopped off to ease review._

  Peeking at the underlying socket file descriptor of `Sock` and checkig if it is `INVALID_SOCKET` is bad encapsulation and stands in the way of testing/mocking/fuzzing.

  Instead use an empty `unique_ptr` to denote that there is no valid socket where appropriate or outright remove such checks where they are not necessary.

  The default constructor `Sock::Sock()` is unnecessary now after recent changes, thus remove it.

ACKs for top commit:
  ajtowns:
    ACK 7df4508369
  jonatack:
    ACK 7df4508369

Tree-SHA512: 9742aeeeabe8690530bf74caa6ba296787028c52f4a3342afd193b05dbbb1f6645935c33ba0a5230199a09af01c666bd3c7fb16b48692a0d185356ea59a8ddbf
2023-10-03 09:57:46 -04:00
Pieter Wuille
b815cce50e net: expose transport types/session IDs of connections in RPC and logs
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-10-02 18:11:11 -04:00
dhruv
c73cd42363 rpc: addnode arg to use BIP324 v2 p2p
Co-authored-by: Pieter Wuille <bitcoin-dev@wuille.net>
2023-10-02 18:10:30 -04:00
Pieter Wuille
abf343b320 net: advertise NODE_P2P_V2 if CLI arg -v2transport is on
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-10-02 18:09:53 -04:00
Andrew Chow
e7b0004b37
Merge bitcoin/bitcoin#27596: assumeutxo (2)
edbed31066 chainparams: add signet assumeutxo param at height 160_000 (Sjors Provoost)
b8cafe3871 chainparams: add testnet assumeutxo param at height 2_500_000 (Sjors Provoost)
99839bbfa7 doc: add note about confusing HaveTxsDownloaded name (James O'Beirne)
7ee46a755f contrib: add script to demo/test assumeutxo (James O'Beirne)
42cae39356 test: add feature_assumeutxo functional test (James O'Beirne)
0f64bac603 rpc: add getchainstates (James O'Beirne)
bb05857794 refuse to activate a UTXO snapshot if mempool not empty (James O'Beirne)
ce585a9a15 rpc: add loadtxoutset (James O'Beirne)
62ac519e71 validation: do not activate snapshot if behind active chain (James O'Beirne)
9511fb3616 validation: assumeutxo: swap m_mempool on snapshot activation (James O'Beirne)
7fcd21544a blockstorage: segment normal/assumedvalid blockfiles (James O'Beirne)
4c3b8ca35c validation: populate nChainTx value for assumedvalid chainstates (James O'Beirne)
49ef778158 test: adjust chainstate tests to use recognized snapshot base (James O'Beirne)
1019c39982 validation: pruning for multiple chainstates (James O'Beirne)
373cf91531 validation: indexing changes for assumeutxo (James O'Beirne)
1fffdd76a1 net_processing: validationinterface: ignore some events for bg chain (James O'Beirne)
fbe0a7d7ca wallet: validationinterface: only handle active chain notifications (James O'Beirne)
f073917a9e validationinterface: only send zmq notifications for active (James O'Beirne)
4d8f4dcb45 validation: pass ChainstateRole for validationinterface calls (James O'Beirne)
1e59acdf17 validation: only call UpdatedBlockTip for active chainstate (James O'Beirne)
c6af23c517 validation: add ChainstateRole (James O'Beirne)
9f2318c76c validation: MaybeRebalanceCaches when chain leaves IBD (James O'Beirne)
434495a8c1 chainparams: add blockhash to AssumeutxoData (James O'Beirne)
c711ca186f assumeutxo: remove snapshot during -reindex{-chainstate} (James O'Beirne)
c93ef43e4f bugfix: correct is_snapshot_cs in VerifyDB (James O'Beirne)
b73d3bbd23 net_processing: Request assumeutxo background chain blocks (Suhas Daftuar)

Pull request description:

  - Background and FAQ: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal
  - Prior progress/project: https://github.com/bitcoin/bitcoin/projects/11
  - Replaces https://github.com/bitcoin/bitcoin/pull/15606, which was closed due to Github slowness. Original description and commentary can be found there.

  ---

  This changeset finishes the first phase of the assumeutxo project. It makes UTXO snapshots loadable via RPC (`loadtxoutset`) and adds `assumeutxo` parameters to chainparams. It contains all the remaining changes necessary to both use an assumedvalid snapshot chainstate and do a full validation sync in the background.

  This may look like a lot to review, but note that
  - ~200 lines are a (non-essential) demo shell script
  - Many lines are functional test, documentation, and relatively dilute RPC code.

  So it shouldn't be as burdensome to review as the linecount might suggest.

  - **P2P**: minor changes are made to `init.cpp` and `net_processing.cpp` to make simultaneous IBD across multiple chainstates work.
  - **Pruning**: implement correct pruning behavior when using a background chainstate
  - **Blockfile separation**: to prevent "fragmentation" in blockfile storage, have background chainstates use separate blockfiles from active snapshot chainstates to avoid interleaving heights and impairing pruning.
  - **Indexing**: some `CValidationInterface` events are given with an additional parameter, ChainstateRole, and all indexers ignore events from ChainstateRole::ASSUMEDVALID so that indexation only happens sequentially.
  - Have `-reindex` properly wipe snapshot chainstates.
  - **RPC**: introduce RPC commands `loadtxoutset` and (hidden) `getchainstates`.
  - **Release docs & first assumeutxo commitment**: add notes and a particular assumeutxo hash value for first AU-enabled release.
    - This will complete the project and allow use of UTXO snapshots for faster node bootstrap.

  The next phase, if it were to be pursued, would be coming up with a way to distribute the UTXO snapshots over the P2P network.

  ---

  ### UTXO snapshots

  Create your own with `./contrib/devtools/utxo_snapshot.sh`, e.g.
  ```shell
  ./contrib/devtools/utxo_snapshot.sh 788000 utxo.dat ./src/bitcoin-cli -datadir=$(pwd)/testdata`)
  ```
  or use the pre-generated ones listed below.

  - Testnet: **2'500'000** (Sjors):
    - torrent: `magnet:?xt=urn:btih:511e09f4bf853aefab00de5c070b1e031f0ecbe9&dn=utxo-testnet-2500000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`
    - sha256: `79db4b025448cc0ac388d8589a28eab02de53055d181e34eb47391717aa16388`
  - Signet: **160'000** (Sjors):
    - torrent: `magnet:?xt=urn:btih:9da986cb27b3980ea7fd06b21e199b148d486880&dn=utxo-signet-160000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`
    - sha256: `eeeca845385ba91e84ef58c09d38f98f246a24feadaad57fe1e5874f3f92ef8c`
  - Mainnet: **800'000** (Sjors):
    - Note: this needs the following commit cherry-picked in: 24deb2022b
    - torrent: `magnet:?xt=urn:btih:50ee955bef37f5ec3e5b0df4cf0288af3d715a2e&dn=utxo-800000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`

  ### Testing

  #### For fun (~5min)

  If you want to do a quick test, you can run `./contrib/devtools/test_utxo_snapshots.sh` and follow the instructions. This is mostly obviated by the functional tests, though.

  #### For real (longer)

  If you'd like to experience a real usage of assumeutxo, you can do that too.
  I've cut a new snapshot at height 788'000 (http://img.jameso.be/utxo-788000.dat - but you can do it yourself with `./contrib/devtools/utxo_snapshot.sh` if you want). Download that, and then create a datadir for testing:
  ```sh
  $ cd ~/src/bitcoin  # or whatever

  # get the snapshot
  $ curl http://img.jameso.be/utxo-788000.dat > utxo-788000.dat

  # you'll want to do this if you like copy/pasting
  $ export AU_DATADIR=/home/${USER}/au-test # or wherever

  $ mkdir ${AU_DATADIR}
  $ vim ${AU_DATADIR}/bitcoin.conf

  dbcache=8000  # or, you know, something high
  blockfilterindex=1
  coinstatsindex=1
  prune=3000
  logthreadnames=1
  ```
  Obtain this branch, build it, and then start bitcoind:
  ```sh
  $ git remote add jamesob https://github.com/jamesob/bitcoin
  $ git fetch jamesob assumeutxo
  $ git checkout jamesob/assumeutxo

  $ ./configure $conf_args && make  # (whatever you like to do here)

  # start 'er up and watch the logs
  $ ./src/bitcoind -datadir=${AU_DATADIR}
  ```
  Then, in some other window, load the snapshot
  ```sh
  $ ./src/bitcoin-cli -datadir=${AU_DATADIR} loadtxoutset $(pwd)/utxo-788000.dat
  ```

  You'll see some log messages about headers retrieval and waiting to see the snapshot in the headers chain. Once you get the full headers chain, you'll spend a decent amount of time (~10min) loading the snapshot, checking it, and flushing it to disk. After all that happens, you should be syncing to tip in pretty short order, and you'll see the occasional `[background validation]` log message go by.

  In yet another window, you can check out chainstate status with
  ```sh
  $ ./src/bitcoin-cli -datadir=${AU_DATADIR} getchainstates
  ```
  as well as usual favorites like `getblockchaininfo`.

ACKs for top commit:
  achow101:
    ACK edbed31066

Tree-SHA512: 6086fb9a38dc7df85fedc76b30084dd8154617a2a91e89a84fb41326d34ef8e7d7ea593107afba01369093bf8cc91770621d98f0ea42a5b3b99db868d2f14dc2
2023-10-02 17:09:44 -04:00
0xb10c
da384a286b
rpc: getrawaddrman for addrman entries
Exposing address manager table entries in a hidden RPC allows to introspect
addrman tables in tests and during development.

As response JSON object the following FORMAT1 is choosen:
{
  "table": {
    "<bucket>/<position>": { "address": "..", "port": .., ... },
    "<bucket>/<position>": { "address": "..", "port": .., ... },
    "<bucket>/<position>": { "address": "..", "port": .., ... },
    ...
  }
}

An alternative would be FORMAT2
{
  "table": {
    "bucket": {
      "position": { "address": "..", "port": .., ... },
      "position": { "address": "..", "port": .., ... },
      ..
    },
    "bucket": {
      "position": { "address": "..", "port": .., ... },
      ..
    },
  }
}

FORMAT1 and FORMAT2 have different encodings for the location of the
address in the address manager. While FORMAT2 might be easier to process
for downstream tools, it also mimics internal addrman mappings, which
might change at some point. Users not interested in the address location
can ignore the location key. They don't have to adapt to a new RPC
response format, when the internal addrman layout changes. Additionally,
FORMAT1 is also slightly easier to to iterate in downstream tools. The
RPC response-building implemenation complexcity is lower with FORMAT1
as we can more easily build a "<bucket>/<position>" key than a multiple
"bucket" objects with multiple "position" objects (FORMAT2).
2023-10-02 15:34:28 +02:00
fanquake
48b8910d12
Merge bitcoin/bitcoin#28508: refactor: Remove SER_GETHASH, hard-code client version in CKeyPool serialize
fac29a0ab1 Remove SER_GETHASH, hard-code client version in CKeyPool serialize (MarcoFalke)
fa72f09d6f Remove CHashWriter type (MarcoFalke)
fa4a9c0f43 Remove unused GetType() from OverrideStream, CVectorWriter, SpanReader (MarcoFalke)

Pull request description:

  Removes a bunch of redundant, dead or duplicate code.

  Uses the idea from and finishes the idea https://github.com/bitcoin/bitcoin/pull/28428 by theuni

ACKs for top commit:
  ajtowns:
    ACK fac29a0ab1
  kevkevinpal:
    added one nit but otherwise ACK [fac29a0](fac29a0ab1)

Tree-SHA512: cc805e2f38e73869a6691fdb5da09fa48524506b87fc93f05d32c336ad3033425a2d7608e317decd3141fde3f084403b8de280396c0c39132336fe0f7510af9e
2023-10-02 12:33:54 +02:00
glozow
e32ba1599c [txpackages] IsChildWithParentsTree()
Many edge cases exist when parents in a child-with-parents package can
spend each other. However, this pattern should also be uncommon in
normal use cases.
2023-10-02 10:13:38 +01:00
James O'Beirne
0f64bac603 rpc: add getchainstates
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-09-30 06:41:23 -04:00
James O'Beirne
ce585a9a15 rpc: add loadtxoutset
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
2023-09-30 06:41:23 -04:00
James O'Beirne
62ac519e71 validation: do not activate snapshot if behind active chain
Most easily reviewed with

  git show --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-09-30 06:41:21 -04:00
James O'Beirne
9511fb3616 validation: assumeutxo: swap m_mempool on snapshot activation
Otherwise we will not receive transactions during background sync until
restart.
2023-09-30 06:40:17 -04:00
James O'Beirne
49ef778158 test: adjust chainstate tests to use recognized snapshot base
In future commits, loading the block index while making use of a
snapshot is contingent on the snapshot being recognized by chainparams.

Ensure all existing unittests that use snapshots use a recognized
snapshot (at height 110).

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-09-30 06:40:17 -04:00
James O'Beirne
4d8f4dcb45 validation: pass ChainstateRole for validationinterface calls
This allows consumers to decide how to handle events from background or
assumedvalid chainstates.
2023-09-30 06:38:47 -04:00
James O'Beirne
434495a8c1 chainparams: add blockhash to AssumeutxoData
This allows us to reference assumeutxo configuration by blockhash as
well as height; this is helpful in future changes when we want to
reference assumeutxo configurations before the block index is loaded.
2023-09-30 06:38:47 -04:00
Andrew Chow
d18a8f6f69
Merge bitcoin/bitcoin#28525: net: Drop v2 garbage authentication packet
e3720bca39 net: Simplify v2 recv logic by decoupling AAD from state machine (Tim Ruffing)
b0f5175c04 net: Drop v2 garbage authentication packet (Tim Ruffing)

Pull request description:

  Note that this is a breaking change, see also https://github.com/bitcoin/bips/pull/1498

  The benefit is a simpler implementation:
   - The protocol state machine does not need separate states for garbage authentication and version phases.
   - The special case of "ignoring the ignore bit" is removed.
   - The freedom to choose the contents of the garbage authentication packet is removed. This simplifies testing.

ACKs for top commit:
  naumenkogs:
    ACK e3720bca39
  sipa:
    ACK e3720bca39. Re-ran the v2 transport fuzzer overnight.
  ajtowns:
    ACK e3720bca39 - simpler and more flexible, nice
  achow101:
    ACK e3720bca39
  Sjors:
    utACK e3720bca39
  theStack:
    Code-review ACK e3720bca39

Tree-SHA512: 16600ed868c8a346828de075c4072e37cf86440751d08ab099fe8b58ff71d8b371a90397d6a4247096796db68794275e7e0403f218859567d04838e0411dadd6
2023-09-29 09:50:02 -04:00
glozow
6619d6a8dc
Merge bitcoin/bitcoin#28450: Add package evaluation fuzzer
262ab8ef78 Add package evaluation fuzzer (Greg Sanders)

Pull request description:

  This fuzzer target caught the issue in https://github.com/bitcoin/bitcoin/pull/28251 within 5 minutes on master branch, and an additional issue which I've applied a preliminary patch to cover.

  Fuzzer target does the following:

  1) Picks mempool confgs, including max package size, count, mempool size, etc
  2) Generates 1 to 26 transactions with arbitrary coins/fees, the first N-1 spending only confirmed outpoints
  3) Nth transaction, if >1, sweeps all unconfirmed outpoints in mempool
  4) If N==1, it may submit it through single-tx submission path, to allow for more interesting topologies
  5) Otherwise submits through package submission interface
  6) Repeat 1-5  a few hundred times per mempool instance

  In other words, it ends up building chains of txns in the mempool using parents-and-children packages, which is currently the topology supported on master.

  The test itself is a direct rip of tx_pool.cpp, with a number of assertions removed because they were failing for unknown reasons, likely due to the notification changes of single tx submission to package, which is used to track addition/removal of transactions in the test. I'll continue working on re-adding these assertions for further invariant testing.

ACKs for top commit:
  murchandamus:
    ACK 262ab8ef78
  glozow:
    reACK 262ab8ef78
  dergoegge:
    tACK 262ab8ef78

Tree-SHA512: 190784777d0f2361b051b3271db8f79b7927e3cab88596d2c30e556da721510bd17f6cc96f6bb03403bbf0589ad3f799fa54e63c1b2bd92a2084485b5e3e96a5
2023-09-28 12:05:24 +01:00
Greg Sanders
262ab8ef78 Add package evaluation fuzzer 2023-09-27 16:27:05 -04:00
Tim Ruffing
b0f5175c04 net: Drop v2 garbage authentication packet
See also https://github.com/bitcoin/bips/pull/1498

The benefit is a simpler implementation:
 - The protocol state machine does not need separate states for garbage
   authentication and version phases.
 - The special case of "ignoring the ignore bit" is removed.
 - The freedom to choose the contents of the garbage authentication
   packet is removed. This simplifies testing.
2023-09-27 12:19:54 +02:00
fanquake
c9f288244b
Merge bitcoin/bitcoin#28483: refactor: Return CAutoFile from BlockManager::Open*File()
fa56c421be Return CAutoFile from BlockManager::Open*File() (MarcoFalke)
9999b89cd3 Make BufferedFile to be a CAutoFile wrapper (MarcoFalke)
fa389d902f refactor: Drop unused fclose() from BufferedFile (MarcoFalke)

Pull request description:

  This is required for https://github.com/bitcoin/bitcoin/pull/28052, but makes sense on its own, because offloading logic to `CAutoFile` instead of re-implementing it allows to delete code and complexity.

ACKs for top commit:
  TheCharlatan:
    Re-ACK fa56c421be
  willcl-ark:
    tACK fa56c421be

Tree-SHA512: fe4638f3a6bd3f9d968cfb9ae3259c9d6cd278fe2912cbc90289851311c8c781099db4c160e775960975c4739098d9af801a8d2d12603f371f8edfe134d8f85a
2023-09-26 14:01:44 +01:00
fanquake
ac9fa6ec78
Merge bitcoin/bitcoin#28385: [refactor] rewrite DisconnectedBlockTransactions to not use boost
4313c77400 make DisconnectedBlockTransactions responsible for its own memory management (glozow)
cf5f1faa03 MOVEONLY: DisconnectedBlockTransactions to its own file (glozow)
2765d6f343 rewrite DisconnectedBlockTransactions as a list + map (glozow)
79ce9f0aa4 add std::list to memusage (glozow)
59a35a7398 [bench] DisconnectedBlockTransactions (glozow)
925bb723ca [refactor] batch-add transactions to DisconnectedBlockTransactions (glozow)

Pull request description:

  Motivation
  - I think it's preferable to use stdlib data structures instead of depending on boost if we can achieve the same thing.
  - Also see #28335 for further context/motivation. This PR simplifies that one.

  Things done in this PR:
  - Add a bench for `DisconnectedBlockTransactions` where we reorg and the new chain has {100%, 90%, 10%} of the same transactions. AFAIU in practice, it's usually close to 100%.
  - Rewrite `DisconnectedBlockTransactions` as a `std::list` + `unordered_map` instead of a boost multi index container.
    - On my machine, the bench suggests the performance is very similar.
  - Move `DisconnectedBlockTransactions` from txmempool.h to its own kernel/disconnected_transactions.h. This struct isn't used by txmempool and doesn't have much to do with txmempool. My guess is that it's been living there for convenience since the boost includes are there.

ACKs for top commit:
  ismaelsadeeq:
    Tested ACK 4313c77400
  stickies-v:
    ACK 4313c77400
  TheCharlatan:
    ACK 4313c77400

Tree-SHA512: 273c80866bf3acd39b2a039dc082b7719d2d82e0940e1eb6c402f1c0992e997256722b85c7e310c9811238a770cfbdeb122ea4babbc23835d17128f214a1ef9e
2023-09-23 18:42:36 +01:00
Andrew Chow
2303fd2f43
Merge bitcoin/bitcoin#28471: Fix virtual size limit enforcement in transaction package context
eb8f58f5e4 Add functional test to catch too large vsize packages (Greg Sanders)
1a579f9d01 Handle over-sized (in virtual bytes) packages with no in-mempool ancestors (Greg Sanders)
bc013fe8e3 Bugfix: Pass correct virtual size to CheckPackageLimits (Luke Dashjr)
533660c58a Replace MAX_PACKAGE_SIZE with MAX_PACKAGE_WEIGHT to avoid vbyte confusion (Greg Sanders)

Pull request description:

  (Alternative) Minimal subset of https://github.com/bitcoin/bitcoin/pull/28345 to:

  1) Replace MAX_PACKAGE_SIZE with MAX_PACKAGE_WEIGHT which accounts for additional WU necessary to not exclude default chain limit transactions that would have been accepted individually. Avoids sigops vbyte confusion.
  2) pass correct vsize to chain limit evaluations in package context
  3) stop overly-large packages that have no existing mempool ancestors (also a bugfix by itself if someone sets non-standard chain limits)

  This should fix the known issues while not blocking additional refactoring later.

ACKs for top commit:
  achow101:
    ACK eb8f58f5e4
  ariard:
    Re-Code ACK eb8f58f5e
  glozow:
    reACK eb8f58f5e4

Tree-SHA512: 1b5cca1a526207e25d387fcc29a776a3198c3a013dc2b35c6275b9d5a64db2476c154ebf52e3a1aed0b9924c75613f21a946577aa760de28cadf0c9c7f68dc39
2023-09-21 12:13:52 -04:00
Andrew Chow
ff564c75e7
Merge bitcoin/bitcoin#27511: rpc: Add test-only RPC getaddrmaninfo for new/tried table address count
28bac81a34 test: add functional test for getaddrmaninfo (stratospher)
c8eb8dae51 rpc: Introduce getaddrmaninfo for count of addresses stored in new/tried table (stratospher)

Pull request description:

  implements https://github.com/bitcoin/bitcoin/issues/26907. split off from #26988 to keep RPC, CLI discussions separate.

  This PR introduces a new RPC `getaddrmaninfo`which returns the count of addresses in the new/tried table of a node's addrman broken down by network type. This would be useful for users who want to see the distribution of addresses from different networks across new/tried table in the addrman.

  ```jsx
  $ getaddrmaninfo

  Result:
  {                   (json object) json object with network type as keys
    "network" : {     (json object) The network (ipv4, ipv6, onion, i2p, cjdns)
      "new" : n,      (numeric) number of addresses in new table
      "tried" : n,    (numeric) number of addresses in tried table
      "total" : n     (numeric) total number of addresses in both new/tried tables from a network
    },
    ...
  }
  ```

  ### additional context from [original PR](https://github.com/bitcoin/bitcoin/pull/26988)

  1. network coverage tests were skipped because there’s a small chance that addresses from different networks could hash to the same bucket and cause count of different network addresses in the tests to fail. see https://github.com/bitcoin/bitcoin/pull/26988#discussion_r1137596851.
  2. #26988 uses this RPC in -addrinfo CLI. Slight preference for keeping the RPC hidden since this info will mostly be useful to only super users. see https://github.com/bitcoin/bitcoin/pull/26988#discussion_r1173964808.

ACKs for top commit:
  0xB10C:
    ACK 28bac81a34
  willcl-ark:
    reACK 28bac81a34
  achow101:
    ACK 28bac81a34
  brunoerg:
    reACK 28bac81a34
  theStack:
    Code-review ACK 28bac81a34

Tree-SHA512: 346390167e1ebed7ca5c79328ea452633736aff8b7feefea77460e04d4489059334ae78a3f757f32f5fb7827b309d7186bebab3c3760b3dfb016d564a647371a
2023-09-20 08:25:20 -04:00
Greg Sanders
533660c58a Replace MAX_PACKAGE_SIZE with MAX_PACKAGE_WEIGHT to avoid vbyte confusion
While allowing submitted packages to be slightly larger than what
may be allowed in the mempool to allow simpler reasoning
about contextual-less checks vs chain limits.
2023-09-20 08:10:30 -04:00
fanquake
1bf915db99
Merge bitcoin/bitcoin#28470: fuzz: Rework addr fuzzing
fad52baf1e fuzz: Rework addr fuzzing (MarcoFalke)
fa5b6d29ee fuzz: Drop unused params from serialize helpers (MarcoFalke)

Pull request description:

  Some minor fixups to addr fuzzing

ACKs for top commit:
  dergoegge:
    utACK fad52baf1e

Tree-SHA512: 6a2b07fb1a65cf855d5e7c0a52bfcb81d46dbc5d4b3e72cef359987cbd28dbfeb2fc54f210e9737cb131b40ac5f88a90e9af284e441e0b37196121590bbaf015
2023-09-20 11:38:56 +00:00
fanquake
53313c49d6
Merge bitcoin/bitcoin#28246: wallet: Use CTxDestination in CRecipient instead of just scriptPubKey
ad0c469d98 wallet: Use CTxDestination in CRecipient rather than scriptPubKey (Andrew Chow)
07d3bdf4eb Add PubKeyDestination for P2PK scripts (Andrew Chow)
1a98a51c66 Allow CNoDestination to represent a raw script (Andrew Chow)
8dd067088d Make WitnessUnknown members private (Andrew Chow)

Pull request description:

  For silent payments, we want to provide a `SilentPaymentsDestination` to be used as the recipient, which requires `CRecipient` to use something other than just the `scriptPubKey` as we cannot know the output script for a silent payment prior to transaction creation. `CTxDestination` seems like the obvious place to add a `SilentPaymentsDestination` as it is our internal representation of an address.

  In order to still allow paying to arbitrary scriptPubKeys (e.g. for data carrier outputs, or the user hand crafted a raw transaction that they have given to `fundrawtransaction`), `CNoDestination` is changed to contain raw scripts.

  Additionally, P2PK scripts are now interpreted as a new `PubKeyDestination` rather than `PKHash`. This results in some things that would have given an address for P2PK scripts to no longer do so. This is arguably more correct.

  `ExtractDestination`'s behavior is slightly changed for the above. It now returns `true` for those destinations that have addresses, so P2PK scripts now result in `false`. Even though it returns false for `CNoDestination`, the script will now be included in that `CNoDestination`.

  Builds on #28244

ACKs for top commit:
  josibake:
    ACK ad0c469d98

Tree-SHA512: ef3f8f3c7284779d9806c77c85b21caf910a79a1f7e7f1b51abcc0d7e074f14e00abf30f625a13075e41d94dad6202c10ddff462c0ee74c2ca4aab585b145a52
2023-09-19 16:48:43 +00:00
MarcoFalke
fa72f09d6f
Remove CHashWriter type
The type is only ever set, but never read via GetType(), so remove it.
Also, remove SerializeHash to avoid silent merge conflicts and use the
already existing GetHash() boilerplate consistently.
2023-09-19 16:38:08 +02:00
MarcoFalke
fa4a9c0f43
Remove unused GetType() from OverrideStream, CVectorWriter, SpanReader
GetType() is never called, so it is completely unused and can be
removed.
2023-09-19 14:19:57 +00:00
MarcoFalke
fa33b2c889
fuzz: Add missing PROVIDE_FUZZ_MAIN_FUNCTION guard to __AFL_FUZZ_INIT 2023-09-19 13:41:24 +02:00
stratospher
c8eb8dae51 rpc: Introduce getaddrmaninfo for count of addresses stored in new/tried table 2023-09-19 16:02:55 +05:30
fanquake
372e7b6510
Merge bitcoin/bitcoin#28489: tests: fix incorrect assumption in v2transport_test
3f4e1bb9ae tests: fix incorrect assumption in v2transport_test (Pieter Wuille)

Pull request description:

  One part of the current `v2transport_test` introduced in #28196 assumes that if a bit gets modified in a message, failure should instantly be detected after sending that message. This is not correct in case the length descriptor is modified, as that may cause the receiver to need more data first. Fix this by sending more messages until failure actually occurs.

  Discovered in https://github.com/bitcoin/bitcoin/pull/27495#issuecomment-1719934041.

ACKs for top commit:
  theStack:
    ACK 3f4e1bb9ae

Tree-SHA512: faa90bf91996cbaaef62d764e746cb222eaf6796316b0d0e13709e528750b7c0ef09172f7fecfe814dbb8c136c5259f65ca1ac79318e6768a0bfc4e626a63249
2023-09-16 12:15:16 +01:00
fanquake
5c7cdda992
Merge bitcoin/bitcoin#28473: refactor: Serialization parameter cleanups
fb6a2ab63e scripted-diff: use SER_PARAMS_OPFUNC (Anthony Towns)
5e5c8f86b6 serialize: add SER_PARAMS_OPFUNC (Anthony Towns)
33203f59b4 serialize: specify type for ParamsWrapper not ref (Anthony Towns)
bf147bfffa serialize: move ser_action functions out of global namespace (Anthony Towns)

Pull request description:

  Cleanups after #25284:

   * ser_action namespacing - https://github.com/bitcoin/bitcoin/pull/25284#discussion_r1316189977
   * make reference implicit - https://github.com/bitcoin/bitcoin/pull/25284#discussion_r1316277030
   * function notation - https://github.com/bitcoin/bitcoin/pull/25284#issuecomment-1710714821

ACKs for top commit:
  MarcoFalke:
    lgtm ACK fb6a2ab63e 💨
  TheCharlatan:
    ACK fb6a2ab63e

Tree-SHA512: aacca2ee9cfec360ade6b394606e13d1dfe05bc29c5fbdd48a4e6992bd420312d4ed0d32218d95c560646af326e9977728dc2e759990636298e326947f6f9526
2023-09-15 14:27:20 +01:00
MarcoFalke
fa56c421be
Return CAutoFile from BlockManager::Open*File()
This is a refactor.
2023-09-15 14:34:24 +02:00
MarcoFalke
9999b89cd3
Make BufferedFile to be a CAutoFile wrapper
This refactor allows to forward some calls to the underlying CAutoFile,
instead of re-implementing the logic in the buffered file.
2023-09-15 14:34:17 +02:00
MarcoFalke
fa389d902f
refactor: Drop unused fclose() from BufferedFile
This was only explicitly used in the tests, where it can be replaced by
wrapping the original raw file pointer into a CAutoFile on creation and
then calling CAutoFile::fclose().

Also, it was used in LoadExternalBlockFile(), where it can also be
replaced by the (implicit call to the) CAutoFile destructor after
wrapping the original raw file pointer in a CAutoFile.
2023-09-15 14:33:51 +02:00