0
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-02-05 10:17:30 -05:00
bitcoin-bitcoin-core/src/test/fuzz
W. J. van der Laan 5c4f0c4d46
Merge bitcoin/bitcoin#21261: p2p: update inbound eviction protection for multiple networks, add I2P peers
1b1088d52f test: add combined I2P/onion/localhost eviction protection tests (Jon Atack)
7c2284eda2 test: add tests for inbound eviction protection of I2P peers (Jon Atack)
ce02dd1ef1 p2p: extend inbound eviction protection by network to I2P peers (Jon Atack)
70bbc62711 test: add combined onion/localhost eviction protection coverage (Jon Atack)
045cb40192 p2p: remove unused m_is_onion member from NodeEvictionCandidate struct (Jon Atack)
310fab4928 p2p: remove unused CompareLocalHostTimeConnected() (Jon Atack)
9e889e8a5c p2p: remove unused CompareOnionTimeConnected() (Jon Atack)
787d46bb2a p2p: update ProtectEvictionCandidatesByRatio() doxygen docs (Jon Atack)
1e15acf478 p2p: make ProtectEvictionCandidatesByRatio() fully ratio-based (Jon Atack)
3f8105c4d2 test: remove combined onion/localhost eviction protection tests (Jon Atack)
38a81a8e20 p2p: add CompareNodeNetworkTime() comparator struct (Jon Atack)
4ee7aec47e p2p: add m_network to NodeEvictionCandidate struct (Jon Atack)
7321e6f2fe p2p, refactor: rename vEvictionCandidates to eviction_candidates (Jon Atack)
ec590f1d91 p2p, refactor: improve constness in ProtectEvictionCandidatesByRatio() (Jon Atack)
4a19f501ab test: add ALL_NETWORKS to test utilities (Jon Atack)
519e76bb64 test: speed up and simplify peer_eviction_test (Jon Atack)
1cde800523 p2p, refactor: rm redundant erase_size calculation in SelectNodeToEvict() (Jon Atack)

Pull request description:

  Continuing the work in #20197 and #20685, this pull updates and abstracts our inbound eviction protection to make it fully ratio-based and easily extensible to peers connected via high-latency privacy networks that we newly support, like I2P and perhaps others soon, as these peers are disadvantaged by the latency criteria of our eviction logic.

  It then adds eviction protection for peers connected over I2P.  As described in https://github.com/bitcoin/bitcoin/pull/20685#issuecomment-767486499, we've observed over the past few months that I2P peers have a min ping latency similar to or greater than that of onion peers.

  The algorithm is a basically a multi-pass knapsack:

  - Count the number of eviction candidates in each of the disadvantaged
    privacy networks.

  - Sort the networks from lower to higher candidate counts, so that
    a network with fewer candidates will have the first opportunity
    for any unused slots remaining from the previous iteration.  In
    the case of a tie in candidate counts, priority is given by array
    member order from first to last, guesstimated to favor more unusual
    networks.

  - Iterate through the networks in this order.  On each iteration,
    allocate each network an equal number of protected slots targeting
    a total number of candidates to protect, provided any slots remain
    in the knapsack.

  - Protect the candidates in that network having the longest uptime,
    if any in that network are present.

  - Continue iterating as long as we have non-allocated slots
    remaining and candidates available to protect.

  The goal of this logic is to favorise the diversity of our peer connections.

  The individual commit messages describe each change in more detail.

  Special thank you to Vasil Dimov for the excellent review feedback and the algorithm improvement that made this change much better than it would have been otherwise. Thanks also to Antoine Riard, whose review feedback nudged this change to protect disadvantaged networks having fewer, rather than more, eviction candidates.

ACKs for top commit:
  laanwj:
    Code review re-ACK 1b1088d52f
  vasild:
    ACK 1b1088d52f

Tree-SHA512: 722f790ff11f2969c79e45a5e0e938d94df78df8687e77002f32e3ef5c72a9ac10ebf8c7a9eb7f71882c97ab0e67b2778191effdb747d9ca54d7c23c2ed19a90
2021-06-14 15:04:32 +02:00
..
addition_overflow.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
addrdb.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
addrman.cpp fuzz: Call const member functions in addrman fuzz test only once 2021-06-13 13:52:21 +02:00
asmap.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
asmap_direct.cpp test: add missing netaddress include headers 2021-03-16 19:52:37 +01:00
autofile.cpp scripted-diff: remove Optional & nullopt 2021-03-15 10:41:30 +08:00
banman.cpp Merge bitcoin/bitcoin#22005: fuzz: Speed up banman fuzz target 2021-06-01 11:32:02 +02:00
base_encode_decode.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
bech32.cpp Implement Bech32m encoding/decoding 2021-03-15 17:26:35 -07:00
block.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
block_header.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
blockfilter.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
bloom_filter.cpp scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext 2021-02-22 10:27:22 +01:00
buffered_file.cpp scripted-diff: remove Optional & nullopt 2021-03-15 10:41:30 +08:00
chain.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
checkqueue.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
coins_view.cpp fuzz: Initialize a TestingSetup for test_one_input 2021-06-10 15:04:39 -04:00
connman.cpp Merge bitcoin/bitcoin#21186: net/net processing: Move addr data into net_processing 2021-05-24 20:28:31 +08:00
crypto.cpp scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext 2021-02-22 10:27:22 +01:00
crypto_aes256.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
crypto_aes256cbc.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
crypto_chacha20.cpp scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext 2021-02-22 10:27:22 +01:00
crypto_chacha20_poly1305_aead.cpp scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext 2021-02-22 10:27:22 +01:00
crypto_common.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
crypto_hkdf_hmac_sha256_l32.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
crypto_poly1305.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
cuckoocache.cpp build: Fix fuzz-cuckoocache cross-compiling for Windows with DEBUG=1 2021-03-03 12:23:41 +02:00
data_stream.cpp Move MakeNoLogFileContext to common libtest_util, and use it in bench 2021-03-03 09:17:37 +01:00
decode_tx.cpp scripted-diff: Bump copyright headers 2020-12-31 09:45:41 +01:00
descriptor_parse.cpp scripted-diff: remove MakeUnique<T>() 2021-03-11 13:45:14 +08:00
deserialize.cpp p2p, rpc, fuzz: various tiny follow-ups 2021-06-06 15:49:22 +02:00
eval_script.cpp scripted-diff: remove MakeUnique<T>() 2021-03-11 13:45:14 +08:00
fee_rate.cpp refactor: Make CFeeRate constructor architecture-independent 2021-05-18 07:13:25 +02:00
fees.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
flatfile.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
float.cpp fuzz: Increase branch coverage of the float fuzz target 2021-06-07 13:41:14 +02:00
fuzz.cpp fuzz: Terminate immediately if a fuzzing harness ever tries to perform a DNS lookup (belts and suspenders) 2021-05-21 19:41:43 +00:00
fuzz.h scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext 2021-02-22 10:27:22 +01:00
FuzzedDataProvider.h fuzz: Bump FuzzedDataProvider.h 2021-03-09 12:56:44 +01:00
golomb_rice.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
hex.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
http_request.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
i2p.cpp scripted-diff: Replace GetDataDir() calls with gArgs.GetDataDirNet() calls 2021-05-24 10:29:58 +02:00
integer.cpp fuzz: Remove incorrect float round-trip serialization test 2021-05-12 14:42:41 +02:00
key.cpp scripted-diff: remove MakeUnique<T>() 2021-03-11 13:45:14 +08:00
key_io.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
kitchen_sink.cpp scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext 2021-02-22 10:27:22 +01:00
load_external_block_file.cpp fuzz: Initialize a TestingSetup for test_one_input 2021-06-10 15:04:39 -04:00
locale.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
merkleblock.cpp scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext 2021-02-22 10:27:22 +01:00
message.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
muhash.cpp scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext 2021-02-22 10:27:22 +01:00
multiplication_overflow.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
net.cpp [net processing] Move addr relay data and logic into net processing 2021-04-30 11:29:14 +01:00
net_permissions.cpp scripted-diff: rename NetPermissionFlags enumerators 2021-05-12 16:13:30 +02:00
netaddress.cpp scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext 2021-02-22 10:27:22 +01:00
netbase_dns_lookup.cpp p2p, refactor: pass and use uint16_t CService::port as uint16_t 2021-03-16 19:52:31 +01:00
node_eviction.cpp p2p: remove unused m_is_onion member from NodeEvictionCandidate struct 2021-06-14 13:58:05 +02:00
p2p_transport_serialization.cpp [fuzz] Occasional valid magic bytes for transport serialization test 2021-05-25 08:20:43 -07:00
parse_hd_keypath.cpp scripted-diff: Bump copyright headers 2020-12-31 09:45:41 +01:00
parse_iso8601.cpp fuzz: Limit ParseISO8601DateTime fuzzing to 32-bit 2021-05-09 10:04:01 +02:00
parse_numbers.cpp test: add ParseUInt16() test coverage 2021-03-19 23:50:36 +01:00
parse_script.cpp scripted-diff: Bump copyright headers 2020-12-31 09:45:41 +01:00
parse_univalue.cpp scripted-diff: remove MakeUnique<T>() 2021-03-11 13:45:14 +08:00
policy_estimator.cpp Move MakeNoLogFileContext to common libtest_util, and use it in bench 2021-03-03 09:17:37 +01:00
policy_estimator_io.cpp Move MakeNoLogFileContext to common libtest_util, and use it in bench 2021-03-03 09:17:37 +01:00
pow.cpp fuzz: [refactor] Use PickValue where possible 2021-03-24 06:57:55 +01:00
prevector.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
primitives_transaction.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
process_message.cpp net: Restrict period when cs_vNodes mutex is locked 2021-04-22 17:28:39 +03:00
process_messages.cpp net: Restrict period when cs_vNodes mutex is locked 2021-04-22 17:28:39 +03:00
protocol.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
psbt.cpp fuzz: Extend psbt fuzz target a bit 2021-04-09 13:17:37 +02:00
random.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
rbf.cpp Merge #20560: fuzz: Link all targets once 2020-12-15 19:00:36 +01:00
rolling_bloom_filter.cpp scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext 2021-02-22 10:27:22 +01:00
rpc.cpp fuzz: Avoid timeout in EncodeBase58 2021-05-11 21:24:49 +02:00
script.cpp Merge bitcoin/bitcoin#18847: compressor: use a prevector in CompressScript serialization [ZAP1] 2021-04-28 21:13:44 +02:00
script_assets_test_minimizer.cpp fuzz: Limit toxic test globals to their respective scope 2021-05-04 09:24:17 +02:00
script_bitcoin_consensus.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
script_descriptor_cache.cpp refactor: post Optional<> removal cleanups 2021-03-17 14:56:20 +08:00
script_flags.cpp fuzz: Ensure prevout is consensus-valid 2021-04-25 10:36:00 +02:00
script_interpreter.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
script_ops.cpp fuzz: Run const CScript member functions only once 2021-05-12 10:20:59 +02:00
script_sigcache.cpp Merge #20560: fuzz: Link all targets once 2020-12-15 19:00:36 +01:00
script_sign.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
scriptnum_ops.cpp scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext 2021-02-22 10:27:22 +01:00
secp256k1_ec_seckey_import_export_der.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
secp256k1_ecdsa_signature_parse_der_lax.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
signature_checker.cpp fuzz: [refactor] Use IsValidFlagCombination in signature_checker fuzz target 2021-03-30 10:42:45 +02:00
signet.cpp Move MakeNoLogFileContext to common libtest_util, and use it in bench 2021-03-03 09:17:37 +01:00
socks5.cpp test: add missing netaddress include headers 2021-03-16 19:52:37 +01:00
span.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
spanparsing.cpp scripted-diff: Bump copyright headers 2020-12-31 09:45:41 +01:00
string.cpp test: add missing netaddress include headers 2021-03-16 19:52:37 +01:00
strprintf.cpp fuzz: Remove strprintf test cases that are known to fail 2021-05-09 10:25:21 +02:00
system.cpp scripted-diff: remove Optional & nullopt 2021-03-15 10:41:30 +08:00
timedata.cpp fuzz: Limit max insertions in timedata fuzz test 2021-05-11 08:54:24 +02:00
torcontrol.cpp fuzz: Add missing include (test/util/setup_common.h) 2021-03-04 08:39:27 +01:00
transaction.cpp fuzz: Speed up transaction fuzz target 2021-05-20 17:26:24 +02:00
tx_in.cpp scripted-diff: Bump copyright headers 2020-12-31 09:45:41 +01:00
tx_out.cpp scripted-diff: Bump copyright headers 2020-12-31 09:45:41 +01:00
tx_pool.cpp Merge bitcoin/bitcoin#20833: rpc/validation: enable packages through testmempoolaccept 2021-05-27 22:40:24 +02:00
txrequest.cpp fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
util.cpp fuzz: Avoid excessively large min fee rate in tx_pool 2021-05-09 10:53:24 +02:00
util.h fuzz: Remove confusing return keyword from CallOneOf 2021-06-07 13:41:13 +02:00
utxo_snapshot.cpp scripted-diff: Replace GetDataDir() calls with gArgs.GetDataDirNet() calls 2021-05-24 10:29:58 +02:00
validation_load_mempool.cpp fuzz: Initialize a TestingSetup for test_one_input 2021-06-10 15:04:39 -04:00
versionbits.cpp versionbits: simplify state transitions 2021-04-12 11:14:49 +10:00