7d8e1dec3b net: fix use-after-free in tests (Vasil Dimov)
Pull request description:
In PeerLogicValidation::PeerLogicValidation() we would schedule a lambda
function to execute later, capturing the local variable
`consensusParams` by reference.
Presumably this was considered safe because `consensusParams` is a
reference itself to a global variable which is not supposed to change,
but it can in tests.
Fixes https://github.com/bitcoin/bitcoin/issues/18372
<!--
*** Please remove the following help text before submitting: ***
Pull requests without a rationale and clear improvement may be closed
immediately.
-->
<!--
Please provide clear motivation for your patch and explain how it improves
Bitcoin Core user experience or Bitcoin Core developer experience
significantly:
* Any test improvements or new tests that improve coverage are always welcome.
* All other changes should have accompanying unit tests (see `src/test/`) or
functional tests (see `test/`). Contributors should note which tests cover
modified code. If no tests exist for a region of modified code, new tests
should accompany the change.
* Bug fixes are most welcome when they come with steps to reproduce or an
explanation of the potential issue as well as reasoning for the way the bug
was fixed.
* Features are welcome, but might be rejected due to design or scope issues.
If a feature is based on a lot of dependencies, contributors should first
consider building the system outside of Bitcoin Core, if possible.
* Refactoring changes are only accepted if they are required for a feature or
bug fix or otherwise improve developer experience significantly. For example,
most "code style" refactoring changes require a thorough explanation why they
are useful, what downsides they have and why they *significantly* improve
developer experience or avoid serious programming bugs. Note that code style
is often a subjective matter. Unless they are explicitly mentioned to be
preferred in the [developer notes](/doc/developer-notes.md), stylistic code
changes are usually rejected.
-->
<!--
Bitcoin Core has a thorough review process and even the most trivial change
needs to pass a lot of eyes and requires non-zero or even substantial time
effort to review. There is a huge lack of active reviewers on the project, so
patches often sit for a long time.
-->
ACKs for top commit:
sipa:
ACK 7d8e1dec3b
practicalswift:
ACK 7d8e1dec3b
MarcoFalke:
ACK 7d8e1dec3b
Tree-SHA512: fe0f6e5fac1976d38dfb249517eef142dcb8837e178d7d199e5e854e3ab428822c6da9d96fe312293d39b6c6cac0c97896f3b5760013db200cccd729ae1b0710
5e47b19e50 tests: Add harness which fuzzes EvalScript and VerifyScript using a fuzzed signature checker (practicalswift)
Pull request description:
Add harness which fuzzes `EvalScript` and `VerifyScript` using a fuzzed signature checker.
Test this PR using:
```
$ make distclean
$ ./autogen.sh
$ CC=clang CXX=clang++ ./configure --enable-fuzz \
--with-sanitizers=address,fuzzer,undefined
$ make
$ src/test/fuzz/signature_checker
…
```
Closes #17986.
Top commit has no ACKs.
Tree-SHA512: a9988f8fa7919fe470756ca3e4e75764a589f590769aab452c8f4c254cf41667793e52131d470a12629ec3681fa7fc20091f371b8f3e3eec105674c2769e7d7e
Change comment from `The reason is that if the number of hashes in the list at a given time
is odd`, to ` The reason is that if the number of hashes in the list at a given level
is odd` (to be a bit more precise)
In PeerLogicValidation::PeerLogicValidation() we would schedule a lambda
function to execute later, capturing the local variable
`consensusParams` by reference.
Presumably this was considered safe because `consensusParams` is a
reference itself to a global variable which is not supposed to change,
but it can in tests.
Fixes https://github.com/bitcoin/bitcoin/issues/18372
fa36f3a295 refactor: move DUMP_BANS_INTERVAL to banman.h (MarcoFalke)
fadafb83cf scheduler: Make schedule* methods type safe (MarcoFalke)
fa70ccc6c4 scheduler: Use C++11 member initialization, add shutdown assert (MarcoFalke)
Pull request description:
Main benefit is that stuff like `15 * 60 * 1000` is replaced by `minutes{15}`
ACKs for top commit:
vasild:
ACK fa36f3a (code review, not tested)
ajtowns:
ACK fa36f3a295
jonatack:
ACK fa36f3a
Tree-SHA512: f35f1a1d643dfa676bd47474659f6492ed05cca04cdb556064b126f654a6a44a4b93fcaddcdcd41faf81b8f11439c11e5c7ab88685ba2eef12f7188843d17ad8
ec30a79f1c Fix UB with bench on genesis block (Gregory Sanders)
Pull request description:
During the loading of the genesis block, the bench print lines in ConnectTip divide by zero due to early return in ConnectBlock.
ACKs for top commit:
practicalswift:
ACK ec30a79f1c
sipa:
utACK ec30a79f1c
promag:
ACK ec30a79, `nBlocksTotal` is only used in logging.
Tree-SHA512: b3bdbb58d10d002a2293d7f99196b227ed9f4ca8c6cd08981e95cc964be47efed98b91fad276ee6da5cf7e6684610998ace7ce9bace172dd6c51c386d985b83c
fac52253f8 rpc: Document an RPCResult for all calls; Enforce at compile time (MarcoFalke)
fadd99f610 rpc: Add missing newline in RPCResult description (MarcoFalke)
Pull request description:
This documents the RPC Result (type and description, if applicable) everywhere it was missing. The patch can be reviewed with the `git diff` option `-W`/`--function-context`.
Also, code won't compile without having an RPCResult documented.
ACKs for top commit:
laanwj:
Lightly tested ACK fac52253f8
promag:
Tested ACK fac52253f8, built and verified listunspent help output.
Tree-SHA512: af2c1af1432beb944993776026c320814bfaecaf202f47359f5758849096ca7051ec6560395a2cc6678dcc111e7c9cf4917d0f0b221bdcf3ed1642e14d0e5b3c
fa7fea3654 refactor: Remove mempool global from net (MarcoFalke)
Pull request description:
To increase modularisation and simplify testing, remove the mempool global from net in favour of a mempool member.
This is done in the same way it was done for the connection manager global.
ACKs for top commit:
jnewbery:
code review ACK fa7fea3654
Tree-SHA512: 0e3e1eefa8d6e46367bc6991d5f36c636b15ae4a3bda99b6fe6715db3240771c3d87943c6eb257d69f31929fa2f1d0973e14fc9d1353a27551dbe746eae36857
fb15bfd99e Fix nit in getblockchaininfo (Steven Roose)
Pull request description:
Noticed that the statistics are not always shown.
ACKs for top commit:
laanwj:
ACK fb15bfd99e
promag:
ACK fb15bfd99e.
Tree-SHA512: bccbfdff03107d14967f6530eec0bcada7ba8eb16c61b829119533a73f2ead742a0da6a473b7962b15e25cd685c8f155506ab16d4a95b20352d3fd1b4b0164a3
7df0cf719f Replace remaining literals BTC with CURRENCY_UNIT (Daniel Kraft)
Pull request description:
This replaces one remaining instance of the literal `"BTC"` string with the `CURRENCY_UNIT` constant, as is done in most of the codebase already.
After this change, no instance of literal `"BTC"` remains anywhere in the RPC help texts.
ACKs for top commit:
MarcoFalke:
ACK 7df0cf719f
laanwj:
ACK 7df0cf719f
Tree-SHA512: 7f7d52b366e084c93a7d6a3c45b1bbfc4f4f50bca6956594077e6d46295977c8cc18499232878869869c73a8ab9a1c41245029ae7425a87cec2ccb0cb52eea13
686c5456f2 Fix missing header in sync.h (João Barbosa)
Pull request description:
`std::string` is referenced in `sync.h` but the relevant header is not explicitly included as required by current guideline. Furthermore on osx 10.14.6 with clang-900.0.31 the following error occurs:
```
In file included from threadinterrupt.cpp:6:
In file included from ./threadinterrupt.h:8:
./sync.h:206:21: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
std::string lockname;
```
ACKs for top commit:
practicalswift:
ACK 686c5456f2
laanwj:
ACK 686c5456f2
Tree-SHA512: 7c1acdfa5b0dd148d1114e14c9450d5907006e63e1a04e82ed8a1e29757925476e6f8ef6024b0c6d1bb596623115209ad580d5035be1e4785337bd01b738c9f2
e6e622e5a0 Implement O(1) OP_IF/NOTIF/ELSE/ENDIF logic (Pieter Wuille)
d0e8f4d5d8 [refactor] interpreter: define interface for vfExec (Anthony Towns)
89fb241c54 Benchmark script verification with 100 nested IFs (Pieter Wuille)
Pull request description:
While investigating what mechanisms are possible to maximize the per-opcode verification cost of scripts, I noticed that the logic for determining whether a particular opcode is to be executed is O(n) in the nesting depth. This issue was also pointed out by Sergio Demian Lerner in https://bitslog.wordpress.com/2017/04/17/new-quadratic-delays-in-bitcoin-scripts/, and this PR implements a variant of the O(1) algorithm suggested there.
This is not a problem currently, because even with a nesting depth of 100 (the maximum possible right now due to the 201 ops limit), the slowdown caused by this on my machine is around 70 ns per opcode (or 0.25 s per block) at worst, far lower than what is possible with other opcodes.
This PR mostly serves as a proof of concept that it's possible to avoid it, which may be relevant in discussions around increasing the opcode limits in future script versions. Without it, the execution time of scripts can grow quadratically with the nesting depth, which very quickly becomes unreasonable.
This improves upon #14245 by completely removing the `vfExec` vector.
ACKs for top commit:
jnewbery:
Code review ACK e6e622e5a0
MarcoFalke:
ACK e6e622e5a0🐴
fjahr:
ACK e6e622e5a0
ajtowns:
ACK e6e622e5a0
laanwj:
concept and code review ACK e6e622e5a0
jonatack:
ACK e6e622e5a0 code review, build, benches, fuzzing
Tree-SHA512: 1dcfac3411ff04773de461959298a177f951cb5f706caa2734073bcec62224d7cd103767cfeef85cd129813e70c14c74fa8f1e38e4da70ec38a0f615aab1f7f7
This replaces one remaining instance of the literal "BTC" string with
the CURRENCY_UNIT constant, as is done in most of the codebase already.
The other remaining instance (which is just part of a log message and thus
not really user-visible) is just removed.
After this change, no instance of literal "BTC" remains anywhere in the
non-Qt and non-test codebase.
09e25071f4 Cache parent xpub inside of BIP32PubkeyProvider (Andrew Chow)
deb791c7ba Only cache xpubs that have a hardened last step (Andrew Chow)
f76733eda5 Cache the immediate derivation parent xpub (Andrew Chow)
58f54b686f Add DescriptorCache* read_cache and DescriptorCache* write_cache to Expand and GetPubKey (Andrew Chow)
66c2cadc91 Rename BIP32PubkeyProvider.m_extkey to m_root_extkey (Andrew Chow)
df55d44d0d Track the index of the key expression in PubkeyProvider (Andrew Chow)
474ea3b927 Introduce DescriptorCache struct which caches xpubs (Andrew Chow)
Pull request description:
Improves the descriptor cache by changing it from a `std::vector<unsigned char>` to a newly introduced `DescriptorCache` class. Instead of serializing pubkeys and whatever else we would want to cache in a way that may not be backwards compatible, we instead create a `DescriptorCache` object and populate it. This object contains only an xpub cache. Since the only `PubkeyProvider` that used the cache is the `BIP32PubkeyProvider` we just have it store the xpubs instead of the pubkeys. This allows us to have both the parent xpub and the child xpubs in the same container. The map is keyed by `KeyOriginInfo`.
Sine we are caching `CExtPubKey`s in `DescriptorCache`, `BIP32PubKeyProviders` can use the cached parent xpubs to derive the children if unhardened derivation is used in the last step. This also means that we can still derive the keys for a `BIP32PubkeyProvider` that has hardened derivation steps. When combined with descriptor wallets, this should allow us to be able to import a descriptor with an `xprv` and hardened steps and still be able to derive from it. In that sense, this is an alternative to #18163
To test that this works, the tests have been updated to do an additional `Expand` at the `i + 1` position. This expansion is not cached. We then do an `ExpandFromCache` at `i + 1` and use the cache that was produced by the expansion at `i`. This way, we won't have the child xpubs for `i + 1` but we will have the parent xpubs. So this checks whether the parent xpubs are being stored and can be used to derive the child keys. Descriptors that have a hardened last step are skipped for this part of the test because that will always require private keys.
ACKs for top commit:
instagibbs:
code review re-re-ACK 09e25071f4
Sjors:
re-ACK 09e25071f4
Tree-SHA512: 95c8d0092274cdf115ce39f6d49dec767679abf3758d5b9e418afc308deca9dc6f67167980195bcc036cd9c09890bbbb39ec1dacffbfacdc03efd72a7e23b276
c8e24ddce3 [REFACTOR] Abstract out script execution out of VerifyWitnessProgram() (Pieter Wuille)
Pull request description:
This is a refactoring cherry-picked out of #17977. As it touches consensus code, I don't think this would ordinarily meet the bar for review cost vs benefit. However, it simplifies the changes for Taproot significantly, and if it's going to be necessitated by inclusion of that code, I may as well give it some additional attention by PRing it independently.
ACKs for top commit:
fjahr:
Re-ACK c8e24ddce3
theStack:
re-ACK c8e24ddce3
Empact:
Code Review Re-ACK c8e24ddce3
ajtowns:
ACK c8e24ddce3
jnewbery:
ACK c8e24ddce3
jonatack:
ACK c8e24dd
Tree-SHA512: 96c2aa5d2f9c7c802bcc008f5cde55b1dfedfaf42e34101331e6c0d594acdf6437661102dc939718f0877c20451336855dfbaa8aa8f57d9e722a7fa7329e3a46
a33cffbeab util: HelpExampleRpc formatting fixup (Jon Atack)
Pull request description:
Minor visual fixup of the HelpExampleRpc template; conforms to the JSON-RPC spec as per https://www.jsonrpc.org/specification#examples. (I'm... somewhat embarassed to open such a minor change, but this is what is shown in all the CLI/RPC help docs.)
ACKs for top commit:
laanwj:
ACK a33cffbeab
Tree-SHA512: 8f1dee080c224742fff60a33fec6f5fb1d59c9fa51f3f2a67bf2e1837dbfa25f12a69e34518936588940013b0e61f55378b4f1a571c47c3cb081ca5b245e1091
NotifyEntryAdded never had any subscribers so can be removed.
Since ConnectTrace no longer subscribes to NotifyEntryRemoved, there are
now no subscribers.
The CValidationInterface TransactionAddedToMempool and
TransactionRemovedFromMempool methods can now provide this
functionality. There's no need for a special notifications framework for
the mempool.
ConnectTrace used to subscribe to the mempool's NotifyEntryRemoved
callback to be notified of transactions removed for conflict. Since
PerBlockConnectTrace no longer tracks conflicted transactions,
ConnectTrace no longer requires these notifications.
The only CValidationInterface client that cares about transactions that
are removed from the mempool because of CONFLICT is the wallet.
Start using the TransactionRemovedFromMempool method to notify about
conflicted transactions instead of using the vtxConflicted vector in
BlockConnected.
8a2a652e6f Remove redundant type information from rpc docs (David O'Callaghan)
Pull request description:
Simple edit of the RPC calls to remove redundant text ("A json object/array ...") from the beginning of help.
Fixes: #18258
Top commit has no ACKs.
Tree-SHA512: cbbf760e0b7b4eda61c40b420ed77f5d878318e37b0eb13e63567212240b2c4ecc15d84030e98075e21c9ae9016539adfd201e5661ea824166a76d335180c32f
3e32499909 Change example addresses to bech32 (Yusuf Sahin HAMZA)
Pull request description:
This is a follow-up PR to #18197 that fixes RPCExamples.
Fixes #18185.
ACKs for top commit:
MarcoFalke:
ACK 3e32499909
jonatack:
ACK 3e32499
Tree-SHA512: c7a6410ef8b6e169016c2c5eac3e6b9501caabd0e8a0871ec31e56bfc44589f056d3f5cb55b5a13bba36f6c15136c2352f883e30e4dcc0997ffd36b27f9173b9
fab7d14ea5 test: Check that wait_until returns if time point is in the past (MarcoFalke)
Pull request description:
Add an explicit regression test for the condvar bug (#18227), so that this doesn't happen again
ACKs for top commit:
laanwj:
ACK fab7d14ea5
Tree-SHA512: 6ec0d0b3945cae87a001e367af34cca1953a8082b4a0d9f8a20d30acd1f36363e98035d4eb173ff786cf6692d352d41f960633415c46394af042eb44e3b5ad71
