30a6c99935 rpc: access some args by name (stickies-v)
bbb31269bf rpc: add named arg helper (stickies-v)
13525e0c24 rpc: add arg helper unit test (stickies-v)
Pull request description:
Adds string overloads for the `RPCHelpMan::Arg` and `RPCHelpMan::MaybeArg` helpers to be able to access RPC arguments by name instead of index number. Especially in RPCs with a large number of parameters, this can be quite helpful.
Example usage:
```cpp
const auto action{self.Arg<std::string>("action")};
```
Most of the LoC is adding test coverage and documentation updates. No behaviour change.
An alternative approach to #27788 with significantly less overhaul.
ACKs for top commit:
fjahr:
Code review ACK 30a6c99935
maflcko:
ACK 30a6c99935🥑
ryanofsky:
Code review ACK 30a6c99935. Nice change! Implementation is surprisingly simple and additional unit test coverage is welcome, too.
Tree-SHA512: 4904f5f914fe1d421d32f60edb7c5a028c8ea0f140a2f207a106b4752d441164e073066a6bf2e17693f859fe847815a96609d3cf521e0ac4178d8cd09362ea3d
fae0db555c refactor: Use chrono type for g_mock_time (MarcoFalke)
fa382d3dd0 test: Add missing Assert(mock_time_in >= 0s) to SetMockTime (MarcoFalke)
Pull request description:
Seems odd to have the assert in the *deprecated* function, but not in the other.
Fix this by adding it to the other, and by inlining the deprecated one.
Also, use chrono type for the global mocktime variable.
ACKs for top commit:
davidgumberg:
crACK fae0db555c
stickies-v:
ACK fae0db555c
Tree-SHA512: 630c2917422ff2a7fa307114f95f22ad3c205429ffe36e67f0b2650733e40c876289c1aecebe882a9123d3106db7606bd6eff067ed6e2ecb95765984d3fe8612
Remove obsolete `check_output` references in the comments and remove
the numbering of the Popen API methods, as they don't seem to provide a
value and just make diffs larger for future changes.
Checking the interface name is kind of brittle. In the age of network
namespaces and containers, there is no reason a loopback interface can't
be called differently.
Check for the `IFF_LOOPBACK` flag to detect loopback interface instead.
It's very hard to randomly construct a transaction that would be the
parent of an existing orphanage tx. For functions like
AddChildrenToWorkSet and GetChildren that take orphan parents, use a tx
that was previously constructed.
Copying util::Result values is less efficient than moving them because they
allocate memory and contain strings. Also this is needed to avoid compile
errors in https://github.com/bitcoin/bitcoin/pull/25722 which adds a
std::unique_ptr member to util::Result which implicity disables copying.
fd81a37239 net: attempts to connect to all resolved addresses when connecting to a node (Sergi Delgado Segura)
Pull request description:
This is a follow-up of #28155 motivated by https://github.com/bitcoin/bitcoin/pull/28155#discussion_r1362677038
## Rationale
Prior to this, when establishing a network connection via `CConnman::ConnectNode`, if the connection needed address resolution, a single address would be picked at random from the resolved addresses and our node would try to connect to it. However, this would lead to the behavior of `ConnectNode` being unpredictable when the address was resolved to various ips (e.g. the address resolving to IPv4 and IPv6, but we only support one of them).
This patches the aforementioned behavior by going over all resolved IPs until a valid one is found or until we
exhaust them.
ACKs for top commit:
mzumsande:
re-ACK fd81a37239 (just looked at diff, only small logging change)
achow101:
ACK fd81a37239
vasild:
ACK fd81a37239
Tree-SHA512: fa1ebc5c84fe61dd0a7fe1113ae2d594a75ad661c43ed8984a31fc9bc50f166b2759b0d8d84ee5dc247691eff78c8156fac970af797bbcbf67492eec0353fb58
`util::Result` objects are aggregates that can hold multiple fields with
different information. Currently Result objects can only hold a success value
of an arbitrary type or a single bilingual_str error message. In followup PR
https://github.com/bitcoin/bitcoin/pull/25722, Result objects may be able to
hold both success and failure values of different types, plus error and warning
messages.
Having a Result::operator= assignment operator that completely erases all
existing Result information before assigning new information is potentially
dangerous in this case. For example, code that looks like it is assigning a
warning value could erase previously-assigned success or failure values.
Conversely, code that looks like it is just assigning a success or failure
value could erase previously assigned error and warning messages.
To prevent potential bugs like this, disable Result::operator= assignment
operator.
It is possible in the future we may want to re-enable operator= in limited
cases (such as when implicit conversions are not used) or add a Replace() or
Reset() method that mimicks default operator= behavior. Followup PR
https://github.com/bitcoin/bitcoin/pull/25722 also adds a Result::Update()
method providing another way to update an existing Result object.
Co-authored-by: stickies-v <stickies-v@protonmail.com>
65951e0418 index: race fix, lock cs_main while 'm_synced' is subject to change (Ryan Ofsky)
Pull request description:
Fixes #29831 and #29863. Thanks to Marko for the detailed description of the issue.
The race occurs because a block could be connected and its event signaled in-between reading the 'next block' and setting `m_synced` during the index initial synchronization. This is because `cs_main` is not locked through the process of determining the final index sync state.
To address the issue, the `m_synced` flag set has been moved under `cs_main` guard.
ACKs for top commit:
fjahr:
Code review ACK 65951e0418
achow101:
ACK 65951e0418
ryanofsky:
Code review ACK 65951e0418
Tree-SHA512: 77286e22de164a27939d2681b7baa6552eb75e99c541d3b9631f4340d7dd01742667c86899b6987fd2d97799d959e0a913a7749b2b69d9e50505128cd3ae0e69
992c714451 common: Don't terminate on null character in UrlDecode (Fabian Jahr)
099fa57151 scripted-diff: Modernize name of urlDecode function and param (Fabian Jahr)
8f39aaae41 refactor: Remove hooking code for urlDecode (Fabian Jahr)
650d43ec15 refactor: Replace libevent use in urlDecode with our own code (Fabian Jahr)
46bc6c2aaa test: Add unit tests for urlDecode (Fabian Jahr)
Pull request description:
Fixes #29654 (as a side-effect)
Removing dependencies is a general goal of the project and the xz backdoor has been an additional wake up call recently. Libevent shows many of the same symptoms, few maintainers and slow releases. While libevent can not be removed completely over night we should start removing it’s usage where it's possible, ideally with the end goal to removing it completely.
This is a pretty easy win in that direction. The [`evhttp_uridecode` function from libevent](e0a4574ba2/http.c (L3542)) we were using in `urlDecode` could be easily emulated in fewer LOC. This also ports the [applicable test vectors over from libevent](https://github.com/libevent/libevent/blob/master/test/regress_http.c#L3430).
ACKs for top commit:
achow101:
ACK 992c714451
theStack:
Code-review ACK 992c714451
maflcko:
ACK 992c714451👈
stickies-v:
ACK 992c714451
Tree-SHA512: 78f76ae7ab3b6710eab2aaac20f55eb0da7803e057eaa6220e865f328666a5399ef1a479702aaf630b2f974ad3aa15e2b6adac9c11bc8c3d4be21e8af1667fea
The previous behavior was the result of casting the result returned from the libevent function evhttp_uridecode to std:string but this was probably not intended.
The point of this was to be able to build bitcoin-tx and bitcoin-wallet without libevent, see #18504.
Now that we use our own implementation of urlDecode this is not needed anymore.
Co-authored-by: stickies-v <stickies-v@protonmail.com>
b22901dfa9 Avoid explicitly computing diagram; compare based on chunks (Pieter Wuille)
Pull request description:
This merges the `BuildDiagramFromChunks` and `CompareFeeRateDiagram` introduced in #29242 into a single `CompareChunks` function, which operates on sorted chunk data rather than diagrams, instead computing the diagram on the fly.
This avoids the need for the construction of an intermediary diagram object, and removes the slightly arbitrary "all diagrams must start at (0, 0)" requirement.
Not a big deal, but I think the result is a bit cleaner and not really more complicated.
ACKs for top commit:
glozow:
reACK b22901d
instagibbs:
reACK b22901dfa9
Tree-SHA512: ca37bdf61d9a9cb5435f4da73e97ead33bf65828ad9af49b87336b1ece70db8ced1c21f517fc6eb6d616311c91f3da75ecae6b9bd42547133e3a3c5320b7816d
08f756bd37 Replace locale-dependent `std::strerror` with `SysErrorString` (Hennadii Stepanov)
d8e4ba4d05 refactor: Rename `subprocess.hpp` to follow our header name conventions (Hennadii Stepanov)
Pull request description:
This PR renames the header `*.hpp` --> `*.h` and adjusts the header guard name, which makes it available for processing by linters.
Fixed the following linter warning:
```
The locale dependent function strerror(...) appears to be used:
src/util/subprocess.h: std::runtime_error( err_msg + ": " + std::strerror(err_code) )
Unnecessary locale dependence can cause bugs that are very tricky to isolate and fix. Please avoid using locale-dependent functions if possible.
Advice not applicable in this specific case? Add an exception by updating the ignore list in /bitcoin/test/lint/lint-locale-dependence.py
^---- failure generated from lint-locale-dependence.py
```
ACKs for top commit:
TheCharlatan:
ACK 08f756bd37
Tree-SHA512: 57a2f01c20eb9552481e428a4969bd59e9ada9f784fe1a45cb62aa9c9152c8e950d336854f45af0e2e5dc7c7b2a1fb216c8f832e3d6ccfb457ad71b6e423231e
4d8d21320e sign: don't assume we are parsing a sane Miniscript (Antoine Poinsot)
Pull request description:
The script provided for signature might be externally provided, for instance by way of 'finalizepsbt'. Therefore the script might be ill-crafted, so don't assume pubkeys are always 32 bytes.
Thanks to Niklas for finding this.
FIxes https://github.com/bitcoin/bitcoin/issues/29851.
ACKs for top commit:
achow101:
ACK 4d8d21320e
furszy:
ACK 4d8d21320e with a small nuance that could be tackled in a follow-up by someone else (or never).
Tree-SHA512: 29b7948b56e6dc05eac1014d684f2129ab1d19cb1e5d304216c826b7057c0e1d84ceb18731b91124b680e17d90e38de9f9a5526e4f6ecc3ea816881a6599bb47
fa6ab0d020 rpc: Reword SighashFromStr error message (MarcoFalke)
Pull request description:
Put quotes around the parameter. In theory, `std::quoted` should be used, but that seems overkill.
This should avoid error messages such as `A valid sighash parameter is not a valid sighash parameter. (code -8)`.
Also, it should fix fuzz false positives when searching for internal bugs in the `rpc` fuzz target. For example, `ZGVzY3JpcHRvcnByb2Nlc3Nwc2J0XP9ce1tdXOVJbnRlcm5hbCBidWcgZGV0ZWN0ZWQAXQ0AHfcAXQ1p7TJv`.
ACKs for top commit:
dergoegge:
ACK fa6ab0d020
brunoerg:
utACK fa6ab0d020
Tree-SHA512: e2c0cc0126de61873a863af38b7b0a23d2dadd596ca0418dae2ad091e8acfb6a9d657c376d59187bb008989dc78c6b44fe518590e5217e4049a867b220c9fb18
fb4cc5f423 netbase: clean up Proxy logging (Matthew Zipkin)
Pull request description:
Follow up to #27375 and see https://github.com/bitcoin/bitcoin/pull/29649#issuecomment-2057456834
This removes an extra log message when we can't connect to our own proxy, and another when the proxy is invalid.
## Before #27375 if proxy is unreachable
```
2024-04-15T17:54:51Z connect() to 127.0.0.1:9999 failed after wait: Connection refused (61)
2024-04-15T17:54:52Z connect() to 127.0.0.1:9999 failed after wait: Connection refused (61)
2024-04-15T17:54:52Z connect() to 127.0.0.1:9999 failed after wait: Connection refused (61)
2024-04-15T17:54:53Z connect() to 127.0.0.1:9999 failed after wait: Connection refused (61)
2024-04-15T17:54:53Z connect() to 127.0.0.1:9999 failed after wait: Connection refused (61)
```
## After #27375 if unix proxy is unreachable:
```
2024-04-15T17:54:03Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T17:54:03Z Cannot connect to socket for /Users/matthewzipkin/Desktop/tor
2024-04-15T17:54:04Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T17:54:04Z Cannot connect to socket for /Users/matthewzipkin/Desktop/tor
2024-04-15T17:54:04Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T17:54:04Z Cannot connect to socket for /Users/matthewzipkin/Desktop/tor
2024-04-15T17:54:05Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T17:54:05Z Cannot connect to socket for /Users/matthewzipkin/Desktop/tor
```
## After this PR:
```
2024-04-15T18:18:51Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T18:18:51Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T18:18:52Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T18:18:52Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
```
ACKs for top commit:
tdb3:
CR ACK for fb4cc5f423
laanwj:
ACK fb4cc5f423
Tree-SHA512: f07b9f7f2ea9f4bc01780c09f0b076547108294a1fa7d158a0dd48d6d7351569e461e5cccf232b7b1413ce2e3679668e523e5a7c89cd58c909da76d3dcbc34de
4357158c47 wallet: return and display signer error (Sjors Provoost)
dc55531087 wallet: compare address returned by displayaddress (Sjors Provoost)
6c1a2cc09a test: use h marker for external signer mock (Sjors Provoost)
Pull request description:
* HWI returns the requested address: as a sanity check, we now compare that to what we expected
* external signer documentation now reflects that HWI alternatives must implement this check
* both RPC and GUI will now return an error text, rather than just fail (the GUI even failed silently in some cases)
ACKs for top commit:
brunoerg:
ACK 4357158c47
achow101:
ACK 4357158c47
Tree-SHA512: 4f56edf3846745c8e7d08ef55cf29e8bb468256457149377c5f02da097931f9ca0c06bdbd856dc2385cde4fd11e4dc3b634c5a48814ff27f5562c8a25d43da93
13adbf733f remove unneeded environment option from cpp-subprocess (Sebastian Falbesoner)
2088777ba0 remove unneeded cwd option from cpp-subprocess (Sebastian Falbesoner)
03ffb09c31 remove unneeded bufsize option from cpp-subprocess (Sebastian Falbesoner)
79c3036373 remove unneeded close_fds option from cpp-subprocess (Sebastian Falbesoner)
62db8f8e5a remove unneeded session_leader option from cpp-subprocess (Sebastian Falbesoner)
80d008c66d remove unneeded defer_spawn option from cpp-subprocess (Sebastian Falbesoner)
cececad7b2 remove unneeded preexec function option from cpp-subprocess (Sebastian Falbesoner)
633e45b2e2 remove unneeded shell option from cpp-subprocess (Sebastian Falbesoner)
Pull request description:
The newly introduced cpp-subprocess library provides a good number of options for the `Popen` class:
0de63b8b46/src/util/subprocess.hpp (L1009-L1020)
Some of them are either not fully implemented (`shell`, missing an implementation on Windows), implemented in an ugly way (e.g. using "Impoverished, meager, needy, truly needy version of type erasure" for `preexec_func` according to the author's own words) or simply unlikely to be ever needed for our external signer use-case (`defer_spawn`). Instead of maintaining incomplete and/or unneeded code, I'd suggest to get rid of it and only keep support for options if there is a strong reason for it.
ACKs for top commit:
achow101:
ACK 13adbf733f
hebasto:
re-ACK 13adbf733f.
Tree-SHA512: 8270da27891cb659da2ef6062a23f4b86331859b15ac27b79ae7433b14f5bd7efaba621f2b3ba1953708d0f38377a8bd23ef1cc0f28b9c152ac8958dd9eec6b0
c3e632b441 Bugfix: bitcoin-cli: Check length of peer.transport_protocol_type (Luke Dashjr)
Pull request description:
"v" would dereference beyond the string length, and "v10" would show as '1'
Turn both of these cases into a blank, like anything else unexpected currently is.
ACKs for top commit:
sipa:
utACK c3e632b441.
hernanmarino:
utACK c3e632b441
alfonsoromanz:
ACK c3e632b441
achow101:
ACK c3e632b441
Tree-SHA512: f641e4412521adae7c8c8e1f268bdaaa223d9048d8286e3df4b13905faaa0d601155ce581cd649f760cab2acc4122356fa94a44714f1f190845552100105eda0
The script provided for signature might be externally provided, for
instance by way of 'finalizepsbt'. Therefore the script might be
ill-crafted, so don't assume pubkeys are always 32 bytes.
Thanks to Niklas for finding this.
f2e3662e57 net: Decrease nMaxIPs when learning from DNS seeds (laanwj)
Pull request description:
Limit number of IPs learned from a single DNS seed to 32, to prevent the results from one DNS seed from dominating AddrMan. Note that the number of results from a UDP DNS query is bounded to 33 already, but it is possible for it to use TCP where a larger number of results can be returned.
Closes #16070.
ACKs for top commit:
Sjors:
utACK f2e3662e57
achow101:
ACK f2e3662e57
1440000bytes:
utACK f2e3662e57
mzumsande:
utACK f2e3662e57
Tree-SHA512: 3f108c2baba7adfedb8019daaf60aa00e628b38d3942e1319c7183a4683670be01929ced9e6372c8e983c902e8633f81fbef12d7cdcaadd7f77ed729c1019942
21d0e6c7b7 doc: release notes for PR 27679 (Matthew Zipkin)
791dea204e test: cover unix sockets in zmq interface (Matthew Zipkin)
c87b0a0ff4 zmq: accept unix domain socket address for notifier (Matthew Zipkin)
Pull request description:
This is a follow-up to https://github.com/bitcoin/bitcoin/pull/27375, allowing ZMQ notifications to be published to a UNIX domain socket.
Fortunately, libzmq handles unix sockets already, all we really have to do to support it is allow the format in the actual option.
[libzmq](https://libzmq.readthedocs.io/en/latest/zmq_ipc.html) uses the prefix `ipc://` as opposed to `unix:` which is [used by Tor](https://gitlab.torproject.org/tpo/core/tor/-/blob/main/doc/man/tor.1.txt?ref_type=heads#L1475) and now also by [bitcoind](a85e5a7c9a/doc/release-notes-27375.md (L5)) so we need to switch that internally.
As far as I can tell, [LND](d20a764486/zmq.go (L38)) supports `ipc://` and `unix://` (notice the double slashes).
With this patch, LND can connect to bitcoind using unix sockets:
Example:
*bitcoin.conf*:
```
zmqpubrawblock=unix:/tmp/zmqsb
zmqpubrawtx=unix:/tmp/zmqst
```
*lnd.conf*:
```
bitcoind.zmqpubrawblock=ipc:///tmp/zmqsb
bitcoind.zmqpubrawtx=ipc:///tmp/zmqst
```
ACKs for top commit:
laanwj:
Code review ACK 21d0e6c7b7
tdb3:
crACK for 21d0e6c7b7. Changes lgtm. Will follow up with some testing within the next few days as time allows.
achow101:
ACK 21d0e6c7b7
guggero:
Tested and code review ACK 21d0e6c7b7
Tree-SHA512: ffd50222e80dd029d903e5ddde37b83f72dfec1856a3f7ce49da3b54a45de8daaf80eea1629a30f58559f4b8ded0b29809548c0638cd1c2811b2736ad8b73030
016ed248ba fuzz: explicitly cap the vsize of RBFs for diagram checks (Greg Sanders)
Pull request description:
In master we are hitting a case where vsize transactions much larger than max standard size are causing an overflow in not-yet-exposed RBF diagram checking code: https://github.com/bitcoin/bitcoin/pull/29757#issuecomment-2049220195
`ConsumeTxMemPoolEntry` is creating entries with tens of thousands of sigops cost, causing the resulting RBFs to be "overly large".
To fix this I cause the fuzz test to stop adding transactions to the mempool when we reach a potential overflow of `int32_t`.
ACKs for top commit:
glozow:
ACK 016ed248ba
marcofleon:
ACK 016ed248ba. I ran libFuzzer on `package_rbf` on the current master branch until the overflow was encountered. Then I built the PR branch and ran the fuzzer using the crash input.
Tree-SHA512: b3ffc98d2c4598eb3010edd58b9370aab1441aafbb1044c83b2b90c17dfe9135b8de9dba475dd0108863c1ffedede443cd978e95231a41cf1f0715629197fa51
Prior to this, when establishing a network connection via CConnman::ConnectNode,
if the connection needed address resolution, a single address would be picked
at random from the resolved addresses and our node will try to connect to it. However,
this would lead to the behavior of ConnectNode being unpredictable when the address
was resolved to various ips (e.g. the address resolving to IPv4 and IPv6, but we only
support one of them).
This patches the aforementioned behavior by going over all resolved IPs until we find one
we can connect to or until we exhaust them.
