c3fa043ae5 doc: build: Fix instructions for msvc gui builds (David Gumberg)
Pull request description:
If the instructions in `doc/build-windows-msvc.md` are followed as-is, and "Developer (PowerShell|Command Prompt) for VS 2022" is used to execute the suggested build commands, the root directory of vcpkg (e.g. in VS 2022 Community edition: `C:\Program Files\Microsoft Visual Studio\2022\Community\VC\vcpkg`), is too long, and when vcpkg attempts to build any of the QT packages, it will fail because of build steps that require path lengths greater than Windows' `MAX_PATH` 260 character limit. This can be avoided without needing to move the vcpkg root dir by setting [`--x-buildtrees-root`](https://learn.microsoft.com/en-us/vcpkg/commands/common-options#buildtrees-root) to a short path, like `C:\vcpkg`.
See e.g. https://github.com/microsoft/vcpkg/issues/28451, https://github.com/microsoft/vcpkg/issues/28083, https://github.com/microsoft/vcpkg/issues/24751.
ACKs for top commit:
achow101:
ACK c3fa043ae5
hebasto:
ACK c3fa043ae5.
TheCharlatan:
ACK c3fa043ae5
Tree-SHA512: 7de11d38b9125de63e72415f79d82f18818123a1b37f679f2229c4c82f5628dd7d1039dbc5dcdf1bc1c8c382e3e29de74a31d256e73872cbf1fa2687c52185ca
c9136ca906 validation: fix issue with an interrupted -reindex (Martin Zumsande)
a2675897e2 validation: Don't loop over all chainstates in LoadExternalBlock (Martin Zumsande)
Pull request description:
If a user interrupts a reindex while it is iterating over the block files, it will continue to reindex with the next node start (if the `-reindex` arg is dropped, otherwise it will start reindexing from scratch).
However, due to an early call to `ActivateBestChainState()` that only exists to connect the genesis block during
the original `-reindex`, it wil start connecting blocks immediately before having iterated through all block files.
Because later headers above the minchainwork threshold won't be loaded in this case, `-assumevalid` will not
be applied and the process is much slower due to script validation being done.
Fix this by only calling `ActivateBestChainState()` here if Genesis is not connected yet (equivalent to `ActiveHeight() == -1`).
Also simplify this spot by only doing this for the active chainstate instead of looping over all chainstates (first commit).
This issue was discussed in the thread below https://github.com/bitcoin/bitcoin/pull/31346#discussion_r1856824817, the impact on assumevalid was found by l0rinc.
The fix can be tested by manually aborting a `-reindex` e.g. on signet and observing in the debug log the order in which blockfiles are indexed / blocks are connected with this branch vs master.
ACKs for top commit:
achow101:
ACK c9136ca906
ryanofsky:
Code review ACK c9136ca906. Only comments changed since last review. Appreciate the new comments, I think they make a little clearer what things code is trying to do and what things are just side-effects.
TheCharlatan:
Re-ACK c9136ca906
Tree-SHA512: 6f34abc317ad7e605ccc0c2f4615e4ea6978223d207f80f768f39cc135a9ac0adf31681fadfa2aed45324a5d27a4f68c5e118ee7eec18ca5c40ef177caa9cc47
fa8e0956c2 rpc: Remove deprecated dummy alias for listtransactions::label (MarcoFalke)
Pull request description:
The RPC arg is not a dummy, but a label, so offering an undocumented alias is inconsistent with all other label interfaces and confusing at best, if not entirely unused.
Fix it by removing the deprecated alias.
This pull is a breaking change, but it should be limited, because it only affects someone using the deprecated named arg on this RPC. I can't imagine anyone doing this, because in all other places where label args are accepted, they are called `label`. If someone really didn't use `label` here as named arg, it would be trivial and less confusing for them to fix it up.
ACKs for top commit:
achow101:
ACK fa8e0956c2
rkrux:
tACK fa8e0956c2
ryanofsky:
Code review ACK fa8e0956c2
Tree-SHA512: 0d0f3f53237ff9fac8c065b7d0a4245f5ff86efa427dbeeca711765494b7315a9d72b44751d346c76422847daf3d7ff90dbccb5ba200b089fb96128bd95da9f0
f0e5e4cdbe test: Add test for rpcwhitelistdefault (naiyoma)
Pull request description:
This PR adds tests for `rpcwhitelistdefault.` The implementation is a continuation of this [PR](https://github.com/bitcoin/bitcoin/pull/17805).
Applied suggestions to include the tests in` rpc_whitelist.py` and to use a single node.
PR covers three test cases:
- rpcwhitelistdefault = 0, no permissions
- rpcwhitelistdefault = 1, no permissions
- rpcwhitelistdefault = 1, with user permissions
I didn't add tests for rpcwhitelistdefault = 0 with user permissions since that is already tested here: [rpc_whitelist.py#L77](https://github.com/bitcoin/bitcoin/blob/master/test/functional/rpc_whitelist.py#L77).
ACKs for top commit:
achow101:
ACK f0e5e4cdbe
ryanofsky:
Code review ACK f0e5e4cdbe. PR seems very clear and simple, moving 1 test and adding 3 new tests.
ismaelsadeeq:
Tested and Code review ACK f0e5e4cdbe
Tree-SHA512: c3652940d2f23746e769ebe834e43dee47b7af8f258cbb133e38663aa8a05a1a8d0194d3008c3a10b0c54d11b5b95420c9cad0aa761c0fc1b9559277443b0696
a759ea3e92 doc: Improve dependencies documentation (Nicola Leonardo Susca)
Pull request description:
Initially there was a distinction between the compiler dependencies and
other required dependencies (refs https://github.com/bitcoin/bitcoin/pull/23565) but the distinction was
removed (refs https://github.com/bitcoin/bitcoin/pull/24585) which is why having two distinct tables could lead
to confusion now.
ACKs for top commit:
achow101:
ACK a759ea3e92
hodlinator:
re-ACK a759ea3e92
rkrux:
ACK a759ea3e92
Tree-SHA512: 14aaf9356d65bd150c9993dcbc51b1b98c835a760b95e6d91e69460c97c18f1dd10eb52b9f1d70129e6aa5e361af3a55619fd35787ed4e1ec48909568adbb604
cd4bfaee10 net: reduce CAddress usage to CService or CNetAddr (Vasil Dimov)
Pull request description:
Using `CAddress` when only `CService` or `CNetAddr` is needed is excessive and confusing. Fix those occurrences to use the class they need:
* `CConnman::CalculateKeyedNetGroup()` needs `CNetAddr`, not `CAddress`, thus change its argument.
* Both callers of `CConnman::CreateNodeFromAcceptedSocket()` create a dummy `CAddress` from `CService`, so use `CService` instead.
* `GetBindAddress()` only needs to return `CService`.
* `CNode::addrBind` only needs to be `CService`.
ACKs for top commit:
Sjors:
ACK cd4bfaee10
achow101:
ACK cd4bfaee10
hodlinator:
ACK cd4bfaee10
laanwj:
Code review ACK cd4bfaee10
Tree-SHA512: 0b41c1519784eeeaf9926c6a4d24f583b90c3376741f37a3199a3808b0dd6d143d3f929bd7c06f87b031f4fc1c2bd7a6dfc7d715ec1f79bf36b862c00fd67085
81c174e318 cmake: Refer to the configure log instead of printing PIE test error (Hennadii Stepanov)
65a0920ca6 cmake: Add `CheckLinkerSupportsPIE` module (Hennadii Stepanov)
Pull request description:
This new module is a wrapper around CMake's `CheckPIESupported` module that fixes an upstream bug.
See: https://gitlab.kitware.com/cmake/cmake/-/issues/26463.
Fixes https://github.com/bitcoin/bitcoin/issues/30771.
ACKs for top commit:
theuni:
utACK 81c174e318.
vasild:
ACK 81c174e318
Tree-SHA512: 77d7022238551a4e69c59d1fe6b78975bb552cbbed5339459853d7ebf0086813036081f464fed230be330b3bd7d6cf8590b536b064028d2f786d6ae40f342f95
9b033bebb1 cmake: rename Kernel component to bitcoinkernel for consistency (Cory Fields)
2e0c92558e cmake: add and use install_binary_component (Cory Fields)
0264c5d86c cmake: use per-target components for bitcoin-qt and bitcoin-gui (Cory Fields)
fb0546b1c5 ci: don't try to install for a fuzz build (Cory Fields)
Pull request description:
This makes it possible to build/install only the desired binaries regardless of the configuration.
For consistency, the component names match the binary names. `Kernel` and `GUI` have been renamed.
Additionally it fixes #31762 by installing only the manpages for the configured targets (and includes them in the component installs for each).
Also fixes #31745.
Alternative to #31765 which is (imo) more correct/thorough.
Can be tested using (for ex):
```bash
$ cmake -B build
$ cmake --build build -t bitcoind -t bitcoin-cli
$ cmake --install build --component bitcoind
$ cmake --install build --component bitcoin-cli
```
ACKs for top commit:
hebasto:
ACK 9b033bebb1.
TheCharlatan:
Re-ACK 9b033bebb1
stickies-v:
re-ACK 9b033bebb1
Tree-SHA512: fd4818e76f190dbeafbf0c246b466f829771902c9d6d7111ed917093b811c8a5536a4a45e20708f73e7f581d6cb77c8e61cfa69e065788dcf0886792f553a355
dead908654 cmake: Improve compatibility with Python version managers (Hennadii Stepanov)
Pull request description:
This PR resolves the issue [highlighted](https://github.com/bitcoin/bitcoin/pull/31411#issuecomment-2516745547) in https://github.com/bitcoin/bitcoin/pull/31411:
> Here's another case where CMake just picks some other Python...
The fix leverages two [hints](https://cmake.org/cmake/help/latest/module/FindPython3.html#hints) for the CMake `FindPython3` module:
1. `Python3_FIND_FRAMEWORK` is set to `LAST`. This ensures that Unix-style package components are preferred over frameworks on macOS. As a side effect, the `FindPython3` module reports a shim or symlink (e.g., from `pyenv`) rather than the underlying framework's binary. The module's output aligns with the result of the `which` command.
2. `Python3_FIND_UNVERSIONED_NAMES` is set to `FIRST`. This supports scenarios where tools like `pyenv`—which use shims—have multiple Python versions installed.
Here are examples of output on my macOS 15.1.1 (Intel) with installed Homebrew's [Python 3.13.0](https://formulae.brew.sh/formula/python@3.13):
- without any Python version manager:
```
% which -a python3
/usr/local/bin/python3
/usr/bin/python3
% cmake -B build
<snip>
-- Found Python3: /usr/local/bin/python3 (found suitable version "3.13.0", minimum required is "3.10") found components: Interpreter
<snip>
```
- using `pyenv`:
```
% pyenv versions
system
* 3.10.14 (set by /Users/hebasto/dev/bitcoin/.python-version)
3.12.8
3.13.1
% which -a python3
/Users/hebasto/.pyenv/shims/python3
/usr/local/bin/python3
/usr/bin/python3
% cmake -B build
<snip>
-- Found Python3: /Users/hebasto/.pyenv/shims/python3 (found suitable version "3.10.14", minimum required is "3.10") found components: Interpreter
<snip>
```
Both variables, `Python3_FIND_FRAMEWORK` and `Python3_FIND_UNVERSIONED_NAMES`, can still be overridden by the user via the command line if needed.
ACKs for top commit:
theuni:
No opinion on the python selection changes themselves, but code-review ACK dead908654
willcl-ark:
ACK dead908654
Tree-SHA512: 69f8541223e5b6c35c892b4ba2a2dcfc24b41a10cf20accc75d3008b16434db8a9240c99c886c3a4566ba24269c5b0e0d856357891811f0a77b39f4afbee3634
70398ae05b mapport: make ProcessPCP void (Antoine Poinsot)
9e6cba2988 mapport: remove unnecessary 'g_mapport_enabled' (Antoine Poinsot)
8fb45fcda0 mapport: remove unnecessary 'g_mapport_current' variable (Antoine Poinsot)
1b223cb19b mapport: merge DispatchMapPort into StartMapPort (Antoine Poinsot)
9bd936fa34 mapport: drop unnecessary function (Antoine Poinsot)
2a6536ceda mapport: rename 'use_pcp' to 'enable' (Antoine Poinsot)
c4e82b854c mapport: make 'enabled' and 'current' bool (Antoine Poinsot)
Pull request description:
Followup to #31130, this does a couple cleanups to `src/mapport.*` to clarify the logic now that there is a single protocol option for port mapping.
ACKs for top commit:
laanwj:
Code review ACK 70398ae05b
TheCharlatan:
ACK 70398ae05b
Tree-SHA512: d9a3ab4fcd59a7cf4872415c40cc7ac3a98dfc5aa25e195d4df880bb588bac286c30c3471e9d9499de379a75f45dcd0a82019eba3cb9f342004ae1482d0ba075
2434aeab62 depends: avoid an unset CMAKE_OBJDUMP (fanquake)
Pull request description:
Similar to #31840, currently our Linux toolchain file contains:
```bash
set(CMAKE_AR "aarch64-linux-gnu-ar")
set(CMAKE_RANLIB "aarch64-linux-gnu-ranlib")
set(CMAKE_STRIP "aarch64-linux-gnu-strip")
set(CMAKE_OBJCOPY "aarch64-linux-gnu-objcopy")
set(CMAKE_OBJDUMP "")
```
`objdump` is currently only used for the macOS cross build, where it's `llvm-objdump`, but we should be consistent in producing a toolchain file that points to actual tools, rather than leaving variables unset.
ACKs for top commit:
hebasto:
ACK 2434aeab62.
theuni:
utACK 2434aeab62
Tree-SHA512: 65f6b7b9cae79e9c0784c108709139125e52d8f2818afbea5f719bc1b6dc338b870abbdfcb174ae541c0027a7ac07cb56012735b7a37b58b9a6e55a48c0257cf
0a02e7fdea test: deduplicates p2p_tx_download constants (Sergi Delgado Segura)
Pull request description:
Some of the networking constants defined in p2p_tx_download.py are more generally defined in p2p.py
ACKs for top commit:
i-am-yuvi:
re-ACK 0a02e7fdea
maflcko:
review ACK 0a02e7fdea🔖
danielabrozzoni:
re-ACK 0a02e7fdea
tdb3:
re ACK 0a02e7fdea
Tree-SHA512: 05fc114a32b6b42a7c57563a38f1a8921e0bb224c4b124ae9d395c3a1105ae6e9cdfc62f603f4f2dee55cef5f6a6ed400d328740ad84fbd3093c5e0f3fb2982a
bb0879ddab test: check `scanning` field from `getwalletinfo` (brunoerg)
Pull request description:
During a rescan, check that `getwalletinfo` returns properly information (the scanning field) about it.
ACKs for top commit:
maflcko:
lgtm ACK bb0879ddab
arejula27:
ACK [`bb0879d`](bb0879ddab)
achow101:
ACK bb0879ddab
BrandonOdiwuor:
Code Review ACK bb0879ddab
Prabhat1308:
re-ACK [`bb0879d`](bb0879ddab)
Tree-SHA512: 9bca1c1e813bf4f61a5621bdc0a5f5c2bcfb388ffe9dfacb821bf6954f6e0880140d72258dc93ab6b84efb54f55c682a17aebd42f6559d6cfac9998e6bc4e5b9
af76664b12 test: Test migration of a solvable script with no privkeys (Ava Chow)
17f01b0795 test: Test migration of taproot output scripts (Ava Chow)
1eb9a2a39f test: Test migration of miniscript in legacy wallets (Ava Chow)
e8c3efc7d8 wallet migration: Determine Solvables with CanProvide (Ava Chow)
fa1b7cd6e2 migration: Skip descriptors which do not parse (Ava Chow)
440ea1ab63 legacy spkm: use IsMine() to extract watched output scripts (Ava Chow)
b777e84cd7 legacy spkm: Move CanProvide to LegacyDataSPKM (Ava Chow)
b1ab927bbf tests: Test migration of additional P2WSH scripts (Ava Chow)
c39b3cfcd1 test: Extra verification that migratewallet migrates (Ava Chow)
Pull request description:
The legacy wallet `IsMine()` is essentially a black box that would tell us whether the wallet is watching an output script. In order to migrate legacy wallets to descriptor wallets, we need to be able to compute all of the output scripts that a legacy wallet would watch. The original approach for this was to understand `IsMine()` and write a function which would be its inverse. This was partially done in the original migration code, and attempted to be completed in #30328. However, further analysis of `IsMine()` has continued to reveal additional edge cases which make writing an inverse function increasingly difficult to verify correctness.
This PR instead changes migration to utilize `IsMine()` to produce the output scripts by first computing a superset of all of the output scripts that `IsMine()` would watch and testing each script against `IsMine()` to filter for the ones that actually are watched. The superset is constructed by computing all possible output scripts for the keys and scripts in the wallet - for keys, every key could be a P2PK, P2PKH, P2WPKH, and P2SH-P2WPKH; for scripts, every script could be an output script, the redeemScript of a P2SH, the witnessScript of a P2WSH, and the witnessScript of a P2SH-P2WSH.
Additionally, the legacy wallet can contain scripts that are redeemScripts and witnessScripts, while not watching for any output script utilizing that script. These are known as solvable scripts and are migrated to a separate "solvables" wallet. The previous approach to identifying these solvables was similar to identifying output scripts - finding known solvable conditions and computing the scripts. However, this also can miss scripts, so the solvables are now identified in a manner similar to the output scripts but using the function `CanProvide()`. Using the same superset as before, all output scripts which are `ISMINE_NO` are put through `CanProvide()` which will perform a dummy signing and then a key lookup to determine whether the legacy wallet could provide any solving data for the output script. The scripts that pass will have their descriptors inferred and the script included in the solvables wallet.
The main downside of this approach is that `IsMine()` and `CanProvide()` can no longer be deleted. They will need to be refactored to be migration only code instead in #28710.
Lastly, I've added 2 test cases for the edge cases that prompted this change of approach. In particular, miniscript witnessScripts and `rawtr()` output scripts are solvable and signable in a legacy wallet, although never `ISMINE_SPENDABLE`.
ACKs for top commit:
sipa:
Code review ACK af76664b12d8611b606a7e755a103a20542ee539; I did not review the tests in detail.
brunoerg:
code review ACK af76664b12
rkrux:
ACK af76664b12
Tree-SHA512: 7f58a90de6f38fe9801fb6c2a520627072c8d66358652ad0872ff59deb678a82664b99babcfd874288bebcb1487d099a77821f03ae063c2b4cbf2d316e77d141
7afeaa2469 test: `-debug=0` and `-debug=none` behave similarly to `-nodebug` (Daniela Brozzoni)
a8fedb36a7 logging: Ensure -debug=0/none behaves consistently with -nodebug (Daniela Brozzoni)
d39d521d86 test: `-nodebug` clears previously set debug options (Daniela Brozzoni)
Pull request description:
Previously, -nodebug cleared all prior -debug configurations in the command line while allowing subsequent debug options to be applied.
However, -debug=0 and -debug=none completely disabled debugging, even for categories specified afterward.
This commit ensures consistency by making -debug=0 and -debug=none behave like -nodebug: they now clear previously set debug configurations but do not disable debugging for categories specified later.
See https://github.com/bitcoin/bitcoin/pull/30529#discussion_r1930956563
ACKs for top commit:
hodlinator:
re-ACK 7afeaa2469
ryanofsky:
Code review ACK 7afeaa2469. Nicely implemented change with test and release notes, and I like how the test is implemented as the first commit.
maflcko:
review ACK 7afeaa2469👡
Tree-SHA512: c69b17ff10da6c88636bd01918366dd408832e70f2d0a7b951e9619089e89c39282db70398ba2542d3aa69a2fe6b6a0a01638b3225aff79d234d84d3067f2caa
Similar to #31840, currently our Linux toolchain file contains:
```bash
set(CMAKE_AR "aarch64-linux-gnu-ar")
set(CMAKE_RANLIB "aarch64-linux-gnu-ranlib")
set(CMAKE_STRIP "aarch64-linux-gnu-strip")
set(CMAKE_OBJCOPY "aarch64-linux-gnu-objcopy")
set(CMAKE_OBJDUMP "")
```
`objdump` is currently only used for the macOS cross build, where it's
`llvm-objdump`, but we should be consistent in producing a toolchain
file that points to actual tools, rather than leaving variables unset.
* `CConnman::CalculateKeyedNetGroup()` needs `CNetAddr`, not `CAddress`,
thus change its argument.
* Both callers of `CConnman::CreateNodeFromAcceptedSocket()` create a
dummy `CAddress` from `CService`, so use `CService` instead.
* `GetBindAddress()` only needs to return `CService`.
* `CNode::addrBind` only needs to be `CService`.
e3c0152769 cmake: Copy `cov_tool_wrapper.sh.in` to the build tree (Hennadii Stepanov)
Pull request description:
This PR ensures that `cov_tool_wrapper.sh.in` is available when invoking the `Coverage.cmake` script from any directory.
Here is an example of usage on Ubuntu 24.10 with the default GCC 14.2.0:
```
$ cmake -B build -DCMAKE_BUILD_TYPE=Coverage -DCMAKE_CXX_FLAGS="-fprofile-update=atomic" -DCMAKE_C_FLAGS="-fprofile-update=atomic"
$ cmake --build build -j $(nproc)
$ cd ..
$ cmake -DJOBS=$(nproc) -DLCOV_OPTS="--ignore-errors inconsistent,inconsistent" -P bitcoin/build/Coverage.cmake
```
Fixes https://github.com/bitcoin/bitcoin/issues/31638.
ACKs for top commit:
theuni:
utACK e3c0152769
Tree-SHA512: ccfc8e893567f199d9b05ea3916cac06fca89c5892cc7492d5251c331c35408222fd918ed08017515e2dfca10970ae3f633b3917bfb7037db539559e71d7f711
If the instructions are followed as-is, and "Developer
(PowerShell|Command Prompt) for VS 2022" is used to execute the
suggested build commands, the root directory of vcpkg (e.g. in VS 2022
Community edition: `C:\Program Files\Microsoft Visual
Studio\2022\Community\VC\vcpkg`), is too long, and when vcpkg attempts
to build any of the QT packages, it will fail because of build steps
that require path lengths greater than Windows' `MAX_PATH` 260 character
limit. This can be avoided without needing to move the vcpkg root dir by
setting `--x-buildtrees-root` to a short path, like `C:\vcpkg`.
3edaf0b428 depends: add missing Darwin objcopy (fanquake)
Pull request description:
Our CMake toolchain for a Darwin cross build currently contains:
```bash
set(CMAKE_AR "/usr/bin/llvm-ar")
set(CMAKE_RANLIB "/usr/bin/llvm-ranlib")
set(CMAKE_STRIP "/usr/bin/llvm-strip")
set(CMAKE_OBJCOPY "arm64-apple-darwin-objcopy")
set(CMAKE_OBJDUMP "/usr/bin/llvm-objdump")
```
`objcopy` isn't currently used for the Darwin build (only for Linux and splitting the debug symbols), but we shouldn't be producing a toolchain file that refers to nonexistent tools.
ACKs for top commit:
laanwj:
Code review ACK 3edaf0b428
theuni:
utACK 3edaf0b428
Tree-SHA512: b74deb9f3f053c37d03505e698419d4a14131131f12a042dab698a81f2ad76b71fd55c1d1afd5f5822cc50fdaad5acdab15a8b20626c56f705179add1165449f
f89f16846e depends: Fix compiling `libevent` package on NetBSD (Hennadii Stepanov)
Pull request description:
Libevent [introduced](https://github.com/libevent/libevent/pull/909) the [`typeof`](https://gcc.gnu.org/onlinedocs/gcc/Typeof.html) C language extension in the NetBSD-specific code, which was pulled into our depends in https://github.com/bitcoin/bitcoin/pull/21991.
However, GCC [states](https://gcc.gnu.org/onlinedocs/gcc/Alternate-Keywords.html):
> the various `-std` options disable certain keywords.
Due to our use of b042c4f053/depends/hosts/netbsd.mk (L1)
the `typeof` keyword is disabled, resulting in a compilation error:
```
$ gmake -C depends libevent CC=/usr/pkg/gcc14/bin/gcc CXX=/usr/pkg/gcc14/bin/g++
<snip>
[ 37%] Building C object CMakeFiles/event_core_static.dir/kqueue.c.o
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c: In function 'kq_setup_kevent':
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:56:27: error: implicit declaration of function 'typeof' [-Wimplicit-function-declaration]
56 | #define INT_TO_UDATA(x) ((typeof(((struct kevent *)0)->udata))(intptr_t)(x))
| ^~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:190:30: note: in expansion of macro 'INT_TO_UDATA'
190 | out->udata = INT_TO_UDATA(ADD_UDATA);
| ^~~~~~~~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:56:64: error: expected expression before 'intptr_t'
56 | #define INT_TO_UDATA(x) ((typeof(((struct kevent *)0)->udata))(intptr_t)(x))
| ^~~~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:190:30: note: in expansion of macro 'INT_TO_UDATA'
190 | out->udata = INT_TO_UDATA(ADD_UDATA);
| ^~~~~~~~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:56:27: error: called object is not a function or function pointer
56 | #define INT_TO_UDATA(x) ((typeof(((struct kevent *)0)->udata))(intptr_t)(x))
| ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:190:30: note: in expansion of macro 'INT_TO_UDATA'
190 | out->udata = INT_TO_UDATA(ADD_UDATA);
| ^~~~~~~~~~~~
gmake[3]: *** [CMakeFiles/event_core_static.dir/build.make:328: CMakeFiles/event_core_static.dir/kqueue.c.o] Error 1
<snip>
```
This PR resolves this issue by following GCC's [recommendation](https://gcc.gnu.org/onlinedocs/gcc/Typeof.html):
> write `__typeof__` instead of `typeof`.
ACKs for top commit:
fanquake:
ACK f89f16846e
Tree-SHA512: c0d2e535408db120535781f8518c616b0f5a39b1c6babb2a74e8e0565348aaf00b0f5a93cac0af7cf6d6bf028d5d58763fe71b3969ed9c7059fa7c3dca9d084c
12fa9511b5 build: simplify dependency graph (Cory Fields)
c4e498300c build: avoid unnecessary dependencies on generated headers (Cory Fields)
Pull request description:
These changes speed up my build (default config/options/targets) by roughly 10%. I suspect the difference may be more significant in other build configs.
Before:
> $ time cmake --build build -j24
> real3m26.932s
After:
> $ time cmake --build build -j24
> real3m7.556s
Generally they allow for jobservers (either `make -jX` or `ninja`) to be better utilized. This can be verified using `top` while building and looking at the number of compiles running at any given time before/after these changes. Before, it's easy to observe periods of stalling when only one or two compiles are happening. After these changes, the compiler process count should mostly match the number of jobs given (`-jX`) until it falls off at the end.
---
The first commit sets [DEPENDS_EXPLICIT_ONLY](https://cmake.org/cmake/help/latest/command/add_custom_command.html#command:add_custom_command) for commands which generate our test header files. Without this option, `test_bitcoin`'s generated headers won't be built until all of its other dependencies have been built. This introduces a significant stall in the build, though currently only Ninja benefits from this being set, and only CMake >= 3.27 understands it.
Example from a generated `build.ninja`:
Before:
> \# Custom command for src/test/data/base58_encode_decode.json.h
>
> build src/test/data/base58_encode_decode.json.h | ${cmake_ninja_workdir}src/test/data/base58_encode_decode.json.h: CUSTOM_COMMAND /home/cory/dev/bitcoin/src/test/data/base58_encode_decode.json /home/cory/dev/bitcoin/cmake/script/GenerateHeaderFromJson.cmake || libcrc32c.a libcrc32c_sse42.a libleveldb.a libminisketch.a minisketch_clmul src/bitcoin_clientversion src/crypto/libbitcoin_crypto.a src/crypto/libbitcoin_crypto_avx2.a src/crypto/libbitcoin_crypto_sse41.a src/crypto/libbitcoin_crypto_x86_shani.a src/generate_build_info src/libbitcoin_cli.a src/libbitcoin_common.a src/libbitcoin_consensus.a src/libbitcoin_node.a src/secp256k1/src/libsecp256k1.a src/secp256k1/src/secp256k1_precomputed src/test/util/libtest_util.a src/univalue/libunivalue.a src/util/libbitcoin_util.a src/wallet/libbitcoin_wallet.a src/zmq/libbitcoin_zmq.a
After:
> \# Custom command for src/test/data/base58_encode_decode.json.h
>
> build src/test/data/base58_encode_decode.json.h | ${cmake_ninja_workdir}src/test/data/base58_encode_decode.json.h: CUSTOM_COMMAND /home/cory/dev/bitcoin/src/test/data/base58_encode_decode.json /home/cory/dev/bitcoin/cmake/script/GenerateHeaderFromJson.cmake
---
The second commit is more significant. It sets [CMAKE_OPTIMIZE_DEPENDENCIES](https://cmake.org/cmake/help/latest/prop_tgt/OPTIMIZE_DEPENDENCIES.html) globally, which allows the objects of static libs to be built in parallel when one lib depends on the other. This can be set as a per-lib property, ~but I don't see any need for that as we don't currently have any edge-cases where this wouldn't be ok. If those should arise, we could always disable on a per-lib basis~.
Edit: turns out this triggers an [upstream bug](https://gitlab.kitware.com/cmake/cmake/-/issues/24058), which I guess can be considered an edge-case until fixed in CMake. I've added 2 per-lib opt-outs as a result.
Example:
Before:
> \# Link the static library src/libbitcoin_cli.a
>
> build src/libbitcoin_cli.a: CXX_STATIC_LIBRARY_LINKER__bitcoin_cli_RelWithDebInfo src/CMakeFiles/bitcoin_cli.dir/compat/stdin.cpp.o src/CMakeFiles/bitcoin_cli.dir/rpc/client.cpp.o || src/univalue/libunivalue.a
After:
> \# Link the static library src/libbitcoin_cli.a
>
> build src/libbitcoin_cli.a: CXX_STATIC_LIBRARY_LINKER__bitcoin_cli_RelWithDebInfo src/CMakeFiles/bitcoin_cli.dir/compat/stdin.cpp.o src/CMakeFiles/bitcoin_cli.dir/rpc/client.cpp.o
>
ACKs for top commit:
l0rinc:
utACK 12fa9511b5
hebasto:
ACK 12fa9511b5.
Tree-SHA512: f85f507e70cdc06acd07542161d9f9b8edf9ba866f08c8ef17aaaed770fa11530a27521c4413456d863463a6e77d4d6983fa623a64e17bbd602c2bc70aacc112
fa952acdb6 ci: Skip read-write of default env vars (MarcoFalke)
Pull request description:
If they remain unset, they use the default anyway. Except for `USER`, but this seems unused anyway.
Can be checked via:
```
sh-5.2# touch /tmp/empty_env
sh-5.2# podman run --rm --env-file /tmp/empty_env 'ubuntu:24.04' env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
container=podman
HOME=/root
HOSTNAME=19ece5c9e052
ACKs for top commit:
0xB10C:
ACK fa952acdb6
Prabhat1308:
utACK [fa952ac](fa952acdb6)
Tree-SHA512: fe0c173b23cfda3025306303a44ffe32ecc57c2e0e1a2376594696f9887ed22f5105da84e898e790041bf15a4aa42a365fba016710ad269d439dda691977be90
fa3a4eafa1 test: Remove stale gettime test (MarcoFalke)
Pull request description:
The `gettime` test is stale:
* It was added to sanity check the `time` implementation in the mingw toolchain to catch a 32-bit vs 64-bit mismatch in commit eaafa23cbd. However, since commit 0000a63689, `std::chrono::system_clock` is used.
* Even though `system_clock` may also return incorrect values, such an error should affect *all* `GetTime<>` calls (not only the second-precision ones). (I expect such an error to lead to a signed integer overflow in the normal nanosecond precision, so it should be caught by ubsan or by the `assert(ret > 0s)`. If not, the error should be apparent on startup in the debug log.)
So remove it for now. An alternative would be to extend the test to cover `time` again, and adjust the comment to say that the test should be fixed along with the block header timestamp. Since that timestamp can't grow beyond 2106 anyway, see the `_test_y2106` functional test.
ACKs for top commit:
l0rinc:
ACK fa3a4eafa1
laanwj:
ACK fa3a4eafa1
Tree-SHA512: fd485e74962b659ee23ba2952d284fa9d6cfb9d9844a5e70013c8ead495ed77f5b784d5ca3ba0b30c492a5d27b2e81f9e1e0dbc530af7da1494789ac5e055b99
b28917be36 depends: Make default `host` and `build` comparable (Hennadii Stepanov)
Pull request description:
To detect cross-compiling, the host and build platforms are compared. The `build` variable is always an output of `config.sub`, but the `host` is not. This can lead to false results. For example, on OpenBSD:
- host=amd64-unknown-openbsd7.5
- build=x86_64-unknown-openbsd7.5
This PR sets the default value of the `host` variable to the value of `build`, ensuring cross-compiling won't be triggered when the `HOST` variable is not set.
This PR fixes needless triggering of cross-compiling for CMake-built packages in depends on OpenBSD due to this code:eb85cacd29/depends/funcs.mk (L193-L197)
No changes in Guix build.
ACKs for top commit:
laanwj:
Concept and code review ACK b28917be36
theuni:
utACK b28917be36.
Tree-SHA512: 8c5835cb8b739355b71f7cb161b350ad8b038a00e6b1def36354ba228cea3dcb9883df3c9a8e79d7d0143241f6f054129fe90772b1b2579702db51237f9d66ff
56a9b847bb build: set build type and per-build-type flags as early as possible (Cory Fields)
f605f7a9c2 build: refactor: set debug definitions in main CMakeLists (Cory Fields)
Pull request description:
This ensures that most compiler tests are not run with the wrong build type's flags. The initial c++ checks are an exception to that because many internal CMake variables are unset until a language is selected, so it's problematic to change our build type before that.
The difference can be seen in `build/CMakeFiles/CMakeConfigureLog.yaml`. Before, `Debug` was used for many of the earlly checks. After this PR, it's only the first 2 checks.
ACKs for top commit:
hebasto:
ACK 56a9b847bb.
Tree-SHA512: 87947352d6d4fd08554515822cb13634ed3be33fcda2af817c22ef943b1d0856ceb39311ddc01b8221397528fdc09f630dc7e74fc92f5a4a073f09c4ae493596
76c090145e guix: remove test-security/symbol-check scripts (fanquake)
Pull request description:
These scripts are becoming more of nuisance, than a value-add; particularly since we've been building releases using Guix. Adding new (release bin) tests can be harder, because it requires constructing a failing test, which is becoming less easy, e.g trying to disable a feature or protection that has been built into the compiler/toolchain by default.
In the pre-Guix days, these were valuable to sanity-check the environment, because we were pulling that pre-built from Ubuntu, with little control. At this point, it's less clear what these scripts are (sanity) checking.
Note that these also weren't completely ported to CMake (#31698), see also #31715 which contains other fixes that would be needed for these test-tests, to accomodate future changes.
ACKs for top commit:
hebasto:
ACK 76c090145e.
theuni:
utACK 76c090145e
Tree-SHA512: 99b5e7c0645c6966a45b17f411b5bee61df23c64d8258cce0ad9cdea4c3af4d4db32ca5fd80d0df2a3a30ba873eb772cc0d5901c345ff7f0eea13fcb971443b4
Currently the manpages are installed, but that is a bug. An upcoming commit
will avoid installing manpages for targets that aren't configured, which
removes the "install" target for fuzz builds.
0f716f2889 qa: cover PROTOCOL_ERROR variant in PCP unit tests (Antoine Poinsot)
fc700bb47f test: Add tests for PCP and NATPMP implementations (laanwj)
caf9521033 net: Use mockable steady clock in PCP implementation (laanwj)
03648321ec util: Add mockable steady_clock (laanwj)
ab1d3ece02 net: Add optional length checking to CService::SetSockAddr (laanwj)
Pull request description:
Add a NodeSteadyClock, a steady_clock that can be mocked with millisecond precision. Use this in the PCP implementation.
Then add a mock for a simple scriptable UDP server,, which is used to test various code paths (including successful mappings, timeouts and errors) in the PCP and NATPMP implementations.
Includes "net: Add optional length checking to CService::SetSockAddr" from #31014 as a prerequisite.
ACKs for top commit:
darosior:
re-ACK 0f716f2889
i-am-yuvi:
Concept ACK 0f716f2889
achow101:
ACK 0f716f2889
Tree-SHA512: 6f91b24e6fe46a3fded7a13972efd77c98e6ef235f8898e4ae44068c5df32d1cdabb22cb66c351b338dc98cb2073b624e43607a28107f4999302bfbe7a138229
2ffea09820 build: disable bitcoin-node if daemon is not built (Sjors Provoost)
Pull request description:
When building for fuzzing with multiprocess enabled, we were still trying to build `bitcoin-node`. This PR fixes that, by applying a similar check as for `bitcoin-gui`.
Before:
```
cmake -B build -DBUILD_FOR_FUZZING=ON -DWITH_MULTIPROCESS=ON
...
Configure summary
=================
Executables:
bitcoind ............................ OFF
bitcoin-node (multiprocess) ......... ON
bitcoin-qt (GUI) .................... OFF
bitcoin-gui (GUI, multiprocess) ..... OFF
...
cmake --build build
...
[ 84%] Built target bitcoin-node
```
After:
```
bitcoin-node (multiprocess) ......... OFF
```
And no `bitcoin-node` target gets built (not to be confused with `bitcoin_node`).
ACKs for top commit:
hebasto:
ACK 2ffea09820.
ryanofsky:
Code review ACK 2ffea09820
laanwj:
Code review ACK 2ffea09820
Tree-SHA512: bdb0b62049f77929d5c084bf98a076e9933de91eb30853ed89edd23cc81b3d4aec4cd57c9a9e21cf1d6930885f8c408dda830db6884b4e326c7fb348f1fbab4c
Previously, -nodebug cleared all prior -debug configurations in the
command line while allowing subsequent debug options to be applied.
However, -debug=0 and -debug=none completely disabled debugging,
even for categories specified afterward.
This commit ensures consistency by making -debug=0 and -debug=none
behave like -nodebug: they now clear previously set debug configurations
but do not disable debugging for categories specified later.
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
Our CMake toolchain for a Darwin cross build currently contains:
```bash
set(CMAKE_AR "/usr/bin/llvm-ar")
set(CMAKE_RANLIB "/usr/bin/llvm-ranlib")
set(CMAKE_STRIP "/usr/bin/llvm-strip")
set(CMAKE_OBJCOPY "arm64-apple-darwin-objcopy")
set(CMAKE_OBJDUMP "/usr/bin/llvm-objdump")
```
`objcopy` isn't currently used for the Darwin build (only for Linux and
splitting the debug symbols), but we shouldn't be producing a toolchain
file that refers to nonexistent tools.
8fe552fe6e test: add missing sync to p2p_tx_download.py (Martin Zumsande)
Pull request description:
If the node hasn't processed the inv from the outbound peer before the mocktime bump, the peer won't be preferred after the other inv timeouts, failing the test . Therefore, add a sync, just like there is one after the `send_message` calls in the previous lines.
Fixes #31833
ACKs for top commit:
maflcko:
lgtm ACK 8fe552fe6e
instagibbs:
ACK 8fe552fe6e
Tree-SHA512: fda935d8a4081b5ecae96f5a73c04f4bb91feaeb09b5c159ffd45cf16668c4345ff268c57f383ba7c7ff544ee07b21f97aa28f257ade809c18b9310837795e7a
9b7023d31a Fuzz HRP of bech32 as well (Lőrinc)
c1a5d5c100 Split out bech32 separator char to header (Lőrinc)
Pull request description:
Instead of the static "bc" human-readable part, it's now randomly generated based on https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki and the extra restrictions in the code:
> The human-readable part, which is intended to convey the type of data, or anything else that is relevant to the reader. This part MUST contain 1 to 83 US-ASCII characters, with each character having a value in the range [33-126]. HRP validity may be further restricted by specific applications.
Since `bech32::Encode` rejects uppercase letters, we're actually generating values in the `[33-126] - ['A'-'Z']` range.
Split out of https://github.com/bitcoin/bitcoin/pull/30596/files#r1706957219
ACKs for top commit:
sipa:
ACK 9b7023d31a
achow101:
ACK 9b7023d31a
marcofleon:
Code review ACK 9b7023d31a. The separation into two targets and the new `GenerateRandomHRP` seem fine to me.
brunoerg:
code review ACK 9b7023d31a
Tree-SHA512: 22a261b8e7b5516e98f4e7990811954454595438a49a10191ed4ca42b5c71c5054fcc73f2d94e23b498ea833c7f1d5adb225f537ef1a24d15b428259450cdf98
b2e9fdc00f test: expect that files may disappear from /proc/PID/fd/ (Vasil Dimov)
Pull request description:
`get_socket_inodes()` calls `os.listdir()` and then iterates on the results using `os.readlink()`. However a file may disappear from the directory after `os.listdir()` and before `os.readlink()` resulting in a `FileNotFoundError` exception.
It is expected that this may happen for `bitcoind` which is running and could open or close files or sockets at any time. Thus ignore the `FileNotFoundError` exception.
ACKs for top commit:
arejula27:
ACK [`b2e9fdc`](b2e9fdc00f)
sipa:
utACK b2e9fdc00f
achow101:
ACK b2e9fdc00f
theuni:
utACK b2e9fdc00f
hodlinator:
ACK b2e9fdc00f
Tree-SHA512: 8eb05393e4de4307a70af446c3fc7e8f7dc3f08bf9d68d74d02b0e4e900cfd4865249f297be31f1fd7b05ffea45eb855c5cfcd75704167950c1deb4f17109f33
With the exception of the first c++ checks, this ensures that compiler tests
are never run with the wrong build type's flags.
Co-Authored-By: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
This prevents intermittent failures - if the node hasn't processed
the inv from the outbound peer before the mocktime bump, the peer
won't be preferred after the other inv timeouts, failing the test .
Therefore, add a sync, just like there is one after the send_message
calls in the previous lines.
The legacy wallet will be able to solve output scripts where the
redeemScript or witnessScript is known, but does not know any of the
private keys involved in that script. These should be migrated to the
solvables wallet.
