faaba770e1 Sort includes in compat.h (MarcoFalke)
fa91a23d63 remove unused limits.h include in compat.h (MarcoFalke)
fa32af22b3 Replace LocaleIndependentAtoi with ToIntegral (MarcoFalke)
faab76c1c0 iwyu on torcontrol (MarcoFalke)
fa0a60dd93 Remove unused boost signals2 from torcontrol (MarcoFalke)
Pull request description:
Remove unused boost, and other includes, and other legacy functions from torcontrol.
ACKs for top commit:
TheCharlatan:
Re-ACK faaba770e1
achow101:
ACK faaba770e1
dergoegge:
utACK faaba770e1
Tree-SHA512: 440f8d3ae9c3cf4dcc368e35b29459b5fcec8c6d233e8f9be3a854e7624b8633d6ccdde10cb0c6f74f86278e06557c4e9e24de30c3c692826237939265c6160a
9658d0dc17 ci: Run "macOS native x86_64" job on GitHub Actions (Hennadii Stepanov)
Pull request description:
From https://github.com/bitcoin/bitcoin/issues/28098:
> Thus, someone would have to sponsor an amount of roughly 5kUSD/mo for those two tasks.
> If the goal is to stay on a free plan, I think the only option is GitHub Actions CI.
---
**IMPORTANT NOTE**. We currently ship macOS release binaries for both architectures: `x86_64` and `arm64`. If this PR gets merged, only `x86_64` architecture will be tested on CI, which implies some [drawbacks](https://github.com/bitcoin/bitcoin/pull/28187#issuecomment-1658077549).
However, it has never been the case that our CI tested both architectures simultaneously. And we hope that GitHub Actions will soon host macOS `arm64` runners.
Historically, we moved from `x86_64` to `arm64` in https://github.com/bitcoin/bitcoin/pull/26388 less than a year ago.
---
Security concerns:
- https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651432106
- https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651688197
`GITHUB_TOKEN` permissions (from the build log in my personal repo):
```
2023-07-27T07:30:17.8313534Z ##[group]GITHUB_TOKEN Permissions
2023-07-27T07:30:17.8314113Z Contents: read
2023-07-27T07:30:17.8314608Z Metadata: read
2023-07-27T07:30:17.8314957Z Packages: read
2023-07-27T07:30:17.8315233Z ##[endgroup]
```
Comparison of resources:
| Resource | Current, Cirrus CI | Suggested, GitHub Actions |
|---|:-:|:-:|
| CPU | 4 | 4 \*\* |
| RAM, GB | 8 | 14 |
**\*\* NOTE**: However, [docs](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources) are mentioning:
> 3-core CPU (x86_64)
ACKs for top commit:
MarcoFalke:
re-ACK 9658d0dc17🏂
achow101:
ACK 9658d0dc17
jarolrod:
ACK 9658d0dc17
Tree-SHA512: 6123e68e6784cdf4e53c3e77b435709261db21f09091af2c22e667d3816a305fffb9d617297a5bc1bda18aaba84a6e210cec6a75c52afa7746a3780a67b69865
fa776e61cd Add importmempool RPC (MarcoFalke)
fa20d734a2 refactor: Add and use kernel::ImportMempoolOptions (MarcoFalke)
fa8866990d doc: Clarify the getmempoolinfo.loaded RPC field documentation (MarcoFalke)
6888886cec Remove Chainstate::LoadMempool (MarcoFalke)
Pull request description:
Currently it is possible to import a mempool by placing it in the datadir and starting the node. However this has many issues:
* Users aren't expected to fiddle with the datadir, possibly corrupting it
* An existing mempool file in the datadir may be overwritten
* The node needs to be restarted
* Importing an untrusted file this way is dangerous, because it can corrupt the mempool
Fix all issues by adding a new RPC.
ACKs for top commit:
ajtowns:
utACK fa776e61cd
achow101:
ACK fa776e61cd
glozow:
reACK fa776e61cd
Tree-SHA512: fcb1a92d6460839283c546c47a2d930c363ac1013c4c50dc5215ddf9fe5e51921d23fe0abfae0a5a7631983cfc7e2fff3788b70f95937d0a989a203be4d67546
fafa17c00b ci: Use hard-coded root path for CI containers (MarcoFalke)
fa084f5ba5 ci: Only create folders when needed (MarcoFalke)
fab27127f4 ci: Drop BASE_SCRATCH_DIR from LIBCXX_DIR (MarcoFalke)
Pull request description:
Currently the CI system will fail if the git folder that holds the Bitcoin Core source is moved from one location to another.
Fix this by using a single hard-coded root path *inside* the CI system containers.
Steps to test:
* Run the CI system: `MAKEJOBS="-j$(nproc)" FILE_ENV="./ci/test/00_setup_env_win64.sh" ./ci/test_run_all.sh`
* Move the git folder: `pwd && cd .. && mv bitcoin_core_folder_1 bitcoin_core_folder_2 && cd ./bitcoin_core_folder_2 && pwd`
* Run the CI system again: (same cmd as above)
On master (error):
```
STRIPPROG="x86_64-w64-mingw32-strip" /bin/bash /bitcoin_core_folder_2/ci/scratch/build/bitcoin-x86_64-w64-mingw32/build-aux/install-sh -c -s ./src/qt/bitcoin-qt.exe ./release
/bitcoin_core_folder_2/ci/scratch/build/bitcoin-x86_64-w64-mingw32/build-aux/install-sh: ./src/qt/bitcoin-qt.exe does not exist.
make: *** [Makefile:1258: bitcoin-25.99.0-win64-setup.exe] Error 1
```
On this pull: (pass).
ACKs for top commit:
fanquake:
ACK fafa17c00b - somewhat tested. MSAN changes are the same as what we did for tidy.
Tree-SHA512: 2ce693a3773c70fcfca062c2a6f0e5a16b94960b34a6145d10cee1a28f79154829d59d014465ccbb80e1cb9dcd5aa043729cee9afd2c4175b05e9bc945364b79
c2a87bd302 ci: Drop no longer needed `macos_sdk_cache` (Hennadii Stepanov)
Pull request description:
It has been cached in the Docker image since https://github.com/bitcoin/bitcoin/pull/27028.
ACKs for top commit:
MarcoFalke:
lgtm ACK c2a87bd302
Tree-SHA512: fba888d132910f9600db0acccf633400e699f7d5ca802ef109a792546f60f3a04791c503b43d156c8debc3bd0bce2ad911f6209eabc47cd63fce7996a6ae3cfc
93cb8f0380 refactor: add missing headers for BIP324 ciphersuite (stratospher)
d22d5d925c crypto: BIP324 ciphersuite follow-up (stratospher)
Pull request description:
follow-up to #28008.
* move `dummy_tag` variable in FSChaCha20Poly1305 crypto_tests outside of the loop to be reused every time
* use easy to read `cipher.last()` in `AEADChaCha20Poly1305::Decrypt()`
* comment for initiator in `BIP324Cipher::Initialize()`
* systematically damage ciphertext with bit positions in bip324_tests
* use 4095 max bytes for `aad` in bip324 fuzz test
ACKs for top commit:
fanquake:
ACK 93cb8f0380 - thanks for following up here.
Tree-SHA512: 361f3e226d3168fdef69a2eebe6092cfc04ba14ce009420222e762698001eaf8be69a1138dab0be237964509c2b96a41a0b4db5c1df43ef75062f143c5aa741a
e417c988f6 fuzz: coins_view: remove an incorrect assertion (Antoine Poinsot)
c5f6b1db56 fuzz: coins_view: correct an incorrect assertion (Antoine Poinsot)
Pull request description:
The `coins_view` fuzz target would assert in two places that the cache is consistent with the backend. But it's never the case (that's the whole point of using a cache).
The only reason this didn't result in a crash was that we would never actually hit these assertions. I ran into this while introducing a new target with an in-memory `CCoinsViewDB` as the backend view (see https://github.com/bitcoin/bitcoin/pull/28216) which made the code paths with those assertions actually reachable.
ACKs for top commit:
dergoegge:
Code review ACK e417c988f6
Tree-SHA512: 5847bb2744a2f2831dace62d32b79cc491bf54e2af4ce425411d245d566622d9aff816d9be5ec8e830d10851c13f2500bf4f0c004d88b4d7cca1d483ef8960a6
* PERSISTENT_WORKER_TEMPLATE_ENV is not needed at all, because
RESTART_CI_DOCKER_BEFORE_RUN is already set on the persistent worker.
* PERSISTENT_WORKER_TEMPLATE can be replaced by pinning the
previous_releases task to a type of worker. This should make the CI
performance more consistent.
CTxDestination is really our internal representation of an address and
doesn't really have anything to do with standard script types, so move
them to their own file.
Replaces the constructor in CScriptID that converts a ScriptHash with a
function ToScriptID that does the same. This prepares for a move of
CScriptID to avoid a circular dependency.
5364dd8666 test: locked_wallet, skip default fee estimation (furszy)
Pull request description:
Coming from https://github.com/bitcoin/bitcoin/pull/28139#discussion_r1284563239.
No test case in this file is meant to exercise fee estimation. All default wallets have a
custom tx fee set [here](b7138252ac/test/functional/wallet_fundrawtransaction.py (L100)). The only one missing is the one created for `locked_wallet`.
ACKs for top commit:
theStack:
ACK 5364dd8666
Tree-SHA512: 514c02708081d18330d759d10e306cee16c6350de243c68f0973777d2582f5d81968a237393c1f59aba245297e03f3f98d3ae5249a042469d0d016255f568719
Have each TestNode keep track of the last timestamp it called
setmocktime with, and add a bumpmocktime() function to bump by a
number of seconds. Makes it easy to fast forward n seconds without
keeping track of what the last timestamp was.
bb3263d3e3 bitcoin-tidy: fix macOS build (Cory Fields)
Pull request description:
[LLVM uses these options](https://github.com/llvm/llvm-project/blob/main/llvm/cmake/modules/HandleLLVMOptions.cmake#L178) for building as well, so there's precedent.
Also fix the shared library extension which was incorrectly being set to dylib.
Thanks to jonatack for reporting and debugging.
ACKs for top commit:
jonatack:
ACK bb3263d3e3 tested with arm64 macos 13.5, llvm 16.0.6 and cmake 3.27.2
Tree-SHA512: de7bfd497f38f1565a14d217d0b057cbfa788bdda702b5942b7f0b55947ae5e1c05af13e7d6a073ed036bc4db57035868f180034508b6e084ab9b901a5baaf2f
follow-up to #28008.
* move `dummy_tag` variable in FSChaCha20Poly1305 crypto_tests
outside of the loop to be reused every time
* use easy to read `cipher.last()` in `AEADChaCha20Poly1305::Decrypt()`
* comment for initiator in `BIP324Cipher::Initialize()`
* systematically damage ciphertext with bit positions in bip324_tests
* use 4095 max bytes for aad in bip324 fuzz test
d82bb90a5b doc: use llvm-config for bitcoin-tidy example (fanquake)
Pull request description:
An LLVM installation will have `llvm-config` available to query for info. Ask it for the `--cmakedir`, and use that in our bitcoin-tidy example, rather than listing multiple different (potential) paths per distro/OS etc.
ACKs for top commit:
theuni:
ACK d82bb90a5b.
jonatack:
ACK d82bb90a5b
TheCharlatan:
Nice, Re-ACK d82bb90a5b
Tree-SHA512: e07e979231f8f000deafce0751bed4b73ff0eff995bec49e90f579c9051cf5859dac5e49554b8219d33b00c81192db979eed98fee1c643a9205ea8babfce2c5d
The migration process must skip any invalid script inside the legacy
spkm and all the addressbook records linked to them.
These scripts are not being watched by the current wallet, nor should
be watched by the migrated one.
IsMine() returns ISMINE_NO for them.
The legacy wallet allowed to import any raw script, without checking if
it was valid or not. Appending it to the watch-only set.
This causes a crash in the migration process because we are only
expecting to find valid scripts inside the legacy spkm.
These stored scripts internally map to `ISMINE_NO` (same as if they
weren't stored at all..).
So we need to check for these special case, and take into account that
the legacy spkm could be storing invalid not watched scripts.
Which, in code words, means IsMineInner() returning IsMineResult::INVALID
for them.
An LLVM installation will have `llvm-config` available to query for
info. Ask it for the `--cmakedir`, and use that in our bitcoin-tidy
example, rather than listing multiple different (potential) paths per
distro/OS etc.
Closes #21070
v21.0 introduced a behaviour changed noted in #21070 where using a config value
`rpcallowip=::0` no longer also permitted ipv4 ip addresses.
The rpc_bind.py functional test covers this new behaviour already by checking
that the list of bind addresses exactly matches what is expected so this
commit only updates the documentation.
547fa52443 net processing: clamp -blockreconstructionextratxn to uint32_t bounds (stickies-v)
e451d1e3c6 net processing: clamp -maxorphantx to uint32_t bounds (stickies-v)
aa89e04e07 doc: document PeerManager::Options members (stickies-v)
Pull request description:
Avoid out-of-bounds user input for `PeerManager::Options` by safely clamping `-maxorphantx` and `-blockreconstructionextratxn`, and avoid platform-specific behaviour by changing `PeerManager::Options::max_extra_txs` from `size_t` to a `uint32_t`. Addresses https://github.com/bitcoin/bitcoin/pull/27499#pullrequestreview-1544114932.
Also documents all `PeerManager::Options` members, addressing https://github.com/bitcoin/bitcoin/pull/27499#discussion_r1272302469.
ACKs for top commit:
dergoegge:
Code review ACK 547fa52443
glozow:
reACK 547fa52443
Tree-SHA512: 042d47b35bb8a7b29ef3dadd4c0c5d26f13a8f174f33687855d603c19f8de0fcbbda94418453331e149885412d4edd5f402d640d938f6d94b4dcf54e2fdbbcc9
fad0b67c21 ci: Use qemu-user through container engine (MarcoFalke)
Pull request description:
Currently the CI containers always run on the host architecture, and only wrap `bitcoind` into `qemu-user` when needed. This has many issues:
* The `i386` tasks can not be run on non-x86 hosts.
* `config.guess` isn't present when building the CI image, which is fine. But it prints a warning, see https://github.com/bitcoin/bitcoin/pull/27739#pullrequestreview-1446580353
* The python tests are run on the host architecture, making it harder to find architecture specific bugs. See for example https://github.com/bitcoin/bitcoin/pull/27529#issuecomment-1521387810
* All modern container engines support automatic dispatch to qemu-user, so it seems redundant to re-invent the wheel.
Fix all issues by:
* removing `HOST` from `ci/test/00_setup_env.sh`.
* removing `QEMU_USER_CMD` and `ci/test/wrap-qemu.sh`.
* removing `DPKG_ADD_ARCH` where possible, and pruning `PACKAGES` where possible.
* specifying the architecture in `CI_IMAGE_NAME_TAG` to be used by the container engine.
ACKs for top commit:
fanquake:
ACK fad0b67c21 - this seems ok to me, and removes complexity from our CI system.
Tree-SHA512: 85e79f9f570e292d70a629d112fd4a6e6217d96226a1b665ed13485f616d84720ad2126b7d4b22fc603049f72fa7f2163b56a6bc276319fcd8b0496304ea4157
