348e79f7c6 lint: Include test_utxo_snapshots in lint_shell (Fabian Jahr)
Pull request description:
jamesob excluded `test_utxo_snapshots.sh` from the shell linter with this explanation: "Add the script to the shellcheck exception list since the quoted variables rule needs to be violated in order to get bitcoind to pick up on $EARLY_IBD_FLAGS." However, macrofake pointed out that single lines can be excluded from linting.
This fixes one fixable rule violation, excludes the rest of the offending lines from the linter and then removes the exclusion of the `test_utxo_snapshots.sh` file. Also adds documentation.
ACKs for top commit:
Empact:
ACK 348e79f7c6
maflcko:
lgtm ACK 348e79f7c6
pablomartin4btc:
tACK 348e79f7c6
Tree-SHA512: a904cc1cc3c94488dfbd39ea69a3ef17259f991708a797009001669448fef81eed086ecbce1ec433988d88baef293849698e2e0eb86a969b949cc7ef93af7b4b
092daa2f95 contrib: add test for macOS linker version to symbol-check (fanquake)
cefbf0bc20 depends: update LD64_VERSION to 711 (fanquake)
Pull request description:
I forgot to do this in 7d5815293e.
Add a test so it's impossible to forget.
ACKs for top commit:
TheCharlatan:
utACK 092daa2f95
achow101:
ACK 092daa2f95
jarolrod:
ACK 092daa2f95
hebasto:
ACK 092daa2f95.
laanwj:
ACK 092daa2f95
Tree-SHA512: 37f0bdfd6607a7760eabe5efe279532ba0c59c0915161e08d5e3b9a0b7705839d62537d6e17406062f6a0a1db5407575da7cd671e9cb916e422e77f5649c6e2b
fa05a726c2 tidy: modernize-use-emplace (MarcoFalke)
Pull request description:
Constructing a temporary unnamed object only to copy or move it into a container seems both verbose in code and a strict performance penalty.
Fix both issues via the `modernize-use-emplace` tidy check.
ACKs for top commit:
Sjors:
re-utACK fa05a726c2
hebasto:
ACK fa05a726c2.
TheCharlatan:
ACK fa05a726c2
Tree-SHA512: 4408a094f406e7bf6c1468c2b0798f68f4d952a1253cf5b20bdc648ad7eea4a2c070051fed46d66fd37bce2ce6f85962484a1d32826b7ab8c9baba431eaa2765
8b6470a906 gui: disable top bar menu actions during shutdown (furszy)
7066e8996d gui: provide wallet controller context to wallet actions (furszy)
Pull request description:
Small follow-up to #751.
Fixes another crash cause during shutdown. Which occurs when the user hovers over the wallets list.
Future Note:
This surely happen in other places as well, we should re-work the way we connect signals. Register
lambas without any precaution can leave dangling pointers.
ACKs for top commit:
hebasto:
ACK 8b6470a906, I've tested each commit separately on macOS Sonoma 14.0 (Apple M1).
Tree-SHA512: 6fbd1bcd6717a8c1633beb9371463ed22422f929cccf9b791ee292c5364134c501e099329cf77a06b74a84c64c1c3d22539199ec49ccd74b3950036316c0dab3
fa858d63a0 fuzz: Merge with -set_cover_merge=1 (MarcoFalke)
Pull request description:
This should be less controversial than commit 151a2b189c. The overall size of the qa-assets repo is reduced further from 1.9GB to 1.6GB. Also, the runtime to iterate on the resulting folder is reduced further from ~1699s to ~1149s (N=1).
ACKs for top commit:
murchandamus:
crACK fa858d63a0
dergoegge:
ACK fa858d63a0
Tree-SHA512: e23fa93bd48f01d11c551b035004c678bd6d76bc24ac7d0d0a7883060804e6711763cbd0cd0ded3aad3e4c40da764decae81c2703388cc11961def3c89a4f9ba
e44c574650 ci: always prune all dangling bitcoin-ci-test images (stickies-v)
ce1699706e ci: add label to docker images (stickies-v)
Pull request description:
Follow-up from https://github.com/bitcoin/bitcoin/pull/27777#discussion_r1210209382.
Labeling the docker images produced by the CI allows us/the user to apply batch operations to all images (including dangling ones) produced by the ci without affecting other, non-bitcoin-ci images. With labeling, we can safely always prune dangling bitcoin-ci-test images without checking for `RESTART_CI_DOCKER_BEFORE_RUN`, which we enable on our persistent runners.
ACKs for top commit:
fanquake:
utACK e44c574650
Tree-SHA512: 1009fb1be78fbc80b5341ba92eac2991e77d050e1ab6048d1d9a65af73413a6be7afc1e1c764eb3f347f363af31245b93fdb38f6ac016d775aad4a0f36e4c98f
Opening the top bar menu when the app is being destroyed
freezes the GUI shutdown process for no reason. No menu
action can be executed.
Note:
This behavior is consistent with how the tray icon menu
is cleared too.
Two recently added tests (PR #28625 / commit 2e31250027
and PR #28634 / commit 3bb51c29df)
introduced a bug by wrongly using the `assert_debug_log` helper.
Instead of passing the expected debug string in a list as expected, it
was passed as bare string, which is then interpretered as a list of
characters, very likely leading the debug log assertion pass even if the
intended message is not appearing.
In order to avoid bugs like this in the future, enforce that the
`{un}expected_msgs` parameters are lists.
Since all bitcoin-ci-test images are now labeled, we can always
prune all dangling images, regardless of whether we are in
RESTART_CI_DOCKER_BEFORE_RUN.
To be safe, still prune all images if RESTART_CI_DOCKER_BEFORE_RUN
in case the filtering doesn't work, or if images were created on
an earlier version that did not assign labels.
This allows us or the user to perform batch operations on all
images produced by the ci, e.g. to prune all dangling images,
without affecting non-ci images.
36a3004a41 devtools: test_utxo_snapshots.sh sleep cleanup and documentation (Fabian Jahr)
Pull request description:
There were bare sleep statements in the script where it was unclear why they were needed and I think they could also be replaced by existing helpers.
ACKs for top commit:
Sjors:
utACK 36a3004a41
pablomartin4btc:
utACK 36a3004a41
Tree-SHA512: b6e2fc69cd7babcfa7f413f11304f4d648d6d64b3b526862664ccedb0016ad182b9e886aa4e8c33315e4c18824512e11a4fd6365f9c3c95093967d3ef7687e62
61a6c3b0e9 build: add `-mbranch-protection=bti` to aarch64 hardening flags (fanquake)
Pull request description:
This is a simpler (less hardening) version of https://github.com/bitcoin/bitcoin/pull/24123.
You can inspect binaries using `readelf -n`, and look for BTI in a `.note.gnu.property`. i.e
```bash
readelf -n src/bitcoin-cli
Displaying notes found in: .note.gnu.property
Owner Data size Description
GNU 0x00000010NT_GNU_PROPERTY_TYPE_0
Properties: AArch64 feature: BTI
```
Related to https://github.com/bitcoin/bitcoin/issues/19075.
ACKs for top commit:
TheCharlatan:
utACK 61a6c3b0e9
Tree-SHA512: 64504de44e91d853165daf4111dca905d8eb9ef3f4bfb0d447c677b02c9100dbd56f13e6fe6539fb06c2343a094229591ac5d1bd9e184b32b512c0ac3f9bac36
fa2c894cbb ci: move-only CI_CONTAINER_ID to 02_run_container.sh (MarcoFalke)
fa695b4df0 ci: Work around podman stop bug (MarcoFalke)
fa09a031c1 ci: Add set -ex to 02_run_container.sh (MarcoFalke)
fac9abbf47 ci: Rename 04_install to 02_run_container (MarcoFalke)
Pull request description:
Sometimes, it seems that `podman stop` does not work. Presumably, it falls back to `podman kill`, which is async.
Try to work around this intermittent issue by using the `rm --force` over `stop`.
Example failing log https://cirrus-ci.com/task/4549784611061760?logs=ci#L238:
```
Restart docker before run to stop and clear all containers started with --rm
++ podman container stop --all
e4eca0766f87864d89fc230aa884a238c214cfbcd44cf76a4dbdb2a30c982009
++ echo 'Prune all dangling images'
Prune all dangling images
++ docker image prune --force
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
+++ docker run --cap-add LINUX_IMMUTABLE --rm --interactive --detach --tty --mount type=bind,src=/tmp/cirrus-build-1970593815,dst=/tmp/cirrus-build-1970593815,readonly --mount type=volume,src=ci_macos_cross_ccache,dst=/tmp/ccache_dir --mount type=volume,src=ci_macos_cross_depends,dst=/ci_container_base/depends --mount type=volume,src=ci_macos_cross_previous_releases,dst=/ci_container_base/prev_releases --env-file /tmp/env --name ci_macos_cross ci_macos_cross
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
time="2023-09-27T20:55:39Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly"
Error: creating container storage: the container name "ci_macos_cross" is already in use by e4eca0766f87864d89fc230aa884a238c214cfbcd44cf76a4dbdb2a30c982009. You have to remove that container to be able to reuse that name: that name is already in use
ACKs for top commit:
hebasto:
ACK fa2c894cbb, I have reviewed the code and tested it locally.
Tree-SHA512: 31fca340c6bedaadf4dd51fa745d9b3969042cebc0c7c904ef18af3f2f986039ec4354ccdff1422fbf77cf223e4423857368dce53cfa67ef15c76b78d007eace
faa190b1ef test: Fuzz merge with -use_value_profile=0 for now (MarcoFalke)
Pull request description:
Seems odd that this has to be done, but for now there are (unknown) size limits on the qa-assets repo. Also, a larger size means that cloning and iterating over the files takes a longer time.
Not sure how to measure the net impact of this, but with some backups reverting this commit, it can be limited on the downside?
ACKs for top commit:
dergoegge:
ACK faa190b1ef
Tree-SHA512: 9f8b3f4526f60e4ff6fca97859a725d145a8339c216bd15c92fad7e53f84308745fee47727527de459c0245ef9d474a9dc836fee599ab2b556b519bd900b9a33
fa2843eba4 ci: Bump asan (MarcoFalke)
Pull request description:
Needed to bump the EOL date and unlock clang-17.
ACKs for top commit:
fanquake:
ACK fa2843eba4
Tree-SHA512: 7d8b3b30ed65bd26f9640db8d06de8c27a9ad8a8160f9e645a97b99df867aa30508491dd8957a54edad724f8672ef1054041106d90ee826ba3e856176ab52afc
79789ccafe ci: Drop no longer needed `NOLINTNEXTLINE` (Hennadii Stepanov)
Pull request description:
After recent tool updates in the "tidy" CI task, the one instance of `NOLINTNEXTLINE` is not required anymore.
ACKs for top commit:
fanquake:
ACK 79789ccafe
Tree-SHA512: 382f3e0570f36a481498ac118ee30a34aabfd6285b21a67c19c74b240305315c5a118fb70f172b843a086262433462d45998e78306697be0ddfb20094733ee98
4077e43bf6 test: fix usdt undeclared function errors on mantis (willcl-ark)
Pull request description:
This is one way to fix #28600
Recently usage of undeclared functions became an error rather than a warning, in C2x. https://reviews.llvm.org/D122983?id=420290
This change has migrated into the build tools of Ubuntu 23.10 which now causes the USDT tests to fail to compile, see
https://github.com/bitcoin/bitcoin/issues/28600
I think there are various potential fixes:
1. Manually declare the functions we use
2. Fix imports so that manual declarations aren't needed
3. Revert the new C2X behaviour and don't error on implicit function declarations
I would have preferred solution 2, but I believe this will require changes to the upstream bcc package. Having played with the imports I can get things working in a standalone C program, using system headers, but when building the program from a python context as we do in the test it uses its own headers (bundled with the python lib) rather than the system ones, and manually importing (some) system headers results in definition mismatches. I also investigated explicitly importing required headers from the package, which use paths like `#import </virtual/bcc/bcc_helpers.h>`, but this seems more obtuse and brittle than simply ignoring the warning.
Therefore I think that until the upstream python pacakge fixes their declarations, we should fix this by setting `-Wno-error=implicit-function-declaration` for the tracing programs.
cc maflcko 0xB10C
ACKs for top commit:
maflcko:
lgtm ACK 4077e43bf6
Tree-SHA512: 8368bb1155e920a95db128dc893267f8dab64f1ae53f6d63c6d9294e2e4e92bef8515e3697e9113228bedc51c0afdbc5bbcf558c119bf0eb3293dc2ced86b435
850670e3d6 test: don't run old binaries under valgrind (Sjors Provoost)
Pull request description:
Some, but not all, backward compatibility tests fail for me and it seems useless to run old release binaries under valgrind anyway.
Can be tested by running `test/functional/feature_txindex_compatibility.py --valgrind --timeout-factor=10` with and without this PR.
—
The previous version of this PR disabled these test entirely under valgrind. The current version does run the test, but starts the old binaries without valgrind.
ACKs for top commit:
maflcko:
lgtm ACK 850670e3d6
Tree-SHA512: ebdf461083f1292528e6619963b910f486b60b4f6b183f0aea2c8bfcafa98caeb204d138700cd288450643bcec5e49e12b89f2f7537fccdf495a2a33acd9cea0
3bb51c29df test: BIP324: add check for missing garbage terminator detection (Sebastian Falbesoner)
Pull request description:
This PR adds test coverage for the "missing garbage terminator" detection on incoming v2 transport (BIP324) connections:
04265ba937/src/net.cpp (L1205-L1209)
Note that this always happens at the same exact amount of bytes sent in (after 64 + 4095 + 16 = 4175 bytes), if at no point, the last 16 bytes of potential authentication data match the garbage, i.e. all the previous bytes after the ellswift pubkey. To keep it simple, we just send in zero-value bytes here and verify that the detection hits exactly after the last bytes is sent.
AFAICT, with this PR all the v2 transport errors that can be triggered in this simple way of "just open a socket and send in a fixed byte-string" are covered. For more advanced test, we need BIP324 cryptography in the test framework in order to perform a v2 handshake etc. (PRs #28374, #24748).
ACKs for top commit:
sipa:
utACK 3bb51c29df
laanwj:
ACK 3bb51c29df
Tree-SHA512: f88275061c7c377a3d9f2608452671afc26deb6d5bd5be596de987c7e5042555153ffe681760c33bce2b921ae04e50f349ea0128a677e6443a95a079e52cdc5f
This is unnecessary and caused test failures. The backward
compatibility tests are meant to find regressions in the
current codebase, not to detect bugs in older releases.
78d3062b68 ci: Install Android API 31 platform as Qt expects (Hennadii Stepanov)
Pull request description:
When building the `qt` package, it expects that the default (in Qt's view) Android API platform is installed.
During the recent Qt version [update](https://github.com/bitcoin/bitcoin/pull/28561), it has been changed:
```diff
--- a/mkspecs/features/android/sdk.prf
+++ b/mkspecs/features/android/sdk.prf
@@ -1,6 +1,6 @@
API_VERSION_TO_USE = $$(ANDROID_API_VERSION)
isEmpty(API_VERSION_TO_USE): API_VERSION_TO_USE = $$API_VERSION
-isEmpty(API_VERSION_TO_USE): API_VERSION_TO_USE = android-28
+isEmpty(API_VERSION_TO_USE): API_VERSION_TO_USE = android-31
ANDROID_JAR_FILE = $$ANDROID_SDK_ROOT/platforms/$$API_VERSION_TO_USE/android.jar
!exists($$ANDROID_JAR_FILE) {
```
This PR fixes the CI for the Android task and addresses https://github.com/bitcoin/bitcoin/pull/28561#issuecomment-1749180177.
Qt [docs](https://doc.qt.io/qt-5/android.html) still claim that Android API Level 21 and up are supported, however, I did not test every possible configuration.
NOTE: https://github.com/bitcoin/bitcoin/pull/28611 is still valid.
ACKs for top commit:
maflcko:
lgtm ACK 78d3062b68
jarolrod:
tACK 78d3062b68
Tree-SHA512: 781fba6d80aae7e6500854de14af0d30169c258e395b9e482a5430a7b4a2211a6181f8c9ee58543c896b431abf09e3e7c5573b9672ed128658f11f98a2006e7e
2e31250027 test: check that loading snapshot not matching AssumeUTXO parameters fails (Sebastian Falbesoner)
Pull request description:
This PR adds test coverage for the failed loading of an AssumeUTXO snapshot in case the referenced block hash doesn't match the parameters in the chainparams. Right now, I expect this would be the most common error-case for `loadtxoutset` out in the wild, as for mainnet the `m_assumeutxo_data` map is empty and this error condition would obviously always be triggered for any (otherwise valid, correctly encoded) snapshot. Note that this test-case is the simplest scenario and doesn't cover any of the TODO ideas mentioned at the top of the functional test yet.
ACKs for top commit:
jamesob:
ACK 2e31250027
Sjors:
utACK 2e31250027
achow101:
ACK 2e31250027
Tree-SHA512: 8bcb2d525c95fbc95f87d3e978ad717d95bddb1ff67cbe7d3b06e4783f0f1ffba32b17ef451468c39c23bc1b3ef1150baa71148c145275c386f2d4822d790d39
bfa0bd632a test: Use pathlib over os.path #28362 (ns-xvrn)
Pull request description:
In reference to issue #28362 refactoring of functional tests to use pathlib over os.path to reduce verbosity and increase the intuitiveness of managing file access.
ACKs for top commit:
maflcko:
re-ACK bfa0bd632a🐨
willcl-ark:
ACK bfa0bd632a
Tree-SHA512: fb0833c4039d09758796514e47567a93ac831cb0776ff1a7ed8299792ad132e83282ef80bea098a88ae1551d906f2a56093d3e8de240412f9080735d00d496d9
74c77825e5 test: Unit test for inferring scripts with hybrid and uncompressed keys (Andrew Chow)
f895f97014 test: Scripts with hybrid pubkeys are migrated to watchonly (Andrew Chow)
37b9b73477 descriptors: Move InferScript's pubkey validity checks to InferPubkey (Andrew Chow)
b7485f11ab descriptors: Check result of InferPubkey (Andrew Chow)
Pull request description:
`InferDescriptor` was not always checking that the pubkey it was placing into the descriptor was an allowed pubkey. For example, given a P2WPKH script that uses an uncompressed pubkey, it would produce a `wpkh()` with the uncompressed key. Additionally, the hybrid key check was only being done for `pk()` scripts, where it should've been done for all scripts.
This PR moves the key checking into `InferPubkey`. If the key is not valid for the context, then `nullptr` is returned and the inferring will fall through to the defaults of either `raw()` or `addr()`.
This also resolves an issue with migrating legacy wallets that contain hybrid pubkeys as such watchonly scripts will become `raw()` or `addr()` and go to the watchonly wallet. Note that a legacy wallet cannot sign for hybrid pubkeys. A test has been added for the migration case.
Also added unit tests for `InferDescriptor` itself as the edge cases with that function are not covered by the descriptor roundtrip test.
ACKs for top commit:
furszy:
ACK 74c77825
Sjors:
utACK 74c77825e5
Tree-SHA512: ed5f63e42a2e46120245a6b0288b90d2a6912860814c6c08fe393332add1cb364dc5eca72f16980352143570aef0c07bf1a91acd294099463bd028b6ce2fe40c
8735e2c136 ci: use LLVM/Clang 17 in tidy job (fanquake)
ce46b68941 ci: use LLVM 17.0.2 in MSAN jobs (fanquake)
Pull request description:
Also update MSAN to use 17.0.2.
Related to #28465.
ACKs for top commit:
maflcko:
lgtm ACK 8735e2c136
Tree-SHA512: 74452b95326cf065afe8332dc1b5b8e5ac12c8fe05c278a1cee017f87a7f7e0cdb8cac5e39d718c8ef587c8ee229bbaadd847df9f191313d41c5cdcab45e7c76
This is a simpler (less hardening) version of #24123.
Scoped to aarch64 to avoid unused command line option warnings when
building on x86_64.
Related to #19075.
