fa9112aac0 Remove utxo db upgrade code (MarcoFalke)
Pull request description:
It is not possible to upgrade Bitcoin Core pre-segwit (pre-0.13.1) to a recent version without a full IBD from scratch after commit 19a56d1519 (released in version 22.0).
Any Bitcoin Core version with the new database format after commit 1088b02f0c (released in version 0.15), can upgrade to any version that is supported as of today.
This leaves the versions 0.13.1-0.14.x. Even though those versions are unsupported, some users with an existing datadir may want to upgrade to a recent version. However, it seems reasonable to simply ask them to `-reindex` to run a full IBD from scratch. This allows us to remove the utxo db upgrade code.
ACKs for top commit:
Sjors:
re-ACK fa9112aac0
laanwj:
Code review ACK fa9112aac0
Tree-SHA512: 4243bb35df9ac4892f9fad30fe486d338745952bcff4160bcb0937c772d57b13b800647da14695e21e3655e85ee0d95fa3dc7789ee309d59ad84f422297fecb8
2da94a4c6f fuzz: add a fuzz target for Miniscript decoding from Script (Antoine Poinsot)
f8369996e7 Miniscript: ops limit and stack size computation (Pieter Wuille)
2e55e88f86 Miniscript: conversion from script (Pieter Wuille)
1ddaa66eae Miniscript: type system, script creation, text notation, tests (Pieter Wuille)
4fe29368c0 script: expose getter for CScriptNum, add a BuildScript helper (Antoine Poinsot)
f4e289f384 script: move CheckMinimalPush from interpreter to script.h (Antoine Poinsot)
31ec6ae92a script: make IsPushdataOp non-static (Antoine Poinsot)
Pull request description:
Miniscript is a language for writing (a subset of) Bitcoin Scripts in a structured way.
Miniscript permits:
- To safely extend the Output Descriptor language to many more scripting features thanks to the typing system (composition).
- Statical analysis of spending conditions, maximum spending cost of each branch, security properties, third-party malleability.
- General satisfaction of any correctly typed ("valid" [0]) Miniscript. The satisfaction itself is also analyzable.
- To extend the possibilities of external signers, because of all of the above and since it carries enough metadata.
Miniscript guarantees:
- That for any statically-analyzed as "safe" [0] Script, a witness can be constructed in the bounds of the consensus and standardness rules (standardness complete).
- That unless the conditions of the Miniscript are met, no witness can be created for the Script (consensus sound).
- Third-party malleability protection for the satisfaction of a sane Miniscript, which is too complex to summarize here.
For more details around Miniscript (including the specifications), please refer to the [website](https://bitcoin.sipa.be/miniscript/).
Miniscript was designed by Pieter Wuille, Andrew Poelstra and Sanket Kanjalkar.
This PR is an updated and rebased version of #16800. See [the commit history of the Miniscript repository](https://github.com/sipa/miniscript/commits/master) for details about the changes made since September 2019 (TL;DR: bugfixes, introduction of timelock conflicts in the type system, `pk()` and `pkh()` aliases, `thresh_m` renamed to `multi`, all recursive algorithms were made non-recursive).
This PR is also the first in a series of 3:
- The first one (here) integrates the backbone of Miniscript.
- The second one (#24148) introduces support for Miniscript in Output Descriptors, allowing for watch-only support of Miniscript Descriptors in the wallet.
- The third one (#24149) implements signing for these Miniscript Descriptors, using Miniscript's satisfaction algorithm.
Note to reviewers:
- Miniscript is currently defined only for P2WSH. No Taproot yet.
- Miniscript is different from the policy language (a high-level logical representation of a spending policy). A policy->Miniscript compiler is not included here.
- The fuzz target included here is more interestingly extended in the 3rd PR to check a script's satisfaction against `VerifyScript`. I think it could be further improved by having custom mutators as we now have for multisig (see https://github.com/bitcoin/bitcoin/issues/23105). A minified corpus of Miniscript Scripts is available at https://github.com/bitcoin-core/qa-assets/pull/85.
[0] We call "valid" any correctly-typed Miniscript. And "safe" any sane Miniscript, ie one whose satisfaction isn't malleable, which requires a key for any spending path, etc..
ACKs for top commit:
jb55:
ACK 2da94a4c6f
laanwj:
Light code review ACK 2da94a4c6f (mostly reviewed the changes to the existing code and build system)
Tree-SHA512: d3ef558436cfcc699a50ad13caf1e776f7d0addddb433ee28ef38f66ea5c3e581382d8c748ccac9b51768e4b95712ed7a6112b0e3281a6551e0f325331de9167
7b00595d33 build: stop overriding user CXXFLAGS (fanquake)
3e2ef23c3e build: stop overriding user LDFLAGS (fanquake)
35c3fd43c3 build: stop overriding user CPPFLAGS (fanquake)
bc7cc57607 doc: explain why we clear CXXFLAGS with enable-debug (fanquake)
Pull request description:
Historically our build system has hijacked `CXXFLAGS` and friends, and this has always been a source of complaints from users and developers. With this PR, we move away from using `CXXFLAGS`, `CPPFLAGS` and `LDFLAGS`, and instead use `CORE_*FLAGS` variables for our flags / options, leaving autoconfs `FLAG` vars to the user.
Note that there are currently two cases where we will at least clear `CXXFLAGS` (if not alreaddy overridden by the user), when doing debugging or when coverage is enabled, to avoid Autoconfs `-g -O2` CXXFLAG default.
ACKs for top commit:
hebasto:
ACK 7b00595d33
Tree-SHA512: bda936a7aa8f98a1bf1552306845cb4bbab54e19a7a0b9ce3210e10fef70db146e9fe42a0cc8c50b2908506771b5b96f39c334e41323b70ec878e4010373096c
4d4dca43fc test: add regression test for bitcoin-core/gui/issues/567 (Vasil Dimov)
3b82608dd1 options: add a comment for -listenonion and dedup a long expression (Vasil Dimov)
Pull request description:
Add a test that would fail, should https://github.com/bitcoin-core/gui/issues/567 resurface.
Also, add a comment and dedup a long expression.
ACKs for top commit:
jarolrod:
reACK 4d4dca43fc
jonatack:
ACK 4d4dca43fc
hebasto:
ACK 4d4dca43fc, tested with reverting changes from bitcoin-core/gui#568, and getting an expected test failure.
shaavan:
ACK 4d4dca43fc
Tree-SHA512: 59f069bdaa84586bb599e9372f89e4e66a3cafcbf58677fdf913d685c17dfa9c3d5b118829d81021a9a33b4fd8e46d4c7eb68c1dd902cf1c44a41b8e66e2967b
112a7ab9a8 refactor: remove macOS MAP_ANONYMOUS work around (fanquake)
Pull request description:
This was added to support compilation on macOS 10.10, our minimum
required macOS is now 10.15. macOS has also supported it since 10.11.
See https://github.com/bitcoin/bitcoin/pull/9063.
macOS 12.3 manpage for mmap:
```bash
MAP_ANONYMOUS Synonym for MAP_ANON.
MAP_ANON Map anonymous memory not associated with any specific file.
```
ACKs for top commit:
laanwj:
Code review ACK 112a7ab9a8
jarolrod:
ACK 112a7ab9a8
Tree-SHA512: 920744c755d05d813ab312ff27e42eacb27b1297972800e6fb64bbaad1ea14258751a7dd80c07bfa554a172f36960b26a07505f67e82885253c8bf551073c38e
This was added to support compilation on macOS 10.10, our minimum
required macOS is now 10.15. macOS has also supported it since 10.11.
See https://github.com/bitcoin/bitcoin/pull/9063.
0c64401324 Revert "qt: Do not use QObject::tr plural syntax for numbers with a unit symbol" (Luke Dashjr)
Pull request description:
Apparently this got forgotten. Maybe too late for 23.x (it's a bugfix, but changes translation strings).
This reverts commit 3adde72bc9 (#296)
per [GChuf](https://github.com/bitcoin-core/gui/pull/296#issuecomment-962516055)
>I can confirm for slovenian and other slavic languages that we do have 3 or 4 different ways of saying "%n GB needed%, depending on the actual number of gigabytes. Similar to english "is/are". There's no way to cover all cases ... this is exactly why transifex allows you to have more than 2 options.
ACKs for top commit:
hebasto:
ACK 0c64401324, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: c01bae44a32b3ec324f2f9b8e4923bbb2e83bbd1460b745c5c911b98a9b2806fcbf815cfb19a1f1a7038c5c14312e102e7df8744c9002ef784b36d158e08eb14
bf77fea3c1 test: fix incorrect named args in txpackage tests (fanquake)
Pull request description:
Final non-scripted-diff commit split from #24661.
Could be tested with: `./autogen.sh && ./configure CC=clang-12 CXX=clang++-12 && make clean && bear make -j9 && ( cd ./src/ && run-clang-tidy-12 -j9 )`.
Motivation:
> Incorrect named args are source of bugs, like https://github.com/bitcoin/bitcoin/pull/22979.
> To allow them being checked by clang-tidy, use a format it can understand.
ACKs for top commit:
ajtowns:
ACK bf77fea3c1
Tree-SHA512: a13bfb5fc70424b13fbeec7f164d7a0d3b72b27ebec11dfd4115b7782a0037f26e9349e06eef8a6b17b8f529e0c7f43ae37a9c252bde65706dd164704d207d5f
21520b9551 fuzz: add target for coinselection (Martin Zumsande)
Pull request description:
This adds a fuzz target for the coinselection algorithms by creating random `OutputGroup`s and running all three coin selection algorithms for them.
It does not fuzz higher-level wallet logic for selecting eligible coins (as in `SelectCoins()`), thought it probably would make sense to have a fuzz target for that too.
ACKs for top commit:
achow101:
ACK 21520b9551
vasild:
ACK 21520b9551
Tree-SHA512: c763003cf5ff5317f929d3d0b2f06fa739ae41dd642042d9a5c5c96e6cb9b349a6c7aeabc77bc2b846d12c8bcb60e07ee20a9f38539429c65723ab76aeee6b2e
0c12f0116c wallet: Postpone NotifyWalletLoaded() for encrypted wallets (Hennadii Stepanov)
aeee419c6a wallet, refactor: Add wallet::NotifyWalletLoaded() function (Hennadii Stepanov)
Pull request description:
Fixes bitcoin-core/gui#571.
`CWallet::Create()` notifies about wallet loading too early, that results the notification goes before `DescriptorScriptPubKeyMan`s were created and added to an encrypted wallet.
And `interfaces::Wallet::taprootEnabled()` in ecf692b466/src/qt/receivecoinsdialog.cpp (L100-L102) erroneously returns `false` for just created encrypted descriptor wallets.
ACKs for top commit:
Sjors:
utACK 0c12f0116c
achow101:
ACK 0c12f0116c
Tree-SHA512: 2694bacd12748cd5f6c95d9d3bf8bcf4502ee67fecd8d057f33236b72069c61401b08f49deb013fc71c3f1e51ae16bdfd827ddcbc2a083d7044589be7a78982e
71038a151e rpc: Fix documentation assertion for `getrawtransaction` (laanwj)
Pull request description:
When `getrawtransaction` is successfully used on a coinbase transaction, there is an assertion error. This is very unlikely but happens in the `interface_usdt_utxocache.py` test in #24358.
This does the following:
- Add missing "coinbase" documentation.
- Synchronize documentation between `getrawtransaction` and `decoderawtransaction`, the two users of `TxToUniv` that have detailed documentation. `decodepsbt` and `getblock` also uses it but fortunately elides this block.
- Change "vout[].amount" to `STR_AMOUNT` for consistency.
- Add maintainer comment to keep the two places synchronized. It might be possible to get smarter with deduplication, but there are some extra fields that prevent the obvious way.
ACKs for top commit:
jonatack:
ACK 71038a151e
Tree-SHA512: 962236130455d805190ff9a5c971e4e25c17db35614a90ce340264ec953b0ad7fb814eb33ae430b5073955a8a350f72bdd67ba93e35f9c70e5175b836a767a35
9563a645c2 refactor: add stdd:: includes to core_write (fanquake)
8b9efebb0a refactor: use named args when ScriptToUniv or TxToUniv are invoked (Michael Dietz)
22f25a6116 refactor: prefer snake case, TxToUniv arg hashBlock renamed block_hash (Michael Dietz)
828a094ecf refactor: merge ScriptPubKeyToUniv & ScriptToUniv into one function (Michael Dietz)
Pull request description:
I've cherry-picked some of the commits out of #22924, and made minor changes (like fixing named args).
ACKs for top commit:
MarcoFalke:
re-ACK 9563a645c2🕓
Tree-SHA512: 4f0e5b45c14cbf68b9e389bbe1211c125d95cbd3da5205b1cff6a4c44f15b15039ba2a5b25cd7e2580d9169404f1b7ff620d8a7e01f6112e3cb153ecfaef8916
2ef47ba6c5 util/check: stop using lambda for Assert/Assume (Anthony Towns)
7c9fe25c16 wallet: move Assert() check into constructor (Anthony Towns)
Pull request description:
Using a lambda creates a couple of odd namespacing issues, in particular making clang's thread safety analysis less helpful, and confusing gcc when calling member functions. Fix this by not using a lambda.
Fixes #21596
Fixes #24654
ACKs for top commit:
MarcoFalke:
ACK 2ef47ba6c5🚢
jonatack:
Tested re-ACK 2ef47ba6c5
Tree-SHA512: 4bdbf3215f3d14472df0552362c5eebe8b7eea2d0928a8a41109edd4e0c5f95de6f8220eb2fee8506874e352c003907faf5ef344174795939306a618157b1bae
Too early NotifyWalletLoaded() call in CWallet::Create() results the
notification goes before DescriptorScriptPubKeyMans were created and
added to an encrypted wallet.
Co-authored-by: Andrew Chow <achow101-github@achow101.com>
When `getrawtransaction` is successfully used on a coinbase transaction,
there is an assertion error. This is very unlikely but happens in the
test in #24358.
This does the following:
- Add missing "coinbase" documentation.
- Synchronize documentation between `getrawtransaction` and
`decoderawtransaction`, the two users of `TxToUniv` that have detailed
documentation. `decodepsbt` also uses it but fortunately elides this block.
- Change "vout[].amount" to `STR_AMOUNT` for consistency.
- Add maintainer comment to keep the two places synchronized. It might
be possible to get smarter with deduplication, but there are some
extra fields that prevent the obvious way.
bb84b7145b add tests for no recipient and using send_max while inputs are specified (ishaanam)
49090ec402 Add sendall RPC née sweep (Murch)
902793c777 Extract FinishTransaction from send() (Murch)
6d2208a3f6 Extract interpretation of fee estimation arguments (Murch)
a31d75e5fb Elaborate error messages for outdated options (Murch)
35ed094e4b Extract prevention of outdated option names (Murch)
Pull request description:
Add sendall RPC née sweep
_Motivation_
Currently, the wallet uses a fSubtractFeeAmount (SFFO) flag on the
recipients objects for all forms of sending calls. According to the
commit discussion, this flag was chiefly introduced to permit sweeping
without manually calculating the fees of transactions. However, the flag
leads to unintuitive behavior and makes it more complicated to test
many wallet RPCs exhaustively. We proposed to introduce a dedicated
`sendall` RPC with the intention to cover this functionality.
Since the proposal, it was discovered in further discussion that our
proposed `sendall` rpc and SFFO have subtly different scopes of
operation.
• sendall:
Use _given UTXOs_ to pay a destination the remainder after fees.
• SFFO:
Use a _given budget_ to pay an address the remainder after fees.
While `sendall` will simplify cases of spending a given set of
UTXOs such as paying the value from one or more specific UTXOs, emptying
a wallet, or burning dust, we realized that there are some cases in
which SFFO is used to pay other parties from a limited budget,
which can often lead to the creation of change outputs. This cannot be
easily replicated using `sendall` as it would require manual
computation of the appropriate change amount.
As such, sendall cannot replace all uses of SFFO, but it still has a
different use case and will aid in simplifying some wallet calls and
numerous wallet tests.
_Sendall call details_
The proposed sendall call builds a transaction from a specific
subset of the wallet's UTXO pool (by default all of them) and assigns
the funds to one or more receivers. Receivers can either be specified
with a given amount or receive an equal share of the remaining
unassigned funds. At least one recipient must be provided without
assigned amount to collect the remainder. The `sendall` call will
never create change. The call has a `send_max` option that changes the
default behavior of spending all UTXOs ("no UTXO left behind"), to
maximizing the output amount of the transaction by skipping uneconomic
UTXOs. The `send_max` option is incompatible with providing a specific
set of inputs.
---
Edit: Replaced OP with latest commit message to reflect my updated motivation of the proposal.
ACKs for top commit:
achow101:
re-ACK bb84b7145b
Tree-SHA512: 20aaf75d268cb4b144f5d6437d33ec7b5f989256b3daeeb768ae1e7f39dc6b962af8223c5cb42ecc72dc38cecd921c53c077bc0ec300b994e902412213dd2cc3
f05a4cdf5a util: Add inotify_rm_watch to syscall sandbox (AllowFileSystem) (Hennadii Stepanov)
Pull request description:
This PR fixes the current master (3297f5c11c) when running `bitcoin-qt` on Ubuntu 22.04 and quitting:
```
$ ./src/qt/bitcoin-qt -signet -sandbox=log-and-abort
Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
ERROR: The syscall "inotify_rm_watch" (syscall number 255) is not allowed by the syscall sandbox in thread "main". Please report.
terminate called without an active exception
Aborted (core dumped)
```
Also see https://github.com/bitcoin/bitcoin/pull/24659#discussion_r835747166
ACKs for top commit:
fanquake:
ACK f05a4cdf5a - checked that qt is using this in it's filesystem watcher code.
Tree-SHA512: 9c7920a25422cd3a040bc1cbc487c12c3dc2b91358c3757f1030d6a1ff12c18c688a8e5b7466f683da88a5e4f5f15d442975660022d706e47021253c24c58f4a
d4ba2b2cbc compat: remove strnlen back-compat code (fanquake)
Pull request description:
This was needed for mingw (not mingw-w64), and some older versions of
macOS, which we no-longer support.
ACKs for top commit:
hebasto:
ACK d4ba2b2cbc
Tree-SHA512: d1beb9df58464feea3076091361d7d46e4a8901e347644a5fa6f24e052ca24ee0c7c0dd3f2a3d682b0204bf50430fa89eac62121691ea08af6dcf6b907bdec87
a40978dcbd [fuzz] Assert that Peer.m_tx_relay.m_relay_txs has been set correctly (John Newbery)
0bca5f2b46 [net processing] PushNodeVersion() takes a const Peer& (John Newbery)
21154ff927 net_processing: move CNode data access out of lock (John Newbery)
Pull request description:
#21160 ([net/net processing]: Move tx inventory into net_processing) had some unaddressed review comments when it was merged. This branch addresses those comments.
ACKs for top commit:
MarcoFalke:
review ACK a40978dcbd
dergoegge:
ACK a40978dcbd
ajtowns:
ACK a40978dcbd
Tree-SHA512: 46624e275f918c5f32d0adab0766e9b3ef8ebdbc74a3c8886d8a2e2ff1079029dcc371b40ef0d787609e9c05219b7456f3e2dfe4fb0cb7bf23ef966769aef1a1
_Motivation_
Currently, the wallet uses a fSubtractFeeAmount (SFFO) flag on the
recipients objects for all forms of sending calls. According to the
commit discussion, this flag was chiefly introduced to permit sweeping
without manually calculating the fees of transactions. However, the flag
leads to unintuitive behavior and makes it more complicated to test
many wallet RPCs exhaustively. We proposed to introduce a dedicated
`sendall` RPC with the intention to cover this functionality.
Since the proposal, it was discovered in further discussion that our
proposed `sendall` rpc and SFFO have subtly different scopes of
operation.
• sendall:
Use _specific UTXOs_ to pay a destination the remainder after fees.
• SFFO:
Use a _specific budget_ to pay an address the remainder after fees.
While `sendall` will simplify cases of spending from specific UTXOs,
emptying a wallet, or burning dust, we realized that there are some
cases in which SFFO is used to pay other parties from a limited budget,
which can often lead to the creation of change outputs. This cannot be
easily replicated using `sendall` as it would require manual computation
of the appropriate change amount.
As such, sendall cannot replace all uses of SFFO, but it still has a
different use case and will aid in simplifying some wallet calls and
numerous wallet tests.
_Sendall call details_
The proposed sendall call builds a transaction from a specific subset of
the wallet's UTXO pool (by default all of them) and assigns the funds to
one or more receivers. Receivers can either be specified with a specific
amount or receive an equal share of the remaining unassigned funds. At
least one recipient must be provided without assigned amount to collect
the remainder. The `sendall` call will never create change. The call has
a `send_max` option that changes the default behavior of spending all
UTXOs ("no UTXO left behind"), to maximizing the output amount of the
transaction by skipping uneconomic UTXOs. The `send_max` option is
incompatible with providing a specific set of inputs.
532c64a726 build: Fix Boost.Process test for Boost 1.78 (Hennadii Stepanov)
Pull request description:
Rebased #24415 with Luke's suggestion.
Fixes #24413.
ACKs for top commit:
hebasto:
ACK 532c64a726, tested on Mac mini (M1, 2020) + macOS Monterey 12.3 (21E230).
Tree-SHA512: 74f779695f6bbc45a2b7341a1402f747cc0d433d74825c7196cb9f156db0c0299895365f01665bd0bff12a8ebb5ea33a29b9a52f5eac0007ec35d1dca6544705
21db4eb3ff test: fix incorrect named args in wallet tests (fanquake)
8b0e776718 test: fix incorrect named args in coin_selection tests (fanquake)
6fc00f7331 bench: fix incorrect named args in coin_selection bench (fanquake)
Pull request description:
Should be one of the last changes split from #24661.
Motivation:
> Incorrect named args are source of bugs, like https://github.com/bitcoin/bitcoin/pull/22979.
> To allow them being checked by clang-tidy, use a format it can understand.
ACKs for top commit:
MarcoFalke:
Concept ACK 21db4eb3ff
Tree-SHA512: c29743a70f6118cf73dc37b56b30f45da55b7d7b3b8ed36859ad59f602c3e6692eb755e05d9a4dd17f05085bcd6cb5b8c4007090a76e4fbfb053f925322cf985
fc892c3a80 rpc: Fail to return undocumented or misdocumented JSON (MarcoFalke)
f4bc4a705a rpc: Add m_skip_type_check to RPCResult (MarcoFalke)
Pull request description:
This avoids documentation shortcomings such as the ones fixed in commit e7b6272b30, 138d55e6a0, 577bd51a4b, f8c84e047c, 0ee9a00f90, 13f41855c5, or faecb2ee0a
ACKs for top commit:
fanquake:
ACK fc892c3a80 - tested that this catches issue, i.e #24691:
Tree-SHA512: 9d0d7e6291bfc6f67541a4ff746d374ad8751fefcff6d103d8621c0298b190ab1d209ce96cfc3a0d4a6a5460a9f9bb790eb96027b16e5ff91f2512e40c92ca84
