fa4c59d65b Move blockstorage option logging to LoadChainstate() (MacroFake)
fa3358b668 Move validation option logging to LoadChainstate() (MacroFake)
Pull request description:
This would allow libbitcoinkernel users to see the options logged as well. Currently they would only be logged for bitcoind. Behavior change suggested in the refactoring pull https://github.com/bitcoin/bitcoin/pull/25704#discussion_r956166460
ACKs for top commit:
ryanofsky:
Code review ACK fa4c59d65b. Only change since last review is moving pruning logprints out of `AppInitParameterInteraction` as suggested
jonatack:
Review ACK fa4c59d65b
Tree-SHA512: f27508ca06a78ef162f002d556cf830df374fe95fd4f10bf22c24b6b48276ce49f52f82ffedc43596c872ddcf08321ca03651495fd3abde16254cb8afab39d33
50996241f2 rpc: sort listdescriptors result (Sjors Provoost)
Pull request description:
This puts receive and change descriptors directly below each other.
The change would be simpler if `UniValue` arrays were sortable.
ACKs for top commit:
achow101:
ACK 50996241f2
S3RK:
reACK 50996241f2
furszy:
utACK 50996241
w0xlt:
reACK 50996241f2
Tree-SHA512: 71246a48ba6f97c3e7c76ee32ff9e958227a14ca5a6eec638215dbfee57264d4e918ea5837f4d030eddc9c797c93df1791ddd55b5a499522ce2a35bcf380670b
b2544d1ee3 qt: Update translation source file for string freeze (Hennadii Stepanov)
Pull request description:
This PR updates the `src/qt/locale/bitcoin_en.xlf` translation source file according to [Release schedule for 24.0](https://github.com/bitcoin/bitcoin/issues/24987).
There were some new strings added since the [recent](https://github.com/bitcoin-core/gui/pull/654) update:
- "Unable to find UTXO for external input" in bitcoin/bitcoin#25679
- "Pre-syncing Headers (%1%)…" in bitcoin/bitcoin#25717
- "Unknown. Pre-syncing Headers (%1, %2%)…" in bitcoin/bitcoin#25717
ACKs for top commit:
jarolrod:
ACK b2544d1ee3
Tree-SHA512: cc3785a3279b2fae8cd477cd3a5b07f5321b25e963f4e03429538a6d43a858f6636e118eccfa64c77dc6279624937ddb6b7dd5d52021193ed93392a98662f25a
1b5bec78e9 init: avoid unsetting service bits from `nLocalServices` (Sebastian Falbesoner)
Pull request description:
This PR is a late follow-up to the [review club session about the PR "Default to NODE_WITNESS in nLocalServices" ](https://bitcoincore.reviews/21090#l-90) (#21090):
```
17:32 <lightlike> hmm, if we are in pruned mode, we first set NODE_NETWORK and then unset it later in init.cpp. that seems a bit strange.
...
17:33 <jnewbery> lightlike: ah yes, you're right. That does seem a bit messy.
```
Rather than setting the service bit `NODE_NETWORK` first and then unset it (if in `fPruneMode`), start with the bare minimum flags that we always serve and only add `NODE_NETWORK` if we are running as a non-pruned node. This seems to be a more logical approach than currently on master.
ACKs for top commit:
naumenkogs:
ACK 1b5bec78e9
stickies-v:
ACK 1b5bec78e9
LarryRuane:
ACK 1b5bec78e9
Tree-SHA512: 2e82d66c4298ffacff41d9e0458b74b83bc156a1fa49e3f3471e942878e5dd2b253b5597ee5ec1d9c8726b432751d05e40f0c580f3976a9e00a7d1f417921ab0
2c05dc7811 Fix link to MurmurHash3.cpp from Austin Appleby (dontbyte)
Pull request description:
Google Code repo doesn't exist anymore
ACKs for top commit:
Zero-1729:
crACK 2c05dc7811
Tree-SHA512: 3e095255757b536f382ffb63e4292413592246c2446d486acbb71c52e4a3ece519d7cfae941685d9e25fd62de5c783510b3d076cd990a3d391496dc3076a0385
fae5bd9200 test: Fix wallet_balance intermittent issue (MacroFake)
Pull request description:
Diff to reproduce:
```diff
index d2ed97ca76..25cc2d5734 100755
--- a/test/functional/wallet_balance.py
+++ b/test/functional/wallet_balance.py
@@ -265,7 +265,7 @@ class WalletTest(BitcoinTestFramework):
self.nodes[0].invalidateblock(block_reorg)
self.nodes[1].invalidateblock(block_reorg)
assert_equal(self.nodes[0].getbalance(minconf=0), 0) # wallet txs not in the mempool are untrusted
- self.generatetoaddress(self.nodes[0], 1, ADDRESS_WATCHONLY, sync_fun=self.no_op)
+ self.generatetoaddress(self.nodes[0], 1, ADDRESS_WATCHONLY)
assert_equal(self.nodes[0].getbalance(minconf=0), 0) # wallet txs not in the mempool are untrusted
# Now confirm tx_orig
```
Example in CI:
```
test 2022-08-24T10:09:22.486000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 133, in main
self.run_test()
File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/wallet_balance.py", line 269, in run_test
assert_equal(self.nodes[0].getbalance(minconf=0), 0) # wallet txs not in the mempool are untrusted
File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/util.py", line 56, in assert_equal
raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not(98.85983340 == 0)
```
https://cirrus-ci.com/task/4981266251513856?logs=ci#L3269
ACKs for top commit:
achow101:
ACK fae5bd9200
w0xlt:
ACK fae5bd9200
Tree-SHA512: 470f366720615c4a9326ec4c581fff569ecce9877f9134bb1975ec3d6f1d13a6403051418a91a80b2a86de617f43e539ec11bbf4f1713d0354d5b0ab98d22437
6b24dfe24d CBlockLocator: performance-move-const-arg Clang tidy fixups (Jon Atack)
Pull request description:
Fix Clang-tidy CI errors on master. See https://cirrus-ci.com/task/4806752200818688?logs=ci#L4696 for an example.
ACKs for top commit:
MarcoFalke:
review ACK 6b24dfe24d
vasild:
ACK 6b24dfe24d
Tree-SHA512: 7a67acf7b42da07b63fbb392236e9a7be8cf35c36e37ca980c4467fe8295c2eda8aef10f41a1e3036cd9ebece47fa957fc3256033f853bd6a97ce2ca42799a0a
fa875349e2 Fix iwyu (MacroFake)
faad673716 Fix issues when calling std::move(const&) (MacroFake)
Pull request description:
Passing a symbol to `std::move` that is marked `const` is a no-op, which can be fixed in two ways:
* Remove the `const`, or
* Remove the `std::move`
ACKs for top commit:
ryanofsky:
Code review ACK fa875349e2. Looks good. Good for univalue to support c++11 move optimizations
Tree-SHA512: 3dc5cad55b93cfa311abedfb811f35fc1b7f30a1c68561f15942438916c7de25e179c364be11881e01f844f9c2ccd71a3be55967ad5abd2f35b10bb7a882edea
28ea4c7039 test: simplify splitment with `sendall` in wallet_basic (brunoerg)
923d24583d test: use `sendall` when emptying wallet (brunoerg)
Pull request description:
In some tests they have used `sendtoaddress` in order to empty a wallet. With the addition of `sendall`, it makes sense to use it for that.
ACKs for top commit:
achow101:
ACK 28ea4c7039
ishaanam:
utACK 28ea4c7039
w0xlt:
ACK 28ea4c7039
Tree-SHA512: 903136d7df5c65d3c02310d5a84241c9fd11070f69d932b4e188b8ad45c38ab5bc1bd5a9242b3e52d2576665ead14be0a03971a9ad8c00431fed442eba4ca48f
f345dc3960 tidy: enable bugprone-use-after-move (fanquake)
94f2235f85 test: work around bugprone-use-after-move warnings in util tests (fanquake)
Pull request description:
Would have caught #25640.
Currently `// NOLINT`s around:
```bash
test/util_tests.cpp:2513:34: error: 't2' used after it was moved [bugprone-use-after-move,-warnings-as-errors]
BOOST_CHECK(v2[0].origin == &t2);
^
test/util_tests.cpp:2511:15: note: move occurred here
auto v2 = Vector(std::move(t2));
^
test/util_tests.cpp:2519:34: error: 't2' used after it was moved [bugprone-use-after-move,-warnings-as-errors]
BOOST_CHECK(v3[1].origin == &t2);
^
test/util_tests.cpp:2516:15: note: move occurred here
auto v3 = Vector(t1, std::move(t2));
^
test/util_tests.cpp:2527:34: error: 't3' used after it was moved [bugprone-use-after-move,-warnings-as-errors]
BOOST_CHECK(v4[2].origin == &t3);
^
test/util_tests.cpp:2523:15: note: move occurred here
auto v4 = Vector(std::move(v3[0]), v3[1], std::move(t3));
```
See: https://releases.llvm.org/14.0.0/tools/clang/tools/extra/docs/clang-tidy/checks/bugprone-use-after-move.html
ACKs for top commit:
ryanofsky:
Code review ACK f345dc3960. Only change since last review is switching to NOLINT directives
Tree-SHA512: afadecbaf1069653f4be5d6e66a5800ffd975c0b1a960057abc6367b616c181cd518897a874a8f3fd5e5e1f45fcc165f7a9a3171136cd4deee641214c4b765b8
3add234546 ui: show header pre-synchronization progress (Pieter Wuille)
738421c50f Emit NotifyHeaderTip signals for pre-synchronization progress (Pieter Wuille)
376086fc5a Make validation interface capable of signalling header presync (Pieter Wuille)
93eae27031 Test large reorgs with headerssync logic (Suhas Daftuar)
355547334f Track headers presync progress and log it (Pieter Wuille)
03712dddfb Expose HeadersSyncState::m_current_height in getpeerinfo() (Suhas Daftuar)
150a5486db Test headers sync using minchainwork threshold (Suhas Daftuar)
0b6aa826b5 Add unit test for HeadersSyncState (Suhas Daftuar)
83c6a0c524 Reduce spurious messages during headers sync (Suhas Daftuar)
ed6cddd98e Require callers of AcceptBlockHeader() to perform anti-dos checks (Suhas Daftuar)
551a8d957c Utilize anti-DoS headers download strategy (Suhas Daftuar)
ed470940cd Add functions to construct locators without CChain (Pieter Wuille)
84852bb6bb Add bitdeque, an std::deque<bool> analogue that does bit packing. (Pieter Wuille)
1d4cfa4272 Add function to validate difficulty changes (Suhas Daftuar)
Pull request description:
New nodes starting up for the first time lack protection against DoS from low-difficulty headers. While checkpoints serve as our protection against headers that fork from the main chain below the known checkpointed values, this protection only applies to nodes that have been able to download the honest chain to the checkpointed heights.
We can protect all nodes from DoS from low-difficulty headers by adopting a different strategy: before we commit to storing a header in permanent storage, first verify that the header is part of a chain that has sufficiently high work (either `nMinimumChainWork`, or something comparable to our tip). This means that we will download headers from a given peer twice: once to verify the work on the chain, and a second time when permanently storing the headers.
The p2p protocol doesn't provide an easy way for us to ensure that we receive the same headers during the second download of peer's headers chain. To ensure that a peer doesn't (say) give us the main chain in phase 1 to trick us into permanently storing an alternate, low-work chain in phase 2, we store commitments to the headers during our first download, which we validate in the second download.
Some parameters must be chosen for commitment size/frequency in phase 1, and validation of commitments in phase 2. In this PR, those parameters are chosen to both (a) minimize the per-peer memory usage that an attacker could utilize, and (b) bound the expected amount of permanent memory that an attacker could get us to use to be well-below the memory growth that we'd get from the honest chain (where we expect 1 new block header every 10 minutes).
After this PR, we should be able to remove checkpoints from our code, which is a nice philosophical change for us to make as well, as there has been confusion over the years about the role checkpoints play in Bitcoin's consensus algorithm.
Thanks to Pieter Wuille for collaborating on this design.
ACKs for top commit:
Sjors:
re-tACK 3add234546
mzumsande:
re-ACK 3add234546
sipa:
re-ACK 3add234546
glozow:
ACK 3add234546
Tree-SHA512: e7789d65f62f72141b8899eb4a2fb3d0621278394d2d7adaa004675250118f89a4e4cb42777fe56649d744ec445ad95141e10f6def65f0a58b7b35b2e654a875
```bash
test/util_tests.cpp:2513:34: error: 't2' used after it was moved [bugprone-use-after-move,-warnings-as-errors]
BOOST_CHECK(v2[0].origin == &t2);
^
test/util_tests.cpp:2511:15: note: move occurred here
auto v2 = Vector(std::move(t2));
^
test/util_tests.cpp:2519:34: error: 't2' used after it was moved [bugprone-use-after-move,-warnings-as-errors]
BOOST_CHECK(v3[1].origin == &t2);
^
test/util_tests.cpp:2516:15: note: move occurred here
auto v3 = Vector(t1, std::move(t2));
^
test/util_tests.cpp:2527:34: error: 't3' used after it was moved [bugprone-use-after-move,-warnings-as-errors]
BOOST_CHECK(v4[2].origin == &t3);
^
test/util_tests.cpp:2523:15: note: move occurred here
auto v4 = Vector(std::move(v3[0]), v3[1], std::move(t3));
```
9816dc96b7 net: note CNode members that are treated as const (Anthony Towns)
ef26f2f421 net: mark CNode unique_ptr members as const (Anthony Towns)
bbec32c9ad net: mark TransportSerializer/m_serializer as const (Anthony Towns)
06ebdc886f net/net_processing: add missing thread safety annotations (Anthony Towns)
Pull request description:
Adds `GUARDED_BY` and `const` annotations to document how we currently ensure various members of `CNode` and `Peer` aren't subject to race conditions.
ACKs for top commit:
MarcoFalke:
review ACK 9816dc96b7📍
jonatack:
utACK 9816dc96b7
hebasto:
ACK 9816dc96b7, I have reviewed the code and it looks OK. In particular, I verified the usage of variables which got `GUARDED_BY` annotations.
Tree-SHA512: fa95bca72435d79caadc736ee7687e505dbe8fbdb20690809e97666664a8d0dea39a7d17cf16f0437d7f5746b9ad98a466b26325d2913252c5d2b520b384b785
m_permissionFlags and m_prefer_evict are treated as const -- they're
only set immediately after construction before any other thread has
access to the object, and not changed again afterwards. As such they
don't need to be marked atomic or guarded by a mutex; though it would
probably be better to actually mark them as const...
Dereferencing a unique_ptr is not necessarily thread safe. The reason
these are safe is because their values are set at construction and do
not change later; so mark them as const and set them via the initializer
list to guarantee that.
The (V1)TransportSerializer instance CNode::m_serializer is used from
multiple threads via PushMessage without protection by a mutex. This
is only thread safe because the class does not have any mutable state,
so document that by marking the methods and the object as "const".
This makes a number of changes:
- Get rid of the verification_progress argument in the node interface
NotifyHeaderTip (it was always 0.0).
- Instead of passing a CBlockIndex* in the UI interface's NotifyHeaderTip,
send separate height, timestamp fields. This is becuase in headers presync,
no actual CBlockIndex object is available.
- Add a bool presync argument to both of the above, to identify signals
pertaining to the first headers sync phase.
Delay sending SENDHEADERS (BIP 130) message until we know our peer's best
header's chain has more than nMinimumChainWork. This reduces inadvertent
headers messages received during initial headers sync due to block
announcements, which throw off our sync algorithm.
In order to prevent memory DoS, we must ensure that we don't accept a new
header into memory until we've performed anti-DoS checks, such as verifying
that the header is part of a sufficiently high work chain. This commit adds a
new argument to AcceptBlockHeader() so that we can ensure that all call-sites
which might cause a new header to be accepted into memory have to grapple with
the question of whether the header is safe to accept, or needs further
validation.
This patch also fixes two places where low-difficulty-headers could have been
processed without such validation (processing an unrequested block from the
network, and processing a compact block).
Credit to Niklas Gögge for noticing this issue, and thanks to Sjors Provoost
for test code.
Avoid permanently storing headers from a peer, unless the headers are part of a
chain with sufficiently high work. This prevents memory attacks using low-work
headers.
Designed and co-authored with Pieter Wuille.
5ef8c2c9fc test: fix typo for MaybeResendWalletTxs (stickies-v)
fbba4a1316 wallet: trigger MaybeResendWalletTxs() every minute (stickies-v)
Pull request description:
ResendWalletTransactions() only executes every [12-36h (24h average)](1420547ec3/src/wallet/wallet.cpp (L1947)). Triggering it every second is excessive, once per minute should be plenty.
The goal of this PR is to reduce the amount of (unnecessary) schedule executions by ~60x without meaningfully altering transaction rebroadcast logic/assumptions which would require more significant review.
ACKs for top commit:
achow101:
ACK 5ef8c2c9fc
1440000bytes:
ACK 5ef8c2c9fc
Tree-SHA512: 4a077e3579b289c11c347eaa0d3601ef2dbb9fee66ab918d56b4a0c2e08222560a0e6be295297a74831836e001a997ecc143adb0c132faaba96a669dac1cd9e6
835bd27e9a Wallet::SetMinVersion - Log the new minversion (Ali Sherief)
Pull request description:
This change prints a single additional line in the debug.log when bitcoin-cli loads a wallet using `loadwallet` (*not* `createwallet`).
When Bitcoin Core creates a wallet, it's `minversion` is set to `FEATURE_BASE`, which is 10500. However, once the wallet is unloaded using `unloadwallet` or through program termination, and subsequently loaded again, `loadwallet` updates the `minversion` in the wallet.dat file to `FEATURE_LATEST`, currently 169900.
The current logging format prints the very old wallet version during `createwallet`, and then the actual version in calls to `loadwallet`. This has confused at least one person ([reference](https://bitcointalk.org/index.php?topic=5410650.0) - I was the one who asked there if there were plans to change that behavior, and was subsequently redirected here by achow), so it will be very helpful to users to explicitly specify in the logs what the walletdb is doing.
ACKs for top commit:
achow101:
ACK 835bd27e9a
Tree-SHA512: 967c8c617e06a84915ddb147378ec3c8b0343e45f43145ec78df9cbc0201867f49c8e11cd068c403eb5ec06e07d38c3c0d3864dad8edc5efbb134a3fb30be41f
59aa54f731 i2p: log "SAM session" instead of "session" (Vasil Dimov)
d7ec30b648 doc: add release notes about the I2P transient addresses (Vasil Dimov)
47c0d02f12 doc: document I2P transient addresses usage in doc/i2p.md (Vasil Dimov)
3914e472f5 test: add a test that -i2pacceptincoming=0 creates a transient session (Vasil Dimov)
ae1e97ce86 net: use transient I2P session for outbound if -i2pacceptincoming=0 (Vasil Dimov)
a1580a04f5 net: store an optional I2P session in CNode (Vasil Dimov)
2b781ad66e i2p: add support for creating transient sessions (Vasil Dimov)
Pull request description:
Add support for generating a transient, one-time I2P address for ourselves when making I2P outbound connection and discard it once the connection is closed.
Background
---
In I2P connections, the host that receives the connection knows the I2P address of the connection initiator. This is unlike the Tor network where the recipient does not know who is connecting to them, not even the initiator's Tor address.
Persistent vs transient I2P addresses
---
Even if an I2P node is not accepting incoming connections, they are known to other nodes by their outgoing I2P address. This creates an opportunity to white-list given nodes or treat them differently based on their I2P address. However, this also creates an opportunity to fingerprint or analyze a given node because it always uses the same I2P address when it connects to other nodes. If this is undesirable, then a node operator can use the newly introduced `-i2ptransientout` to generate a transient (disposable), one-time I2P address for each new outgoing connection. That address is never going to be reused again, not even if reconnecting to the same peer later.
ACKs for top commit:
mzumsande:
ACK 59aa54f731 (verified via range-diff that just a typo / `unique_ptr` initialisation were fixed)
achow101:
re-ACK 59aa54f731
jonatack:
utACK 59aa54f731 reviewed range diff, rebased to master, debug build + relevant tests + review at each commit
Tree-SHA512: 2be9b9dd7502b2d44a75e095aaece61700766bff9af0a2846c29ca4e152b0a92bdfa30f61e8e32b6edb1225f74f1a78d19b7bf069f00b8f8173e69705414a93e
207abc147c doc: Add my key to SECURITY.md (Andrew Chow)
Pull request description:
I was recently added to the security list, and I think it would make sense to have more people who are on the list to be publicly listed as security contacts, so adding myself to the doc.
ACKs for top commit:
glozow:
ACK 207abc147c, matches the entry in trusted-keys.
Tree-SHA512: 33f91d8ea618d7dfdeb372695aff3092f2f2e3df8503eafff18fc3756b3da566a27d6f83fdaf01a749c3d71c7a17a8ae43af2495721b969442924ff773930290
e90a445d7e scripted-diff: rpc: fix rescan RPC name (s/rescanwallet/rescanblockchain/) (Sebastian Falbesoner)
Pull request description:
There is no RPC call named `rescanwallet`, i.e. fix this by renaming to the actual RPC called `rescanblockchain`.
ACKs for top commit:
achow101:
ACK e90a445d7e
aureleoules:
ACK e90a445d7e.
promag:
ACK e90a445d7e
Tree-SHA512: abf1d1c18de32d87c29e4ff2b782dfb0e4a46dc2c2cc51ab616d12674a0f4a5d22214e00955663ae897cbb88f4f6ced913850f28ea3f5c1b3a54577a25fbf399
