e71c51b27d refactor: rename command -> message type in comments in the src/net* files (Shashwat)
2b09593bdd scripted-diff: Rename message command to message type (Shashwat)
Pull request description:
This PR is a follow-up to #24078.
> a message is not a command, but simply a message of some type
The first commit covers the message_command variable name and comments not addressed in the original PR in `src/net*` files.
The second commit goes beyond the original `src/net*` limit of #24078 and does similar changes in the `src/rpc/net.cpp` file.
ACKs for top commit:
MarcoFalke:
review ACK e71c51b27d💥
Tree-SHA512: 24015d132c00f15239e5d3dc7aedae904ae3103a90920bb09e984ff57723402763f697d886322f78e42a0cb46808cb6bc9d4905561dc6ddee9961168f8324b05
b42643c253 doc: update init.cpp -conf help text (josibake)
970b9987ad doc: update devtools, release-process readmes (josibake)
50635d27b4 build: include bitcoin.conf in build outputs (josibake)
6aac946f49 doc: update bitcoin-conf.md (Josiah Baker)
1c7e820ded script: add script to generate example bitcoin.conf (josibake)
b483084d86 doc: replace bitcoin.conf with placeholder file (josibake)
Pull request description:
create a script for parsing the output from `bitcoind --help` to create an example conf file for new users
## problem
per #10746 , `bitcoin.conf` not being put into the data directory during installation causes some confusion for users when running bitcoin. in the discussion on the issue, one proposed solution was to have an example config file and instruct users to `cp` it into their data directory after startup. in addition to #10746 , there have been other requests for a "skeleton config file" (https://github.com/bitcoin/bitcoin/issues/19641) to help users get started with configuring bitcoind.
the main issue with an example config file is that it creates a second source of truth regarding what options are available for configuring bitcoind. this means any changes to the options (including the addition or removal of options) would have to be updated for the command line and also updated in the example file.
this PR addresses this issue by providing a script to generate an example file directly from the `bitcoind --help` on-demand by running `contrib/devtools/gen-bitcoin-conf.sh`. this solution was originally proposed on #10746 and would also solve #19641 . this guarantees any changes made to the command-line options or the command-line options help would also be reflected in the example file after compiling and running the script.
the main purpose of this script is to generate a config file to be included with releases, same as `gen-manpages.sh`. this ensures every release also includes an up-to-date, full example config file for users to edit. the script is also available for users who compile from source for generating an example config for their compiled binary.
## special considerations
this removes the `bitcoin.conf` example file from the repo as it is now generated by this script. the original example file did contain extra text related to how to use certain options but going forward all option help docs should be moved into `init.cpp`
this also edits `init.cpp` to have the option help indicate that `-conf` is not usable from the config file. this is similar to how `-includeconf` 's help indicates it cannot be used from the command line
ACKs for top commit:
laanwj:
Tested and code review ACK b42643c253
Tree-SHA512: 4546e0cef92aa1398da553294ce4712d02e616dd72dcbe0b921af474e54f24750464ec813661f1283802472d1e8774e634dd1cc26fbf1f13286d3e0406c02c09
e3a06a3c6c test: Add `strerror` to locale-dependence linter (laanwj)
f00fb1265a util: Increase buffer size to 1024 in SysErrorString (laanwj)
718da302c7 util: Refactor SysErrorString logic (laanwj)
e7f2f77756 util: Use strerror_s for SysErrorString on Windows (laanwj)
46971c6dbf util: Replace non-threadsafe strerror (laanwj)
Pull request description:
Some uses of non-threadsafe `strerror` have snuck into the code since they were removed in #4152. Add a wrapper `SysErrorString` for thread-safe strerror alternatives (with code from `NetworkErrorString`) and replace all uses of `strerror` with this.
Edit: I've also added a commit that refactors the code so that buf[] is never read at all if the function fails, making some fragile-looking code unnecessary.
Edit2: from the linux manpage:
```
ATTRIBUTES
For an explanation of the terms used in this section, see attributes(7).
┌───────────────────┬───────────────┬─────────────────────────┐
│Interface │ Attribute │ Value │
├───────────────────┼───────────────┼─────────────────────────┤
│strerror() │ Thread safety │ MT-Unsafe race:strerror │
├───────────────────┼───────────────┼─────────────────────────┤
…
├───────────────────┼───────────────┼─────────────────────────┤
│strerror_r(), │ Thread safety │ MT-Safe │
│strerror_l() │ │ │
└───────────────────┴───────────────┴─────────────────────────┘
```
As the function can be called from any thread at any time, using a non-thread-safe function is unacceptable.
ACKs for top commit:
jonatack:
ACK e3a06a3c6c
Tree-SHA512: 20e71ebb9e979d4e1d8cafbb2e32e20c2a63f09115fe72cdde67c8f80ae98c531d286f935fd8a6e92a18b72607d7bd3e846b2d871d9691a6036b0676de8aaf25
e5d1831517 [netgroup] Use nStartByte as offset for the last byte of the group (dergoegge)
Pull request description:
This addresses my review [comments](https://github.com/bitcoin/bitcoin/pull/22910#discussion_r856095896) I left on #22910.
This has no effect on the current logic as `nStartByte` is only used for internal addresses which only ever add 10 whole bytes to the returned group. However to avoid future bugs, I think we should use `nStartByte` as offset for the last byte as well, in case we ever add a new address type that makes makes use of `nStartByte` and adds fractional bytes to the group.
ACKs for top commit:
jnewbery:
Code review ACK e5d1831517
theStack:
Concept and code-review ACK e5d1831517
Tree-SHA512: 4c08c7d6cb38b553e998798b3e3b790177aaa2141a48e277dfd538e01a7fccadf644329e93c5b0fb5e7e4037494c8dfe061b94eb52c6b31dc21bdf99eb0e311a
f849e63bad fuzz: SplitString with multiple separators (Martin Leitner-Ankerl)
d1a9850102 http: replace boost::split with SplitString (Martin Leitner-Ankerl)
0d7efcdf75 core_read: Replace boost::split with SplitString (Martin Leitner-Ankerl)
b7ab9db545 Extend Split to work with multiple separators (Martin Leitner-Ankerl)
Pull request description:
As a followup of #22953, this removes the remaining occurrences of `boost::split` and replaces them with our own `SplitString`. To be able to do so, this extends the function `spanparsing::Split` to work with multiple separators. Finally this removes 3 more files from `lint-includes.py`.
ACKs for top commit:
theStack:
Code-review ACK f849e63bad
Tree-SHA512: f37d4dbe11cab2046e646045b0f018a75f978d521443a2c5001512737a1370e22b09247d5db0e5c9e4153229a4e2d66731903c1bba3713711c4cae8cedcc775d
4cb9d21434 blockstorage: add LIFETIMEBOUND to GetFirstStoredBlock()::start_time (Jon Atack)
Pull request description:
Suggested in https://github.com/bitcoin/bitcoin/pull/25016#discussion_r862330288, the lifetimebound attribute here indicates that a resource owned by the `start_block` param of `CBlockIndex* BlockManager::GetFirstStoredBlock()` can be retained by the method's return value, which enables detecting the use of out-of-scope stack memory (ASan `stack-use-after-scope`) at compile time.
See https://releases.llvm.org/12.0.0/tools/clang/docs/AttributeReference.html#lifetimebound and #22278 for related discussion, and #25040 for a similar example.
ACKs for top commit:
MarcoFalke:
review ACK 4cb9d21434
Tree-SHA512: a3f5ef83ebb6f08555d7c89f2437a682071b4ad77a7aa3326b6d2282c909bf9fcf4dac6bf05ee1d9931f2102cad4a02df5468bde1cf377d7126e84e8541604dc
fa758f9bc5 scripted-diff: Rename rpc/misc.cpp to rpc/node.cpp (MacroFake)
fa87eb8ce1 rpc: Move output script RPCs to separate file (MacroFake)
Pull request description:
RPCs handling output scripts (addresses, scriptPubKeys, and output script descriptors) should not be placed in a file called `misc.cpp`, so move them out, then rename `misc.cpp`.
ACKs for top commit:
pk-b2:
ACK fa758f9bc5
vincenzopalazzo:
ACK fa758f9bc5
Tree-SHA512: 0cf8b5b8456361015513e93d3e604ea07d998dd578415b1d0e2918fb401fc44547fc1bb80b7c33c2086f6268e7b8f59837d2955f57434f646ea7921f0158b32d
fa4652ce59 Pass lifetimebound reference to SingleThreadedSchedulerClient (MacroFake)
Pull request description:
Currently a pointer is passed, which is confusing and requires run-time asserts to avoid nullptr dereference.
All call sites can pass a reference, so do that. Also mark it LIFETIMEBOUND to avoid call sites passing a temporary. Also, unrelated cleanup in touched lines.
ACKs for top commit:
pk-b2:
ACK fa4652ce59
jonatack:
Code review ACK fa4652ce59 rebased to master, debug build, unit tests
vincenzopalazzo:
ACK fa4652ce59
Tree-SHA512: cd7ec77347e195d659b8892d34c1e9644d4f88552a4d5fa310dc1756eb27050a99d3098b0b0d27f8474230f82c178fd9e22e7018d8248d5e47a7f4caad395e25
Note that `SplitString` doesn't support token compression, but in this case
it does not matter as empty strings are already skipped anyways.
Also removes split.hpp and classification.hpp from expected includes
f64aa9c411 Disallow more unsafe string->path conversions allowed by path append operators (Ryan Ofsky)
Pull request description:
Add more `fs::path` `operator/` and `operator+` overloads to prevent unsafe string->path conversions on Windows that would cause strings to be decoded according to the current Windows locale & code page instead of the correct string encoding.
Update application code to deal with loss of implicit string->path conversions by calling `fs::u8path` or `fs::PathFromString` explicitly, or by just changing variable types from `std::string` to `fs::path` to avoid conversions altogether, or make them happen earlier.
In all cases, there's no change in behavior either (1) because strings only contained ASCII characters and would be decoded the same regardless of what encoding was used, or (2) because of the 1:1 mapping between paths and strings using the `PathToString` and `PathFromString` functions.
Motivation for this PR was just that I was experimenting with #24469 and noticed that operations like `fs::path / std::string` were allowed, and I thought it would be better not to allow them.
ACKs for top commit:
hebasto:
ACK f64aa9c411
Tree-SHA512: 944cce49ed51537ee7a35ea4ea7f5feaf0c8fff2fa67ee81ec5adebfd3dcbaf41b73eb35e49973d5f852620367f13506fd12a7a9b5ae3a7a0007414d5c9df50f
88044a14d9 Guard `#include <config/bitcoin-config.h>` (Hennadii Stepanov)
Pull request description:
A fix for builds when the `HAVE_CONFIG_H` macro is not defined.
ACKs for top commit:
Empact:
Code Review ACK 88044a14d9
Tree-SHA512: f2bf1693c7671d7113dccaf66ae34a84719d86cb3271fa18b36611deab93a48d787b3ccfbd735d3b763017d709971cb1151d8d7f30390720009e6e2a6275b5b0
fa753abd7c rpc: Move fee estimation RPCs to separate file (MacroFake)
Pull request description:
Fee estimation is generally used by wallets when creating txs. It doesn't have anything to do with creating or submitting blocks.
ACKs for top commit:
pk-b2:
ACK fa753abd7c
brunoerg:
crACK fa753abd7c
Tree-SHA512: 81e0edc936198a0baf0f5bfa8cfedc12db51759c7873bb0082dfc5f0040d7f275b35f639c6f5b86fa1ea03397b0d5e757c2ce1b6b16f1029880a39b9c3aaceda
e5485e8e4b test, bench: make prevector and checkqueue swap member functions noexcept (Jon Atack)
abc1ee5090 validation: make CScriptCheck and prevector swap member functions noexcept (Jon Atack)
Pull request description:
along with those seen elsewhere in the codebase (prevector and checkqueue units/fuzz/bench).
A swap must not fail; when a class has a swap member function, it should be declared noexcept.
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#c84-a-swap-function-must-not-fail
ACKs for top commit:
pk-b2:
ACK e5485e8e4b
w0xlt:
ACK e5485e8e4b
Tree-SHA512: c82359d5e13f9262ce45efdae9baf71e41ed26568e0aff620e2bfb0ab37a62b6d56ae9340a28a0332c902cc1fa87da3fb72d6f6d6f53a8b7e695a5011f71f7f1
778343a379 scripted-diff: Rename PeerManagerImpl members (dergoegge)
91c339243e [net processing] Move nHighestFastAnnounce into PeerManagerImpl (dergoegge)
10b83e2aa3 [net processing] Move block cache state into PeerManagerImpl (dergoegge)
a4c55a93ef [net processing] Inline and simplify UpdatePreferredDownload (dergoegge)
490c08f96a [net processing] Move nPreferredDownload into PeerManagerImpl (dergoegge)
a292df283a [net processing] Move mapNodeState into PeerManagerImpl (dergoegge)
37ecaf3e7a [net processing] Move CNodeState declaration above PeerManagerImpl (dergoegge)
Pull request description:
This PR moves the remaining net processing globals into `PeerManagerImpl`. This will make testing the peer manager in isolation easier and also acts as a code clean up.
ACKs for top commit:
jnewbery:
Code review ACK 778343a379
MarcoFalke:
ACK 778343a379 🗒
Tree-SHA512: 4f22105d1de37b94c3ef349f38784a30cf8d450d394a6a7849e5bd78940a71e3edbffa3d25e8efb35d7f698fd255f199de7bd4c33e23af5621a6e4e67ed43cb5
fad35e9afd test: Remove boost::split from rpc_tests.cpp (MacroFake)
Pull request description:
No need for boost, as there are no tabs.
Can be tested with:
```diff
diff --git a/src/test/rpc_tests.cpp b/src/test/rpc_tests.cpp
index 50b5078110..ad6a888ad0 100644
--- a/src/test/rpc_tests.cpp
+++ b/src/test/rpc_tests.cpp
@@ -29,6 +29,7 @@ public:
UniValue RPCTestingSetup::CallRPC(std::string args)
{
+Assert(args.find('\t')==std::string::npos);
std::vector<std::string> vArgs;
boost::split(vArgs, args, boost::is_any_of(" \t"));
std::string strMethod = vArgs[0];
ACKs for top commit:
fanquake:
utACK fad35e9afd
Tree-SHA512: 3df789a222b407d61ad549adc4bbded00705d7c3db07472c31ce0e82216fe3ae27724b7f0ee3e85084bdf405cc28185e85487c9a7001620d6654fda77bab8eb3
e2b954e87f rpc: use GetBlockTime() for getblockchaininfo#time (Jon Atack)
86ce844d3b blockstorage, refactor: pass GetFirstStoredBlock() start_block by reference (Jon Atack)
ed12c0a49d blockstorage, refactor: make GetFirstStoredBlock() a member of BlockManager (Jon Atack)
Pull request description:
Picks up the remaining review feedback in #21726 and #24956.
- make the global function `GetFirstStoredBlock()` a member of the `BlockManager` class
- pass the `start_block` param of `GetFirstStoredBlock()` by reference instead of a pointer
- use `GetBlockTime()` for RPC getblockchaininfo#time
ACKs for top commit:
MarcoFalke:
ACK e2b954e87f
Tree-SHA512: 546e3c2e18245996b5b286829a605ae919eff3510963ec71b7c9ede521b1f501697e5b2f9d35d7a0606a74cbc8907201c58acf1e2cf7daaa86eefe2e3a8e296b
fa2102e239 test: Split MempoolAncestryTests into two (MacroFake)
Pull request description:
The two tests don't share any state, so it seems clearer to put them in separate scopes.
ACKs for top commit:
jnewbery:
Code review ACK fa2102e239
Tree-SHA512: 6669f50f8d5944fed55ecc88aa1bd139bddf6a40e3c2e8f88c3cc7e70cf6d4650c0dd652c7f304813893827c3930d626268655cd9b3f17ff9c9a1a02f0359714
fa60169811 rpc: Move signmessage RPC util to new file (MacroFake)
fa9425177e Remove cs_main from verifymessage (MacroFake)
Pull request description:
The `verifymessage` RPC has several issues:
* It takes `cs_main` for no reason, blocking progress on removing the `cs_main` global mutex.
* It is located in a file called `misc`, which is not a very helpful name.
Fix all issues.
ACKs for top commit:
vincenzopalazzo:
ACK fa60169811
Tree-SHA512: c71a1f481b828e0a544405fecbbc7ca44e66ea46b498d7aed1f1c584d6a99724deb13e89d90b9d5cdeecbce293e6a41e9f7ae299543f6d761bf9e7a839b6c7f3
fa10c9f5a1 Crash debug builds on PCKG_MEMPOOL_ERROR (MacroFake)
Pull request description:
Would be nice to allow fuzz targets to meaningfully cover this code
ACKs for top commit:
glozow:
utACK fa10c9f5a1
vincenzopalazzo:
ACK fa10c9f5a1
Tree-SHA512: 68efacedbf72f67cf3dc0bb9927a698492cdc1b08df91ef6af863ad8828b78058a64e52d64d244a5b2966cb9e63797b2647d1bb222677bf83b26fca6e4b1dbf0
5f213213cb tests: add tests for cross-chain wallet use prevention (Seibart Nedor)
968765973b wallet: ensure wallet files are not reused across chains (Seibart Nedor)
Pull request description:
This implements a proposal in #12805 and is a rebase of #14533.
This seems to be a working approach, but I'm not sure why the `p2p_segwit.py` functional test needed a change, so I'll look into it more.
ACKs for top commit:
achow101:
ACK 5f213213cb
dongcarl:
Code Review ACK 5f213213cb
[deleted]:
tACK 5f213213cb
Tree-SHA512: 2c934300f113e772fc31c16ef5588526300bbc36e4dcef7d77bd0760c5c8f0ec77f766b1bed5503eb0157fa26dc900ed54d2ad1b41863c1f736ce5c1f3b67bec
2052e3aa9a wallet: ignore chainStateFlushed notifications while attaching chain (Martin Zumsande)
Pull request description:
Fixes #24487
When a rescan is performed during `CWallet::AttachChain()` (e.g. when loading an old wallet) but this is interrupted by a shutdown signal, the wallet will currently stop the rescan, receive a `chainStateFlushed` signal, set the saved best block to the tip and shut down. At next startup, the rescan is not continued or repeated because of this. But some blocks have never been scanned by the wallet, which could lead to an incorrect balance.
Fix this by ignoring `chainStateFlushed` notifications until the chain is attached. Since `CWallet::chainStateFlushed` is being manually called by `AttachChain()` anyway after finishing with the rescan, it is not a problem if intermediate notifications are ignored.
Manual rescans started / aborted by the `rescanblockchain` / `abortrescan` RPCs are not affected by this.
I didn't choose alternative ways of fixing this issue that would delay the validationinterface registration or change anything else about the handling of `blockConnected` signals for the reasons mentioned in [this existing comment](https://github.com/bitcoin/bitcoin/blob/master/src/wallet/wallet.cpp#L2937-L2944).
ACKs for top commit:
achow101:
ACK 2052e3aa9a
ryanofsky:
Code review ACK 2052e3aa9a. This is a straightforward fix for the bug described in #24487 where a wallet could skip scanning blocks if is shut down in the middle of a sync and a chainStateFlushed notification was received during the sync. It would be nice to write a test for this but probably would be tricky to write.
w0xlt:
Code Review ACK 2052e3aa9a
Tree-SHA512: a6186173d72b26bd4adbf2315e11af365004a723ea5565a0f7b868584dc47c321a6572eafaeb2420bd21eed1c7ad92b47e6218c5eb72313a3c6bee58364e2247
fab34d392c Call CHECK_NONFATAL only once where needed (MarcoFalke)
Pull request description:
Now that `CHECK_NONFATAL` is the identity function starting with commit b1c5991eeb, it can be called less often in places where it was called more than once on the same value.
ACKs for top commit:
jonatack:
Review ACK fab34d392c
Tree-SHA512: ae221d7ee81f8d0be7ab21ce54d5d209e691df8a5c7f4a6f6db282453391904f87f533a2b7f85d6259827de8b85dacd9e0d9dbeecc4245a338247e0893ff3459
035fa1f07a build: Remove LIBTOOL_APP_LDFLAGS for bitcoin-chainstate (Cory Fields)
3f0595095d docs: Add libbitcoinkernel_la_SOURCES explanation (Carl Dong)
94ad45deb2 ci: Build libbitcoinkernel (Carl Dong)
26b2e7ffb3 build: Extract the libbitcoinkernel library (Carl Dong)
1df44dd20c b-cs: Define G_TRANSLATION_FUN in bitcoinkernel.cpp (Carl Dong)
83a0bb7cc9 build: Separate lib_LTLIBRARIES initialization (Carl Dong)
c1e16cb31f build: Create .la library for bitcoincrypto (Carl Dong)
8bdfe057c7 build: Create .la library for leveldb (Carl Dong)
05d1525b6d build: Create .la library for crc32c (Carl Dong)
64caf94479 build: Remove vestigial LIBLEVELDB_SSE42 (Carl Dong)
1392e8e2d8 build: Don't add unrelated libs to LIBTEST_* (Carl Dong)
Pull request description:
Part of: #24303
This PR introduces a `libbitcoinkernel` static library linking in the minimal list of files necessary to use our consensus engine as-is. `bitcoin-chainstate` introduced in #24304 now will link against `libbitcoinkernel`.
Most of the changes are related to the build system.
Please read the commit messages for more details.
ACKs for top commit:
theuni:
This may be my favorite PR ever. It's a privilege to ACK 035fa1f07a.
Tree-SHA512: b755edc3471c7c1098847e9b16ab182a6abb7582563d9da516de376a770ac7543c6fdb24238ddd4d3d2d458f905a0c0614b8667aab182aa7e6b80c1cca7090bc
fa82a1ed83 lint: Mention NONFATAL_UNREACHABLE in lint-assertions.py (MacroFake)
Pull request description:
Follow up to commit b1c5991eeb. Also remove empty newline added in that commit.
ACKs for top commit:
fanquake:
ACK fa82a1ed83
Tree-SHA512: cf398eceb135672137183bfa19ee57a82553a3dbcbce74db954c6fcd79f9606092cc0d8217610fe6cd67b7ef2d4f01d90329f0f568516d9b14aa2cd0f0715478
7ab07e0332 validation: Prune UnloadBlockIndex and callees (Carl Dong)
7d99d725cd validation: No mempool clearing in UnloadBlockIndex (Carl Dong)
572d831927 Clear {versionbits,warning}cache in ~Chainstatemanager (Carl Dong)
eca4ca4d60 style-only: Use std::clamp for check_ratio, rename (Carl Dong)
fe96a2e4bd style-only: Use for instead of when loading Chainstate (Carl Dong)
5921b863e3 init: Reset mempool and chainman via reconstruction (Carl Dong)
6e747e80e7 validation: default initialize and guard chainman members (Anthony Towns)
98f4bdae81 refactor: Convert warningcache to std::array (Carl Dong)
Pull request description:
Fixes #22964
-----
This is a small part of the work to accomplish what I described in 972c5166ee:
```
Over time, we should probably move these mutable global state variables
into ChainstateManager or CChainState so it's easier to reason about
their lifecycles.
```
`::UnloadBlockIndex` manually resets a subset of our mutable globals in addition to unloading the `ChainstateManager` and clearing the mempool. The need for this manual reset (AFAICT) arises out of the fact that many of these globals are closely related to the block index (hence `::UnloadBlockIndex`), and need to be reset with it.
I've shot this "manual reset" gun at my foot several times while doing the de-globalize chainman work.
Thankfully, now that we have a `BlockManager` class that owns the block index, these globals should be moved under that class so that they can live and die with the block index. These moves, along with making the block index non-heap-based, eliminates:
1. 3585b52139 The need to reason about when we need to manually call `::UnloadBlockIndex` (this decision can at times seem almost arbitrary)
2. f741623c25 The need to have an `::UnloadBlockIndex` or explicit `~ChainstateManager` at all
ACKs for top commit:
MarcoFalke:
ACK 7ab07e0332👘
ajtowns:
ACK 7ab07e0332
ryanofsky:
Code review ACK 7ab07e0332. This all looks good and simplifies things nicely. I left some minor suggestions below but feel free to ignore.
Tree-SHA512: a36ee3fc122ce0b4e8d1c432662d7009df06264b724b793252978a1e409dde7a7ef1f78b9ade3f8bfb5388213f10ae2d058d57a7a46ae563e9034d7d33a52b69
