foster/bitcoin-bitcoin-core
0
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-02-08 10:31:50 -05:00
Code Issues Activity

Merge #21064: refactor: use std::shared_mutex & remove Boost Thread

Browse source 
060a2a64d4 ci: remove boost thread installation (fanquake)
06e1d7d81d build: don't build or use Boost Thread (fanquake)
7097add83c refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake)
8e55981ef8 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake)

Pull request description:

  This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock).

  Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in #16684 with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in #21022, I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible.

  Two other points:

  [Cory mentioned in #21022](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179):
  > It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet.

  Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on:
  * The version of Boost.
  * The platform you're building for.
  * Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences).
  * Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`.

  A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point.

  With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc.

  Previous similar PRs were #19183 & #20922. The authors are included in the commits here.
  Also related to #21022 - pthread sanity checking.

ACKs for top commit:
  laanwj:
    Code review ACK 060a2a64d4
  vasild:
    ACK 060a2a64d4

Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
This commit is contained in:
Wladimir J. van der Laan 2021-02-12 11:21:05 +01:00
commit 9996b1806a
No known key found for this signature in database
GPG key ID: 1E4AED62986CD25D
15 changed files with 28 additions and 217 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.fuzzbuzz.yml
View file

@ -5,7 +5,7 @@ environment:
- CXXFLAGS=-fcoverage-mapping -fno-omit-frame-pointer -fprofile-instr-generate -gline-tables-only -O1
setup:
- sudo apt-get update
- sudo apt-get install -y autoconf bsdmainutils clang git libboost-all-dev libc++1 libc++abi1 libc++abi-dev libc++-dev libclang1 libclang-dev libdb5.3++ libevent-dev libllvm-ocaml-dev libomp5 libomp-dev libqt5core5a libqt5dbus5 libqt5gui5 libtool llvm llvm-dev llvm-runtime pkg-config qttools5-dev qttools5-dev-tools software-properties-common
- sudo apt-get install -y autoconf bsdmainutils clang git libboost-system-dev libboost-filesystem-dev libboost-test-dev libc++1 libc++abi1 libc++abi-dev libc++-dev libclang1 libclang-dev libdb5.3++ libevent-dev libllvm-ocaml-dev libomp5 libomp-dev libqt5core5a libqt5dbus5 libqt5gui5 libtool llvm llvm-dev llvm-runtime pkg-config qttools5-dev qttools5-dev-tools software-properties-common
- ./autogen.sh
- CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined --enable-danger-fuzz-link-all
- make

187
build-aux/m4/ax_boost_thread.m4
View file

@ -1,187 +0,0 @@
# ===========================================================================
# https://www.gnu.org/software/autoconf-archive/ax_boost_thread.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_BOOST_THREAD
#
# DESCRIPTION
#
# Test for Thread library from the Boost C++ libraries. The macro requires
# a preceding call to AX_BOOST_BASE. Further documentation is available at
# <http://randspringer.de/boost/index.html>.
#
# This macro calls:
#
# AC_SUBST(BOOST_THREAD_LIB)
#
# And sets:
#
# HAVE_BOOST_THREAD
#
# LICENSE
#
# Copyright (c) 2009 Thomas Porschberg <thomas@randspringer.de>
# Copyright (c) 2009 Michael Tindal
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 33
AC_DEFUN([AX_BOOST_THREAD],
[
AC_ARG_WITH([boost-thread],
AS_HELP_STRING([--with-boost-thread@<:@=special-lib@:>@],
[use the Thread library from boost -
it is possible to specify a certain library for the linker
e.g. --with-boost-thread=boost_thread-gcc-mt ]),
[
if test "$withval" = "yes"; then
want_boost="yes"
ax_boost_user_thread_lib=""
else
want_boost="yes"
ax_boost_user_thread_lib="$withval"
fi
],
[want_boost="yes"]
)
if test "x$want_boost" = "xyes"; then
AC_REQUIRE([AC_PROG_CC])
AC_REQUIRE([AC_CANONICAL_BUILD])
CPPFLAGS_SAVED="$CPPFLAGS"
CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS"
export CPPFLAGS
LDFLAGS_SAVED="$LDFLAGS"
LDFLAGS="$LDFLAGS $BOOST_LDFLAGS"
export LDFLAGS
AC_CACHE_CHECK(whether the Boost::Thread library is available,
ax_cv_boost_thread,
[AC_LANG_PUSH([C++])
CXXFLAGS_SAVE=$CXXFLAGS
case "x$host_os" in
xsolaris )
CXXFLAGS="-pthreads $CXXFLAGS"
break;
;;
xmingw32 )
CXXFLAGS="-mthreads $CXXFLAGS"
break;
;;
*android* )
break;
;;
* )
CXXFLAGS="-pthread $CXXFLAGS"
break;
;;
esac
AC_COMPILE_IFELSE([
AC_LANG_PROGRAM(
[[@%:@include <boost/thread/thread.hpp>]],
[[boost::thread_group thrds;
return 0;]])],
ax_cv_boost_thread=yes, ax_cv_boost_thread=no)
CXXFLAGS=$CXXFLAGS_SAVE
AC_LANG_POP([C++])
])
if test "x$ax_cv_boost_thread" = "xyes"; then
case "x$host_os" in
xsolaris )
BOOST_CPPFLAGS="-pthreads $BOOST_CPPFLAGS"
break;
;;
xmingw32 )
BOOST_CPPFLAGS="-mthreads $BOOST_CPPFLAGS"
break;
;;
*android* )
break;
;;
* )
BOOST_CPPFLAGS="-pthread $BOOST_CPPFLAGS"
break;
;;
esac
AC_SUBST(BOOST_CPPFLAGS)
AC_DEFINE(HAVE_BOOST_THREAD,,
[define if the Boost::Thread library is available])
BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'`
LDFLAGS_SAVE=$LDFLAGS
case "x$host_os" in
*bsd* )
LDFLAGS="-pthread $LDFLAGS"
break;
;;
esac
if test "x$ax_boost_user_thread_lib" = "x"; then
for libextension in `ls -r $BOOSTLIBDIR/libboost_thread* 2>/dev/null | sed 's,.*/lib,,' | sed 's,\..*,,'`; do
ax_lib=${libextension}
AC_CHECK_LIB($ax_lib, exit,
[link_thread="yes"; break],
[link_thread="no"])
done
if test "x$link_thread" != "xyes"; then
for libextension in `ls -r $BOOSTLIBDIR/boost_thread* 2>/dev/null | sed 's,.*/,,' | sed 's,\..*,,'`; do
ax_lib=${libextension}
AC_CHECK_LIB($ax_lib, exit,
[link_thread="yes"; break],
[link_thread="no"])
done
fi
else
for ax_lib in $ax_boost_user_thread_lib boost_thread-$ax_boost_user_thread_lib; do
AC_CHECK_LIB($ax_lib, exit,
[link_thread="yes"; break],
[link_thread="no"])
done
fi
if test "x$ax_lib" = "x"; then
AC_MSG_ERROR(Could not find a version of the Boost::Thread library!)
fi
if test "x$link_thread" = "xno"; then
AC_MSG_ERROR(Could not link against $ax_lib !)
else
BOOST_THREAD_LIB="-l$ax_lib"
case "x$host_os" in
*bsd* )
BOOST_LDFLAGS="-pthread $BOOST_LDFLAGS"
break;
;;
xsolaris )
BOOST_THREAD_LIB="$BOOST_THREAD_LIB -lpthread"
break;
;;
xmingw32 )
break;
;;
*android* )
break;
;;
* )
BOOST_THREAD_LIB="$BOOST_THREAD_LIB -lpthread"
break;
;;
esac
AC_SUBST(BOOST_THREAD_LIB)
fi
fi
CPPFLAGS="$CPPFLAGS_SAVED"
LDFLAGS="$LDFLAGS_SAVED"
fi
])

3
build_msvc/bitcoin_config.h
View file

@ -56,9 +56,6 @@
/* define if the Boost::System library is available */
#define HAVE_BOOST_SYSTEM /**/
/* define if the Boost::Thread library is available */
#define HAVE_BOOST_THREAD /**/
/* define if the Boost::Unit_Test_Framework library is available */
#define HAVE_BOOST_UNIT_TEST_FRAMEWORK /**/

1
build_msvc/vcpkg.json
View file

@ -8,7 +8,6 @@
"boost-process",
"boost-signals2",
"boost-test",
"boost-thread",
"sqlite3",
"double-conversion",
{

2
ci/test/00_setup_env_native_asan.sh
View file

@ -7,7 +7,7 @@
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_asan
export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libqrencode-dev libsqlite3-dev"
export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libqrencode-dev libsqlite3-dev"
export DOCKER_NAME_TAG=ubuntu:20.04
export NO_DEPENDS=1
export GOAL="install"

2
ci/test/00_setup_env_native_fuzz.sh
View file

@ -8,7 +8,7 @@ export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:20.04"
export CONTAINER_NAME=ci_native_fuzz
export PACKAGES="clang llvm python3 libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev"
export PACKAGES="clang llvm python3 libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev"
export NO_DEPENDS=1
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false

2
ci/test/00_setup_env_native_fuzz_with_valgrind.sh
View file

@ -8,7 +8,7 @@ export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:20.04"
export CONTAINER_NAME=ci_native_fuzz_valgrind
export PACKAGES="clang llvm python3 libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev valgrind"
export PACKAGES="clang llvm python3 libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev valgrind"
export NO_DEPENDS=1
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false

2
ci/test/00_setup_env_native_valgrind.sh
View file

@ -7,7 +7,7 @@
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_valgrind
export PACKAGES="valgrind clang llvm python3-zmq libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libsqlite3-dev"
export PACKAGES="valgrind clang llvm python3-zmq libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libsqlite3-dev"
export USE_VALGRIND=1
export NO_DEPENDS=1
export TEST_RUNNER_EXTRA="--exclude rpc_bind" # Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547

5
configure.ac
View file

@ -657,7 +657,7 @@ case $host in
AC_MSG_ERROR("windres not found")
fi
CPPFLAGS="$CPPFLAGS -D_MT -DWIN32 -D_WINDOWS -DBOOST_THREAD_USE_LIB -D_WIN32_WINNT=0x0601 -D_WIN32_IE=0x0501 -DWIN32_LEAN_AND_MEAN"
CPPFLAGS="$CPPFLAGS -D_MT -DWIN32 -D_WINDOWS -D_WIN32_WINNT=0x0601 -D_WIN32_IE=0x0501 -DWIN32_LEAN_AND_MEAN"
dnl libtool insists upon adding -nostdlib and a list of objects/libs to link against.
dnl That breaks our ability to build dll's with static libgcc/libstdc++/libssp. Override
@ -1387,7 +1387,6 @@ if test x$want_boost = xno; then
fi
AX_BOOST_SYSTEM
AX_BOOST_FILESYSTEM
AX_BOOST_THREAD
dnl Opt-in to boost-process
AS_IF([ test x$with_boost_process != x ], [ AX_BOOST_PROCESS ], [ ax_cv_boost_process=no ] )
@ -1401,7 +1400,7 @@ dnl counter implementations. In 1.63 and later the std::atomic approach is defau
m4_pattern_allow(DBOOST_AC_USE_STD_ATOMIC) dnl otherwise it's treated like a macro
BOOST_CPPFLAGS="-DBOOST_SP_USE_STD_ATOMIC -DBOOST_AC_USE_STD_ATOMIC $BOOST_CPPFLAGS"
BOOST_LIBS="$BOOST_LDFLAGS $BOOST_SYSTEM_LIB $BOOST_FILESYSTEM_LIB $BOOST_THREAD_LIB"
BOOST_LIBS="$BOOST_LDFLAGS $BOOST_SYSTEM_LIB $BOOST_FILESYSTEM_LIB"
fi
dnl Check for reduced exports

2
depends/packages/boost.mk
View file

@ -22,7 +22,7 @@ $(package)_toolset_$(host_os)=clang
else
$(package)_toolset_$(host_os)=gcc
endif
$(package)_config_libraries=filesystem,system,thread,test
$(package)_config_libraries=filesystem,system,test
$(package)_cxxflags=-std=c++17 -fvisibility=hidden
$(package)_cxxflags_linux=-fPIC
$(package)_cxxflags_android=-fPIC

2
doc/build-unix.md
View file

@ -82,7 +82,7 @@ Build requirements:
Now, you can either build from self-compiled [depends](/depends/README.md) or install the required dependencies:
sudo apt-get install libevent-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev
sudo apt-get install libevent-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev
BerkeleyDB is required for the wallet.

2
src/Makefile.am
View file

@ -687,7 +687,7 @@ endif
bitcoin_util_SOURCES = bitcoin-util.cpp
bitcoin_util_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES)
bitcoin_util_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)
bitcoin_util_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS)
bitcoin_util_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) $(PTHREAD_FLAGS)
if TARGET_WINDOWS
bitcoin_util_SOURCES += bitcoin-util-res.rc

12
src/script/sigcache.cpp
View file

@ -12,8 +12,10 @@
#include <cuckoocache.h>
#include <boost/thread/lock_types.hpp>
#include <boost/thread/shared_mutex.hpp>
#include <algorithm>
#include <mutex>
#include <shared_mutex>
#include <vector>
namespace {
/**
@ -29,7 +31,7 @@ private:
CSHA256 m_salted_hasher_schnorr;
typedef CuckooCache::cache<uint256, SignatureCacheHasher> map_type;
map_type setValid;
boost::shared_mutex cs_sigcache;
std::shared_mutex cs_sigcache;
public:
CSignatureCache()
@ -64,13 +66,13 @@ public:
bool
Get(const uint256& entry, const bool erase)
{
boost::shared_lock<boost::shared_mutex> lock(cs_sigcache);
std::shared_lock<std::shared_mutex> lock(cs_sigcache);
return setValid.contains(entry, erase);
}
void Set(const uint256& entry)
{
boost::unique_lock<boost::shared_mutex> lock(cs_sigcache);
std::unique_lock<std::shared_mutex> lock(cs_sigcache);
setValid.insert(entry);
}
uint32_t setup_bytes(size_t n)

19
src/test/cuckoocache_tests.cpp
View file

@ -1,15 +1,18 @@
// Copyright (c) 2012-2020 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <boost/test/unit_test.hpp>
#include <boost/thread/lock_types.hpp>
#include <boost/thread/shared_mutex.hpp>
#include <cuckoocache.h>
#include <deque>
#include <random.h>
#include <script/sigcache.h>
#include <test/util/setup_common.h>
#include <boost/test/unit_test.hpp>
#include <deque>
#include <mutex>
#include <shared_mutex>
#include <thread>
#include <vector>
/** Test Suite for CuckooCache
*
@ -201,11 +204,11 @@ static void test_cache_erase_parallel(size_t megabytes)
* "future proofed".
*/
std::vector<uint256> hashes_insert_copy = hashes;
boost::shared_mutex mtx;
std::shared_mutex mtx;
{
/** Grab lock to make sure we release inserts */
boost::unique_lock<boost::shared_mutex> l(mtx);
std::unique_lock<std::shared_mutex> l(mtx);
/** Insert the first half */
for (uint32_t i = 0; i < (n_insert / 2); ++i)
set.insert(hashes_insert_copy[i]);
@ -219,7 +222,7 @@ static void test_cache_erase_parallel(size_t megabytes)
/** Each thread is emplaced with x copy-by-value
*/
threads.emplace_back([&, x] {
boost::shared_lock<boost::shared_mutex> l(mtx);
std::shared_lock<std::shared_mutex> l(mtx);
size_t ntodo = (n_insert/4)/3;
size_t start = ntodo*x;
size_t end = ntodo*(x+1);
@ -234,7 +237,7 @@ static void test_cache_erase_parallel(size_t megabytes)
for (std::thread& t : threads)
t.join();
/** Grab lock to make sure we observe erases */
boost::unique_lock<boost::shared_mutex> l(mtx);
std::unique_lock<std::shared_mutex> l(mtx);
/** Insert the second half */
for (uint32_t i = (n_insert / 2); i < n_insert; ++i)
set.insert(hashes_insert_copy[i]);

2
test/lint/lint-includes.sh
View file

@ -65,8 +65,6 @@ EXPECTED_BOOST_INCLUDES=(
boost/signals2/optional_last_value.hpp
boost/signals2/signal.hpp
boost/test/unit_test.hpp
boost/thread/lock_types.hpp
boost/thread/shared_mutex.hpp
)
for BOOST_INCLUDE in $(git grep '^#include <boost/' -- "*.cpp" "*.h" | cut -f2 -d: | cut -f2 -d'<' | cut -f1 -d'>' | sort -u); do