Merge #18581: ci: Print ccache stats, add pip cache, and cleanups

73f8303629 ci: Add pip cache (Hennadii Stepanov) 6d480667b2 ci: Print ccache statistics summary (Hennadii Stepanov) a6eea715ff ci: Drop unused WINEDEBUG variable (Hennadii Stepanov) Pull request description: The Travis [pip cache](https://docs.travis-ci.com/user/caching/#pip-cache) is free and saves a dozen of seconds :) Here are some excerpts from the Travis logs with `ccache` statistics (I found useful): 2) [Arm64](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507749) ``` ccache version 3.6 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Fri Apr 10 18:21:06 2020 stats zeroed Fri Apr 10 18:03:04 2020 cache hit (direct) 156 cache hit (preprocessed) 198 cache miss 143 cache hit rate 71.23 % called for link 8 cleanups performed 9 files in cache 1255 cache size 80.6 MB max cache size 100.0 MB ``` 3) [s390x](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507750) ``` ccache version 3.4.1 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats zero time Fri Apr 10 17:15:35 2020 cache hit (direct) 115 cache hit (preprocessed) 163 cache miss 94 cache hit rate 74.73 % called for link 6 cleanups performed 0 files in cache 804 cache size 304.8 MB max cache size 5.0 GB ``` 4) [Win64](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507751) ``` ccache version 3.4.1 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats zero time Fri Apr 10 17:38:07 2020 cache hit (direct) 147 cache hit (preprocessed) 199 cache miss 138 cache hit rate 71.49 % called for link 7 cleanups performed 7 files in cache 1242 cache size 87.0 MB max cache size 100.0 MB ``` 5) [CentOS 7](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507752) ``` ccache version 3.7.7 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Fri Apr 10 17:45:59 2020 stats zeroed Fri Apr 10 17:34:27 2020 cache hit (direct) 152 cache hit (preprocessed) 200 cache miss 144 cache hit rate 70.97 % called for link 8 cleanups performed 4 files in cache 1352 cache size 86.1 MB max cache size 100.0 MB ``` 6) [bionic](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507753) ``` ccache version 3.4.1 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats zero time Fri Apr 10 17:21:10 2020 cache hit (direct) 136 cache hit (preprocessed) 35 cache miss 330 cache hit rate 34.13 % called for link 8 cleanups performed 18 files in cache 1302 cache size 83.8 MB max cache size 100.0 MB ``` 7) [xenial](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507754) ``` ccache version 3.2.4 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf cache hit (direct) 161 cache hit (preprocessed) 174 cache miss 95 called for link 7 files in cache 3411 cache size 1.1 GB max cache size 5.0 GB ``` 10) [focal](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507757) ``` ccache version 3.7.7 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Fri Apr 10 17:35:57 2020 stats zeroed Fri Apr 10 17:21:32 2020 cache hit (direct) 78 cache hit (preprocessed) 125 cache miss 107 cache hit rate 65.48 % called for link 120 cleanups performed 0 files in cache 6218 cache size 1.8 GB max cache size 5.0 GB ``` 14) [macOS 10.12](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507761) ``` ccache version 3.4.1 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats zero time Fri Apr 10 17:47:19 2020 cache hit (direct) 28 cache hit (preprocessed) 1 cache miss 469 cache hit rate 5.82 % called for link 8 cleanups performed 17 files in cache 1946 cache size 82.6 MB max cache size 100.0 MB ``` 15) [macOS 10.14 native](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507762) ``` ccache version 3.7.8 cache directory /Users/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /Users/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /usr/local/Cellar/ccache/3.7.8/etc/ccache.conf stats updated Fri Apr 10 17:38:44 2020 stats zeroed Fri Apr 10 17:22:36 2020 cache hit (direct) 213 cache hit (preprocessed) 293 cache miss 144 cache hit rate 77.85 % called for link 11 called for preprocessing 56 compile failed 30 preprocessor error 56 bad compiler arguments 14 autoconf compile/link 77 no input file 66 cleanups performed 0 files in cache 1564 cache size 284.6 MB max cache size 5.0 GB ``` Top commit has no ACKs. Tree-SHA512: f9f2c6d72680bafc7a982efc04dc37ab9bba77ffd2a4cbbc56bb0e86c5c3501d57aa2f0ebeac0113e9d8daac1ee4eac1eddf74c1172c267b37ee2d9324366c8a
2025-03-05 14:06:27 -05:00 · 2020-04-13 11:44:59 -04:00 · 2020-04-13 11:44:59 -04:00 · 6ef45bc968
commit 6ef45bc968
parent 130b64415f 73f8303629
4 changed files with 8 additions and 4 deletions
--- a/.travis.yml
+++ b/.travis.yml
@ -71,7 +71,7 @@ jobs:
    - stage: lint
      name: 'lint'
      env:
-      cache: false
+      cache: pip
      language: python
      python: '3.5' # Oldest supported version according to doc/dependencies.md
      install:
--- a/ci/test/00_setup_env.sh
+++ b/ci/test/00_setup_env.sh
@ -24,7 +24,7 @@ echo "Fallback to default values in env (if not yet set)"
 export MAKEJOBS=${MAKEJOBS:--j4}
 # A folder for the ci system to put temporary files (ccache, datadirs for tests, ...)
 # This folder only exists on the ci host.
-export BASE_SCRATCH_DIR=${BASE_SCRATCH_DIR:-$BASE_ROOT_DIR/ci/scratch/}
+export BASE_SCRATCH_DIR=${BASE_SCRATCH_DIR:-$BASE_ROOT_DIR/ci/scratch}
 # What host to compile for. See also ./depends/README.md
 # Tests that need cross-compilation export the appropriate HOST.
 # Tests that run natively guess the host
@ -53,7 +53,6 @@ export DEPENDS_DIR=${DEPENDS_DIR:-$BASE_ROOT_DIR/depends}
 export BASE_OUTDIR=${BASE_OUTDIR:-$BASE_SCRATCH_DIR/out/$HOST}
 export PREVIOUS_RELEASES_DIR=${PREVIOUS_RELEASES_DIR:-$BASE_ROOT_DIR/releases/$HOST}
 export SDK_URL=${SDK_URL:-https://bitcoincore.org/depends-sources/sdks}
-export WINEDEBUG=${WINEDEBUG:-fixme-all}
 export DOCKER_PACKAGES=${DOCKER_PACKAGES:-build-essential libtool autotools-dev automake pkg-config bsdmainutils curl ca-certificates ccache python3 rsync git procps}
 export GOAL=${GOAL:-install}
 export DIR_QA_ASSETS=${DIR_QA_ASSETS:-${BASE_SCRATCH_DIR}/qa-assets}
--- a/ci/test/04_install.sh
+++ b/ci/test/04_install.sh
@ -26,7 +26,7 @@ export ASAN_OPTIONS="detect_stack_use_after_return=1:check_initialization_order=
 export LSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/lsan"
 export TSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/tsan:log_path=${BASE_SCRATCH_DIR}/sanitizer-output/tsan"
 export UBSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1"
-env | grep -E '^(BITCOIN_CONFIG|BASE_|QEMU_|CCACHE_|WINEDEBUG|LC_ALL|BOOST_TEST_RANDOM|CONFIG_SHELL|(ASAN|LSAN|TSAN|UBSAN)_OPTIONS|TEST_PREVIOUS_RELEASES|PREVIOUS_RELEASES_DIR)' | tee /tmp/env
+env | grep -E '^(BITCOIN_CONFIG|BASE_|QEMU_|CCACHE_|LC_ALL|BOOST_TEST_RANDOM|CONFIG_SHELL|(ASAN|LSAN|TSAN|UBSAN)_OPTIONS|TEST_PREVIOUS_RELEASES|PREVIOUS_RELEASES_DIR)' | tee /tmp/env
 if [[ $HOST = *-mingw32 ]]; then
  DOCKER_ADMIN="--cap-add SYS_ADMIN"
 elif [[ $BITCOIN_CONFIG = *--with-sanitizers=*address* ]]; then # If ran with (ASan + LSan), Docker needs access to ptrace (https://github.com/google/sanitizers/issues/764)
--- a/ci/test/06_script_a.sh
+++ b/ci/test/06_script_a.sh
@ -7,6 +7,7 @@
 export LC_ALL=C.UTF-8

 BITCOIN_CONFIG_ALL="--disable-dependency-tracking --prefix=$DEPENDS_DIR/$HOST --bindir=$BASE_OUTDIR/bin --libdir=$BASE_OUTDIR/lib"
+DOCKER_EXEC "command -v ccache > /dev/null && ccache --zero-stats"
 if [ -z "$NO_DEPENDS" ]; then
  DOCKER_EXEC ccache --max-size=$CCACHE_SIZE
 fi
@ -44,3 +45,7 @@ trap 'DOCKER_EXEC "cat ${BASE_SCRATCH_DIR}/sanitizer-output/* 2> /dev/null"' ERR
 BEGIN_FOLD build
 DOCKER_EXEC make $MAKEJOBS $GOAL || ( echo "Build failure. Verbose build follows." && DOCKER_EXEC make $GOAL V=1 ; false )
 END_FOLD
+
+BEGIN_FOLD ccache_stats
+DOCKER_EXEC "command -v ccache > /dev/null && ccache --version | head -n 1 && ccache --show-stats"
+END_FOLD