Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use official boost release in examples / readme #304

Merged
merged 1 commit into from
Dec 5, 2021
Merged

Conversation

TheLartians
Copy link
Member

As discussed in #300, we can now use the official Boost library, as it appears that proper CMake support has been added to the library!

@TheLartians TheLartians enabled auto-merge (squash) November 28, 2021 00:52
@thebashpotato
Copy link

thebashpotato commented Nov 28, 2021

UPDATE

Turns out its not hanging just takes quite a while to download

I have tried using the update-boost branch from this PR, to see if It was working. How ever; it just hangs when attempting to add the package. Any one else have this problem?
Below is the CPMAddPackage function I used also from the PR

CPMAddPackage(
  NAME Boost
  VERSION 1.77.0
  GITHUB_REPOSITORY "boostorg/boost"
  GIT_TAG "boost-1.77.0"
)

my cmake version is 3.22.0 installed through pip on Ubuntu 20.04

@TheLartians
Copy link
Member Author

Turns out its not hanging just takes quite a while to download

Yeah, it seems the Boost repo is organised in a bunch of submodules that total to a total repo size of 2gb, which explains why the cloning takes a very long time. Afaik, the old version used the release zip packages, avoiding such enormous git clones. While being much more efficient, in general users would except to be able to get the most recent official package, so I think I prefer this solution for now.

@TheLartians TheLartians changed the title Use official boost release Use official boost release in examples / readme Nov 28, 2021
@TheLartians TheLartians merged commit fed5f8e into master Dec 5, 2021
@TheLartians TheLartians deleted the update-boost branch December 5, 2021 15:08
@DNKpp
Copy link
Contributor

DNKpp commented Dec 19, 2021

It took me some time and code digging to figure out, how to explicitly include/exclude libraries from build process to minimize the build time. Usually find_package(boost) can be used with find_package(boost COMPONENTS multi_array json). I wasn't able to figure out how to forward this via CPM, thus I searched for a workaround.
Adding set(BOOST_INCLUDE_LIBRARIES multi_array json) before CPMAddPackage will explicitly add only the listed libraries (and their dependencies) while BOOST_EXCLUDE_LIBRARIES explicitly excludes the listed. Perhaps it is worth adding those into the example.

@TheLartians
Copy link
Member Author

Hey, thanks for figuring that out! Usually I would have expected for CMake to automatically detect which boost libraries to build based on the provided libraries (and using the EXCLUDE_FROM_ALL flag), however it seems that boost doesn't support that. Is there any official documentation available on the BOOST_EXCLUDE_LIBRARIES variable?

@DNKpp
Copy link
Contributor

DNKpp commented Dec 19, 2021

Well, I haven't found any. I found it by simply following the boost cmake include paths, until I reached this file: https://github.com/boostorg/cmake/blob/bd981bfaff20ae4275ecbeba711c90e7fc9d0316/include/BoostRoot.cmake

As there is also a BoostFetch module next to it, I hoped that this variable also enables the modular fetching, but unfortunatly that's currently not the case. I wasn't even able to finde a caller for the boost_fetch function, thus it might be a preparation for further releases.

@iboB
Copy link
Member

iboB commented Jan 14, 2022

The boost cmake support is still a work in progress. It's not complete yet, though with 1.78 there has been a lot of progress, compared to 1.77 at least. With 1.78 EXCLUDE_FROM_ALL and BOOST_INCLUDE_LIBRARIES work as expected. Perhaps we should update the example?

Though with the submodule structure of the project and the way FetchContent works, the download is still huge (downloading the entire history of all submodules).

@ClausKlein
Copy link
Contributor

I experimented with a mixt solution. At the official boost tar file the main CMakeLists.txt is missing.
So I use boost-make again to bootstrap the boost distribution without git clone.
see #359

@alexchandel
Copy link

So what is currently the recommended way to use Boost with CPM? The example uses GitHub for Boost 1.81.0, but does CPM support Boost's release zipfiles too?

What about for older versions of Boost? Does CPM support e.g. Boost 1.60.0?

@ClausKlein
Copy link
Contributor

ClausKlein commented Feb 17, 2023

So what is currently the recommended way to use Boost with CPM? The example uses GitHub for Boost 1.81.0, but does CPM support Boost's release zipfiles too?

Yes, and it is much faster too use!

The boost git repo use git submodules!

The fasted way I know is this #359

What about for older versions of Boost? Does CPM support e.g. Boost 1.60.0?

No, IMHO: you should always use the current stable version!

@alexchandel
Copy link

alexchandel commented Feb 17, 2023

Yes, and it is much faster too use!
The boost git repo use git submodules!

So how do you use it with zipflies? Just with URL and URL_HASH? The docs seem to warn that Boost zipfile releases don't contain the right CMakeLists.txt, is that true?

The fasted way I know is this #359

How does one use that? The PR comment does not specify.

No, IMHO: you should always use the current stable version!

Sadly many people cannot always distribute bleeding edge libs. This project has to work with older libs, and the best I can do is build them with modern CMake.

@ClausKlein
Copy link
Contributor

The fasted way I know is this #359

How does one use that? The PR comment does not specify.
see https://github.com/ClausKlein/boost-cmake/blob/6986986011d7012de6aba44613a825a5f37266b8/CMakeLists.txt

@ClausKlein
Copy link
Contributor

ClausKlein commented Feb 17, 2023

The current boost example takes about 11 min to clone the repo and 2 min to build all dependent libs?

The build with the boost source tar file takes about 2 min to extract the tar archive and about 10 sec to build the example!

@ClausKlein
Copy link
Contributor

ClausKlein commented Feb 17, 2023

My version of named CPM cache entries

If you use this to build the example, it takes about 1 min!

if(EXISTS "$ENV{CPM_SOURCE_CACHE}/boost_1_80_0")
  set(FETCHCONTENT_SOURCE_DIR_BOOST "$ENV{CPM_SOURCE_CACHE}/boost_1_80_0" CACHE PATH "Boost source DIR")
  set(CPM_Boost_SOURCE ${FETCHCONTENT_SOURCE_DIR_BOOST} CACHE PATH "Manual override")
endif()

You must only install the official boost source tar file at this location in CPM.cmake cache.

@ClausKlein
Copy link
Contributor

bash-3.2$ time doit.sh
+ CMAKE_PRESET='-G Ninja -D CMAKE_CXX_COMPILER_LAUNCHER=/usr/local/bin/ccache -D CMAKE_BUILD_TYPE=Debug -D CMAKE_PREFIX_PATH=/Users/clausklein/cmake/TheLartians/CPM.cmake/examples/stagedir -D BUILD_TESTING=1 -D CMAKE_CXX_STANDARD=20 -DBUILD_SHARED_LIBS=YES'
+ export CPM_SOURCE_CACHE=/Users/clausklein/.cache/CPM
+ CPM_SOURCE_CACHE=/Users/clausklein/.cache/CPM
+ export CPM_USE_LOCAL_PACKAGES=NO
+ CPM_USE_LOCAL_PACKAGES=NO
+ export CC
+ export CXX
+ export GCOV
++ uname -m
+ MACHINE=x86_64
++ basename /Users/clausklein/cmake/TheLartians/CPM.cmake/examples/boost
+ PROJECT_NAME=boost
+ BUILD_DIR=./cmake-build-boost-x86_64-Debug
+ '[' -d ./cmake-build-boost-x86_64-Debug -o -d CMakeFiles ']'
+ rm -rf ./cmake-build-boost-x86_64-Debug CMakeFiles .build build stage
+ '[' -d cmake ']'
+ '[' -f CMakeLists.txt ']'
+ find . -name '*.cmake' -o -name CMakeLists.txt -o -name '*.in'
+ cmake -S . -B ./cmake-build-boost-x86_64-Debug/install -G Ninja -D CMAKE_CXX_COMPILER_LAUNCHER=/usr/local/bin/ccache -D CMAKE_BUILD_TYPE=Debug -D CMAKE_PREFIX_PATH=/Users/clausklein/cmake/TheLartians/CPM.cmake/examples/stagedir -D BUILD_TESTING=1 -D CMAKE_CXX_STANDARD=20 -DBUILD_SHARED_LIBS=YES -Wdev -Wdeprecated
-- The C compiler identification is AppleClang 14.0.0.14000029
-- The CXX compiler identification is AppleClang 14.0.0.14000029
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/local/bin/git (found version "2.39.0") 
-- Fetching Boost
-- Fetching Boost - done
-- Boost found: 1.79.0 /Users/clausklein/.cache/CPM/boost_1_79_0
-- Looking for __linux__
-- Looking for __linux__ - not found
-- Looking for _WIN32
-- Looking for _WIN32 - not found
-- Looking for __APPLE__
-- Looking for __APPLE__ - found
-- Looking for __ANDROID__
-- Looking for __ANDROID__ - not found
-- Looking for __FreeBSD__
-- Looking for __FreeBSD__ - not found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- The ASM compiler identification is Clang with GNU-like command-line
-- Found assembler: /Applications/Xcode.app/Contents/Developer/usr/bin/gcc
-- Found BZip2: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk/usr/lib/libbz2.tbd (found version "1.0.8") 
-- Looking for BZ2_bzCompressInit
-- Looking for BZ2_bzCompressInit - found
-- Found ZLIB: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk/usr/lib/libz.tbd (found version "1.2.11") 
-- The following ICU libraries were not found:
--   uc (required)
--   dt (required)
--   i18n (required)
-- Failed to find all ICU components (missing: ICU_LIBRARY _ICU_REQUIRED_LIBS_FOUND) (found version "70.1")
-- Found iconv library: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk/usr/lib/libiconv.tbd
-- HAVE_NATIVE_SYSLOG
-- BOOST_LOG_WITHOUT_EVENT_LOG
-- BOOST_LOG_WITHOUT_DEBUG_OUTPUT
-- Found MPI_C: /usr/local/Cellar/open-mpi/4.1.4_2/lib/libmpi.dylib (found version "3.1") 
-- Found MPI_CXX: /usr/local/Cellar/open-mpi/4.1.4_2/lib/libmpi.dylib (found version "3.1") 
-- Found MPI: TRUE (found version "3.1")  
-- CPM: boost-cmake: Adding package PackageProject.cmake@1.9.0 (v1.9.0 at /Users/clausklein/.cache/CPM/packageproject.cmake/e3848b31ae54d78683c7e47eb420383f05024bdf)
-- CPM: Adding package boost-cmake@1.79.3 (v1.79.3 at /Users/clausklein/.cache/CPM/boost-cmake/803a32763e322499da1fe5027eaa15bc994edfd0)
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    BUILD_TESTING


-- Build files have been written to: /Users/clausklein/cmake/TheLartians/CPM.cmake/examples/boost/cmake-build-boost-x86_64-Debug/install
+ cmake --build ./cmake-build-boost-x86_64-Debug/install
[2/2] Linking CXX executable CPMExampleAsio
+ cmake --build ./cmake-build-boost-x86_64-Debug/install --target test
[0/1] Running tests...
Test project /Users/clausklein/cmake/TheLartians/CPM.cmake/examples/boost/cmake-build-boost-x86_64-Debug/install
    Start 1: CPMExampleAsio
1/1 Test #1: CPMExampleAsio ...................   Passed    1.90 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   1.90 sec
+ cmake --install ./cmake-build-boost-x86_64-Debug/install --prefix /Users/clausklein/cmake/TheLartians/CPM.cmake/examples/stagedir
-- Install configuration: "Debug"
+ exit

real	0m9.105s
user	0m3.724s
sys	0m2.950s
bash-3.2$ 

@DNKpp
Copy link
Contributor

DNKpp commented Feb 18, 2023

Just for the reference, it seems like as of 1.81 the cmake support has been highly extended. At the release page https://github.com/boostorg/boost/releases the packages contain a CMakeLists.txt file and at https://github.com/boostorg/cmake/tree/69f16e2819c4686450bdac807e3eb58346fb2ac2 there is a general guide how to use boost with cmake. Perhaps its worth having a "legacy" and a "up2date" example in the official docs here :)

@ClausKlein
Copy link
Contributor

@DNKpp read please this notes about building-boost-with-cmake

and #448

@alexchandel
Copy link

alexchandel commented Feb 19, 2023

Neither the 1.81 repo nor the 1.81 release seems to build properly.

set(EXTERN_DIR ${CMAKE_CURRENT_LIST_DIR}/opt)
include(CPM)

CPMAddPackage(
    NAME                Boost
    VERSION             1.81.0
    URL                 "https://github.com/boostorg/boost/releases/download/boost-1.81.0/boost-1.81.0.7z"
    URL_HASH            SHA256=86a1961606c433d15c5af55e1270d1d6d9afa770a243981e040f902c4c6ef154
    DOWNLOAD_DIR        ${EXTERN_DIR}
    SOURCE_DIR          ${EXTERN_DIR}/boost
    EXCLUDE_FROM_ALL    1
)

find_package(Boost 1.81.0 REQUIRED)

add_executable(TestBoost main.cpp)
target_link_libraries(TestBoost
    Boost::regex
)
// main.cpp
#include <boost/regex.hpp>
int main(int argc, char const *argv[]) { return 0; }

They both error like,

VERBOSE=1 cmake --build .
/usr/local/Cellar/cmake/3.25.2/bin/cmake -P /Users/dev/Desktop/test-boost/build/CMakeFiles/VerifyGlobs.cmake
/usr/local/Cellar/cmake/3.25.2/bin/cmake -S/Users/dev/Desktop/test-boost -B/Users/dev/Desktop/test-boost/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/Cellar/cmake/3.25.2/bin/cmake -E cmake_progress_start /Users/dev/Desktop/test-boost/build/CMakeFiles /Users/dev/Desktop/test-boost/build//CMakeFiles/progress.marks
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/Makefile2 all
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/TestBoost.dir/build.make CMakeFiles/TestBoost.dir/depend
cd /Users/dev/Desktop/test-boost/build && /usr/local/Cellar/cmake/3.25.2/bin/cmake -E cmake_depends "Unix Makefiles" /Users/dev/Desktop/test-boost /Users/dev/Desktop/test-boost /Users/dev/Desktop/test-boost/build /Users/dev/Desktop/test-boost/build /Users/dev/Desktop/test-boost/build/CMakeFiles/TestBoost.dir/DependInfo.cmake --color=
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/TestBoost.dir/build.make CMakeFiles/TestBoost.dir/build
Building CXX object CMakeFiles/TestBoost.dir/main.cpp.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++  -I/Users/dev/Desktop/test-boost/opt/boost/libs/regex/include -I/Users/dev/Desktop/test-boost/opt/boost/libs/config/include -I/Users/dev/Desktop/test-boost/opt/boost/libs/throw_exception/include -I/Users/dev/Desktop/test-boost/opt/boost/libs/assert/include -I/Users/dev/Desktop/test-boost/opt/boost/libs/predef/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -mmacosx-version-min=12.6 -MD -MT CMakeFiles/TestBoost.dir/main.cpp.o -MF CMakeFiles/TestBoost.dir/main.cpp.o.d -o CMakeFiles/TestBoost.dir/main.cpp.o -c /Users/dev/Desktop/test-boost/main.cpp
In file included from /Users/dev/Desktop/test-boost/main.cpp:2:
In file included from /Users/dev/Desktop/test-boost/opt/boost/libs/regex/include/boost/regex.hpp:32:
In file included from /Users/dev/Desktop/test-boost/opt/boost/libs/regex/include/boost/regex/v4/regex.hpp:32:
/Users/dev/Desktop/test-boost/opt/boost/libs/regex/include/boost/regex/v4/regex_workaround.hpp:42:10: fatal error: 'boost/scoped_ptr.hpp' file not found
#include <boost/scoped_ptr.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/TestBoost.dir/main.cpp.o] Error 1
make[1]: *** [CMakeFiles/TestBoost.dir/all] Error 2
make: *** [all] Error 2

scoped_ptr.hpp is in smart_ptr (and a few others), but that isn't included. Is CMake not yet capable of tracking Boost dependencies, or building Boost dependencies as needed?

Using the source release (https://github.com/boostorg/boost/releases/download/boost-1.81.0/boost-1.81.0.7z) doesn't work either, as it lacks a root CMakeLists.txt. Copying in the root CMakeLists.txt from the repo yields a compiler error unable to find boost/regex.hpp, as that release copies them all into /boost/.

Note I don't want to build all of Boost if possible, just a few libraries (hence EXCLUDE_FROM_ALL 1).

@ClausKlein
Copy link
Contributor

ClausKlein commented Feb 19, 2023

There is a clear error message from CMake, because it can't work this way!

cmake_minimum_required(VERSION 3.21...3.26)

project(MyLibrary VERSION 1.0.0 LANGUAGES CXX)

include(cmake/CPM.cmake)

if(EXISTS "$ENV{CPM_SOURCE_CACHE}/boost_1_81_0")
    set(FETCHCONTENT_SOURCE_DIR_BOOST "$ENV{CPM_SOURCE_CACHE}/boost_1_81_0" CACHE PATH "Boost source DIR")
    set(CPM_Boost_SOURCE ${FETCHCONTENT_SOURCE_DIR_BOOST} CACHE PATH "Manual override")
endif()

CPMAddPackage(
    NAME                Boost
    VERSION             1.81.0
    URL                 "https://github.com/boostorg/boost/releases/download/boost-1.81.0/boost-1.81.0.7z"
    URL_HASH            SHA256=86a1961606c433d15c5af55e1270d1d6d9afa770a243981e040f902c4c6ef154
    EXCLUDE_FROM_ALL    1
    DOWNLOAD_ONLY       1
)

if(Boost_ADDED)
    add_subdirectory(${Boost_SOURCE_DIR})
else()
    find_package(Boost 1.81.0 REQUIRED)
endif()

add_executable(TestBoost main.cpp)
target_link_libraries(TestBoost
    Boost::regex
)
cmake -S . -B ./cmake-build-boost-x86_64-Debug/install -G Ninja \
  -D CMAKE_CXX_COMPILER_LAUNCHER=/usr/local/bin/ccache -D CMAKE_BUILD_TYPE=Debug \
  -D CMAKE_PREFIX_PATH=/Users/clausklein/tmp/stagedir  -D CMAKE_CXX_STANDARD=20 
-- The CXX compiler identification is AppleClang 14.0.0.14000029
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CPM: Adding package Boost@ (/Users/clausklein/.cache/CPM/boost_1_81_0)
-- Configuring done
CMake Error at CMakeLists.txt:27 (target_link_libraries):
  Target "TestBoost" links to:

    Boost::regex

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.
bash-3.2$ 

or

-- CPM: Adding package Boost@ (/Users/clausklein/.cache/CPM/boost_1_81_0)
CMake Error at CMakeLists.txt:22 (add_subdirectory):
  add_subdirectory not given a binary directory but the given source
  directory "/Users/clausklein/.cache/CPM/boost_1_81_0" is not a subdirectory
  of "/Users/clausklein/tmp/boost".  When specifying an out-of-tree source a
  binary directory must be explicitly specified.


-- Configuring incomplete, errors occurred!
See also "/Users/clausklein/tmp/boost/cmake-build-boost-x86_64-Debug/install/CMakeFiles/CMakeOutput.log".
bash-3.2$ 

@alexchandel
Copy link

@ClausKlein Except my example works when I add Boost::filesystem as an additional target dependency:

target_link_libraries(TestBoost
    Boost::regex
    Boost::filesystem
)

So it seems that CMake is not resolving dependencies for certain Boost libraries.

@ClausKlein
Copy link
Contributor

@alexchandel Try this please:

cmake_minimum_required(VERSION 3.21...3.26)

project(MyLibrary VERSION 1.0.0 LANGUAGES CXX)

include(cmake/CPM.cmake)

CPMAddPackage("gh:ClausKlein/boost-cmake@1.80.0")

add_executable(TestBoost main.cpp)
target_link_libraries(TestBoost Boost::regex)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants