'Re: [CMake] Adding compile and build type tests to CMake/CTest'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       cmake
Subject:    Re: [CMake] Adding compile and build type tests to CMake/CTest
From:       P F via CMake <cmake () cmake ! org>
Date:       2017-09-07 3:39:37
Message-ID: 405D2C82-58F6-46C7-90AC-75D4C51314F4 () yahoo ! com
[Download RAW message or body]

The `add_test` function can run whatever command you want it to, including compiling \
a target:

add_library(foo_compile_test STATIC EXCLUDE_FROM_ALL foo_compile_test.cpp)
add_test(NAME foo_compile_test
    COMMAND ${CMAKE_COMMAND} --build . --target foo_compile_test --config \
$<CONFIGURATION>  WORKING_DIRECTORY ${CMAKE_BINARY_DIR})

Then you set cmake to expect failure:

set_tests_properties(foo_compile_test PROPERTIES WILL_FAIL TRUE)

You can also check for output instead of just expecting failure:

set_tests_properties(mytest PROPERTIES
  PASS_REGULAR_EXPRESSION "foo failed"
)

This is especially useful to make sure that the error is from the static_assert \
message and not from another compile error.

Of course, cmake could provide a module which provides a function to do this, which \
additionally could help workaround the caveats of this approach. This is essentially \
what `bcm_test` in the boost cmake modules do. Hopefully, in the future this module \
could be merged upstream into cmake.

> On Sep 5, 2017, at 11:44 AM, Edward Diener <eldlistmailingz@tropicsoft.com> wrote:
> 
> On 9/5/2017 2:47 AM, Dvir Yitzchaki wrote:
> > There's already CheckCXXSourceCompiles and friends.
> > The only problem is that try_compile is not scriptable otherwise you could let \
> > the test invoke ${CMAKE_COMMAND} -P check_source_compiles.cmake.
> 
> To put it succinctly CMake should adding compile-time testing so that when some \
> compilation succeeds the test is successful and if the compilation fails the test \
> is not successful, with the proviso that you can reverse the result as a compile \
> should fail type of test. Similarly a build type testing, without having to run \
> anything should be added along the same lines. 
> In modern C++ it is perfectly feasible, especially with template programming, to do \
> compile time testing, invoking a compile-time static assert as a compile-time \
> failure. Boost has had this for years and modern C++ has it as part of the latest \
> version of the C++ standard. CMake needs to update itself to the reality that pure \
> compile-time testing is a reality for modern C++ and should update itself \
> accordingly. Only having run-time testing is an artifact of the past. Hopefully \
> CMake developers will get the message and make the necessary update to CMake/CTest. \
> 
> > -----Original Message-----
> > From: CMake [mailto:cmake-bounces@cmake.org] On Behalf Of Roger Leigh
> > Sent: Monday, September 4, 2017 16:51
> > To: cmake@cmake.org
> > Subject: Re: [CMake] Adding compile and build type tests to CMake/CTest
> > On 04/09/17 14:40, Edward Diener wrote:
> > > Boost Build has tests for running an application successfully or not,
> > > for compiling one or more source files successfully or not, and for
> > > building one or more source files into an exe or not. These tests in
> > > Boost Build are the run/run-fail, compile/compile-fail, and
> > > link/link-fail rules.
> > > 
> > > CMake/CTest has the exact equivalent to the run/run-fail rule in its
> > > add_test framework, but there is not add_test equivalent to the other
> > > two sets of rules. It sure would be nice, when Boost transitions to
> > > using CMake/CTest instead of Boost Build, if CMake/CTest had the
> > > equivalent of the other two sets of types of test in its add_test
> > > framework.
> > > 
> > > Is there any consensus that these other two types of tests might be
> > > valuable for CMake/CTest, or any way to make this happen ?
> > I've certainly wished for them.  Particularly when testing templated code where \
> > you want to test that certain things fail correctly, e.g. via static_assert or \
> > simply being invalid. I understand it's possible to make this work partially, by \
> > creating targets which aren't built by default, and then add tests which invoke \
> > the targets.  But this appears to have some caveats, such as potential \
> > misbehaviour with parallel testing.  Or you can have a separate CMake build for \
> > each individual target, but it's extra complexity.  Having a proper means of \
> > registering such tests would be very, very nice. Regards,
> > Roger
> 
> 
> -- 
> 
> Powered by www.kitware.com
> 
> Please keep messages on-topic and check the CMake FAQ at: \
> http://www.cmake.org/Wiki/CMake_FAQ 
> Kitware offers various services to support the CMake community. For more \
> information on each offering, please visit: 
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
> 
> Visit other Kitware open-source projects at \
> http://www.kitware.com/opensource/opensource.html 
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: \
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information \
on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at \
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

[prev in list] [next in list] [prev in thread] [next in thread] 