microbenchmark.hpp File Reference

Functions to perform (multithreaded) timing measurement on a given functor. More...

Go to the source code of this file.

Description

Functions to perform (multithreaded) timing measurement on a given functor.

This helper simplifies micro benchmarks of isolated implementation details. The test subject, given as function object or lambda, is invoked numerous times within a tight loop. In the multithreaded variant, the lambda is copied into N threads and performed in each thread in parallel; after waiting on termination of the test threads, results are summed up and then averaged into milliseconds per single invocation. The actual timing measurement relies on chrono::duration, which means to count micro ticks of the OS.

Warning

care has to bee taken when optimisation is involved! Optimisation usually has quite some impact on the results, but since this function is inline, the lambda can typically be inlined and the loop possibly be optimised away altogether. A simple workaround is to define a volatile variable in the call context, close the lambda by reference, and perform a comparison with that volatile variable in each invocation. The compiler is required actually to access the value of the volatile each time.

Remarks

some interesting observations (in my setup, 8 core AMD FX-8350)

• if we replace the global volatile by a local variable within the test subject, the initialisation of that local typically costs +5ns per invocation.
• incrementing the volatile costs +10ns
• multithreaded (unlocked) incrementing of the global volatile creates massive overhead and increases the running time by factor 100. This nicely confirms that the x86_64 platform has strong cache coherence.

Definition in file microbenchmark.hpp.

#include "lib/meta/function.hpp"
#include "lib/scoped-collection.hpp"
#include "lib/sync-barrier.hpp"
#include "lib/thread.hpp"
#include "lib/test/microbenchmark-adaptor.hpp"
#include <chrono>

Namespaces
namespace	lib
	Implementation namespace for support and library code.
namespace	lib::test
	Unit tests for the Lumiera support library.
namespace	lib::test::anonymous_namespace{microbenchmark.hpp}

Typedefs
using	CLOCK_SCALE = std::micro

Functions
template<class FUN >
double	benchmarkTime (FUN const &invokeTestCode, const size_t repeatCnt=1)
	Helper to invoke a functor or λ to observe its running time.
template<class FUN >
size_t	benchmarkLoop (FUN const &testSubject, const size_t repeatCnt=DEFAULT_RUNS)
	Benchmark building block to invoke a functor or λ in a tight loop, passing the current loop index and capturing a result checksum value.
template<class FUN >
auto	microBenchmark (FUN const &testSubject, const size_t repeatCnt=DEFAULT_RUNS)
	perform a simple looped microbenchmark.
template<size_t nThreads, class FUN >
auto	threadBenchmark (FUN const &subject, const size_t repeatCnt=DEFAULT_RUNS)
	perform a multithreaded microbenchmark.

Variables
constexpr size_t	DEFAULT_RUNS = 10'000'000