P0412R0 Benchmarking Primitives

Motivation

#include <chrono>
#include <iostream>

double perform_computation(int);

void benchmark()
{
    using namespace std;
    auto start = chrono::high_resolution_clock::now();
    double answer = perform_computation(42);
    auto delta = chrono::high_resolution_clock::now() - start;
    cout << "The answer is: " << answer << ". The computation took "
         << chrono::duration_cast<chrono::milliseconds>(delta).count()
         << " ms";
}
		

The above example has some problems, because the compiler is allowed to:

• Reorder calls
• Perform constant propagation and constant folding
• If the result was not used, eliminate the computation completely

#include <chrono>
#include <iostream>

double perform_computation(int);

void benchmark()
{
	using namespace std;
	// reorder: place perform_computation here
	auto start = chrono::high_resolution_clock::now();
	double answer = perform_computation(42);	// perform constant-folding
	auto delta = chrono::high_resolution_clock::now() - start;
	// or reorder: place perform_computation here
	cout << "The answer is: " << answer << ". The computation took "
			 << chrono::duration_cast<chrono::milliseconds>(delta).count()
			 << " ms";
}
	

The proposal

The paper proposes to add two new functions to the standard library:

• touch — mark a glvalue as "modified"
• keep — mark a value as "read"

#include <chrono>
#include <iostream>
#include <benchmark>

double perform_computation(int);

void benchmark()
{
    using namespace std;
    auto start = chrono::high_resolution_clock::now();
    int value = 42;
    experimental::benchmark::touch(value);
    double answer = perform_computation(value);
    experimental::benchmark::keep(answer);
    auto delta = chrono::high_resolution_clock::now() - start;
    cout << "The answer is: " << answer << ". The computation took "
         << chrono::duration_cast<chrono::milliseconds>(delta).count()
         << " ms";
}
		

Existing libraries

Some benchmarking libraries already have keep-like functions:

• do_not_optimize — in Google benchmark
• do_not_optimize_away — in Celero, Folly

Folly also provides a function similar to touch:

• make_unpredictable

Implementation experience

A prototype implementation exists in two flavors:

• Complete implementation (uses a GCC/Clang-specific feature)
• Portable implementation for POD types without padding

Compiler intrinsics are required for proper implementation of this feature.

Discussion

• Which standard (C++ or Library Fundamentals)?
• Namespace
• Naming
• How should touch treat const-ness