Benchmarks

Methodology

The reproducible benchmark generates equivalent 5,000-test suites for each framework. C frameworks are compiled as C11 with clang -O2; C++ frameworks are compiled as C++11 with clang++ -O2. Keeping the languages in separate tables avoids treating results from different compilers and language runtimes as directly comparable.

Each suite is measured in three runtime modes: one filtered passing test, all passing tests, and all failing tests. The report also records preprocessed line count, compile time, object size, and executable size. Runtime output is suppressed while measuring. Compile time uses one warmup and the median of three measured runs; runtime uses two warmups and the median of ten measured runs.

The compile metric covers the generated test translation unit. Unity and GoogleTest normally use a separately compiled support library, so that one-time library build is excluded from compile time. Its object bytes are still included in the object-size total, and its linked code is included in the executable size. Header-only frameworks compile their runner as part of the measured translation unit. All runtime measurements execute tests in-process; Acutest is invoked with --no-exec to disable its default per-test child-process isolation.

#define JC_TEST_IMPLEMENTATION
#include <jc_test.h>

TEST(Math, Multiply)
{
    ASSERT_EQ(4, 2 * 2);
}

int main(int argc, char** argv)
{
    jc_test_init(&argc, argv);
    return jc_test_run_all();
}

#define JC_TEST_IMPLEMENTATION
#include <jc_test.hpp>

TEST(MyTest, Multiplication) {
    ASSERT_EQ(4, 2 * 2);
}
TEST(MyTest, Division) {
    ASSERT_EQ(2, 4 / 2);
}

int main(int argc, char *argv[]) {
    // Parses and removes any jc_test specific arguments
    jc_test_init(&argc, argv);
    // ... Do your test initialization
    return jc_test_run_all();
}

The all-fail jctest cases retain and print every failure again in the final recap. They therefore do additional allocation and formatting work that the comparison frameworks may not perform. Results are informational rather than timing gates in CI.

Reproduce the benchmark

python3 test/comparisons/benchmark.py \
    --utest-dir /path/to/utest.h-checkout \
    --greatest-dir /path/to/greatest-checkout \
    --unity-dir /path/to/Unity-checkout \
    --acutest-dir /path/to/acutest-checkout \
    --gtest-dir /path/to/googletest-checkout \
    --doctest-dir /path/to/doctest-checkout \
    --catch2-dir /path/to/Catch2-v2-checkout \
    --output build/comparisons/results.md

The report records the platform, compiler versions, workload, and exact dependency commits. The generated sources, object files, and executables are kept under build/comparisons for inspection.

Reference run: 2026-07-23

Apple Clang 21.0.0 on arm64 macOS 27.0. GoogleTest 1.12.1 and Catch2 2.13.10 are the last release lines of those frameworks that support C++11. Exact revisions for every dependency are recorded by the generated report.

C frameworks

C++ frameworks

Real life examples

In our case (the Defold game engine), we wanted to replace GTEST with something new.

Here’s a comparison between the GTEST and jctest frameworks when building the game engine tests. We see a decrease of the compile time, ending up at ~54% of the original time.

The footprint on disc is also important, as it takes up both space but also affects writing/reading times. In this scenario the file sizes ended up at ~45% of the original sizes.