Unreal provides a few different ways of writing tests as part of its automation framework. My personal favorite are spec tests. I’ve been using them extensively as part of my work at Lumeto as well as in my personal projects.
Spec (short for specification) tests are based on the Behavior-Driven Development (BDD) methodology. Specs provide an intuitive, human-readable syntax for writing tests that focus on validating the expectations of a public API rather than its implementation.
Specs can be used to implement unit, integration and functional tests and can optionally be written to match user stories — such as “As a user I should be able to…” or “As a developer I should be able to…” — allowing to test slices of functionality.
By convention specs in Unreal are implemented in files called
<FeatureName>.spec.cpp. No header file is required. Tests can be stored anywhere in the project. Personally I like to group them in a
Tests folder in the same module where features are implemented.
A minimal test file looks like this:
Under the hood the macros take care of creating a
FExampleSpec class that registers our tests with the engine so we run them (more on this in a bit).
EAutomationTestFlags work the same as with other automated tests and help the engine understand the type of test that is being implemented.
Everything is wrapped in the
#if WITH_AUTOMATION_TESTS conditional to prevent tests from being included in Shipping builds. Individual test cases (AKA expectations) can be defined inside the
Specs provide an expressive, lambda-based interface to define expectations. The following is a simple unit test:
Expectations are defined using the
It() function and should contain logic that validates specific behavior. The
Describe() function can be used to group multiple expectations together. Grouping expectations is optional, but very useful as it makes tests easier to organize, locate and run.
Specs can be run just like other tests in Unreal, including:
- From the Session Frontend (found in the Tools menu) in the editor:
- From the command line:
- From Rider (or — I think — recent versions of Visual Studio):
It’s good practice to make sure each test runs in a controlled environment that provides everything needed, but nothing more. The objects that define the environment and data for a test are usually called fixtures.
Fixtures are normally created before running a test and destroyed afterwards to make sure each test runs in isolation — preventing false positives and side effects. Chances are, however, that groups of tests that validate similar functionality need to run in a similar environment.
Specs provide two handy functions called
AfterEach() that run before and after each expectation and that can be used to create and destroy fixtures:
AfterEach() are relative to their scope. In the example above, they will run only for expectations defined inside the parent
Describe() block, making it possible to set up different environments for different groups of tests.
Specs support async execution out-of-the-box, allowing to test asynchronous game logic, integration with remote services, etc. This is done by using latent versions of the functions we’ve seen above, such as
Testing gameplay features
With some additional work Specs can be used to write tests for many gameplay features directly from C++, without having to rely exclusively on Blueprint functional tests (which in my experience are slower to implement and run).
As an example, let’s create some tests for an actor that should explode when it receives damage. Since actors can only exist inside a world, we need to create one as part of our test fixtures.
The following is a fixture class that creates a temporary
UWorld suitable for running tests:
The world created by the fixture is completely empty, which is ideal to make sure tests run in isolation and to avoid the overhead of loading a real level with meshes, textures, etc. The world and everything it contains are automatically destroyed when the fixture is destructed.
The same approach can be used to write fixtures that spawn actors, components or, generally, set up complex environments or data before running a test, and tear them down when the test is over.
Testing the actor
The following is a barebone implementation of the actor and its tests:
- Specs, just like other automated tests (excluding Gauntlet tests), can’t be used to implement multiplayer tests at least out-of-the-box, although engineers at Rare were able to overcome this limitation.
- Specs can only be implemented in C++, which might be a good or bad thing depending on context and constraints.
- The official Automation Framework and Automation Specs docs provide a very good introduction to the system.
- The Automated Testing of Gameplay Features in Sea of Thieves and Automated Testing at Scale in Sea of Thieves talks are a gold mine of hands-on information about automated testing in Unreal.
- Posts from Jessica Baker, one of the speakers of the sessions above: Tests and Testability and How to Write Great Unit Tests.
- For more information about running tests from the command line see UE4 Unit Tests in Jenkins.
- Searching for files ending with
.spec.cppin the engine source code will reveal all the specs available in the engine. As with everything else in Unreal, looking at existing code is one the best ways to learn.
- A Gearbox Plugin for Codeless Automated Tests: presentation at Unreal Fest 2023 about a different approach to automated testing (similar to Unreal functional tests) specifically for gameplay features.
- 26 Jan. 2024: improved01 code listings based on this Mastodon conversation.
- 15 Feb. 2024: added link to Unreal Fest 2023 presentation.