Automated Testing

Introduction

The Kanda SDK includes a comprehensive testing framework designed to facilitate automated testing within Unity. This framework supports a variety of testing paradigms to ensure high code coverage and robust functionality. Utilizing the Unity Test Runner, developers are encouraged to achieve over 80% test coverage, even though this is not a strict requirement.

Framework Overview

The Kanda Foundation Tests Framework, provided under the Kanda.Foundation.Tests namespace, includes a batteries-included setup for testing. This framework offers basic scaffolds and templates that can be used to quickly set up test classes. Additionally, several base classes and test helpers are provided to streamline the testing process.

Key Features

1. Base System Tests:
   • Description: Provides a scaffold for testing Entities systems by creating a dedicated ECS world and injecting desired systems.
   • Example Use: Simplifying setup and teardown of test environments for system testing.
2. Entity Setup Helpers:
   • Description: Shorthand methods to set up entities with various component configurations.
   • Example Use: Quickly creating entities representing client connections or setting up in-game states.
3. Mock Setup Helpers:
   • Description: Utilities for creating mock instances of services.
   • Example Use: Mocking services like IAppLifecycleService to isolate components under test.
4. Scene Setup Helpers:
   • Description: Methods for creating and managing scenes for testing.
   • Example Use: Creating new scenes or managing scene assets during tests.
5. Random Test Values:
   • Description: Methods for generating random values for testing purposes.
   • Example Use: Creating random network IDs or weak references for testing.

Testing Guidelines

Prefer Edit-Mode Tests

These are preferred over play mode tests due to their faster execution and higher stability. Play mode tests, while useful for high-level integration testing, tend to be slow and brittle, especially across different platforms.

Prefer Unit Tests

Units, in this context, should be understood as behaviours of MonoBehaviours, systems, or services.

This does not necessarily mean test every public method of every class. Instead, focusing on the high-level behaviors of a component under different circumstances will usually provide the required coverage on low-level methods.

In our experience, unit-component tests offer the best balance between development time, coverage, and test robustness.

Prefer Behavioural Testing

In Unity, many functions produce side effects rather than return values. Test will typically focus on these side effects resulting from component behaviours by checking outcomes in states or downstream actions.

Structure tests using If-When-Then

A well-structured test often follows this template:

• If: Given a set of circumstances or inputs.
• When: Some work is performed.
• Then: We expect something to happen.

To help future readers, annotate your tests using If-When-Then comments as demonstrated in later sections. This is a variant on the Arrange-Act-Assert pattern where the If-When-Then convention helps encourage natural and easy-to-read language.

Testing for Side Effects

1. State Verification: Set up initial state, perform an action, then verify the resulting world state.

   [Test]
   public void StateVerificationTest() {
       // If: An entity with SomeComponent exists
       var world = new World("TestWorld");
       var entityManager = world.EntityManager;
       var entity = entityManager.CreateEntity(typeof(SomeComponent));
    
       // When: SomeSystem updates
       var system = world.GetOrCreateSystem<SomeSystem>();
       system.Update();
    
       // Then: Entity should have ExpectedComponent
       Assert.IsTrue(entityManager.HasComponent<ExpectedComponent>(entity));
   }

2. Action Verification: Use mocks to verify that certain actions trigger expected downstream behaviors. This is sometimes necessary when the side-effect produced is not a return value, or data artifact, but an action triggered in the Unity engine.

   [Test]
   public void ActionVerificationTest() {
       // If: Component is instantiated with a (mocked) scene manager
       var mockSceneManager = new Mock<ISceneManager>();
       var component = new ComponentUnderTest(mockSceneManager.Object);
       
       // When: Some method is executed
       component.SomeMethodThatTriggersSceneLoaded();
       
       // Then: A scene load should have been triggered
       mockService.Verify(service => service.LoadSceneAsync(It.IsAny<string>()), Times.Once);
   }

Use Abstractions When Necessary

Use interfaces and object wrappers to abstract Unity APIs that are difficult to mock, such as static classes. This allows for more flexible and testable code.

public interface ISceneManager
{
    Scene GetActiveScene();
}
 
public class UnitySceneManager : ISceneManager
{
    public Scene GetActiveScene()
    {
        // Wrap the static Unity API call
        return SceneManager.GetActiveScene();
    }
}

In tests, you can now swap out the concrete implementation with a mock.

var mockSceneManager = new Mock<ISceneManager>();
mockSceneManager.Setup(s => s.GetActiveScene()).Returns(someTestScene);
var componentUnderTest = new ComponentUnderTest(_mockSceneManager.Object);

Avoid Scripting Symbols in Runtime Code

Custom scripting symbols such as #if UNITY_EDITOR or #if UNITY_SERVER compile code conditionally depending on platform.

This makes it hard to write good edit-mode tests, as testing code compiled for a specific platform typically requires a player to be built for the given platform. It also makes code hard to read and errors or typos for platforms (other than your current one) can be difficult to spot.

Unless the code specifically depends on APIs only available on some platforms, you can use IPlatformInfoService to get runtime information about the current platform.

public void SomeMethodWithPlatformCode(IPlatformInfoService platformService)
{
    var context = platformService.GetContext();
    var device = platformService.GetDevice();
    
    if (context.HasFlag(PlatformContextFlags.DedicatedServer) && PlatformDevice.Windows) 
    {
        // Do something on Windows servers only
    }
}
 
// Even though we can't verify SomeMethodWithPlatformCode in its entirety,
// we can still test the individual methods provided it does not depend on native APIs.
 
[Test]
public void SomeMethodWithPlatformCode_DoesWhatYouExpected() {
    // If: The current platform is Windows server
    _mockPlatformInfo
        .Setup(p => p.GetContext())
        .Returns(PlatformContextFlags.DedicatedServer);
    
    _mockPlatformInfo
        .Setup(p => p.GetDevice())
        .Returns(PlatformDevice.Windows);
    
    // When: Server method executes
    component.SomeMethodWithPlatformCode(_mockPlatformInfo.Object)
 
    // Then: Something should have happened
}

Utilize Services

Services are utility classes that can be accessed across applications or scenes. Instead of using singletons, consider using services to avoid making code hard to test. Use the Service Containers and App Service Locator framework to register and manage services.

public interface IExampleService {
    void ExecuteTask();
}
 
public class ExampleService : IExampleService {
    public void ExecuteTask() {
        // Implementation here
    }
}

Register the service at application startup and access it using the App Service Locator.

public static class AppServices {
    public void RegisterAll() {
        AppServiceLocator.Register<IExampleService>(new ExampleService());
    }
}

During tests, replace these services with mocks or fakes to isolate the code being tested.

[Test]
public void ExampleServiceTest() {
    // If: IExampleService is registered in AppServiceLocator 
    var mockService = new Mock<IExampleService>();
    AppServiceLocator.Register<IExampleService>(mockService.Object);
    
    // When: The component is instantiated and executes some method
    var componentUnderTest = new ComponentUnderTest();
    componentUnderTest.SomeMethod();
 
    // Then: The underlying service should have been triggered
    mockService.Verify(service => service.ExecuteTask(), Times.Once);
}

Conclusion

The testing framework and guidelines are designed to facilitate efficient and effective automated testing in Unity. By focusing on edit-mode unit tests, behavioral testing, and utilizing abstractions and services, developers can achieve high test coverage and maintain robust, reliable code.