Hello USD - Part 13: Test, Test, Test

This article is part of a series.

“The best time to build a test was before the first commit. The next best time is now.” – Proverbious the Commitor, probably

The SketchPad output needs to be validated on three levels:

This posts documents setting up unit tests, which I’ll use to address the first step.

Overviews

Update Package File and Make Tests Directory

Update package file and create a new directory

cd $PROJECT_DIR
mkdir Tests
touch Tests/FileBuilderTests/MyFirstTest.swift

├── Package.swift
├── README.md
├── Sources
│   └── <Source Files>
└── Tests
    └── FileBuilderTests
        └── MyFirstTest.swift

    targets: [
        .target(
            name: "SketchPad"
        ),
        .testTarget(name: "FileBuilderTests", dependencies: ["SketchPad"]),
        .executableTarget(
            name: "SketchPadCLI",
            dependencies: [
                "SketchPad",
                .product(name: "ArgumentParser", package: "swift-argument-parser"),
            ]
        )
    ]

Test That Tests Run

The basic Test template for a package that Xcode auto-generates looks like:

import XCTest
@testable import $LIBRARY_NAME

final class SomeNameTests: XCTestCase {
    func testExample() throws {
        // This is an example of a functional test case.
        // Use XCTAssert and related functions to verify your tests produce the correct
        // results.
        XCTAssertEqual($SOME_TEXT_VAR, "Hello, World!")
    }
}

@testable makes functions that would normally not be available outside the package available for testing. Which I know I want in this case.

I’m starting with the below, a guaranteed pass to make sure everything works.

import XCTest
@testable import SketchPad

final class FileBuilderTests: XCTestCase {
    func testExample() throws {
        XCTAssertEqual("Hello, World!", "Hello, World!")
    }
}

Then ran the tests from the command line with:

swift test

Generating the following in the console:

Building for debugging...
[4/4] Linking SketchPadPackageTests
Build complete! (0.70s)
Test Suite 'All tests' started at 2023-07-18 10:42:38.303
Test Suite 'SketchPadPackageTests.xctest' started at 2023-07-18 10:42:38.304
Test Suite 'FileBuilderTests' started at 2023-07-18 10:42:38.304
Test Case '-[FileBuilderTests.FileBuilderTests testExample]' started.
Test Case '-[FileBuilderTests.FileBuilderTests testExample]' passed (0.001 seconds).
Test Suite 'FileBuilderTests' passed at 2023-07-18 10:42:38.305.
         Executed 1 test, with 0 failures (0 unexpected) in 0.001 (0.001) seconds
Test Suite 'SketchPadPackageTests.xctest' passed at 2023-07-18 10:42:38.305.
         Executed 1 test, with 0 failures (0 unexpected) in 0.001 (0.001) seconds
Test Suite 'All tests' passed at 2023-07-18 10:42:38.305.
         Executed 1 test, with 0 failures (0 unexpected) in 0.001 (0.002) seconds

Note that the command ran All the tests. In the future I will not want to run all the tests all the time. I’ll be able to run just a subset with:

swift test --filter $TEST_TARGET_NAME
## For example:
swift test --filter FileBuilderTests

Now let’s see what what happens after adding a mismatch to the test and running just the FileBuilderTests target.

import XCTest
@testable import SketchPad

final class FileBuilderTests: XCTestCase {
    func testExample() throws {
        XCTAssertEqual("Hello, Dolly!", "Hello, World!")
    }
}

… the output will be:

Building for debugging...
[4/4] Linking SketchPadPackageTests
Build complete! (0.69s)
Test Suite 'Selected tests' started at 2023-07-18 10:51:53.822
Test Suite 'SketchPadPackageTests.xctest' started at 2023-07-18 10:51:53.823
Test Suite 'FileBuilderTests' started at 2023-07-18 10:51:53.823
Test Case '-[FileBuilderTests.FileBuilderTests testExample]' started.
/Users/carlynorama/Developer/GitHub/SketchPad/Tests/FileBuilderTests/MyFirstTest.swift:16: error: -[FileBuilderTests.FileBuilderTests testExample] : XCTAssertEqual failed: ("Hello, Dolly!") is not equal to ("Hello, World!")
Test Case '-[FileBuilderTests.FileBuilderTests testExample]' failed (0.012 seconds).
Test Suite 'FileBuilderTests' failed at 2023-07-18 10:51:53.835.
         Executed 1 test, with 1 failure (0 unexpected) in 0.012 (0.012) seconds
Test Suite 'SketchPadPackageTests.xctest' failed at 2023-07-18 10:51:53.835.
         Executed 1 test, with 1 failure (0 unexpected) in 0.012 (0.012) seconds
Test Suite 'Selected tests' failed at 2023-07-18 10:51:53.835.
         Executed 1 test, with 1 failure (0 unexpected) in 0.012 (0.013) seconds

That’s still a lot to go digging through to find the error.

Setting up XCode to Test a Package

I’ve been doing all my building for this project in the command line using swift run. Seeing the dense results of swift test makes me want to run to XCode to scan for those little red diamonds when they crop up.

Since I added the testTarget by hand, for XCode I needed to also add a Scheme by hand as well. To do so through the menu choose Product > Scheme > New Scheme

The New Schema window’s dropdown menu already showed FileBuildTests as selected. Leaving the name unchanged, I hit OK. Once setting the active Schema to FileBuildTests, I then had no problems running a test by clicking on the diamond in the number gutter next to its declaration.

Cropped screenshot of the area at the top of an XCode window where one can select the Scheme

Switching the selected Scheme to the cli (sketchpad) would let me run the default script by pressing the Run button. By default the the output goes to the console. I played around with adding a shortcut to open terminal, but had no luck setting up the XCode Scheme to receive a command or options arguments. YMMV.

First Actually Useful Test

I changed the name of the file to HelperStructsTests.swift and began writing the tests for Document.

import XCTest
@testable import SketchPad

final class DocumentTests: XCTestCase {
    func testExample() throws {
        let result = Document {
            "Hello, 
            World!"
        }
        XCTAssertEqual(result.render(style: .minimal), "Hello, World!")
    }
}

I’ll eventually match each file in Sources with a file in Tests.

< INSERT TEST WRITING MONTAGE, with motivational music of course >

https://github.com/carlynorama/SketchPad/tree/main/Tests/FileBuilderTests

Lessons learned

import XCTest
@testable import SketchPad
import RegexBuilder

final class USDFileBuilderTests: XCTestCase {

static let defaultBuilder = USDAFileBuilder()
    
    @available(macOS 13.0, *)
    func headerMatch(_ toTest:String) -> Bool {

        let regex = Regex {
            One("#usda 1.0\n(\n\t")
            OneOrMore(.any)
            One("\n)")
        }
        
        if toTest.firstMatch(of: regex) != nil {
            return true }

        return false
    }
    
    @available(macOS 13.0, *)
    func test_getHeaderStructure() {
        let documentRender = Document {
            Self.defaultBuilder.generateHeader(defaultPrimID: "MyPrim")
        }.render(style: .multilineIndented)
        
        let testResult = headerMatch(documentRender)
        XCTAssert(testResult)
        
        XCTAssert(documentRender.contains("defaultPrim ="))
        XCTAssert(documentRender.contains("upAxis ="))
        XCTAssert(documentRender.contains("metersPerUnit ="))
    }

}

extension StringNode: Equatable {
    static func == (lhs: StringNode, rhs: StringNode) -> Bool {
        switch (lhs, rhs) {
        case (.container(let lhs), .container(let rhs)):
            return lhs == rhs
        case (.content(let lhs), .content(let rhs)):
            return lhs == rhs
        case (.list(let lhs), .list(let rhs)):
            return lhs == rhs
        default:
            return false
        }
    }
}

Difficulty With Finding out Code Coverage (unresolved)

Can’t really figure out how to gauge my coverage. Not that worried about it, but do love a data point.

Reference Dump

Misc & Future Topics

This article is part of a series.