Hello USD - Part 16: Swift just sipping OpenUSD through... a pipe!

This article is part of a series.

I’ve mostly been working with OpenUSD via the command line, so I decided to take initial pass at interfacing with the OpenUSD library in Swift trough a Swift launched shell.

I don’t need much. I’d like to:

Related post: Part 7: Where my error messages at????

I’m going to write the library package and a cli executable package separately a lá the SwiftPNG and clipng pair to keep the cli code far away from the package I’ll eventually bring into SketchPad.

New Repos: USDTestingCLI | USDServiceProvider

The first chunk of the post covers setting up that CLI/Package pair and adding a shell call (Process()). The OpenUSD part starts at “Really getting started: usdcat”

End of Post Tagged Commits:

Make the project scaffolding

Step 1: Executable Package Test

I had XCode open so I didn’t use the command line, I just File > New > Package’d from the menu to create USDTestingCLI and made the following changes

Update USDTestingCLI.swift

@main
public struct USDTestingCLI {

    public static func main() {
        print("Hello world!")
    }
}

Update Package.swift

// swift-tools-version: 5.8
// The swift-tools-version declares the minimum version of Swift required to build this package.


import PackageDescription

let package = Package(
    name: "USDTestingCLI",
    products: [
        .executable(
            name: "myusdtests",
            targets: ["USDTestingCLI"]),
    ],
    dependencies: [ ],
    targets: [
        .executableTarget(
            name: "USDTestingCLI",
            dependencies: []),
         //None yet.
//        .testTarget(
//            name: "USDTestingCLITests",
//            dependencies: ["USDTestingCLI"]),
    ]
)

Hit the Run button (cmd-R) and “Hello world!” showed up in the console. Done.

Step 2: Make Library Package

Again just made another default package from File > New > Package. This time making no changes, so the main file in Sources stays like:

USDServiceProvider.swift

No Changes.

public struct USDServiceProvider {
    public private(set) var text = "Hello, World!"

    public init() {
    }
    
}

USDTestingCLI - USDTestingCLI.swift

Update the main file in the CLI Package to:

import Foundation
import ArgumentParser
import USDServiceProvider


@main
public struct USDTestingCLI:ParsableCommand {
    public static let configuration = CommandConfiguration(
        
        abstract: "A Swift command-line tool to create 3D files from simple instructions",
        version: "0.0.1",
        subcommands: [
            test.self,
        ],
        defaultSubcommand: test.self)
    
    public init() {}
    
    struct test:ParsableCommand {
        func run() throws {
            print(USDServiceProvider().text)
        }
    }
}

USDTestingCLI - Package.swift

Add a local dependency.

// swift-tools-version: 5.8
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription

let package = Package(
    name: "USDTestingCLI",
    products: [
        // Products define the executables and libraries a package produces, and make them visible to other packages.
        .executable(
            name: "myusdtests",
            targets: ["USDTestingCLI"]),
    ],
    dependencies: [
        .package(url: "https://github.com/apple/swift-argument-parser.git", from: "1.2.2"),
        .package(path: "../USDServiceProvider"),
    ],
    targets: [
        .executableTarget(
            name: "USDTestingCLI",
            dependencies: [
                .product(name: "ArgumentParser", package: "swift-argument-parser"),
                .product(name: "USDServiceProvider", package: "USDServiceProvider"),
            ]),
        .testTarget(
            name: "USDTestingCLITests",
            dependencies: ["USDTestingCLI"]),
    ]
)

Again, running via XCode (cmd-R) should work to see “Hello, World!” in the console.

Step 4: Add shell commands

USDServiceProvider - USDServiceProvider.swift

Next I’m going to pull from previous work

Useful links for understanding this code:

I added the following to the USDServiceProvider struct:

     public func echo(_ input:String) -> String {
        if input.isEmpty {
            return "[crickets chirping]"
        } else {
            let message = try? shell("echo \(input)")
            return (message != nil) ? message! : "nothing to say"
        }
        
    }
    
    @discardableResult // Add to suppress warnings when you don't want/need a result
    func shell(_ command: String) throws -> String {
        let task = Process()
        let pipe = Pipe()
        
        task.standardOutput = pipe
        task.standardError = pipe
        task.arguments = ["-c", command]
        
        task.standardInput = nil
        task.executableURL = URL(fileURLWithPath: "/bin/bash") //<-- what shell
        try task.run()
        
        let data = pipe.fileHandleForReading.readDataToEndOfFile()
        let output = String(data: data, encoding: .utf8)!
        
        return output
    }
    
    public func whatsInMyBin() {
        let testPath = "/usr/bin"
        let ls = Process()
        //https://manned.org/env.1
        ls.executableURL = URL(fileURLWithPath: "/usr/bin/env")
        ls.arguments = ["ls", "-al", testPath]
        do{
            try ls.run()
        } catch {
            print(error)
        }
    }

And changed the CLI test struct to:

    struct test:ParsableCommand {
        func run() throws {
            print(USDServiceProvider().whatsInMyBin())
            print(USDServiceProvider().echo("can you hear me?"))
        }
    }

In a shell program (because the output is long):

cd ~/Developer/GitHub/USDTestingCLI & swift run myusdtests

I get a very very very long list of commands I could potentially use in my program!

Really getting started: usdcat

usdcat is a freestanding binary with no python dependencies. usdchecker, on the other hand, is a python script with a shebang on the top. While it is “clickable”, it is not contained. Getting usdcat to run on a known valid .usda file will be much easier to implement than usdchecker. The goal:

usdcat -o $OUTPUT_NAME --flatten $INPUT_ROOT_FILE

Since I know the location of the OpenUSD build I want to use, just to see if it works, I can change USDServiceProvider to:

public struct USDServiceProvider {
    public private(set) var pathToBin:String
    
    public init(_ pathToUSD:String? = nil) {
       pathToBin = "/Users/USERNAME/opd/USD_nousdview_0722/bin"
    }
    
    public func usdcatHelp() -> String {
        let message = try? Self.shell("\(pathToBin)/usdcat -h")
        return (message != nil) ? message! : "nothing to say"
    }
    
    //NOTE: Changed to static
    @discardableResult // Add to suppress warnings when you don't want/need a result
    static func shell(_ command: String) throws -> String {
        let task = Process()
        let pipe = Pipe()
        
        task.standardOutput = pipe
        task.standardError = pipe
        task.arguments = ["-c", command]
        
        task.standardInput = nil
        task.executableURL = URL(fileURLWithPath: "/bin/bash") //<-- what shell
        try task.run()
        
        let data = pipe.fileHandleForReading.readDataToEndOfFile()
        let output = String(data: data, encoding: .utf8)!
        
        return output
    }
}

I can call it!

But can I use it?

    public func makeUSDC() {
        let inputFile = "/Users/USERNAME/Developer/GitHub/USDTestingCLI/compress_me.usda"
        let outputFile = "/Users/USERNAME/Developer/GitHub/USDTestingCLI/compressed.usdc"
        print(try? shell("pwd"))
        let message = try? shell("\(pathToBin)/usdcat -o \(outputFile) --flatten \(inputFile)")
        print(message)
    }

Success!!!

Note the call to pwd there. When using the XCode Run / Cmd-R button the print out reads:

Optional("/Users/USERNAME/Library/Developer/Xcode/DerivedData/USDTestingCLI-dabxknpcmjbcpiexpxknncyhvmiy/Build/Products/Debug\n")

But swift run myusdtests from a shell within the CLI’s directory prints out the more reasonable:

Optional("/Users/USERNAME/Developer/GitHub/USDTestingCLI\n")

Just something to keep in mind when setting the input and output file locations. XCode runs a CLI in a quarantined area, like an App, not in the pwd of the project. Using full paths made it easier to run this quick test.

Using The Python for usdchecker

usdchecker does not work with the default flavor of python installed on MacOS (3.11), although that appears to be fixable (see aside).

ASIDE: boost problem w/ 3.11 (https://github.com/boostorg/python/pull/385), according to comment in OpenUSD build script can set a different version of boost via a build arg? --build-args boost,cxxflags=

IF the caller’s python environment could be dependably be the same as the build’s python environment, simply adding the following to the shell function would be fine:

    var environment =  ProcessInfo.processInfo.environment
    environment["PYTHONPATH"] = "\(pathToPython)"
    task.environment = environment
    print(task.environment ?? "")

I don’t want to have to remember to only ever call my script from the perfect environment. My code can remember that for me. I already have script I can work from documented in a SETUP file.

USDService provider gets a new extension

extension USDServiceProvider {
    
    func environmentWrap(_ newCommand:String, python:PythonEnvironment) -> String {
        """
        \(python.setString)
        export PATH=$PATH:\(pathToBaseDir)/bin;
        export PYTHONPATH=$PYTHONPATH:\(pathToBaseDir)/lib/python
        \(newCommand)
        """
    }
    
    public enum PythonEnvironment {
        case defaultSystem
        case pyenv(String)
        case systemInstallMacOS(String)
        case customPath(String)
        
        var setString:String {
            switch self {
                
            case .defaultSystem:
                return ""
            case .pyenv(let v):
                return setPythonWithPyEnv(version: v)
            case .systemInstallMacOS(let v):
                return setPythonSystemMacOS(version: v)
            case .customPath(let p):
                return prependPath(customString: p)
            }
        }
        
        func setPythonWithPyEnv(version:String) -> String {
            """
            export PYENV_ROOT="$HOME/.pyenv"
            command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
            eval "$(pyenv init -)"
            export PYENV_VERSION=\(version)
            """
        }
        
        func setPythonSystemMacOS(version:String) -> String {
            """
            PATH="/Library/Frameworks/Python.framework/Versions/\(version)/bin:${PATH}"
            export PATH
            """
        }
        
        func prependPath(customString:String) -> String {
            """
            PATH="\(customString):${PATH}"
            export PATH
            """
        }
    }

}

A new initializer

    public init(pathToUSDBuild:String, pythonEnv:PythonEnvironment) {
        self.pathToBaseDir = pathToUSDBuild
        self.pythonEnv = pythonEnv
    }

and an update to the shell command

        task.standardError = pipe
        //BELOW: NEW WRAPPER FOR TASK ARG
        task.arguments = ["-c", environmentWrap(command, python: pythonEnv)]
        
        task.standardInput = nil

The CLI gets a new command

    struct checkncrate:ParsableCommand {
        
        @Argument(help: "The input file") var inputFile: String
        @Argument(help: "The output file") var outputFile: String?
        
        func run() throws {
            
            //TODO: fragile. if cli sticks around, improve.
            let outputFilePath = outputFile ?? inputFile.replacingOccurrences(of: ".usda", with: ".usdc")
            
            let usdSP = USDServiceProvider(pathToUSDBuild: USDBuild, pythonEnv: .pyenv("3.10"))
            
            print("hello")
            
            let result = usdSP.check(inputFile)
            print(result)
            usdSP.makeCrate(from: inputFile, outputFile: outputFilePath)
        }
    }

cd ~/Developer/GitHub/USDTestingCLI & swift run myusdtests compress_me.usda

compress_me.usda

results in…

FAILURE!!! I mean SUCCESS!!! I mean getting info about a failure IS success!!

Building for debugging… [5/5] Linking myusdtests Build complete! (0.91s) hello Stage does not specify its linear scale in metersPerUnit. (fails ‘StageMetadataChecker’) Stage has missing or invalid defaultPrim. (fails ‘StageMetadataChecker’) Failed!

WHEW!

Still a bit of a way to go before USDServiceProvider actually provides the services I want for SketchPad, but a nice start.

Wait Wait… one more thing.

A OpenUSD build isn’t confirmed working until the hello-world script is run…

Added to USDTestingCLI.swift

    struct helloworld:ParsableCommand {
        @Argument(help: "The output file") var outputFile: String?
        
        func run() throws {
            
            //TODO: also fragile. if cli sticks around, improve.
            let outputFilePath = outputFile ?? "~/Documents/hello_world.usda"
            
            let usdSP = USDServiceProvider(pathToUSDBuild: USDBuild, pythonEnv: pythonEnv)
            usdSP.saveHelloWorld(to: outputFilePath)
        }
    }

Added to USDServiceProvider.swift

    // Works from Terminal other shell program, not so much XCode b/c of pyenv
    // TODO: Try on computer with system python USD Build
    public func saveHelloWorld(to outputLocation:String) {
        let current = URL(string: #file)
        let dir = current!.deletingLastPathComponent()
        let message = try? shell("python3 \(dir)/python_scripts/hello_world.py \(outputLocation)")
        print("message:\(message ?? "no message")")
    }

NEW: Sources/USDServiceProvider/python_scripts/hello_world.py

from pxr import Usd, UsdGeom
import sys

def main(output):
    stage = Usd.Stage.CreateNew(output)
    xformPrim = UsdGeom.Xform.Define(stage, '/hello')
    spherePrim = UsdGeom.Sphere.Define(stage, '/hello/world')
    stage.GetRootLayer().Save()

if __name__ == "__main__":
    main(sys.argv[1])

Hello Sphere!!

Composite of two screenshots: A text editor with basic hello world text, and a QuickView window with our friend the sphere, once again.

This article is part of a series.