Hello USD - Part 7: Where my error messages at????

Yesterday my script outputted file(s) passed the Pixar Library’s usdchecker but not the Apple’s USDZ Tools usdARKitChecker. usdARKitChecker was very coy about why.

The TL:DR:

System Info:

What’s in the Apple script?

First thing I wanted to know was why the verbose mode usdARKitChecker really wasn’t that chatty. No clarity of why my file failed. Also, since I’m new to so much of this I wanted a lot of feedback as to what was being checked and what could go wrong.

So I went poking through the file.

Code Apple’s (license in linked file later in post). Comments mine.

## should by python3? 
#!/usr/bin/python

## Python libraries
import subprocess, sys, os, argparse
## pixar usd library python scripts
from pxr import *
## validateMesh.py is file in same directory
from validateMesh import validateMesh
## validateMaterial.py is file in same directory
from validateMaterial import validateMaterial
## Note - these files are linked to later in post

def validateFile(file, verbose, errorData):
    ## UsdStage "owns the scenegraph and provides access to a composition." 
    ## https://github.com/PixarAnimationStudios/OpenUSD/blob/b53573ea2a6b29bc4a6b129f604bbb342c35df5c/pxr/usd/usd/stage.cpp#L866
    ## https://openusd.org/release/api/usd_page_front.html
    stage = Usd.Stage.Open(file)

    ## true until an error fires
    success = True

    ## https://github.com/PixarAnimationStudios/OpenUSD/blob/b53573ea2a6b29bc4a6b129f604bbb342c35df5c/pxr/usd/usd/primFlags.h#L576
    ## https://github.com/PixarAnimationStudios/OpenUSD/blob/b53573ea2a6b29bc4a6b129f604bbb342c35df5c/pxr/usd/usd/primFlags.h#L135
    ## Prims appear to have several booleans (flags) relating to their configuration / how deep they are, etc.  This "Usd_PrimFlagsPredicate" seems to gather them all together in a set and lets the summoner filter out non-relevant prims. (I suspect `Usd.PrimIsActive` etc are in fact bit masks as this really is a boolean and, not a logical and)
    predicate = Usd.TraverseInstanceProxies(Usd.PrimIsActive & Usd.PrimIsDefined & ~Usd.PrimIsAbstract)
    for prim in stage.Traverse(predicate):
        if prim.GetTypeName() == "Mesh":
            success = validateMesh(prim, verbose, errorData) and success
        if prim.GetTypeName() == "Material":
            success = validateMaterial(prim, verbose, errorData) and success
    return success

def runValidators(filename, verboseOutput, errorData):
    ## https://github.com/PixarAnimationStudios/OpenUSD/blob/b53573ea2a6b29bc4a6b129f604bbb342c35df5c/pxr/usd/usdUtils/complianceChecker.py#L912
    ## RIGHT HERE THE ABOVE IS THE THING TO READ.
    checker = UsdUtils.ComplianceChecker(arkit=True, 
            skipARKitRootLayerCheck=False, rootPackageOnly=False, 
            skipVariants=False, verbose=False) ## <-- Here is (a) verbose problem?
    # TODO: checker.DumpAllRules()?
    checker.CheckCompliance(filename)
    errors = checker.GetErrors()
    failedChecks = checker.GetFailedChecks()
    for rule in checker._rules:
        error = rule.__class__.__name__
        failures = rule.GetFailedChecks()
        if len(failures) > 0:
            errorData.append({ "code": "PXR_" + error })
            errors.append(error)

    usdCheckerResult = len(errors) == 0

    ## Where are the errors being printed? 

    ## Do the custom checks from the other files
    ## TODO: understand them
    mdlValidation = validateFile(filename, verboseOutput, errorData)

    success = usdCheckerResult and mdlValidation
    print("usdARKitChecker: " + ("[Pass]" if success else "[Fail]") + " " + filename)

def main(argumentList, outErrorList=None):
    ## Read in any arguments / options
    parser = argparse.ArgumentParser()
    parser.add_argument("--verbose", "-v", action='store_true', help="Verbose mode.")
    parser.add_argument('files', nargs='*')
    args=parser.parse_args(argumentList)

    verboseOutput = args.verbose

    ## Validation succeeds until it fails. 
    totalSuccess = True

    for filename in args.files:
        errorData = []
        ## filename: from the loop
        ## verboseOutput: From the flag
        ## errorData: Just defined. 
        runValidators(filename, verboseOutput, errorData)
        ## outErrorList: defined in function call as None by default
        ## When is it ever updated to not None? Is it just an optional out pipe? 
        if outErrorList is not None:
        	outErrorList.append({ "file": filename, "errors": errorData })
        # We're still good only if no new errors and no old errors
        totalSuccess = totalSuccess and len(errorData) == 0

    ## Program exits with no errors (code 0) if totalSuccess
    if totalSuccess:
        return 0
    else:
        return 1

if __name__ == '__main__':
    argumentList = sys.argv[1:]
    sys.exit(main(argumentList))

What’s in the Pixar script

An invaluable clue in the usdARKitChecker script - how to find the Pixar script that drives usdchecker, complianceChecker.py.

So much information! I do not understand a lot of these terms yet, but I feel like I’m getting closer. Text copied from file linked to above:

Getting more error messages out usdARKitChecker

Running

usdchecker --arkit ~/Developer/GitHub/USDHelloWorld/Part_06/multiball_20230709T222354.usda

supplies the following error:

The stage uses 2 layers. It should contain a single usdc layer to be compatible with ARKit's implementation of usdz. (fails 'ARKitRootLayerChecker')
Root layer of the stage '~/Developer/GitHub/USDHelloWorld/Part_06/multiball_20230709T222354.usda' does not have the '.usdc' extension. (fails 'ARKitRootLayerChecker')

Why was I not getting that from usdARKitChecker --verbose? Well now I am! ALL THE MESSAGES ALL THE TIME!!!

Links to raw:

Also: https://github.com/carlynorama/USDHelloWorld/tree/main/explorations/Part_07

#!/usr/bin/python3
## changed to python3

import subprocess, sys, os, argparse
from pxr import *
from validateMesh import validateMesh
from validateMaterial import validateMaterial

## No changes
def validateFile(file, verbose, errorData):
    stage = Usd.Stage.Open(file)
    success = True
    predicate = Usd.TraverseInstanceProxies(Usd.PrimIsActive & Usd.PrimIsDefined & ~Usd.PrimIsAbstract)
    for prim in stage.Traverse(predicate):
        if prim.GetTypeName() == "Mesh":
            success = validateMesh(prim, verbose, errorData) and success
        if prim.GetTypeName() == "Material":
            success = validateMaterial(prim, verbose, errorData) and success
    return success

## More verbose verbose mode
def runValidators(filename, verboseOutput, errorData):
    checker = UsdUtils.ComplianceChecker(arkit=True, 
            skipARKitRootLayerCheck=False, rootPackageOnly=False, 
            skipVariants=False, verbose=verboseOutput) #<- changed to pass verboseOutput through

    
    checker.CheckCompliance(filename)

    if verboseOutput:
        print("Rules Being Checked by usdchecker:")
        # checker.DumpAllRules() line failed with error: 
        # for ruleNum, rule in enumerate(GetBaseRules()):   
        # NameError: name 'GetBaseRules' is not defined
        for rule in checker._rules:
            print(rule)
        print("-----")

    errors = checker.GetErrors()
    failedChecks = checker.GetFailedChecks()
    warnings = checker.GetWarnings()
    
    if verboseOutput:
        for warning in warnings:
            print(warning)
        for error in errors:
            print(error)
        for failure in failedChecks:
            print(failure)
        print("-----")
    for rule in checker._rules:
        error = rule.__class__.__name__
        failures = rule.GetFailedChecks()
        if len(failures) > 0:
            errorData.append({ "code": "PXR_" + error })
            errors.append(error)
    if verboseOutput:
        print("Will be exported to outErrorList")
        print(errorData)
        print("----")

    usdCheckerResult = len(errors) == 0
    mdlValidation = validateFile(filename, verboseOutput, errorData)

    success = usdCheckerResult and mdlValidation
    print("usdARKitChecker: " + ("[Pass]" if success else "[Fail]") + " " + filename)

## No changes
def main(argumentList, outErrorList=None):
    parser = argparse.ArgumentParser()
    parser.add_argument("--verbose", "-v", action='store_true', help="Verbose mode.")
    parser.add_argument('files', nargs='*')
    args=parser.parse_args(argumentList)

    verboseOutput = args.verbose
    totalSuccess = True
    for filename in args.files:
        errorData = []
        runValidators(filename, verboseOutput, errorData)
        if outErrorList is not None:
            outErrorList.append({ "file": filename, "errors": errorData })
        totalSuccess = totalSuccess and len(errorData) == 0

    if totalSuccess:
        return 0
    else:
        return 1

if __name__ == '__main__':
    argumentList = sys.argv[1:]
    sys.exit(main(argumentList))

Why use usdARKitChecker at all?

Unclear to me, someone new to the party, what hasn’t been ported to the Pixar library yet. I haven’t tested anything with materials or meshes and I don’t know if the extra files provided in the USDZ Tools still give a needed extra layer. They seem to check models fairly extensively. But cross referencing usdchecker on some of the terms seems to reflect the scripts are feeling around similar areas in some cases. ALSO haven’t even begun to poke at ARKit, ModelIO, the new betas, etc to see what tools are baked in to the frameworks that might be more on the beaten path.

Can I fix my scripts compliance?

My problems with making my files ARKit compliant are both easier and harder to fix than I was hoping.

Easier in that I can still use primitives and don’t need to make a Scene, the problems I thought I was having. Switching over to a meshes and embedding things into scenes or creating assemblies would have been new territory.

Harder because I am not going to be writing a Swift USDA -> USDC compressor algorithm anytime in the near future and if I want to embed a .usda generator in a project that will punt it to ARKit I don’t know how to do that yet and now it seems without usdc, there will be no joy. I’d like to do a demo project of getting a Swift wrapper around C++, but the OpenUSD Pixar tools have a lot going on.

In the mean time I will live with a little post processing.

My first successful attempt at force-fixing my files was:

usdview ../Part_06/multiball_20230709T222354.usda
# File > Save flattened as... (choose USDC option)
usdchecker --arkit multiball_20230709T222354_flattened.usdc
> Success!
python3 usdARKitChecker_revised.py multiball_20230709T222354_flattened.usdc
> usdARKitChecker: [Pass] multiball_20230709T222354_flattened.usdc

This requires actual interaction from a human, so not tenable in the long run.

Second successful attempt:

# note just the ONE input file name, the root scene file
usdcat -o output_test_2.usdc --flatten ../Part_06/multiball_20230709T222354.usda
python3 usdARKitChecker_revised.py output_test_2.usdc
> usdARKitChecker: [Pass] output_test_2.usdc

See.. its a USDC now!

Group of 13 spheres, all different colors. Kind of looks like the picture from yesterday.

Size savings: (5KB + 415 bytes) for uncompressed 3KB for compressed.

Failed attempts involved:

Before I found the right order / combination of usdcat parameters I started to write a shell script. It works, but I leave it here only for the record since the one liner is better.

#!/bin/sh

# Usage: give the script the name of the root file and
# the root file only, not the full list. 
# can be replaced with 
# `usdcat -o $OUTPUT_NAME --flatten $INPUT_ROOT_FILE``

#PATH_TO_USDCAT="somepath"
REMOVE_TMP_FLAG=false
TMP_FILE_NAME="smash_tmp.usda"
OUTPUT_NAME="smash_output.usdc"

# using $@ (all args) is iffy here since really
# should be passing _one_ file name after all.
usdcat --flatten $@ > $TMP_FILE_NAME
usdcat -o $OUTPUT_NAME $TMP_FILE_NAME
if $REMOVE_TMP_FLAG; then
    rm $TMP_FILE_NAME
fi