What if instead of a CLI, Plugins? Part 4, Prebuild plugins & misc

This article is part of a series.

All the plugins run their tools inside a sandbox. A command plugin’s tools can be given permission to leave via the Package manifest. Build plugins (in-build and prebuild) plugin tools cannot. Not without removing the whole sandbox. (5.9, 01/2024).

The sandbox makes it easier to bring a new plugin into the system because whether the fear is poorly written code or nefarious code… well it’s in a sandbox.

I bring this up because I couldn’t come up with a single solitary interesting idea of something to do for a prebuild plugin that wouldn’t have been better off as a command plugin (needed permissions to write to the source folder) or in-build plugin (expensive work). So I decided to try leaving the sandbox.

The Create Swift Package Plugins( min 19) video shows generating localization resources. Now that, as Boris Buegling said, is generally useful. I recommend that example.

It takes advantage of the fact that prebuild plugins don’t need any input information other than the arguments for the tool and only ask for an output directory. The lack of specificity on inputs and outputs means the plugin’s executable tool will run every build. The Package Manger has no list to check against to make sure everything expected of it has already been handled, so the tool gets run to be sure.

My example comes in two parts:

Set Up

Making a prebuild plugin will be pretty straight forward. Similar enough to a build plugin, it just takes a little editing.

It still gets added to Package.swift the same way

//Plugin and target in same package example 
        .executableTarget(
            name:"plugin-tester",
            dependencies: [
                .product(name: "ArgumentParser", package: "swift-argument-parser"),
            ],
            path:"Sources/PluginTesterCLI",
            exclude: ["Data"],
            plugins: ["MyInBuildPlugin", "MyPreBuildPlugin"]
        ),
        //...
        .plugin(name: "MyPreBuildPlugin", capability: .buildTool()),
        //...

And it still lives in the Plugins folder.

└── MyFancyPackage
    └── Plugins
    │   └── MyPreBuildPlugin
    │       └──plugin.swift
    └── Package.swift
    └── .git
    └── .gitignore

plugin.swift

Let’s look at my ill advised prebuild command that I’ve wrapped in a function returning a PackagePlugin.Command. (below)

zsh (a shell program) plays the roll of the executable. The tool can be any executable the SPM can see, so I went for it to see if I could just shove the shell in there.

My very long shell command zips up the requested directory (folderToZip) with a timestamped archive name and then pares down the stored archives to only the 5 most recent. folderToZip can be a path to anywhere local, although I did not do extensive testing. Prebuild commands don’t track inputs and the sandbox doesn’t reign them in. The outputDir does get limited by the sandbox. It expects to be within the plugin’s working directory in the build folder.

   func zipFileCommand(outputDir:Path, folderToZip:Path) -> PackagePlugin.Command {
        let parentDirectory = folderToZip.removingLastComponent().string
        let folderOnly = folderToZip.lastComponent

        let zipNCleanCommand = "cd \(parentDirectory) && zip -r \(outputDir)/snapshot_$(date +'%Y-%m-%dT%H-%M-%S').zip \(folderOnly) && cd - && cd \(outputDir) && ls -1t | tail -n +6 | xargs rm -f"
        
        return .prebuildCommand(
            displayName: "------------ MyPreBuildPlugin ------------",
            executable: .init("/bin/zsh"), //also Path("/bin/zsh")
            arguments: ["-c", zipNCleanCommand],
            //environment: [:], in case it's needed... 
            outputFilesDirectory: outputDir)
        
    }

The function can slot in to a plugin that plays nicely with swift run and Xcode.

 //...
@main
struct MyPreBuildPlugin:BuildToolPlugin {
    func createBuildCommands(context: PackagePlugin.PluginContext, target: PackagePlugin.Target) async throws -> [PackagePlugin.Command] {   
        let outputDir = context.pluginWorkDirectory
        let folderToZip = Path("/Users/{your path here}/TestZipFolder") 
        print("from MPBP:", outputDir)
        return [ zipFileCommand(outputDir: outputDir, folderToZip: folderToZip)]
    }
}
#if canImport(XcodeProjectPlugin)
import XcodeProjectPlugin

extension MyPreBuildPlugin: XcodeBuildToolPlugin {
    // Entry point for creating build commands for targets in Xcode projects.
    func createBuildCommands(context: XcodePluginContext, target: XcodeTarget) throws -> [Command] {
        let outputDir = context.pluginWorkDirectory
        let folderToZip = context.xcodeProject.directory.appending([target.displayName])
       print("from MPBP:", outputDir)
       return [ zipFileCommand(outputDir: outputDir, folderToZip: folderToZip)]
    }
}

#endif

The report generated from running the plugin attached to a package from within Xcode looks like: (from report Inspector, shown in previous post)

/usr/bin/sandbox-exec -p "(version 1)
(deny default)
(import \"system.sb\")
(allow file-read*)
(allow process*)
(allow mach-lookup (global-name \"com.apple.lsd.mapdb\"))
(allow file-write*
    (subpath \"/private/tmp\")
    (subpath \"/private/var/folders/px/cd_t0mkd6gq0ntb0v6kqj__00000gn/T\")
)
(deny file-write*
    (subpath \"/Users/{...}/Developer/GitHub/PluginExplorer\")
)
(allow file-write*
    (subpath \"/Users/{...}/DerivedData/PluginExplorer-dikuycmkyjcfkhffudvbwxrkjxih/SourcePackages/plugins/pluginexplorer.output/plugin-tester/MyPreBuildPlugin\")
)
" /bin/zsh -c "cd /Users/{...}/Developer/GitHub && zip -r /Users/{...}/Library/Developer/Xcode/DerivedData/PluginExplorer-dikuycmkyjcfkhffudvbwxrkjxih/SourcePackages/plugins/pluginexplorer.output/plugin-tester/MyPreBuildPlugin/snapshot_$(date +'%Y-%m-%dT%H-%M-%S').zip TestZipFolder && cd - && cd /Users/{...}/Library/Developer/Xcode/DerivedData/PluginExplorer-dikuycmkyjcfkhffudvbwxrkjxih/SourcePackages/plugins/pluginexplorer.output/plugin-tester/MyPreBuildPlugin && ls -1t | tail -n +6 | xargs rm -f"

  adding: TestZipFolder/ (stored 0%)
  adding: TestZipFolder/robot_stylized_bc.png (deflated 3%)
  adding: TestZipFolder/robot_realistic_m.png (deflated 0%)
  adding: TestZipFolder/robot_stylized_m.png (deflated 24%)
  adding: TestZipFolder/robot_papercraft_r.png (deflated 0%)

There are clearly defined allowed and not allowed places to save files.

Now lets try scenario 2, writing an archive of the current source files to a storage folder.

        // could also try some  Path("/Users/{---}/")
        let outputDir = context.package.directory.appending(["Storage"])
        
        let inputDir = target.directory

Make sure /$TARGET_PACKAGE/Storage actually exists and added to .gitignore to be safe. Run the Scheme again. This build should fail. At the bottom of the MyPreBuildPlugin report will be:

zip I/O error: Operation not permitted
zip error: Could not create output file

Open Terminal and

## plugin-tester is the name of the executable target the 
## prebuild plugin is attached to. 
cd $TARGET_PACKAGE
swift run  plugin-tester ## get same message
swift run --disable-sandbox plugin-tester ## smooth sailing
ls Storage ## archive should be listed

Xcode 15.2 cannot run packages outside of safe mode. I have read about build plugins being run outside of safe mode when linked to an actual xcproj. I gave a cursory attempt, but could not make that work. I am just not very familiar with customizing builds for Xcode projects. Again a topic for another day.

Other options

I got to wondering if there could be another way to approach this without busting out of the sandbox. In general the sandbox makes sure one doesn’t do dumb things, and turning it off for all of the code when it just needs to be off for a small part of it seems unwise.

Command plugins have way more flexibility with their permissions. With the help from the swift forums I have the start of a command plugin that could be used in the future anytime I’m tempted to write a build plugin that needs to be run out of the sandbox because it manages its own build.

It uses a proxy for the PackageManager, and it seems to be a widely adopted approach.

It only works for packages and even in package mode does seem to have trouble in Xcode 15.2 (the only IDE I’ve tried.) The implementation of the proxy in that Xcode does not seem to be complete? It throws a unspecified("internalError(\"unimplemented\")") error. That’s a draw back but I’m hoping it will be resolved in the coming months since this seems to be a more officially supported path?

I’ve dumped the code that would be in plugin.swift at the bottom of the post. It offers two options. After doing the permission-needing the work the plugin can:

Summary

Not a lot to add just on prebuild plugins because they aren’t that different than build plugins. It was fun to try playing around with throwing different commands I had in my bin folder into the executable parameter. I even wrote a command plugin that could cd to another folder and do a swift run. With the sandbox off it actually worked, but super ill advised. Was glad to learn about that crazy meta way in to packageManger. That’s amazing.

//
//  CustomBuildNRun.swift
//
//
//  Created by Carlyn Maw on 1/29/24.
//  
//

//Use style ONE  or TWO, not both like the below.

import PackagePlugin
import Foundation

@main
struct CustomBuildNRun: CommandPlugin {
    // Entry point for command plugins applied to Swift Packages.
    func performCommand(context: PackagePlugin.PluginContext, arguments: [String]) async throws {

        //DO WORK
        
        let targetToRunName = "DemoFruitStore"
        let arguments = ["fruit_list", "citrus"]
        
        let parameters = PackageManager.BuildParameters()
        //parameters.logging = printCommands ? .verbose : .concise
        //parameters.configuration = release ? .release : .debug
        //print(parameters)
        
        //------------------------------------  STYLE ONE
        //rebuilds EVERYBODY, result has all the artifacts from every
        //plugin, etc.
        let result = try packageManager.build(.all(includingTests: false), parameters: parameters)
        print(result.logText)
        print("-----------")
        print(result.builtArtifacts)
        if result.succeeded {
            if let resultToRun = result.builtArtifacts.first(where: {
                $0.kind == .executable && $0.path.lastComponent == targetToRunName
            }) {
                let message = try runProcess(URL(fileURLWithPath: resultToRun.path.string), arguments: arguments)
                print(message)
            }
        }
        
        //------------------------------------  END STYLE ONE
        
        //------------------------------------  STYLE TWO
        //the .build() function can also filter on product.
        //Does not rebuild plugins, etc.
        
        let targets = try context.package.targets(named: [targetToRunName])
        
       let targetsToRun = try targets.flatMap {
           try extractRunnableTargets($0, parameters: parameters)
        }
        
       try targetsToRun.forEach { targetToRun in
            let message = try runProcess(URL(fileURLWithPath: targetToRun.path.string), arguments: arguments)
            print(message)
        }
        
        //------------------------------------  END STYLE TWO
    }
    
    func extractRunnableTargets(_ target:Target, parameters:PackageManager.BuildParameters) throws -> [PackageManager.BuildResult.BuiltArtifact] {
        print("trying for target... \(target.name)")
        let result = try packageManager.build(.target(target.name), parameters: parameters)
        print(result.logText)
        if result.succeeded  {
           return result.builtArtifacts.filter({$0.kind == .executable })
        } else {
            return []
        }
        
    }
}

@discardableResult
func runProcess(_ tool:URL, arguments:[String] = [], workingDirectory:URL? = nil) throws -> String {
    let task = Process()
    let pipe = Pipe()
    
    task.standardOutput = pipe
    task.standardError = pipe
    task.arguments = arguments
    
    if let workingDirectory {
        task.currentDirectoryURL = workingDirectory
    }
    //task.qualityOfService
    //task.environment

    task.standardInput = nil
    task.executableURL = tool
    try task.run()
    
    let data = pipe.fileHandleForReading.readDataToEndOfFile()
    let output = String(data: data, encoding: .utf8)!
    
    task.waitUntilExit()

    if task.terminationStatus == 0 || task.terminationStatus == 2 {
      return output
    } else {
      print(output)
      throw CustomProcessError.unknownError(exitCode: task.terminationStatus)
    }
}

enum CustomProcessError: Error {
  case unknownError(exitCode: Int32)
}

This article is part of a series.