What if instead of a CLI, Plugins? Part 4, Prebuild plugins & misc
This article is part of a series.
Related Repo
All the plugins run their tools inside a sandbox. A command plugin’s tools can be given permission to leave via the Package manifest. Build plugins (in-build and prebuild) plugin tools cannot. Not without removing the whole sandbox. (5.9, 01/2024).
The sandbox makes it easier to bring a new plugin into the system because whether the fear is poorly written code or nefarious code… well it’s in a sandbox.
I bring this up because I couldn’t come up with a single solitary interesting idea of something to do for a prebuild plugin that wouldn’t have been better off as a command plugin (needed permissions to write to the source folder) or in-build plugin (expensive work). So I decided to try leaving the sandbox.
The Create Swift Package Plugins( min 19) video shows generating localization resources. Now that, as Boris Buegling said, is generally useful. I recommend that example.
It takes advantage of the fact that prebuild plugins don’t need any input information other than the arguments for the tool and only ask for an output directory. The lack of specificity on inputs and outputs means the plugin’s executable tool will run every build. The Package Manger has no list to check against to make sure everything expected of it has already been handled, so the tool gets run to be sure.
My example comes in two parts:
- Leaving the confines of the package to find resources elsewhere on the computer and then brining them into the build directory as a zip archive. (No sandbox violation)
- Bundling up all the source files and saving them into a “Storage” folder inside the package. (Sandbox violation)
Set Up
Making a prebuild plugin will be pretty straight forward. Similar enough to a build plugin, it just takes a little editing.
It still gets added to Package.swift the same way
//Plugin and target in same package example
.executableTarget(
name:"plugin-tester",
dependencies: [
.product(name: "ArgumentParser", package: "swift-argument-parser"),
],
path:"Sources/PluginTesterCLI",
exclude: ["Data"],
plugins: ["MyInBuildPlugin", "MyPreBuildPlugin"]
),
//...
.plugin(name: "MyPreBuildPlugin", capability: .buildTool()),
//...
And it still lives in the Plugins folder.
└── MyFancyPackage
└── Plugins
│ └── MyPreBuildPlugin
│ └──plugin.swift
└── Package.swift
└── .git
└── .gitignore
plugin.swift
Let’s look at my ill advised prebuild command that I’ve wrapped in a function returning a PackagePlugin.Command
. (below)
zsh
(a shell program) plays the roll of the executable. The tool can be any executable the SPM can see, so I went for it to see if I could just shove the shell in there.
My very long shell command zips up the requested directory (folderToZip
) with a timestamped archive name and then pares down the stored archives to only the 5 most recent. folderToZip
can be a path to anywhere local, although I did not do extensive testing. Prebuild commands don’t track inputs and the sandbox doesn’t reign them in. The outputDir does get limited by the sandbox. It expects to be within the plugin’s working directory in the build folder.
func zipFileCommand(outputDir:Path, folderToZip:Path) -> PackagePlugin.Command {
let parentDirectory = folderToZip.removingLastComponent().string
let folderOnly = folderToZip.lastComponent
let zipNCleanCommand = "cd \(parentDirectory) && zip -r \(outputDir)/snapshot_$(date +'%Y-%m-%dT%H-%M-%S').zip \(folderOnly) && cd - && cd \(outputDir) && ls -1t | tail -n +6 | xargs rm -f"
return .prebuildCommand(
displayName: "------------ MyPreBuildPlugin ------------",
executable: .init("/bin/zsh"), //also Path("/bin/zsh")
arguments: ["-c", zipNCleanCommand],
//environment: [:], in case it's needed...
outputFilesDirectory: outputDir)
}
The function can slot in to a plugin that plays nicely with swift run
and Xcode.
//...
@main
struct MyPreBuildPlugin:BuildToolPlugin {
func createBuildCommands(context: PackagePlugin.PluginContext, target: PackagePlugin.Target) async throws -> [PackagePlugin.Command] {
let outputDir = context.pluginWorkDirectory
let folderToZip = Path("/Users/{your path here}/TestZipFolder")
print("from MPBP:", outputDir)
return [ zipFileCommand(outputDir: outputDir, folderToZip: folderToZip)]
}
}
#if canImport(XcodeProjectPlugin)
import XcodeProjectPlugin
extension MyPreBuildPlugin: XcodeBuildToolPlugin {
// Entry point for creating build commands for targets in Xcode projects.
func createBuildCommands(context: XcodePluginContext, target: XcodeTarget) throws -> [Command] {
let outputDir = context.pluginWorkDirectory
let folderToZip = context.xcodeProject.directory.appending([target.displayName])
print("from MPBP:", outputDir)
return [ zipFileCommand(outputDir: outputDir, folderToZip: folderToZip)]
}
}
#endif
The report generated from running the plugin attached to a package from within Xcode looks like: (from report Inspector, shown in previous post)
/usr/bin/sandbox-exec -p "(version 1)
(deny default)
(import \"system.sb\")
(allow file-read*)
(allow process*)
(allow mach-lookup (global-name \"com.apple.lsd.mapdb\"))
(allow file-write*
(subpath \"/private/tmp\")
(subpath \"/private/var/folders/px/cd_t0mkd6gq0ntb0v6kqj__00000gn/T\")
)
(deny file-write*
(subpath \"/Users/{...}/Developer/GitHub/PluginExplorer\")
)
(allow file-write*
(subpath \"/Users/{...}/DerivedData/PluginExplorer-dikuycmkyjcfkhffudvbwxrkjxih/SourcePackages/plugins/pluginexplorer.output/plugin-tester/MyPreBuildPlugin\")
)
" /bin/zsh -c "cd /Users/{...}/Developer/GitHub && zip -r /Users/{...}/Library/Developer/Xcode/DerivedData/PluginExplorer-dikuycmkyjcfkhffudvbwxrkjxih/SourcePackages/plugins/pluginexplorer.output/plugin-tester/MyPreBuildPlugin/snapshot_$(date +'%Y-%m-%dT%H-%M-%S').zip TestZipFolder && cd - && cd /Users/{...}/Library/Developer/Xcode/DerivedData/PluginExplorer-dikuycmkyjcfkhffudvbwxrkjxih/SourcePackages/plugins/pluginexplorer.output/plugin-tester/MyPreBuildPlugin && ls -1t | tail -n +6 | xargs rm -f"
adding: TestZipFolder/ (stored 0%)
adding: TestZipFolder/robot_stylized_bc.png (deflated 3%)
adding: TestZipFolder/robot_realistic_m.png (deflated 0%)
adding: TestZipFolder/robot_stylized_m.png (deflated 24%)
adding: TestZipFolder/robot_papercraft_r.png (deflated 0%)
There are clearly defined allowed and not allowed places to save files.
Now lets try scenario 2, writing an archive of the current source files to a storage folder.
// could also try some Path("/Users/{---}/")
let outputDir = context.package.directory.appending(["Storage"])
let inputDir = target.directory
Make sure /$TARGET_PACKAGE/Storage actually exists and added to .gitignore to be safe. Run the Scheme again. This build should fail. At the bottom of the MyPreBuildPlugin report will be:
zip I/O error: Operation not permitted
zip error: Could not create output file
Open Terminal and
## plugin-tester is the name of the executable target the
## prebuild plugin is attached to.
cd $TARGET_PACKAGE
swift run plugin-tester ## get same message
swift run --disable-sandbox plugin-tester ## smooth sailing
ls Storage ## archive should be listed
Xcode 15.2 cannot run packages outside of safe mode. I have read about build plugins being run outside of safe mode when linked to an actual xcproj. I gave a cursory attempt, but could not make that work. I am just not very familiar with customizing builds for Xcode projects. Again a topic for another day.
Other options
I got to wondering if there could be another way to approach this without busting out of the sandbox. In general the sandbox makes sure one doesn’t do dumb things, and turning it off for all of the code when it just needs to be off for a small part of it seems unwise.
Command plugins have way more flexibility with their permissions. With the help from the swift forums I have the start of a command plugin that could be used in the future anytime I’m tempted to write a build plugin that needs to be run out of the sandbox because it manages its own build.
It uses a proxy for the PackageManager, and it seems to be a widely adopted approach.
It only works for packages and even in package mode does seem to have trouble in Xcode 15.2 (the only IDE I’ve tried.) The implementation of the proxy in that Xcode does not seem to be complete? It throws a unspecified("internalError(\"unimplemented\")")
error. That’s a draw back but I’m hoping it will be resolved in the coming months since this seems to be a more officially supported path?
I’ve dumped the code that would be in plugin.swift at the bottom of the post. It offers two options. After doing the permission-needing the work the plugin can:
- Build every thing, including dependencies, then run the named target
- Build only the named target then run it.
Summary
Not a lot to add just on prebuild plugins because they aren’t that different than build plugins. It was fun to try playing around with throwing different commands I had in my bin folder into the executable parameter. I even wrote a command plugin that could cd to another folder and do a swift run
. With the sandbox off it actually worked, but super ill advised. Was glad to learn about that crazy meta way in to packageManger. That’s amazing.
//
// CustomBuildNRun.swift
//
//
// Created by Carlyn Maw on 1/29/24.
//
//
//Use style ONE or TWO, not both like the below.
import PackagePlugin
import Foundation
@main
struct CustomBuildNRun: CommandPlugin {
// Entry point for command plugins applied to Swift Packages.
func performCommand(context: PackagePlugin.PluginContext, arguments: [String]) async throws {
//DO WORK
let targetToRunName = "DemoFruitStore"
let arguments = ["fruit_list", "citrus"]
let parameters = PackageManager.BuildParameters()
//parameters.logging = printCommands ? .verbose : .concise
//parameters.configuration = release ? .release : .debug
//print(parameters)
//------------------------------------ STYLE ONE
//rebuilds EVERYBODY, result has all the artifacts from every
//plugin, etc.
let result = try packageManager.build(.all(includingTests: false), parameters: parameters)
print(result.logText)
print("-----------")
print(result.builtArtifacts)
if result.succeeded {
if let resultToRun = result.builtArtifacts.first(where: {
$0.kind == .executable && $0.path.lastComponent == targetToRunName
}) {
let message = try runProcess(URL(fileURLWithPath: resultToRun.path.string), arguments: arguments)
print(message)
}
}
//------------------------------------ END STYLE ONE
//------------------------------------ STYLE TWO
//the .build() function can also filter on product.
//Does not rebuild plugins, etc.
let targets = try context.package.targets(named: [targetToRunName])
let targetsToRun = try targets.flatMap {
try extractRunnableTargets($0, parameters: parameters)
}
try targetsToRun.forEach { targetToRun in
let message = try runProcess(URL(fileURLWithPath: targetToRun.path.string), arguments: arguments)
print(message)
}
//------------------------------------ END STYLE TWO
}
func extractRunnableTargets(_ target:Target, parameters:PackageManager.BuildParameters) throws -> [PackageManager.BuildResult.BuiltArtifact] {
print("trying for target... \(target.name)")
let result = try packageManager.build(.target(target.name), parameters: parameters)
print(result.logText)
if result.succeeded {
return result.builtArtifacts.filter({$0.kind == .executable })
} else {
return []
}
}
}
@discardableResult
func runProcess(_ tool:URL, arguments:[String] = [], workingDirectory:URL? = nil) throws -> String {
let task = Process()
let pipe = Pipe()
task.standardOutput = pipe
task.standardError = pipe
task.arguments = arguments
if let workingDirectory {
task.currentDirectoryURL = workingDirectory
}
//task.qualityOfService
//task.environment
task.standardInput = nil
task.executableURL = tool
try task.run()
let data = pipe.fileHandleForReading.readDataToEndOfFile()
let output = String(data: data, encoding: .utf8)!
task.waitUntilExit()
if task.terminationStatus == 0 || task.terminationStatus == 2 {
return output
} else {
print(output)
throw CustomProcessError.unknownError(exitCode: task.terminationStatus)
}
}
enum CustomProcessError: Error {
case unknownError(exitCode: Int32)
}