Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-14262] Foundation.Process has too much overhead for short-lived processes #3232

Open
typesanitizer opened this issue Feb 22, 2021 · 5 comments

Comments

@typesanitizer
Copy link

Previous ID SR-14262
Radar rdar://problem/74577513
Original Reporter @typesanitizer
Type Bug

Attachment: Download

Environment
  • M1 Macbook Pro running macOS 11.2.1 (20D74)

  • Xcode 12.4 (12D4e), Swiftc (5.3.2 (swiftlang-1200.0.45 clang-1200.0.32.28))

Additional Detail from JIRA
Votes 0
Component/s Foundation
Labels Bug
Assignee None
Priority Medium

md5: 54b4da616e867e4d69e4bdcf8ffd44e1

Issue Description:

I'm trying to use Foundation.Process for invoking git, and its own overhead is significantly more than the time to run the git command itself.

Here is a simple git repo:

$ git init
$ git checkout -b main && touch cake && git add . && git commit -m "Add cake."
$ git checkout -b develop && touch fruit && git add . && git commit -m "Add fruit." 

Here is how I measured the timing (using https://github.com/sharkdp/hyperfine)

$ hyperfine 'git checkout main; git checkout develop'
Benchmark #​1: git checkout main; git checkout develop
  Time (mean ± σ):      11.5 ms ±   0.3 ms    [User: 4.7 ms, System: 4.2 ms]
  Range (minmax):    10.8 ms12.8 ms    205 runs 

Here is the Swift code.

$ cat ../timing.swift
import Foundation
func runCommand(args: [String]) throws {
    let task = Process()
    task.executableURL = URL(fileURLWithPath: "/usr/bin/git")
    task.arguments = args
    let stdout = Pipe()
    let stderr = Pipe()
    task.standardOutput = stdout
    task.standardError = stderr
    try task.run()
    task.waitUntilExit()
}
try runCommand(args: ["checkout", "main"])
try runCommand(args: ["checkout", "develop"]) 

Compiled that with xcrun swiftc -O timing.swift -o ../timing.

$ hyperfine '../timing'
Benchmark #​1: ../timing
  Time (mean ± σ):     149.6 ms ±   4.8 ms    [User: 24.2 ms, System: 18.7 ms]
  Range (minmax):   134.9 ms155.4 ms    21 runs 

I hope I'm not misusing the API or measuring the wrong thing...

@typesanitizer
Copy link
Author

@swift-ci create

@typesanitizer
Copy link
Author

Simplified example, without using Git.

import Foundation
func runCommand() throws {
    let task = Process()
    task.executableURL = URL(fileURLWithPath: "/usr/bin/true")
    try task.run()
    task.waitUntilExit()
}
try runCommand()
$ hyperfine '../timing.swift'
Benchmark #​1: ../timing
  Time (mean ± σ):      82.8 ms ±   4.3 ms    [User: 9.9 ms, System: 7.2 ms]
  Range (minmax):    68.1 ms89.6 ms    42 runs

@typesanitizer
Copy link
Author

Added screenshot for Instruments results for the /usr/bin/true example. (I added a loop that ran it ~100 times to do the Instruments capture.)

@typesanitizer
Copy link
Author

Testing similar Rust code (according to the stack trace, it is also calling __posix_spawn) gives a 10x speedup.

// 0.66 seconds
use std::process::Command;
fn main() {
 for _ in 0 .. 100 {
   let _ = Command::new("/usr/bin/true").status();
 }
}
// 7.01 seconds
import Foundation
func runCommand() throws {
  let task = Process()
  task.executableURL = URL(fileURLWithPath: "/usr/bin/true")
  try task.run()
  task.waitUntilExit()
}
for i in 1 ..< 100 {
  try runCommand()
}

@typesanitizer
Copy link
Author

After talking to Karoy and David, this seems to be down to waitUntilExit. Using terminationHandler is much faster (at the cost of having to CPS your code). Or one can use posix_spawn + wait directly, although that's a bit more dangerous and clunky. There is a nice posix_spawn example on Stack Overflow (in C++) here: https://stackoverflow.com/a/27328610/2682729 .

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant