Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-4856] Swift Array copy is very slow #47433

Open
swift-ci opened this issue May 10, 2017 · 14 comments
Open

[SR-4856] Swift Array copy is very slow #47433

swift-ci opened this issue May 10, 2017 · 14 comments
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. performance standard library Area: Standard library umbrella

Comments

@swift-ci
Copy link
Collaborator

Previous ID SR-4856
Radar None
Original Reporter rmann (JIRA User)
Type Bug

Attachment: Download

Environment

Xcode 8.3.2, macOS 10.12.4, on an early 2013 15" Retina MacBook Pro, 2.8 GHz

Additional Detail from JIRA
Votes 2
Component/s Standard Library
Labels Bug, Performance
Assignee None
Priority Medium

md5: ce755e5a51e1a3472fde13818edfce75

Issue Description:

Creating an Array from a 1 MB [UInt8] array takes 3 seconds (2 MB takes 6 s, 3 MB takes 9 s). This seems very, very slow.

import Foundation

let hugeSize = 1 * 1024 * 1024
let array1 = [UInt8](repeating: 0, count: hugeSize)
print("Starting copy")
let startTime = CFAbsoluteTimeGetCurrent()
let array2 = Array(array1)
let endTime = CFAbsoluteTimeGetCurrent()
let dur = endTime - startTime
print("Copy dur: \(dur) s")

Output:

Starting copy
Copy dur: 2.96815097332001 s

This is in Xcode 8.3.2, macOS 10.12.4, on an early 2013 15" Retina MacBook Pro, 2.8 GHz. I've also seen it on an iPad Pro.

@dabrahams
Copy link
Collaborator

I can't reproduce this, even in -Onone:

swiftc -Onone x.swift -o /tmp/x && /tmp/x

Starting copy
Copy dur: 2.40206718444824e-05 s

Are you sure you didn't leave off the e-05 from the result? ;-)

@swift-ci
Copy link
Collaborator Author

Comment by Rick M (JIRA)

Copied-and-pasted directly from Xcode 8.3.2 (8E2002). Attached Playground file.
SlowArrayCopy.playground.zip

@rintaro
Copy link
Mannequin

rintaro mannequin commented May 17, 2017

Running this in Playground is slow because Playground need to display the results.

let array2 = Array(array1) doesn't even force copying in the current stdlib implementation; array1 and array2 still share the buffer.

@swift-ci
Copy link
Collaborator Author

Comment by Rick M (JIRA)

I don't buy that this is slow because it needs to display results. That doesn't really make any sense. Can you explain why this would be the case?

If you want to more closely reporduce the situation I see running on our iOS device, you can take a slice of array1 when making the copy. That's the only difference. It didn't seem necessary for this example since performance is so bad with the example as given.

@dabrahams
Copy link
Collaborator

Rintaro's is the likeliest explanation. As he says, there is basically no computation required to "copy" or to slice an array.

@dabrahams
Copy link
Collaborator

You could definitely consider this an Xcode/playgrounds bug, but then I think you'd want to file a radar as that code is not part of the open source Swift project.

@swift-ci
Copy link
Collaborator Author

Comment by Rick M (JIRA)

So, this first showed up in code on an iOS app (both Simulator and device). I just tried to get a reproducible case in a Playground. But it's not a Playground bug. What else could it be?

@dabrahams
Copy link
Collaborator

rmann (JIRA User) I have no clue, sorry. I don't think the code you're running in your playground is what's slowing down your iOS app though. I suggest you reduce the app until you can isolate the problem.

@swift-ci
Copy link
Collaborator Author

Comment by Rick M (JIRA)

I can promise you it was this line of code:

thing = Array(self.dataBuffer![0 ..< bigSize])

@dabrahams
Copy link
Collaborator

But that line of code is a very different matter. Copying a slice of an array into an Array is O(N), because the Array has a 1-word representation (pointer-to-buffer) and can't represent a slice of some other buffer.

@swift-ci
Copy link
Collaborator Author

Comment by Rick M (JIRA)

So, in my comment a couple hours ago I said "If you want to more closely reporduce the situation I see running on our iOS device, you can take a slice of array1 when making the copy." The response from you was "there is basically no computation required to "copy" or to slice an array," so I assumed you were saying it wouldn't be making the copy.

Nevertheless, it seems to be a very slow copy. I fully expect it to be O(N), but two seconds to copy a megabyte? That sounds like an unoptimized loop, rather than a call to memcpy() (or similar).

@dabrahams
Copy link
Collaborator

So, in my comment a couple hours ago I said "If you want to more closely reporduce the situation I see running on our iOS device, you can take a slice of array1 when making the copy." The response from you was "there is basically no computation required to "copy" or to slice an array," so I assumed you were saying it wouldn't be making the copy.

Sorry, lots going on here; subtle hints can get lost in the noise.

Nevertheless, it seems to be a very slow copy. I fully expect it to be O(N), but two seconds to copy a megabyte? That sounds like an unoptimized loop, rather than a call to memcpy() (or similar).

That does sound very slow. Is this a debug or release build?

@swift-ci
Copy link
Collaborator Author

Comment by Rick M (JIRA)

I do believe it was a debug build. Not sure what the Playground is doing.

@dabrahams
Copy link
Collaborator

That doesn't surprise me too much then. With this program:

import Foundation

func test() {
  let hugeSize = 1 * 1024 * 1024
  let array1 = [UInt8](repeating: 0, count: hugeSize)
  print("Starting copy")
  let startTime = CFAbsoluteTimeGetCurrent()
  let array2 = Array(array1[1..<array1.endIndex])
  let endTime = CFAbsoluteTimeGetCurrent()
  let dur = endTime - startTime
  print("Copy dur: \(dur) s")
  if CommandLine.arguments.count == 1000 {  print(array2) }
}

test()
$ swiftc -Onone x.swift -o /tmp/x && time /tmp/x
Starting copy
Copy dur: 0.262063026428223 s

real    0m0.274s
user    0m0.256s
sys 0m0.004s
$ swiftc -O x.swift -o /tmp/x && time /tmp/x
Starting copy
Copy dur: 0.00106900930404663 s

real    0m0.012s
user    0m0.004s
sys 0m0.003s

If it's any consolation, when I use the current master branch of Swift in -Onone, I get:

Starting copy
Copy dur: 0.00223302841186523 s

real    0m0.014s
user    0m0.006s
sys 0m0.004s

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. performance standard library Area: Standard library umbrella
Projects
None yet
Development

No branches or pull requests

2 participants