Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-7094] Swift 4.1(?) performance regression #49642

Closed
jepers opened this issue Feb 28, 2018 · 4 comments
Closed

[SR-7094] Swift 4.1(?) performance regression #49642

jepers opened this issue Feb 28, 2018 · 4 comments
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. performance regression standard library Area: Standard library umbrella swift 4.1

Comments

@jepers
Copy link

jepers commented Feb 28, 2018

Previous ID SR-7094
Radar rdar://problem/38030106
Original Reporter @jepers
Type Bug
Status Closed
Resolution Done
Environment

macOS 10.13.3 (17D102), Xcode 9.3 beta 3 (compared to Xcode 9.2)

Additional Detail from JIRA
Votes 0
Component/s Standard Library
Labels Bug, 4.1Regression, Performance
Assignee None
Priority Medium

md5: 9352b8912f15a18329e98931c10d4b9a

Issue Description:

EDIT: As the original code example was not a very good representation of the issue, I'll add this demo program:

// In order to reproduce/understand the issue meant to be
// demonstrated by this program:
//
// Note the difference (in reported time measurements) of declaring
// `let a = Double(1)` at top level or within test()
//
// Also note that `a.binade` is a constant, which I'd expect the optimizer to
// hoist out of the loop, just like it does if we use eg `.magnitude` instead
// of `a.binade`.
//
// Examine the effects of the 2*2 = 4 ways of combining these modifications.
//
// I'm using Development Snapshot 2018-05-30.
// The issues are worse and different with eg def toolchain of Xcode 9.4.

import AppKit

let a = Double(1) // <-- See diff in times when this is within test() instead.

func test() {
    for _ in 0 ..< 5 {
        var checksum = UInt64(0)
        let t0 = CACurrentMediaTime()
        for _ in 0 ..< 1_000_000_000 {
            let v = a.binade // <-- Compare with eg `.magnitude`.
            checksum = checksum &+ v.bitPattern
        }
        let t1 = CACurrentMediaTime()
        print(t1 - t0, "seconds, checksum:", checksum)
    }
}
test()

As noted in the edit above, the following program is the original demo (but it is misleading in that it suggests that 0.01 s is the goal time, the real goal time (on my machine) should be 4e-08 s, which is thousands of times faster.):

//-----------------------------------------------------------------------
// test.swift - Demonstrating a Swift 4.1(?) performance regression
//-----------------------------------------------------------------------
//
// This program demonstrates a huge increase in execution time when
// compiled with recent versions of the compiler. 
//
// Here's what I see on my MacBook Pro, 2 GHz Intel Core i7:
//
// Compiling with Xcode 9.2:
//
// › sudo xcode-select -s /Applications/Xcode.app/
// › xcrun swiftc --version
// Apple Swift version 4.0.3 (swiftlang-900.0.74.1 clang-900.0.39.2)
// Target: x86_64-apple-macosx10.9
// › xcrun swiftc -O -whole-module-optimization -static-stdlib test.swift
// › ./test
// 0.0111292660003528 seconds, checksum: 6917529027641081856
// 0.0125824180140626 seconds, checksum: 13835058055282163712
// 0.0109889880113769 seconds, checksum: 2305843009213693952
// 0.00988745299400762 seconds, checksum: 9223372036854775808
// 0.0114470749977045 seconds, checksum: 16140901064495857664
//
// Compiling with Xcode 9.3 beta 3:
//
// › sudo xcode-select -s /Applications/Xcode-beta.app/
// › xcrun swiftc --version
// Apple Swift version 4.1 (swiftlang-902.0.41 clang-902.0.31)
// Target: x86_64-apple-darwin17.4.0
// › xcrun swiftc -O -whole-module-optimization -static-stdlib test.swift
// › ./test
// 5.08630971799721 seconds, checksum: 6917529027641081856
// 5.10243050198187 seconds, checksum: 13835058055282163712
// 5.09426819198416 seconds, checksum: 2305843009213693952
// 5.06536405498628 seconds, checksum: 9223372036854775808
// 5.08944137202343 seconds, checksum: 16140901064495857664
//
// (The results are the same when doing release builds within Xcode,
//  and I have not tested any versions of the compiler between the default
//  toolchains of Xcode 9.2 and Xcode 9.3 beta 3. Recent dev snapshots
//  show approximately the latter (slow) results).
//
// Note that the particulars (for-loop, `Double`'s `.binade`, etc) of
// this program have been chosen just because they make for a simple
// demonstration, and that the situation where I first encountered the 
// broader issue that I guess is behind this example, was very different,
// and of course not as contrived.
//
// You can for example replace `Double` with `Float` and `.binade` with
// `.nextUp`, `.ulp` or `.significand` and see that the results are
// pretty much the same, but note that eg `.magnitude` and `.squareRoot()`
// are not affected by this regression. I would guess that many other
// types, properties and methods would demonstrate similar erratic
// behavior of the optimizer.
//
// From my perspective as humble Swift programmer, I'd like to be able to
// trust the compiler to for example hoist `let v = a.significand` out of
// the for-loop for me, computing it only once, since it should be possible
// for the compiler to know that it is a constant value. But note that this
// isn't happening even in the "fast" Xcode 9.2 version of the compiler
// (since manually moving `let v = a.significand` out of the inner for-loop
// will further improve the reported times, from 0.01 seconds to around
// 1.0e-08 seconds). But I guess this can't be generally expected until
// Swift offers a proper way to mark and track side effects.
//
// Anyway, I'd like to at least be able to trust a new version of the
// compiler to not suddenly make some of my code 500 times slower!
//
// Therefore I'm wondering if there are any unit tests that catches this
// sort of performance regressions? Perhaps they are not gone through
// until the full testing that is performed for official releases?
//
// Related discussion here:
// https://forums.swift.org/t/missed-optimization-opportunity/10381
//-----------------------------------------------------------------------

import AppKit

let a = Double(1)
var checksum = UInt64(0)
for _ in 0 ..< 5 {
    let start = CACurrentMediaTime()
    for _ in 0 ..< 1_000_000_000 {
        let v = a.binade
        checksum = checksum &+ v.bitPattern
    }
    let stop = CACurrentMediaTime()
    print(stop - start, "seconds, checksum:", checksum)
}
@belkadan
Copy link
Contributor

belkadan commented Mar 1, 2018

cc @moiseev, @stephentyrone

@moiseev
Copy link
Mannequin

moiseev mannequin commented Mar 1, 2018

@swift-cicreate

@belkadan
Copy link
Contributor

Well, I did some things based on the analysis in the Radar, but I don't think they directly addressed the perf regression. It's at least down to ~0.55s on master, but not ~0.01s.

@jepers
Copy link
Author

jepers commented May 31, 2018

Turns out that the way the demo program is formulated above is not representing the issue very well.

NOTE: The time we should be looking for (when the optimizer hoists calculation of .binade out of the loop) is around 4e-08 seconds, rather than 0.01 seconds (which is thousands of times slower).

This forum post will hopefully make it clear what I mean (although it uses .nextDown instead of .binade).

I have also edited the description of this issue, adding a better demo program, since the original demo program is misleading in that it suggests that 0.01 s is the goal time.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. performance regression standard library Area: Standard library umbrella swift 4.1
Projects
None yet
Development

No branches or pull requests

3 participants