Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SR-9222] XCTest performance tests are always much slower on the first measurement #358

Open
swift-ci opened this issue Nov 11, 2018 · 5 comments
Open

[SR-9222] XCTest performance tests are always much slower on the first measurement #358

swift-ci opened this issue Nov 11, 2018 · 5 comments
Labels
bug XCTest

Comments

@swift-ci
Copy link

Previous ID SR-9222
Radar None
Original Reporter bnut (JIRA User)
Type Bug

Attachment: Download

Environment 
Xcode 10.1 (10B61)
MacBook Pro (15-inch, 2016)
2.9 GHz Intel Core i7
16 GB 2133 MHz LPDDR3
Additional Detail from JIRA
Votes 0
Component/s XCTest
Labels Bug
Assignee None
Priority Medium

md5: d535dadca1b8fdfb15ae346fea01370a

Issue Description:

Using the measure block in XCTest always results in a significant overhead on the first measurement.

import XCTest
@testable import xctperf_test


final class xctperf_testTests: XCTestCase {
    func testExample() {
        measure {
            for _ in 0 ... 10_000 {
                XCTAssertEqual("abc", "abc")
            }
        }
    }


    static var allTests = [
        ("testExample", testExample),
    ]
}
@swift-ci
Copy link
Author

Comment by Andrew Bennett (JIRA)

@briancroom this might be something you're interested in, based on other similar tickets.

@briancroom
Copy link
Collaborator

Hi bnut (JIRA User), based on the screenshot, it looks like you are seeing this problem when using Xcode’s XCTest, rather than swift-corelibs-xctest. Could you confirm?

@swift-ci
Copy link
Author

Comment by Andrew Bennett (JIRA)

Yes, that's correct @briancroom - sorry if you were the wrong person to ping.

I created the target with SwiftPM using swift package init which adds some linux support, and then using swift package generate-xcodeproj. I don't think this would lead it to use swift-corelibs-xctest though.

I've verified this by looking at the `XCTest` module, which contains:

public var XCT_UI_TESTING_AVAILABLE: Int32 { get }
//
//  Copyright (c) 2014-2015 Apple Inc. All rights reserved.
//

Which doesn't seem to be in the corelibs version.

@briancroom
Copy link
Collaborator

Hey bnut (JIRA User), sorry for the delay. No worries, I'm a fine person to ask about this!

Yeah, swift-corelibs-xctest is generally only used on non-Apple platforms, so this would be Xcode's version of XCTest that you're using here. I haven't seen this particular behavior before, but it does seem like a bug. Would you be willing to file a Radar about this on https://bugreport.apple.com and attach an Xcode project which reproduces the problem?

@swift-ci
Copy link
Author

Comment by Andrew Bennett (JIRA)

Thanks for the feedback @briancroom, I've created a report at https://bugreport.apple.com/web/?problemID=46153989

Please let me know if you need more information.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug XCTest
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@briancroom @swift-ci