New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SR-1355] [XCTest] Support XCTest performance testing API #389
Labels
Comments
I believe the following methods make up the XCTest performance testing API: class XCTestCase : XCTest {
/// The names of the performance metrics to measure when invoking `XCTestCase.measure()`.
/// Returns `XCTPerformanceMetric_WallClockTime` by default. Subclasses can override this to
/// change the behavior of `XCTestCase.measure()`.
class func defaultPerformanceMetrics() -> [String]
/// Call from a test method to measure resources (`XCTestCase.defaultPerformanceMetrics()`)
/// used by the block in the current process.
/// func testPerformanceOfMyFunction() {
/// measure {
/// // Do that thing you want to measure.
/// myFunction()
/// }
/// }
/// - Parameter block: A closure whose performance to measure.
func measure(_ block: () -> Void)
/// Call from a test method to measure resources (XCTPerformanceMetrics) used by the
/// block in the current process. Each metric will be measured across calls to the block.
/// The number of times the block will be called is undefined and may change in the
/// future. For one example of why, as long as the requested performance metrics do
/// not interfere with each other the API will measure all metrics across the same
/// calls to the block. If the performance metrics may interfere the API will measure
/// them separately.
///
/// func testMyFunction2_WallClockTime() {
/// measureMetrics(self.dynamicType.defaultPerformanceMetrics(), automaticallyStartMeasuring: false) {
/// // Do setup work that needs to be done for every iteration
/// // but you don't want to measure before the call to `XCTestCase.startMeasuring()`.
/// setupSomething()
/// startMeasuring()
/// // Do that thing you want to measure.
/// myFunction()
/// stopMeasuring()
/// // Do teardown work that needs to be done for every iteration
/// // but you don't want to measure after the call to `XCTestCase.stopMeasuring()`.
/// teardownSomething()
/// }
/// }
///
/// Caveats:
/// • If true was passed for `automaticallyStartMeasuring` and `XCTestCase.startMeasuring()`
/// is called anyway, the test will fail.
/// • If false was passed for `automaticallyStartMeasuring` then `XCTestCase.startMeasuring()`
/// must be called once and only once before the end of the block or the test will fail.
/// • If `XCTestCase.stopMeasuring()` is called multiple times during the block the test will fail.
///
/// - Parameter metrics: An array of NSStrings (XCTPerformanceMetrics) to measure. Providing an unrecognized string is a test failure.
/// - Parameter automaticallyStartMeasuring: If false, XCTestCase will not take any measurements until `XCTesrCase.startMeasuring()` is called.
/// - Parameter block: A closure whose performance to measure.
func measureMetrics(_ metrics: [String], automaticallyStartMeasuring automaticallyStartMeasuring: Bool, for block: () -> Void)
/// Call this from within a measure block to set the beginning of the critical section.
/// Measurement of metrics will start at this point.
func startMeasuring()
/// Call this from within a measure block to set the ending of the critical section.
/// Measurement of metrics will stop at this point.
func stopMeasuring()
}
/// Records wall clock time in seconds between startMeasuring/stopMeasuring.
let XCTPerformanceMetric_WallClockTime: String The documentation above is based on the latest Apple headers. |
PR: #109 |
This was pretty much finished in #109 – thanks, @briancroom! Please open JIRA issues for anything not finished in the PR and worth tracking. 🙂 |
Awesome, thanks!! |
I've created https://bugs.swift.org/browse/SR-1609 as a follow-up task to this. |
This issue was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Additional Detail from JIRA
md5: 14030175159ae9057e04dccb414023e0
relates to:
Issue Description:
swift-corelibs-xctest should support the performance testing APIs.
Related to:
https://bugs.swift.org/browse/SR-1354
The text was updated successfully, but these errors were encountered: