Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-9872] swift test --generate-linuxmain is not idempotent #4878

Closed
swift-ci opened this issue Feb 6, 2019 · 3 comments
Closed

[SR-9872] swift test --generate-linuxmain is not idempotent #4878

swift-ci opened this issue Feb 6, 2019 · 3 comments
Labels

Comments

@swift-ci
Copy link
Contributor

swift-ci commented Feb 6, 2019

Previous ID SR-9872
Radar None
Original Reporter featherless (JIRA User)
Type Bug
Status Resolved
Resolution Done
Environment

Apple Swift version 4.2.1 (swiftlang-1000.11.42 clang-1000.11.45.1)
Target: x86_64-apple-darwin18.0.0

Additional Detail from JIRA
Votes 0
Component/s Package Manager
Labels Bug
Assignee None
Priority Medium

md5: 3c9a9df66e3c22015e8cf4d060250389

Issue Description:

The problem

`swift test --generate-linuxmain` is a wonderful aid for ensuring that all tests are ran as part of Linux test suites. I've recently added this command to my project's continuous integration in order to detect if any tests are missing from the Linux environment. Unfortunately, it looks like the `swift test --generate-linuxmain` command's output is not guaranteed to be consistent across multiple invocations, resulting in flaky continuous integration jobs.

Specifically, `LinuxMain.swift`'s imports and `tests +=` statements are what appear to shuffle in their order.

This happens when there are multiple test targets in a single package.

Request

Successive invocations of `swift test --generate-linuxmain` with the same code should not result in different output each time. I suspect sorting the output would help.

@ankitspd
Copy link
Member

ankitspd commented Feb 6, 2019

Can you test this with latest trunk snapshot? I think I put in something to make sure we produce the entries in a stable order.

@swift-ci
Copy link
Contributor Author

swift-ci commented Feb 7, 2019

Comment by Jeff Verkoeyen (JIRA)

I tried it with the latest trunk snapshot and it does appear to be generating the tests in alphabetical order now; thank you!

For reference, the output of this test: https://travis-ci.com/jverkoey/MySqlConnector/jobs/175900687

The pull request testing the change: jverkoey/MySqlClient#14

@swift-ci
Copy link
Contributor Author

Comment by Kohki Miki (JIRA)

This feature is already implemented.

So I added a test case.

#2049

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 4, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants