Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-1281] Code coverage testing with custom toolchains in Xcode doesn't work #43889

Open
swift-ci opened this issue Apr 21, 2016 · 5 comments
Open
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. code coverage Area → source tooling: code coverage off topic Resolution: Is beyond the scope of the Swift project (Xcode, proprietary Apple frameworks, etc.) source tooling Area: IDE support, SourceKit, and other source tooling swift 5.7 unexpected error Bug: Unexpected error Xcode

Comments

@swift-ci
Copy link
Collaborator

Previous ID SR-1281
Radar None
Original Reporter rfdickerson (JIRA User)
Type Bug

Attachment: Download

Environment

swift-DEVELOPMENT-04-12-a, Mac OS X 10.11

Additional Detail from JIRA
Votes 3
Component/s Compiler
Labels Bug, CodeCoverage
Assignee None
Priority Medium

md5: 832192ff9d7246fb2eec64e5038f4b79

Issue Description:

When I add Code Coverage testing to the XCode project generated with 'swift build -X' then following error occurs:

I have also tried adding emiting profiling and coverage to the project with

`swift build -Xcc -fprofile-instr-generate -Xcc -ftest-coverage`

But that did not work, either

@belkadan
Copy link
Contributor

I don't think we support code coverage generation with custom toolchains, because the version of LLVM used in the toolchain is out of sync with the version in Xcode. @vedantk, anything to add?

@belkadan
Copy link
Contributor

I suppose Xcode could provide a better error message. Mind filing a Radar?

@swift-ci
Copy link
Collaborator Author

Comment by Robert F. Dickerson (JIRA)

You are right regarding the wrong version of llvm being used. I can get a default.profraw file from building with:

swift build -Xswiftc -profile-generate -Xswiftc -profile-coverage-mapping

Next, I run the tests

swift test

Next, using the LLVM branch used in the swift compiler, I can run

/Ninja-ReleaseAssert/llvm-macosx-x86_64/bin/llvm-profdata merge -o testprof.profdata default.profraw

The next step of viewing the data using llvm-cov is not working for me.

@ddunbar
Copy link
Member

ddunbar commented Jul 26, 2016

This is unfortunate for SwiftPM's Xcode project generation because it blocks us from enabling code coverage by default in the generated schemes (since we expect many of users to frequently be using toolchains, at least for now).

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@hamishknight
Copy link
Collaborator

We ought to be able to use the llvm-profdata and llvm-cov binaries in the custom toolchain, as they are included in development snapshots. Currently Xcode 14 has a different failure mode for this, it just doesn't show the coverage in the editor. We're tracking this internally as rdar://25754212.

@hamishknight hamishknight changed the title [SR-1281] Code coverage testing in generated XCode projects do not work [SR-1281] Code coverage testing with custom toolchains in Xcode doesn't work Sep 20, 2022
@AnthonyLatsis AnthonyLatsis added source tooling Area: IDE support, SourceKit, and other source tooling Xcode off topic Resolution: Is beyond the scope of the Swift project (Xcode, proprietary Apple frameworks, etc.) unexpected error Bug: Unexpected error swift 5.7 and removed compiler The Swift compiler in itself labels Apr 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. code coverage Area → source tooling: code coverage off topic Resolution: Is beyond the scope of the Swift project (Xcode, proprietary Apple frameworks, etc.) source tooling Area: IDE support, SourceKit, and other source tooling swift 5.7 unexpected error Bug: Unexpected error Xcode
Projects
None yet
Development

No branches or pull requests

5 participants