Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-12303] Swift Package Manager package stopped working after Xcode 11.3, Xcode 12, etc. #4581

Open
swift-ci opened this issue Mar 1, 2020 · 43 comments
Labels

Comments

@swift-ci
Copy link
Contributor

swift-ci commented Mar 1, 2020

Previous ID SR-12303
Radar rdar://problem/59976659
Original Reporter freak4pc (JIRA User)
Type Bug

Attachment: Download

Environment

Xcode Env:
Swift 5.1.3
Xcode 11.3 / 11.4b2
Dependency is RxSwift 5.0.1 (Default product type which should be static)

CLI Env:
ZSH
Swift 5.1.3
CLI Only (Package.swift)
Dependency is RxSwift 5.0.1 (Specific branch altered to have all products dynamic)

Additional Detail from JIRA
Votes 50
Component/s Package Manager
Labels Bug
Assignee None
Priority Medium

md5: 5bf2f37058917142c656b336ed4f560c

Issue Description:

I have two different and similar issues with SPM that seem to be a regression which renders SPM unusable for our project consumers.

A project I'm maintaining, RxSwift, is not working for consumers using SPM in basically any scenario since Xcode 11.3 and the Swift version attached to it (I believe 5.1)

When running in Xcode using the built-in dependency manager, using either a dynamic or static product from RxSwifit doesn't work (see attached reproduction RxSPM.zip and RxSPMDynamic.zip). They build, but crash as soon as you run the unit test (or sometimes even the project itself).

This is the static setup, but the dynamic one has the same associations:

I also tried running in CLI mode (e.g. making a Package.swift manually), which works fine with static, but crashes in the same way for a dynamic product (see attached RxSPMCLI.zip).

Would appreciate your help on this🙂

Thanks,
Shai.

@hborla
Copy link
Member

hborla commented Mar 3, 2020

@swift-ci create

@swift-ci
Copy link
Contributor Author

Comment by Shai Mishali (JIRA)

Still happening on Xcode 11.4 & Swift 5.2

@swift-ci
Copy link
Contributor Author

Comment by Shai Mishali (JIRA)

After checking with a few people, this works in Xcode 11.3 and stopped working in Xcode 11.4 (Still not working in 11.4.1).
This is not a minor issue and is effecting at least a dozen large-scaled open source projects that are depended upon by many developers.

@swift-ci
Copy link
Contributor Author

Comment by Shai Mishali (JIRA)

See also: pointfreeco/swift-composable-architecture#70

@swift-ci
Copy link
Contributor Author

Comment by Shai Mishali (JIRA)

This is still an issue in Xcode 11.5. I'm not sure how this is labeled Medium and not Critical given how widely this affects many libraries and use cases.

The thing a dependency manager solves (resolving dependencies and making sure you don't have duplicate symbols of the same things if there are many consumers of them) is literally not working. This is a community-wide issue of dozens of popular frameworrks, for example RxSwift which is one of the top most popular open-source Swift projects, or Pointfreeco's Composable Architecture who hacked around this by dunamically loading XCTest using NSClassFromString (since they only use XCTFail, this is specific to their use case).

Could anyone estimate if this is anywhere on the SPM team's radar to fix? I think the severity here is higher than you are assuming of it to be.

Thank you,
Shai.

@swift-ci
Copy link
Contributor Author

Comment by Marius Landwehr (JIRA)

This is blocking from using SPM with Xcode in my current Project setup. I would really like to use Swift Package Manager but this is a show stopper for me...

@swift-ci
Copy link
Contributor Author

swift-ci commented Jul 1, 2020

Comment by Brian Drelling (JIRA)

I spoke with a member of the Xcode SPM integration team during WWDC 2020 to bring up this issue and was pointed to the ticket, so this is definitely in front of them and they are aware it's Critical. I don't know why Jira doesn't reflect that. I also don't know how this is not immediately addressed in 11.6 or 12.0 because it's killing us and going to cause a ton of sloppy workarounds to pop up if it isn't addressed fast.

@swift-ci
Copy link
Contributor Author

swift-ci commented Jul 1, 2020

Comment by Marius Landwehr (JIRA)

Yep did a Lab too and I was asked to write a Feedback with much as possible information to tackle this problem. They are well aware of this and it's a general Problem in SwiftPM. A timeline for this wasn't adressed at all. I don't think this will be available with Xcode 12

@swift-ci
Copy link
Contributor Author

swift-ci commented Jul 1, 2020

Comment by Shai Mishali (JIRA)

I'm not sure they view it as critical given I opened this issue three months ago ...
But I talked to some of the SPM members and I know they're aware of it. I'm just not sure what's the resolution plan, and more importantly, the timeline.
:-(

@stephencelis
Copy link
Contributor

I filed FB7841039 after finding that this bug prevents me from using Xcode 12's well-advertised SPM Playground support. Hopefully we see movement on this soon.

@swift-ci
Copy link
Contributor Author

Comment by Shai Mishali (JIRA)

Still happening on Xcode 11.6 and Xcode 12 beta 3. The amount of issues in many projects I'm managing regarding SPM is staggering 🙁

@swift-ci
Copy link
Contributor Author

swift-ci commented Aug 8, 2020

Comment by Shai Mishali (JIRA)

Still happening on Xcode 12 beta 4. I'm wondering if there's any chance we could get some clarity and transparency around the plan for this very critical issue.

@swift-ci
Copy link
Contributor Author

Comment by Brian Drelling (JIRA)

Still happening on Xcode 12 beta 5.

It's also still not in the release notes.

This feels really ridiculous... please at least acknowledge that it is a known issue and provide workarounds. There are a number of workarounds flying around, but having recommended workarounds listed on the release notes will help people standardize and work past them in the meantime.

@swift-ci
Copy link
Contributor Author

Comment by Shai Mishali (JIRA)

Still happening on Xcode 12 beta 6. It would be really helpful, and respectful, to get some visibility around this issue as it's critical to many projects.

@swift-ci
Copy link
Contributor Author

Comment by Marius Landwehr (JIRA)

freak4pc (JIRA User) any news on the GM Seed?

@swift-ci
Copy link
Contributor Author

Comment by Marius Landwehr (JIRA)

Still happening on Xcode Version 12.0 (12A7209). Nothing new here...

@swift-ci
Copy link
Contributor Author

Comment by Shai Mishali (JIRA)

I guess we're stuck with this bug for another solid year 🙁
How heartbreaking.

@swift-ci
Copy link
Contributor Author

swift-ci commented Oct 6, 2020

Comment by Shai Mishali (JIRA)

It has been over 7 months and a GM release since we filed this critical bug and there is still no official response from anyone working on this.
We are releasing RxSwift 6 today and I've had to already close over 20 different GitHub issues so far due to this bug; and we have nothing to do but send them to this SR.

This is really disappointing. Is there any way we could get some recognition of the bug plus some theoretical ETA to fix it? @rballard @neonichu @aciidb0mb3r

@swift-ci
Copy link
Contributor Author

Comment by Alberto López (JIRA)

We have the same problem. My xcode version is 12.3(12C33) and still not working

@swift-ci
Copy link
Contributor Author

Comment by Brian Drelling (JIRA)

I still don't understand how this hasn't been acknowledged or addressed, nor why this is marked as "Priority: Medium". It's responsible for a number of issues, headaches, troubleshooting, and workarounds in most (if not all) of the projects I'm involved with. Not just that, but the projects that do function have the most convoluted dependency graphs now because of this that it becomes unnecessarily difficult to troubleshoot.

@swift-ci
Copy link
Contributor Author

Comment by Shai Mishali (JIRA)

Teams and clients I'm involved with have dropped using SPM.

From the business perspective alone, it doesn't make sense investing effort in a product where you have no technical response for what I would consider a critical bug, for a year.

We will not be using SPM and I recommend anyone I know to not use SPM since it's basically acting as a startup beta product and not a mature Apple-backed product, in my opinion (unfortunately).

@swift-ci
Copy link
Contributor Author

Comment by Lluis Gerard (JIRA)

How is this left unresolved for more than a year? What workarounds are people using, if any?

@typesanitizer
Copy link

Can you verify with Xcode 12.5 beta 2 or newer? Here's the relevant part of the release notes:

The Swift Package Manager now builds package products and targets as dynamic frameworks automatically, if doing so avoids duplication of library code at runtime. (59931771) (FB7608638)

@Mordil
Copy link

Mordil commented Apr 21, 2021

This seems to have been fixed in 12.5 RC (12E262) where the Package's product definition is unspecified if it's dynamic or static.

When setting it to dynamic this fails to work still. When leaving it unspecified Xcode is able to build and run the app & unit tests just fine.

Even in the supplied dynamic projects, I changed the package resolution from the 5.1-spm-dynamic branch to the 5.1.0 tag and things started working again.

@swift-ci
Copy link
Contributor Author

Comment by Simon Whitty (JIRA)

I can confirm that 12.5 RC (12E262) fixes the runtime errors for me, but a compile time error exists with multiple project file setups:

SPMTest.zip

                                  ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐  
                                      Package.swift      
                                  │                   │  
                                     ┌─────────────┐     
┌───────────────────────┐     ┌───┼─▶│   RxSwift   │ ◀┼─┐
│                       │     │      └─────────────┘    │
│       CommonKit       │─────┤   │         ▲         │ │
│                       │     │             │           │
└───────────────────────┘     │   │  ┌─────────────┐  │ │
            ▲                 └─────▶│   RxRelay   │    │
            │                     │  └─────────────┘  │ │
            │                               ▲           │
┌───────────────────────┐         └ ─ ─ ─ ─ ┼ ─ ─ ─ ─ ┘ │
│                       │                   │           │
│      Application      │───────────────────┴───────────┘
│                       │                                
└───────────────────────┘   

Clean builds fail with compile error — No such module 'RxSwift'

It appears the build system fails to copy RxSwift.swiftmodule into $(BUILT_PRODUCTS_DIR)

(raised FB9084743)

There are two workarounds that I mention here for those interested:

1. Manually build CommonKit first, then build Application and it will succeed

or

2. Add Build Setting

FRAMEWORK_SEARCH_PATHS = $(inherited) $(BUILT_PRODUCTS_DIR)/PackageFrameworks

@swift-ci
Copy link
Contributor Author

Comment by Kiel (JIRA)

Thanks for posting your workarounds, simon.whitty (JIRA User)! They helped me today. I used the second solution because it seemed less invasive, risky and prone to failure or regression. The only disadvantage is that if you switch from testing a scheme to building and running it, you'll end up with duplicate symbol logs at runtime.

The exact `No such module 'RxSwift'` occurred for me today but the setup was slightly different for me. I have an app hosting unit tests with two or three levels of dependencies, some of which depend on the RxSwift package and its non-test products like RxRelay and RxCocoa. The unit test bundle depends on the RxSwift package but it only depends on the RxTest and RxBlocking products.

Build the app scheme and it succeeds. Test the app scheme and the building the app target (as opposed to the unit test bundle target) fails because the compiler cannot find the RxSwift.framework in the built products directory (it is instead located in the PackageFrameworks subdirectory).

@swift-ci
Copy link
Contributor Author

Comment by Nicolas Fontaine (JIRA)

Hi,

Thank you for the workarounds, it really helped me understand where the problem exactly was with SPM.

I'm having the same issue here. Hope it's gonna be fixed soon. I'm gonna try and compile a lite library importing RxSwift automatically before my main app so the .swiftmodule folder is in the right spot. If you have any tip for how to automatise this please let me know as using xcodebuild CLI on a pre-build phase will generate another folder for the build in the derivedData... 🙁

Thanks !

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 4, 2022
@tobischw
Copy link

This bug still appears to be around in Xcode 14 (14A309). Ridiculous.

@neonichu
Copy link
Member

neonichu commented Sep 22, 2022

Trying to summarize this thread since it is unclear to me what "this bug" means since there a multiple problems mentioned:

  • there's the original issue (FB7608638) - duplicated symbols, fixed in 12.5
  • an issue with packages in Playgrounds (FB7841039) which might be related and is unresolved
  • there's the follow-up issue (FB9084743) - No such module 'RxSwift', which continued to affect certain projects until it was resolved in 13.0

Which of these issues are we talking about here as still occurring in 14A309 and if you're not talking about FB7841039, is there an example project I could take a look at?

@tobischw
Copy link

tobischw commented Sep 22, 2022

I believe it may still be related to your last bullet point - FB9084743. Trying to use RxSwift within a test target yields No such module 'RxCocoaRuntime' - none of the mentioned workarounds seemed to resolve it. Granted, I am not sure if this is an issue with RxSwift, but this seems relevant: ReactiveX/RxSwift#2057

I'll try and get an example project for you. Thanks for looking into this.

@TLizer
Copy link

TLizer commented Dec 1, 2022

@tobischw I just stumbled on similar issue: building unit tests yielded missing module XYZ, when I linked missing module to test target then another showed up etc...
It happened that issue lied in TEST_HOST build setting which was incorrect.
For some reason xcode creates path to test host as:

TEST_HOST = "$(BUILT_PRODUCTS_DIR)/<ProductName>.app/$(BUNDLE_EXECUTABLE_FOLDER_PATH)/<ProductName>

After checking resolved value for TEST_HOST it was:

"<PathToProducts>/<ProductName>.app//<ProductName>

Which means $(BUNDLE_EXECUTABLE_FOLDER_PATH) is empty string or non existing.
Correcting this build setting manually to remove $(BUNDLE_EXECUTABLE_FOLDER_PATH)/ fixed issue for me and now tests are compiling and running on pure SPM project :)

@minguking
Copy link

minguking commented Dec 4, 2022

if there's anyone still suffering from this error by using RxSwift as SPM, this was the solution in my case, very similar to the solution right above me, but slight difference.
try to fix TEST_HOST of your Tests target
from
TEST_HOST = "$(BUILT_PRODUCTS_DIR)/<ProductName>.app/$(BUNDLE_EXECUTABLE_FOLDER_PATH)/<ProductName>
To
TEST_HOST = "$(BUILT_PRODUCTS_DIR)/<ProductName>.app/<ProductName>
so, just remove the /$(BUNDLE_EXECUTABLE_FOLDER_PATH) part. that's it.

@ncreated
Copy link

👋 We stepped across the same problem in Datadog SDK. Our Datadog (Swift) module depends on _Datadog_Private (Objective-c) one. Even though the modulemap for both is generated properly, the "SwiftEmitModule" build step is unable to find _Datadog_Private.modulemap for the app tests target if the project was created with Xcode 14.x.

Root cause seems to be the TEST_HOST path being wrong as @TLizer explains above. We explained more in DataDog/dd-sdk-ios#1085 - we're happy to give more details 🐶💜🍎, although the reproduction steps are quite straightforward.

It would be great to know if FB9084743 was classified as an SPM / Xcode bug 🐞 or if vendors like us should work on workarounds on their side.

@neonichu
Copy link
Member

As far as I can see, FB9084743 is considered to be fixed and the TEST_HOST issue may not be tracked in FBs at all, yet.

@ncreated
Copy link

Thanks for reply @neonichu. I just filled FB11875280 to report the TEST_HOST problem (rdar://FB11875280).

@jsryudev
Copy link

jsryudev commented Mar 29, 2023

This issue still exist in Xcode 14.3 (14E222b).
To avoid this issue, we need to use a solution from below.

@vani2
Copy link

vani2 commented Aug 9, 2023

The solutions above don't work for me at Xcode 14.3.1.

@neonichu
Copy link
Member

Would suggest filing feedback reports if this is still happening. FB11875280 is also still open as far as I can see.

@tommyming
Copy link

tommyming commented Sep 21, 2023

The same problem still exists, using Xcode 15 (15A240d), used the workaround solution below, trying to import RxSwift and RxTest to the unit test target.

This issue still exist in Xcode 14.3 (14E222b). To avoid this issue, we need to use a solution from below.

Tried to remove the mentioned packages (RxSwift, RxTest) above, but the same error occurred with different package names.

Below is the build result of the test target, suppose I don't need those libraries, but Xcode still detects them as required modules.
Screenshot 2023-09-21 at 12 43 40 PM

Please feel free to correct me if there is anything wrong, and I am glad to help, thanks.

@neonichu
Copy link
Member

Same thing I said in August still applies:

Would suggest filing feedback reports if this is still happening. FB11875280 is also still open as far as I can see.

@tommyming
Copy link

Will do that immediately, thanks for the reply!
If there are any updates please let me know also.

@skv6631
Copy link

skv6631 commented Feb 19, 2024

Any fix for this. Still getting this for DataDog library. @ncreated

@MaxDesiatov
Copy link
Member

@skv6631 is the issue reproducible for you when building on the command line with swift build?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests