Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-6004] Static libraries don't automatically load extensions #48561

Closed
swift-ci opened this issue Sep 26, 2017 · 20 comments
Closed

[SR-6004] Static libraries don't automatically load extensions #48561

swift-ci opened this issue Sep 26, 2017 · 20 comments
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. compiler The Swift compiler in itself

Comments

@swift-ci
Copy link
Collaborator

Previous ID SR-6004
Radar rdar://problem/34686622
Original Reporter soffes (JIRA User)
Type Bug
Status Resolved
Resolution Done

Attachment: Download

Environment

Xcode 9.0 (9A235)

Additional Detail from JIRA
Votes 7
Component/s Compiler
Labels Bug
Assignee None
Priority Medium

md5: a88822ddd333e2719ad59655283f788c

is duplicated by:

  • SR-10507 Dynamic cast to existential fails when conformance is added in extension in static library

Issue Description:

I'm not sure if this is the intended behavior, but a static library doesn't load extensions unless something else in the same file is explicitly referenced. I realize adding `all_load` to the target it is linked into solves this. I'd love to avoid that if possible.

This is more of a question of is this the indeed behavior and if so is doing `all_load` the intended use?

I've attached a sample project reproducing this. Here's an explanation:

  1. Create a static library A containing a protocol and some types that conform to it

  2. Create another static library B that creates some another protocol and adds extensions to types in static library A

  3. In another target that links both A & B, you can observe types from A that conform to a protocol from B not doing so at runtime.

  4. If you explicitly reference something in a file that adds conformation, it will load the extension with the conformance and work as expected.

The attached sample project may make this more clear. Start in ViewController.swift and check out the comments.

@belkadan
Copy link
Contributor

Heh, we have -ObjC to solve this for categories, but that's not the same thing. @jckarter, @gparker42, thoughts?

@gparker42
Copy link
Mannequin

gparker42 mannequin commented Sep 27, 2017

You can use -force_load to get load-everything semantics for a single archive, which is better than -all_load.

-ObjC might just work. The linker will pull in the ObjC category representing the extension, and that might be enough to pull in the rest.

@belkadan
Copy link
Contributor

Not all Swift extensions have corresponding categories, and the protocol conformance info probably isn't attached. I suppose we could start vending dummy categories…

@gparker42
Copy link
Mannequin

gparker42 mannequin commented Sep 27, 2017

If I recall correctly there is some symbol attribute that the compiler can add which tells the linker to pull in that definition unconditionally. Let me look around.

@keith
Copy link
Collaborator

keith commented Sep 27, 2017

There definitely is, at some point I was actually trying to debug an issue with something like that, you can see when they're marked with that attribute using `nm -m`. Example:

% nm -m ./Library.o
...
00000000000002b8 (__DATA,__const) non-external [no dead strip] __TMfV7Library3Foo
...

@gparker42
Copy link
Mannequin

gparker42 mannequin commented Sep 27, 2017

I talked to the linker guys and there is not. Symbols can be marked no-dead-strip, but the lazy-loading semantics of static archives do not honor that. We may need to get linker enhancements if we need a fix for Swift there.

The usual solution is to avoid static archives. Instead, use ld -r to combine your .o files into one big .o file, then dead-strip that. That will remove some of the unused code that you get from static archive linking while also honoring no-dead-strip symbols.

@belkadan
Copy link
Contributor

@swift-ci create

@swift-ci
Copy link
Collaborator Author

swift-ci commented Feb 6, 2019

Comment by Michael Bachand (JIRA)

We are experiencing this issue at Airbnb with Buck-generated Xcode projects that use Swift static libraries. We plan to use @objc (paired with the -ObjC "Other Linker Flag") to work around it.

@swift-ci
Copy link
Collaborator Author

swift-ci commented Feb 6, 2019

Comment by Michael Bachand (JIRA)

Here is a repro in a Buck-generated project: airbnb/BuckSample#38

@keith
Copy link
Collaborator

keith commented May 23, 2019

@belkadan @gparker42 any idea on how we can move forward to a fix for this one? I'd be willing to take a stab at some implementation for this if possible

@AnnaZaks
Copy link
Mannequin

AnnaZaks mannequin commented Aug 9, 2019

soffes (JIRA User), Could you verify if the problem is fixed in the latest Xcode betas and if so move the JIRA to "Closed"?

Thanks!
Anna

@keith
Copy link
Collaborator

keith commented Aug 9, 2019

Did something change in Swift that should have fixed this?

@brentleyjones
Copy link
Contributor

In Swift 5.1, per the Xcode release notes, this is now fixed for us. We use -ObjC though, so that might be it.

@AnnaZaks
Copy link
Mannequin

AnnaZaks mannequin commented Aug 9, 2019

There was a linker side change to address this.

@keith
Copy link
Collaborator

keith commented Aug 9, 2019

Ah interesting, I see this in the release notes "Static libraries are now always force-loaded in their entirety during linking, fixing runtime errors related to demangling failures. (47598583)"

On the surface this sounds pretty scary! Since we're intentionally not force loading all of our libraries, only the ones that experience this problem. But I can try to see if it fixed our case as well

@keith
Copy link
Collaborator

keith commented Aug 11, 2019

The sample project attached to this issue actually still has the original problem with Xcode 11 beta 5

@keith
Copy link
Collaborator

keith commented Aug 11, 2019

Setting GENERATE_MASTER_OBJECT_FILE on the Xcode target (as suggested in order to do the `ld -r` combination above) does fix this case.

@belkadan
Copy link
Contributor

The -ObjC in the original project is on the static libraries rather than the client of the static libraries (the app). If I fix that, and tell ModelsUI not to link with Models, it works in Xcode 11 (beta 6, but should be earlier ones too).

I do notice that there's not a dedicated build setting for -ObjC and people have to add it to "Other Linker Flags", so I'll go talk to some linker and Xcode folks about that. That'll be Apple-internal conversations only, though, sorry.

@keith
Copy link
Collaborator

keith commented Aug 24, 2019

@belkadan to clarify, this new swift linking behavior is enabled by the -ObjC flag?

@keith
Copy link
Collaborator

keith commented Aug 24, 2019

Ok after a bit more testing with this project it looks like -ObjC does enable this new behavior? I updated the sample project locally to make sure DogView wasn't a NSObject subclass since adding -ObjC actually fixed this example even in Xcode 10.3, but it looks like that does enable this new Swift loading behavior.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. compiler The Swift compiler in itself
Projects
None yet
Development

No branches or pull requests

4 participants