Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-662] swiftpm doesn't correctly order .a files when constructing swiftc args #5431

Closed
swift-ci opened this issue Feb 2, 2016 · 8 comments
Assignees
Labels

Comments

@swift-ci
Copy link
Contributor

swift-ci commented Feb 2, 2016

Previous ID SR-662
Radar None
Original Reporter jaybuff (JIRA User)
Type Bug
Status Closed
Resolution Done

Attachment: Download

Additional Detail from JIRA
Votes 2
Component/s Package Manager
Labels Bug
Assignee @mxcl
Priority Medium

md5: f13d5ad12d638b307720fac644c5388f

relates to:

  • SR-387 Undef. symbols when linking SwiftPM

Issue Description:

It looks like llbuild doesn't consider order to pass .a files to swiftc.

$ swift --version
Swift version 2.2-dev (LLVM 3ebdbb2c7e, Clang f66c5bb67b, Swift 42591f7cba)
Target: x86_64-unknown-linux-gnu

# build a package that depends on the swift package manager
$ cat Package.swift
import PackageDescription

let package = Package(
    name: "swift-ld-bug",
    dependencies: [
        .Package(url: "https://github.com/apple/swift-package-manager.git", Version!("0.1.0")),
    ]
)

# try to use the sys.Path method provided by swift pm
$ cat Sources/main.swift
import sys
let path = Path.join("usr", "bin")
print("path is \(path)")

# the build fails with linker issues: sys can't find methods in libc and POSIX
$ swift build
Compiling Swift Module 'libc' (3 sources)
Compiling Swift Module 'PackageDescription' (3 sources)
Linking Library:  .build/debug/libc.a
Compiling Swift Module 'POSIX' (22 sources)
Linking Library:  .build/debug/POSIX.a
Compiling Swift Module 'sys' (9 sources)
Linking Library:  .build/debug/sys.a
Linking Library:  .build/debug/PackageDescription.a
Compiling Swift Module 'dep' (9 sources)
Linking Library:  .build/debug/dep.a
Compiling Swift Module 'swiftbuild' (3 sources)
Linking Executable:  .build/debug/swift-build
Compiling Swift Module 'swiftldbug' (1 sources)
Linking Executable:  .build/debug/swift-ld-bug
/home/jaybuff/swift-ld-bug/.build/debug/sys.a(Path.swift.o): In function `_TFE3sysSSg10isAbsoluteSb':
/home/jaybuff/swift-ld-bug/Packages/swift-package-manager-0.1.0/Sources/sys/Path.swift:194: undefined reference to `_TFE4libcSS9hasPrefixfSSSb'
/home/jaybuff/swift-ld-bug/.build/debug/sys.a(Path.swift.o): In function `_TZFV3sys4Pathg4homeSS':
/home/jaybuff/swift-ld-bug/Packages/swift-package-manager-0.1.0/Sources/sys/Path.swift:48: undefined reference to `_TF5POSIX6getenvFSSGSqSS_'
/home/jaybuff/swift-ld-bug/.build/debug/sys.a(Path.swift.o): In function `_TFE3sysSS7abspathfzT_SS':
/home/jaybuff/swift-ld-bug/Packages/swift-package-manager-0.1.0/Sources/sys/Path.swift:189: undefined reference to `_TF5POSIX6getcwdFzT_SS'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: build had 1 command failures
error: exit(1): ["/home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/bin/swift-build-tool", "-f", "/home/jaybuff/swift-ld-bug/.build/debug/swift-ld-bug.o/llbuild.yaml"]

# run that build with verbose output and see the swiftc command it's using:
$ swift build -v
/home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/bin/swiftc --driver-mode=swift -I /home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/lib/swift/pm -L /home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/lib/swift/pm -lPackageDescription /home/jaybuff/swift-ld-bug/Package.swift
/usr/bin/git -C /home/jaybuff/swift-ld-bug/Packages/swift-package-manager-0.1.0 config --get remote.origin.url
/usr/bin/git -C /home/jaybuff/swift-ld-bug/Packages/swift-package-manager-0.1.0 config --get remote.origin.url
/home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/bin/swiftc --driver-mode=swift -I /home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/lib/swift/pm -L /home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/lib/swift/pm -lPackageDescription /home/jaybuff/swift-ld-bug/Packages/swift-package-manager-0.1.0/Package.swift
/usr/bin/git -C /home/jaybuff/swift-ld-bug/Packages/swift-package-manager-0.1.0 config --get remote.origin.url
/home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/bin/swiftc --driver-mode=swift -I /home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/lib/swift/pm -L /home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/lib/swift/pm -lPackageDescription /home/jaybuff/swift-ld-bug/Packages/swift-package-manager-0.1.0/Package.swift
/home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/bin/swift-build-tool -v -f /home/jaybuff/swift-ld-bug/.build/debug/SwiftPackageManager.o/llbuild.yaml
/home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/bin/swift-build-tool -v -f /home/jaybuff/swift-ld-bug/.build/debug/swift-ld-bug.o/llbuild.yaml
/home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/bin/swiftc -o /home/jaybuff/swift-ld-bug/.build/debug/swift-ld-bug /home/jaybuff/swift-ld-bug/.build/debug/swift-ld-bug.o/Sources/main.swift.o -g /home/jaybuff/swift-ld-bug/.build/debug/dep.a /home/jaybuff/swift-ld-bug/.build/debug/libc.a /home/jaybuff/swift-ld-bug/.build/debug/PackageDescription.a /h
ome/jaybuff/swift-ld-bug/.build/debug/POSIX.a /home/jaybuff/swift-ld-bug/.build/debug/sys.a -L/usr/local/lib
/home/jaybuff/swift-ld-bug/.build/debug/sys.a(Path.swift.o): In function `_TFE3sysSSg10isAbsoluteSb':
/home/jaybuff/swift-ld-bug/Packages/swift-package-manager-0.1.0/Sources/sys/Path.swift:194: undefined reference to `_TFE4libcSS9hasPrefixfSSSb'
/home/jaybuff/swift-ld-bug/.build/debug/sys.a(Path.swift.o): In function `_TZFV3sys4Pathg4homeSS':
/home/jaybuff/swift-ld-bug/Packages/swift-package-manager-0.1.0/Sources/sys/Path.swift:48: undefined reference to `_TF5POSIX6getenvFSSGSqSS_'
/home/jaybuff/swift-ld-bug/.build/debug/sys.a(Path.swift.o): In function `_TFE3sysSS7abspathfzT_SS':
/home/jaybuff/swift-ld-bug/Packages/swift-package-manager-0.1.0/Sources/sys/Path.swift:189: undefined reference to `_TF5POSIX6getcwdFzT_SS'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: build had 1 command failures
/usr/bin/which clang++
error: exit(1): ["/home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/bin/swift-build-tool", "-v", "-f", "/home/jaybuff/swift-ld-bug/.build/debug/swift-ld-bug.o/llbuild.yaml"]

# what if we we run the swiftc, but put sys.a before POSIX.a and libc.a in the args:
$ /home/jaybuff/swift-2.2-SNAPSHOT-2016-01-11-a-ubuntu15.10/usr/bin/swiftc -o /home/jaybuff/swift-ld-bug/.build/debug/swift-ld-bug /home/jaybuff/swift-ld-bug/.build/debug/swift-ld-bug.o/Sources/main.swift.o -g /home/jaybuff/swift-ld-bug/.build/debug/sys.a /home/jaybuff/swift-ld-bug/.build/debug/dep.a /home/jaybuff/swift-ld-bug/.build/debug/libc.a /home/jaybuff/s
wift-ld-bug/.build/debug/PackageDescription.a /home/jaybuff/swift-ld-bug/.build/debug/POSIX.a  -L/usr/local/lib

# it worked!
$  .build/debug/swift-ld-bug
path is usr/bin

The list of .a files and their order is passed from swiftc to clang++, which is finally passed to ld. ld's man page says that order matters:

Normally, an archive is searched only once in the order that it is specified on the command line. If a symbol in that archive is needed to resolve an undefined symbol referred to by an object in an archive that appears later on the command line, the linker would not be able to resolve that reference.

@mxcl
Copy link
Contributor

mxcl commented Feb 3, 2016

This is a package manager bug as we decide the order.

@swift-ci
Copy link
Contributor Author

swift-ci commented Feb 3, 2016

Comment by Jay Buffington (JIRA)

Ah, yes, looks like the bug is somewhere around here (from Sources/dep/llbuild.swift)

                // Target dependencies are *already* reverse sorted.

                let libsInThisPackage = target.dependencies
                    .filter{ $0.type == .Library }
                    .map{ $0.productFilename }
                    .map{ Path.join(params.prefix, $0) }

@swift-ci
Copy link
Contributor Author

swift-ci commented Feb 8, 2016

Comment by Shmuel Kallner (JIRA)

I'd like to add that this bug will only show itself on certain Unix based platforms. In particular on OSX you won't see this issue because LD will scan the list of .a flles multiple times.

One solution for this on Linux albeit not optimal is to place all of the .a's in a group using the -( and -) LD parameters around the .a files. This may not be optimal as the LD documentation claims there is a performance penalty for doing this.

@mxcl
Copy link
Contributor

mxcl commented Feb 9, 2016

We should be able to order them correctly

@mxcl
Copy link
Contributor

mxcl commented Feb 18, 2016

Confirmed this still occurs after the testing refactor. Looking into it.

@swift-ci
Copy link
Contributor Author

Comment by Shmuel Kallner (JIRA)

We have found that the issue seems to crop up if one adds "extra" dependencies beyond the minimal list needed.

By that I mean, if A is dependent on B and B is dependent on C, then in theory A can directly import C, even though A is not directly dependent on C and that dependency may not have been listed in A's dependency list at all. If however C gets listed as a dependency of A and by chance it is listed in the dependency list after B, then the problem occurs. The solution is to make sure that such lists of dependencies are in reverse order, i.e. in the case above C is listed before B

@mxcl
Copy link
Contributor

mxcl commented Feb 18, 2016

Actually, good news, I didn't correctly verify this bug. It seems to work with latest swiftpm:

Let me know if I'm wrong.

@swift-ci
Copy link
Contributor Author

Comment by Jay Buffington (JIRA)

Works for me 👍

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 4, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants