Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-14796] The new swift-driver doesn't pass the '-sdk /' flag correctly to the ClangImporter -sysroot #57145

Closed
finagolfin opened this issue Jun 20, 2021 · 5 comments
Assignees
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. transfer candidate The issue may belong in another repository

Comments

@finagolfin
Copy link
Contributor

Previous ID SR-14796
Radar rdar://problem/79563727
Original Reporter @buttaface
Type Bug
Status Resolved
Resolution Done
Environment

Fedora Core 33 x86_64

Additional Detail from JIRA
Votes 0
Component/s Compiler
Labels Bug
Assignee @buttaface
Priority Medium

md5: 8027fdac6ea6c52cc590c901f8d3aba3

Issue Description:

I came across this bug when trying to build the trunk standalone stdlib with the latest official trunk snapshot build for centos8 on Fedora x86_64. It fails because it passes in -sdk / -resource-dir build/Ninja-Release/swift-linux-x86_64/lib/swift when building the stdlib, which then can't find basic libc headers like sys/types.h.

I'm able to reproduce by trying to build this simple test file:

import Foundation

let fm = FileManager.default
let path = NSTemporaryDirectory() + "testdir(NSUUID().uuidString)"
print("creating temporary directory (path)")
try fm.createDirectory(atPath: path, withIntermediateDirectories: false, attributes: nil)
try? fm.removeItem(atPath: path)
print("removed temporary directory (path)")

If I add the -Xfrontend -dump-clang-diagnostics flag to check the ClangImporter flags, I get

> ./swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a-centos8/usr/bin/swiftc check.swift -Xfrontend -dump-clang-diagnostics
'clang' '-fsyntax-only' '-fblocks' '-D__swift__=50500' '-fretain-comments-from-system-headers' '-isystem' '/home/butta/src/swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a-centos8/usr/lib/swift' '-fPIC' '-fmodules' '-Xclang' '-fmodule-feature' '-Xclang' 'swift' '-Werror=non-modular-include-in-framework-module' '-x' 'c' '-std=gnu11' '-fmodule-map-file=/home/butta/src/swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a-centos8/usr/lib/swift/linux/x86_64/glibc.modulemap' '-Xclang' '-nostdsysteminc' '-fmodules-validate-system-headers' '-Xclang' '-fmodule-format=obj' '-fapinotes-modules' '-fapinotes-swift-version=5' '-iapinotes-modules' '/home/butta/src/swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a-centos8/usr/lib/swift/apinotes' '-target' 'x86_64-unknown-linux-gnu' '<swift-imported-modules>' '-resource-dir' '/home/butta/src/swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a-centos8/usr/lib/swift/clang' '-fansi-escape-codes' '-working-directory' '/home/butta/src'

and that builds fine. If I add the two flags above, it fails:

> ./swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a-centos8/usr/bin/swiftc check.swift -sdk / -resource-dir swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a-centos8/usr/lib/swift -Xfrontend -dump-clang-diagnostics
warning: Could not read SDKSettings.json for SDK at: /home/butta/src
'clang' '-fsyntax-only' '-fblocks' '-D__swift__=50500' '-fretain-comments-from-system-headers' '-isystem' 'swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a-centos8/usr/lib/swift' '-fPIC' '-fmodules' '-Xclang' '-fmodule-feature' '-Xclang' 'swift' '-Werror=non-modular-include-in-framework-module' '-x' 'c' '-std=gnu11' '-fmodule-map-file=swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a-centos8/usr/lib/swift/linux/x86_64/glibc.modulemap' '--sysroot' '/home/butta/src' '-fmodules-validate-system-headers' '-Xclang' '-fmodule-format=obj' '-fapinotes-modules' '-fapinotes-swift-version=5' '-iapinotes-modules' 'swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a-centos8/usr/lib/swift/apinotes' '-target' 'x86_64-unknown-linux-gnu' '<swift-imported-modules>' '-resource-dir' 'swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a-centos8/usr/lib/swift/clang' '-fansi-escape-codes' '-working-directory' '/home/butta/src'
While building module 'CoreFoundation':
In file included from <module-includes>:1:
/home/butta/src/swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a-centos8/usr/lib/swift/CoreFoundation/CoreFoundation.h:25:10: fatal error: 'sys/types.h' file not found

This same failing command works fine when run with the latest official build of the Swift 5.4.1 toolchain for centos8, but fails with the latest official Swift 5.5 snapshot build too. Comparing the working 5.4.1 command, I see that it is correctly passing --sysroot /- to the ClangImporter whereas the trunk and 5.5 snapshot builds incorrectly pass the working directory, ie -sysroot /home/butta/src above.

What gets even stranger is that I tried building the exact same tag from source on Fedora and I could not reproduce the problem, implying this is an issue with how the official builds are produced on the CI. Maybe something changed recently in the C++ toolchain used to build the Swift compiler on the CI or with how the build is prepared?

I've put together a pull on the Github Actions CI to demonstrate the same behavior on Ubuntu 20.04, so this is not specific to my Fedora environment. Note how everything works fine with the official Swift 5.4.1 build, that they have pre-installed on the CI.

@shahmishal, maybe you know what's going on here.

@typesanitizer
Copy link

@swift-ci create

@finagolfin
Copy link
Contributor Author

I've updated the linked pull to use the latest July 15 official trunk snapshot build, which shows the same problem. I recently built the July 9 trunk source snapshot for a pull I just submitted and I still cannot reproduce this with a local build, but still can with the official July 9 trunk snapshot build, which implies this is a problem with the CI toolchain used to produce the official snapshot builds or some later build step like stripping the binaries that isn't run for my local build, which just builds the compiler and corelibs.

Also, I noticed that this doesn't affect cross-compiling the stdlib for Android with these official snapshot builds, so I tried the equivalent sdk in the above failing command `-sdk /usr/..` and that worked fine. Even `-sdk /u` registers properly, though it still fails because `/u/include` doesn't exist.

I then tried `-sdk a` and it dumped `'--sysroot' '/home/butta/src/a'` being passed to ClangImporter, so it weirdly appends single-character sdk paths to the working directory and passes it to the ClangImporter sysroot but not two-character sdk paths or longer. Not sure what build optimization would cause this, but probably worth some linux guy like @drexin digging into.

@finagolfin
Copy link
Contributor Author

Looks like a swift-driver issue, goes away if I add {{-disallow-use-new-driver}} to the prebuilt snapshots. I wasn't building early swift-driver locally, which is why I didn't hit this with the local build. I'll look into a fix.

@finagolfin
Copy link
Contributor Author

Submitted a fix:

apple/swift-driver#776

@finagolfin
Copy link
Contributor Author

Trunk fix was merged, submitted for 5.5:

apple/swift-driver#784

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@AnthonyLatsis AnthonyLatsis added transfer candidate The issue may belong in another repository and removed compiler The Swift compiler in itself labels Nov 6, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. transfer candidate The issue may belong in another repository
Projects
None yet
Development

No branches or pull requests

3 participants