Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-12695] Swift 5.2.2 release optimization fails on Linux #55138

Closed
jdmcd opened this issue Apr 28, 2020 · 23 comments
Closed

[SR-12695] Swift 5.2.2 release optimization fails on Linux #55138

jdmcd opened this issue Apr 28, 2020 · 23 comments
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. compiler The Swift compiler in itself Linux Platform: Linux optimized only Flag: An issue whose reproduction requires optimized compilation

Comments

@jdmcd
Copy link

jdmcd commented Apr 28, 2020

Previous ID SR-12695
Radar rdar://problem/62518020
Original Reporter @jdmcd
Type Bug
Status Resolved
Resolution Done

Attachment: Download

Additional Detail from JIRA
Votes 1
Component/s Compiler
Labels Bug, Linux, OptimizedOnly
Assignee None
Priority Medium

md5: c1cff6fad4b49db2998701ff5f9cdc4a

Issue Description:

This is similar to the problem described in https://bugs.swift.org/browse/SR-12674, however it's being built without the cross module optimization flag. I've attached the build output and this can be replicated easily by running the project from FB7679587 - just modify the Dockerfile to remove the cross module option.

When running, the process never completes and after 10-15 mins of hanging on a compilation step (Almost always on `TranseoCommon ClassLink.swift`) it exits with code 1.

@weissi
Copy link
Member

weissi commented Apr 28, 2020

CC @eeckstein

@weissi
Copy link
Member

weissi commented Apr 28, 2020

@swift-ci create

@jdmcd
Copy link
Author

jdmcd commented Apr 28, 2020

@eeckstein @weissi I've attached a list of files that appear to be compiling successfully to this issue - grabbed via the files in `/tmp`. By the looks of it it seems that my project itself is compiling ok and there's something else going on? Not positive though.

@eeckstein
Copy link
Member

reproduced the long compile time, though eventually the compiler finished.

@jdmcd
Copy link
Author

jdmcd commented Apr 28, 2020

@eeckstein that's good and bad news, I suppose. Is it possible that I'm running out of memory or something? I was also working with someone from the Vapor community who was also able to reproduce the problem

@eeckstein
Copy link
Member

My setup isn't exactly the same. I tried on macOS. I'll do some more experiments.

@jdmcd
Copy link
Author

jdmcd commented Apr 28, 2020

@eeckstein got it - I've always been able to get it to build on Mac, only had the problem on Linux in the Docker container

@jdmcd
Copy link
Author

jdmcd commented Apr 28, 2020

@eeckstein @weissi are there any flags I should be passing to swift build to get more details besides `-v`? Just trying to figure out what else I can dive into since right now the output isn't giving me much

@eeckstein
Copy link
Member

From the "swift build -v" output you can copy the swiftc invocation line which causes the trouble and invoke that separately. Looks like the building the module "App" causes the problem.

It could be that the memory consumption is a problem. This compilation needs > 1GB of memory. I don't know why, yet.

@jdmcd
Copy link
Author

jdmcd commented Apr 28, 2020

@eeckstein Makes sense - I did in fact copy just that command I got the same result (just hanging). I've got it running with strace right now, would that output be helpful to you? Not sure if there's a way to add more memory to Docker - I've got 16gb on my mac so shouldn't be a problem

@jdmcd
Copy link
Author

jdmcd commented Apr 29, 2020

@eeckstein Should I be attempting to debug using steps from here? https://github.com/apple/swift/blob/master/docs/DebuggingTheCompiler.rst#identifying-an-optimizer-bug

The only thing is that it's not crashing after compilation, it's crashing during compilation, so I'm not sure if those instructions are applicable

@eeckstein
Copy link
Member

@jdmcdLet me try to reproduce it on linux. I should know more in a few hours.

@jdmcd
Copy link
Author

jdmcd commented Apr 29, 2020

@eeckstein Awesome thank you!!

@eeckstein
Copy link
Member

@jdmcd On linux, "swift build" fails at the linking stage, but compilation succeeded.
Can you check if this is the case for you as well? Can you attach the output of the failed build?

@jdmcd
Copy link
Author

jdmcd commented Apr 29, 2020

@eeckstein `swift build` (in debug mode) compiles for me fine - how would I check to see if it fails at the linking stage? Also, while debugging with @gwynne and tannernelson (JIRA User) we discovered that it finishes building successfully when running `
swift build -c release -Xswiftc -Xfrontend -Xswiftc -debug-constraints`, which I find very strange.

@weissi
Copy link
Member

weissi commented Apr 29, 2020

@jdmcd just as a random guess: You're sure you're not running out of memory? Because the debug-constraints may make it slow enough to reduce peak memory usage?

@jdmcd
Copy link
Author

jdmcd commented Apr 29, 2020

@weissi that's an interesting point. Although my computer has 16 gb of memory and I'm not running anything else during the process so I think I should be ok? Is there any way to check?

@eeckstein
Copy link
Member

@jdmcd You can check in the docker preferences how much memory it can use.
The swift process to compile the App module needs ~3gb of memory.

@jdmcd
Copy link
Author

jdmcd commented Apr 29, 2020

@eeckstein I bumped my memory for the docker container up to 5gb and that seems to have helped the compilation complete - seems that @weissi was right! But now I think I might be encountering that linking problem you mentioned as later on in the Docker file I get:

docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./Run\": stat ./Run: no such file or directory": unknown.

@jdmcd
Copy link
Author

jdmcd commented Apr 29, 2020

I got this up and running with the help of the Vapor community fixing my Dockerfile, so I'm all set. Not sure if there is more exploration to be done regarding why there's so much memory usage happening but other than that I think I'm set 🙂 Thank you everyone for your help!

@eeckstein
Copy link
Member

👍

@weissi
Copy link
Member

weissi commented Apr 30, 2020

@jdmcd there is also (from outside the container)

docker stats <CONTAINER_NAME>

@weissi
Copy link
Member

weissi commented Apr 30, 2020

gives you continuous live stats like

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
e95f8f7bd55f        boring_liskov       81.79%              16.38MiB / 1.943GiB   0.82%               976B / 0B           0B / 0B             11

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. compiler The Swift compiler in itself Linux Platform: Linux optimized only Flag: An issue whose reproduction requires optimized compilation
Projects
None yet
Development

No branches or pull requests

3 participants