I guess the right answer here would be to pass a list of files down to the frontend, instead of trying to pass them all individually, or to have a mode where we pass them via stdin. We may also want to try supporting something like GCC's @file command-line syntax, but that's (a) more complicated, and (b) potentially dangerous.

Comment by Tomasz Gebarowski (JIRA)

Hi Jordan, thanks for answering! I would probably opt for using pipe/stdin to pass parameters to a spawned process. I've noticed that you opened an internal radar for that issues, any idea if that is a serious issue for you and is going to be fixed anytime soon? It's a bit blocker for our project. Setting -num-threads 0 seems to help, but only from command line. Xcode keeps overriding this value when invoking swiftc so setting

defaults write com.apple.dt.Xcode IDEBuildOperationMaxNumberOfConcurrentCompileTasks 0

doesn't help as the value is simply ignored by Xcode. Any idea how to force Xcode to use -num-threads 0 when invoking swiftc?

You might be able to add -num-threads 0 to your OTHER_SWIFT_FLAGS build setting for now. It's probably a coincidence that that helps, though.

I can't speak to schedule at this point, just that you're not the only one with such a big target (unless the other reports are also from your team). If someone else gets to it first, that's also great.

Comment by Tomasz Gebarowski (JIRA)

I'm the only person from my team who reported this problem, so I guess that more projects are affected.
I tried appending -num-threads 0 to OTHER SWIFT FLAGS, but unfortunately Xcode adds its -num-threads just after values
from OTHER SWIFT FLAGS and it gets overwritten. I solved the problem temporarily by writing a simple wrapper for swiftc in Python (which replaced the swiftc symlink in Xcode archive) that invokes swiftc under the hood and appends -num-threads 0 as last argument. This seems to work, at least the code compiles.

Unfortunately it fails at the linking stage. It seems that -whole-module-optimization is not working at all in Xcode 7.1 and Xcode 7.2.
There is already a radar for that:
rdar://23472326

Even the simplest Xcode project fails at linking stage when -whole-module-optimization is turned on. I will keep investigating this second problem, maybe there is some workaround for that as well.

Anyway thanks for help!

Comment by Tomasz Gebarowski (JIRA)

Small update, I managed to bypass this bug by modifying the way swiftc is invoked from Xcode. With this approach I managed to successfully compile my Xcode project with +1500 files in it (with -whole-module-optimization turned on)

The wrapper script I used is on my github account, more info can be found there as well:
https://github.com/tgebarowski/swiftc-wrapper

It looks like you're just silently throwing away arguments. I can't recommend that.

Ah, I see what it's doing. Okay, I can't call it supported, and it may break other things in Xcode, but it shouldn't be dangerous.

Comment by Tomasz Gebarowski (JIRA)

I am not dropping any arguments, but this is also not a fix, it's just a way to bypass incorrect compiler behavior. Maybe someone facing same problem will find it useful. With this workaround I managed to successfully compile,archive and run my project on both device and simulator. At least it's not blocking me and I can migrate my app to Swift 2 and wait either for (A) getting if fixed in next Xcode release or (B) dropping iOS 7 support in my app and dividing it to smaller components.

I've added a -filelist parameter for input files in ad945426. I'd also like to remove the O(N) arguments from

• swiftmodule inputs to the merge-module build phase

• /output/ files for multithreading single-frontend builds (WMO)

• object file inputs to the linker, at least on OS X

Finshed the rest of the work in commits up to bbc14af.

...but asking Chris to review and waiting for a buildbot cycle before cherry-picking the later commits to the 2.2 branch.

Test fix-up in 9cb775a.

Reviewed by ChrisW and cherry-picked to the 2.2 branch!

It should also be in the next development snapshot from the master branch, if you want to test sooner than Xcode 7.3.

Comment by Tomasz Gebarowski (JIRA)

Thanks Jordan for such a quick reaction. When I have time I will test the snapshot and let you know if the problem is fixed in the snapshot build.

Any updates here, tgebarowski (JIRA User)? If you've been able to compile your 1500+ file Swift/Objective-C project using Xcode 7.3, then I think we can close this issue.

Comment by Tomasz Gebarowski (JIRA)

Hi Brian.

The project compilation started to work with Xcode 7.3 with -Owholemodule (where the fix made by Jordan was first introduced). Compilation with -Onone was still not working, but I think that this was another issue. So yes, I think that you can close this.

Regards,
Tomasz

Cool, thanks! Post a link to the other issue you mentioned, if you can find it. 🙂

Comment by Vladislav Alexeev (JIRA)

I can confirm Xcode 9.2 still invokes swift and it passes all swift files through command line instead of using -filelist option. I understand there is support for -filelist in Swift, but how do people make Xcode work properly with large projects?

This bug was about communication between the swiftc driver and the subprocesses it spawns, not about any tools that need to invoke swiftc themselves. SR-4517 talks about adding general response file support to the driver, but even just acceping a filelist option would be a reasonable task on its own. (This is tracked by rdar://problem/33635183 within Apple.)

Comment by Vladislav Alexeev (JIRA)

Thanks Jordan for references and explanation!

Is this problem fixed? On Xcode 9.2 I have the same problem

See my comment above.

Comment by Vitaliy (JIRA)

Is this problem fixed? On Xcode 10.0 I have the same problem ((

See my comment above.