@swift-ci create

Still unable to build a toolchain on an M1 using a fresh clone. Error messages include:

FAILED: lib/swift/macosx/libswiftCompatibilityDynamicReplacements.a
cd /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/stdlib/toolchain/CompatibilityDynamicReplacements && /Applications/Xcode124.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo -create -output /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/./lib/swift/macosx/libswiftCompatibilityDynamicReplacements.a /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/x86_64/libswiftCompatibilityDynamicReplacements.a /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/arm64/libswiftCompatibilityDynamicReplacements.a /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/arm64e/libswiftCompatibilityDynamicReplacements.a
fatal error: /Applications/Xcode124.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/x86_64/libswiftCompatibilityDynamicReplacements.a and /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/arm64/libswiftCompatibilityDynamicReplacements.a have the same architectures (arm64) and can't be in the same fat output file
[22/1588][ 1%][1.247s] cd /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/stdlib/toolchain/Compatibility50 && /Applications/Xcode124.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo -create -output /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/./lib/swift/macosx/libswiftCompatibility50.a /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/x86_64/libswiftCompatibility50.a /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/arm64/libswiftCompatibility50.a /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/arm64e/libswiftCompatibility50.a
FAILED: lib/swift/macosx/libswiftCompatibility50.a
cd /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/stdlib/toolchain/Compatibility50 && /Applications/Xcode124.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo -create -output /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/./lib/swift/macosx/libswiftCompatibility50.a /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/x86_64/libswiftCompatibility50.a /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/arm64/libswiftCompatibility50.a /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/arm64e/libswiftCompatibility50.a
fatal error: /Applications/Xcode124.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/x86_64/libswiftCompatibility50.a and /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/arm64/libswiftCompatibility50.a have the same architectures (arm64) and can't be in the same fat output file
[23/1588][ 1%][1.256s] cd /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/stdlib/toolchain/Compatibility51 && /Applications/Xcode124.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo -create -output /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/./lib/swift/macosx/libswiftCompatibility51.a /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/x86_64/libswiftCompatibility51.a /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/arm64/libswiftCompatibility51.a /Volumes/Elements/swift-convert/build/buildbot_osx/swift-macosx-arm64/lib/swift/macosx/arm64e/libswiftCompatibility51.a
FAILED: lib/swift/macosx/libswiftCompatibility51.a

I've finally been able to build a toolchain for myself on an M1 Mac. It took a good while 🙂. There are about 4 problems that need to be addressed by someone who know more about CMake than I do. The first is the error above where lipo fails when trying to merge the archives for the three architectures x86_64, arm64 and arm64e as they all seem to have been built for arm64. It's odd that this problem should be specific to build on an arm64 machine as I have built "fat" toolchains recently on an Intel machine but I couldn't investigate it any further. In the end I changed SUPPORTED_OSX_ARCHS in the file cmake/modules/DarwinSDKs.cmake to only include arm64 and not attempt a "fat" build (besides it takes a good deal less time!)

The next problem seems to be related to llvm where you get an undefined symbol for ___isOSVersionAtLeast in the source file stdlib/public/runtime/ImageInspectionMachO.cpp. I tried various things but in the end defined my own version of this function returning true in this file.

The third relatively minor problem is with the Swift package manager where I have filed a straightforward PR apple/swift-package-manager#3500 and, with fixes for these problems the toolchain build proceeds most of the way through until you get to another place where you have to provide a symbol ___isOSVersionAtLeast in indexstore-db/lib/LLVMSupport/Support/Path.cpp and you will be rewarded with a toolchain (if not the symbols archive.)

As you can see these are relatively minor problems which someone with knowledge of the CMake files could make easy work of. I'm not sure what the cause of the llvm problem would be. I'm amazed nobody on the compiler team has made the switch to M1 yet as they are just so blazingly fast – or do they know there is something even better on the horizon 🙂.

Any ideas @compnerd?

I've preformed a new clone and replicated these changes though the following were also necessary:

The missing symbol has changed in the last month from _*isOSVersionAtLeast to *_isPlatformVersionAtLeast

So, in the interim, I've added to swift/stdlib/public/runtime/ImageInspectionMachO.cpp:

extern "C" bool __isPlatformVersionAtLeast();
bool __isPlatformVersionAtLeast()}}{{{}}
{{ return true;}}
{{}}}

You need to copy build/buildbot_osx/lldb-macosx-arm64/bin/lldb-tblgen from an x86_64 build for lldb to build.

The benchmarks fail to run with a undefined swift symbol that doesn't demangle so I've had to:

dyld: Symbol not found: _SayxSicig16_Differentiation14DifferentiableRzlWJrUSpSr

cat >build/buildbot_osx/benchmarks-macosx-arm64/bin/Benchmark_Onone

#!/bin/bash

exit 0

^D

cp build/buildbot_osx/benchmarks-macosx-arm64/bin/Benchmark_Onone build/buildbot_osx/benchmarks-macosx-arm64/bin/Benchmark_O

cp build/buildbot_osx/benchmarks-macosx-arm64/bin/Benchmark_Onone build/buildbot_osx/benchmarks-macosx-arm64/bin/Benchmark_Osize

The complete diff for the various changes I had to make to get a toolchain were:

diff --git a/benchmark/scripts/build_script_helper.py b/benchmark/scripts/build_script_helper.py  
index 53bf7b19f68..8c9f8a5d995 100755

-   -   -   a/benchmark/scripts/build_script_helper.py  
            +++ b/benchmark/scripts/build_script_helper.py  
            @@ -54,10 +54,11 @@ def main():  
            os.makedirs(bin_dir)

swiftbuild_path = os.path.join(args.toolchain, "usr", "bin", "swift-build")

-   perform_build(args, swiftbuild_path, "debug", "Benchmark_Onone", "-Onone")

-   perform_build(args, swiftbuild_path, "release", "Benchmark_Osize", "-Osize")

-   perform_build(args, swiftbuild_path, "release", "Benchmark_O", "-O")  
    +# perform_build(args, swiftbuild_path, "debug", "Benchmark_Onone", "-Onone")  
    +# perform_build(args, swiftbuild_path, "release", "Benchmark_Osize", "-Osize")  
    +# perform_build(args, swiftbuild_path, "release", "Benchmark_O", "-O")

if **name** == "**main**":  
+ exit(0)  
main()  
diff --git a/benchmark/scripts/test_Benchmark_Driver.py b/benchmark/scripts/test_Benchmark_Driver.py  
index 7690aacf31b..b18f0d1147d 100644

-   -   -   a/benchmark/scripts/test_Benchmark_Driver.py  
            +++ b/benchmark/scripts/test_Benchmark_Driver.py  
            @@ -1071,4 +1071,5 @@ class TestBenchmarkDoctor(unittest.TestCase):

if **name** == "**main**":  
+ exit(0)  
unittest.main()  
diff --git a/cmake/modules/DarwinSDKs.cmake b/cmake/modules/DarwinSDKs.cmake  
index 98b79e1f279..73813d48ca9 100644

-   -   -   a/cmake/modules/DarwinSDKs.cmake  
            +++ b/cmake/modules/DarwinSDKs.cmake  
            @@ -14,7 +14,7 @@ set(SUPPORTED_TVOS_ARCHS "arm64")  
            set(SUPPORTED_TVOS_SIMULATOR_ARCHS "x86_64;arm64")  
            set(SUPPORTED_WATCHOS_ARCHS "armv7k;arm64_32")  
            set(SUPPORTED_WATCHOS_SIMULATOR_ARCHS "i386;x86_64;arm64")  
            -set(SUPPORTED_OSX_ARCHS "x86_64;arm64;arm64e")  
            +set(SUPPORTED_OSX_ARCHS "arm64")

is_sdk_requested(OSX swift_build_osx)  
if(swift_build_osx)  
diff --git a/stdlib/public/runtime/ImageInspectionMachO.cpp b/stdlib/public/runtime/ImageInspectionMachO.cpp  
index b21ca9d0e05..bd953673e30 100644

-   -   -   a/stdlib/public/runtime/ImageInspectionMachO.cpp  
            +++ b/stdlib/public/runtime/ImageInspectionMachO.cpp  
            @@ -172,5 +172,10 @@ int swift::lookupSymbol(const void \*address, SymbolInfo \*info) {  
            return 1;  
            }

+extern "C" bool \_\_isPlatformVersionAtLeast();  
+bool \_\_isPlatformVersionAtLeast() {  
+ return true;  
+}  
+  
\#endif // defined(**APPLE**) && defined(**MACH**) &&  
// !defined(SWIFT_RUNTIME_MACHO_NO_DYLD)

   

The lipo issue should be addressed by the following PRs

[5.5] #38445
[main] #38415

Thanks very much Eric

A year on, I've just tried again and basically utils/build-toolchain basically no longer works (on an M1 at least). The issues I mentioned above are still there along with a few new undefined symbols related to half-floats and the bootstrapping process to bring Swift sources into the compiler doesn't seem to work, failing to find a usable version of swiftc. I don't have an intel machine to test if it "just works" on that architecture.

@edymtt Looks like you had a fix for this previously, I wonder if something changed recently.

edymtt's fix made a big difference but there are many other issues, at least on my machine. My clone is from a week ago.

An update on building a toolchain on M1, I've been able to get the build past the previous road blocks I mentioned by performing the whole build under rosetta!

I've not been able to get a toolchain just yet as it trips up at the very last trying to build swift-driver multi-architecture but it does seem to indicate this isn't a holding it wrong issue or specific to my particular clone. The problem is very easy to replicate if you run build-toolchain on an M1. I'm sure someone with the right llvm and CMake knowledge could clear this up in an instant.

To finish the week on a high note, I've been rewarded with a toolchain built on an M1 by running the whole thing inside Rosetta2. I had to edit the build.ninja file for libYams.a inside the swift-driver project to correct the -arch for target arm64 then build on the command line, edit that build out and retry the toolchain script. Seems like an obscure CMake problem for this configuration. Note, the "rosetta" trick only works with recent versions of CMake (I used 3.23.1 and Xcode 13.4)

@johnno1962 I am assuming that normal-ish builds work on M1? Are you only seeing this issue with build-toolchain?

Yes, I believe the build-script with and without --xcode works now. It's strange as the build-toolchain script is just a lightweight wrapper for calling build-script.

@johnno1962 I imagine that the issue here is that build-toolchain I don't think on our CI is being run on M1 machines. So I imagine that some incompatibilities have snuck in. @shahmishal we really should have a bot running build-toolchain on an Apple Silicon machine (maybe once a day) like the status bot.

I couldn't agree more but the first step would be investigating the problems I've been seeing for a year and getting it working.

Bump.

Just did a fresh clone and build-toolchain. Worked 💯 Thanks to everyone who sorted this out.