Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SR-9325] build failure on Amazon Linux June 2018 AMI #51795

Open
swift-ci opened this issue Nov 22, 2018 · 9 comments
Open

[SR-9325] build failure on Amazon Linux June 2018 AMI #51795

swift-ci opened this issue Nov 22, 2018 · 9 comments
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior.

Comments

@swift-ci
Copy link
Collaborator

Previous ID SR-9325
Radar None
Original Reporter johanatan (JIRA User)
Type Bug
Environment

x86_64

Additional Detail from JIRA
Votes 0
Component/s Project Infrastructure
Labels Bug
Assignee None
Priority Medium

md5: 04892248e0dae2f499d11420e1aa95e5

Issue Description:

[ec2-user@ip-172-16-179-22 swift]$ ./utils/build-script -R   
./utils/build-script: note: Using toolchain default
which: no ninja in (/home/ec2-user/swift/cmake-3.13.0-Linux-x86_64/bin:/home/ec2-user/.local/bin:/home/ec2-user/bin:/home/ec2-user/swift:/home/ec2-user/swift/build/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ec2-user/.local/bin:/home/ec2-user/bin:/home/ec2-user/.local/bin:/home/ec2-user/bin:/home/ec2-user/swift:/home/ec2-user/swift/build/bin)
which: no ninja-build in (/home/ec2-user/swift/cmake-3.13.0-Linux-x86_64/bin:/home/ec2-user/.local/bin:/home/ec2-user/bin:/home/ec2-user/swift:/home/ec2-user/swift/build/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ec2-user/.local/bin:/home/ec2-user/bin:/home/ec2-user/.local/bin:/home/ec2-user/bin:/home/ec2-user/swift:/home/ec2-user/swift/build/bin)
+ mkdir -p /home/ec2-user/swift/build/Ninja-ReleaseAssert
+ env HOST_VARIABLE_linux_x86_64__SWIFT_BENCHMARK_TARGETS= HOST_VARIABLE_linux_x86_64__SWIFT_RUN_BENCHMARK_TARGETS= HOST_VARIABLE_linux_x86_64__SWIFT_SDKS=LINUX HOST_VARIABLE_linux_x86_64__SWIFT_STDLIB_TARGETS=swift-test-stdlib-linux-x86_64 HOST_VARIABLE_linux_x86_64__SWIFT_TEST_TARGETS= /home/ec2-user/swift/swift/utils/build-script-impl --workspace /home/ec2-user/swift --build-dir /home/ec2-user/swift/build/Ninja-ReleaseAssert --install-prefix /usr --host-target linux-x86_64 --stdlib-deployment-targets linux-x86_64 --host-cc /home/ec2-user/swift/build/bin/clang --host-cxx /home/ec2-user/swift/build/bin/clang++ --darwin-xcrun-toolchain default --darwin-deployment-version-osx=10.9 --darwin-deployment-version-ios=7.0 --darwin-deployment-version-tvos=9.0 --darwin-deployment-version-watchos=2.0 --cmake /home/ec2-user/swift/cmake-3.13.0-Linux-x86_64/bin/cmake --cmark-build-type Release --llvm-build-type Release --swift-build-type Release --swift-stdlib-build-type Release --lldb-build-type Release --lldb-build-with-xcode 1 --foundation-build-type Release --libdispatch-build-type Release --libicu-build-type Release --xctest-build-type Release --swiftpm-build-type Release --swiftsyntax-build-type Release --skstresstester-build-type Release --llbuild-build-type Release --swift-enable-assertions true --swift-stdlib-enable-assertions true --swift-analyze-code-coverage false --llbuild-enable-assertions true --lldb-assertions true --cmake-generator Ninja --build-jobs 16 '--common-cmake-options=-G Ninja -DCMAKE_C_COMPILER:PATH=/home/ec2-user/swift/build/bin/clang -DCMAKE_CXX_COMPILER:PATH=/home/ec2-user/swift/build/bin/clang++ -DCMAKE_LIBTOOL:PATH=/usr/bin/libtool -DLLVM_VERSION_MAJOR:STRING=7 -DLLVM_VERSION_MINOR:STRING=0 -DLLVM_VERSION_PATCH:STRING=0 -DCLANG_VERSION_MAJOR:STRING=7 -DCLANG_VERSION_MINOR:STRING=0 -DCLANG_VERSION_PATCH:STRING=0 -DCMAKE_MAKE_PROGRAM=/home/ec2-user/swift/build/Ninja-ReleaseAssert/ninja-build/ninja' --build-args=-j16 --cmark-cmake-options= '--llvm-cmake-options=-DLLVM_ENABLE_ASSERTIONS=TRUE -DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64;PowerPC;SystemZ;Mips' '--swift-cmake-options=-DSWIFT_STDLIB_ENABLE_SIL_OWNERSHIP=FALSE -DSWIFT_ENABLE_GUARANTEED_NORMAL_ARGUMENTS=TRUE -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE -DSWIFT_FORCE_OPTIMIZED_TYPECHECKER=FALSE -DSWIFT_STDLIB_ENABLE_STDLIBCORE_EXCLUSIVITY_CHECKING=FALSE' --build-stdlib-deployment-targets all --ninja-bin=/home/ec2-user/swift/build/Ninja-ReleaseAssert/ninja-build/ninja --skip-build-foundation --skip-build-xctest --skip-build-lldb --skip-build-llbuild --skip-build-libdispatch --skip-build-libicu --skip-build-swiftpm --skip-build-swiftsyntax --skip-build-skstresstester --skip-build-playgroundsupport --build-swift-dynamic-stdlib --build-swift-dynamic-sdk-overlay --skip-build-ios-device --skip-build-ios-simulator --skip-build-tvos-device --skip-build-tvos-simulator --skip-build-watchos-device --skip-build-watchos-simulator --skip-build-android --skip-test-swift --skip-test-cmark --skip-test-lldb --skip-test-llbuild --skip-test-swiftpm --skip-test-swiftsyntax --skip-test-skstresstester --skip-test-xctest --skip-test-foundation --skip-test-libdispatch --skip-test-libicu --skip-test-playgroundsupport --skip-test-linux --skip-test-freebsd --skip-test-cygwin --skip-test-osx --skip-test-ios-host --skip-test-ios-simulator --skip-test-tvos-host --skip-test-tvos-simulator --skip-test-watchos-host --skip-test-watchos-simulator --skip-test-android-host --skip-test-benchmarks --skip-test-optimized --skip-test-optimize-for-size --android-deploy-device-path /data/local/tmp --host-libtool /usr/bin/libtool --llvm-lit-args=-sv
Building the standard library for: swift-test-stdlib-linux-x86_64
+ /home/ec2-user/swift/cmake-3.13.0-Linux-x86_64/bin/cmake --build /home/ec2-user/swift/build/Ninja-ReleaseAssert/cmark-linux-x86_64 -- -j16 all
ninja: no work to do.
symlinking the system headers (/usr/include/c++) into the local clang build directory (/home/ec2-user/swift/build/Ninja-ReleaseAssert/llvm-linux-x86_64/include).
+ ln -s -f /usr/include/c++ /home/ec2-user/swift/build/Ninja-ReleaseAssert/llvm-linux-x86_64/include
+ /home/ec2-user/swift/cmake-3.13.0-Linux-x86_64/bin/cmake --build /home/ec2-user/swift/build/Ninja-ReleaseAssert/llvm-linux-x86_64 -- -j16 all
ninja: no work to do.
+ /home/ec2-user/swift/cmake-3.13.0-Linux-x86_64/bin/cmake --build /home/ec2-user/swift/build/Ninja-ReleaseAssert/swift-linux-x86_64 -- -j16 all swift-test-stdlib-linux-x86_64
ninja: error: '/home/ec2-user/swift/build/Ninja-ReleaseAssert/llvm-linux-x86_64/lib/libclangAPINotes.a', needed by 'bin/swift', missing and no known rule to make it
./utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting
@gottesmm
Copy link
Member

Thanks!

@gottesmm
Copy link
Member

A few things:

1. Can you try doing this from the root directory? (i.e. not from within swift). I don't remember if this makes a difference, but generally people do that (just trying to reduce the delta).
2. Can you run ./swift/utils/update-checkout --dump-hashes? Are you using the master scheme?

@swift-ci
Copy link
Collaborator Author

Comment by Jonathan Leonard (JIRA)

[ec2-user@ip-172-16-179-22 swift]$ ./swift/utils/update-checkout --dump-hashes 
{
 "https-clone-pattern": "https://github.com/%s.git", 
 "ssh-clone-pattern": "git@github.com:%s.git", 
 "repos": {
     "compiler-rt": {
         "remote": {
             "id": "apple/swift-compiler-rt"
         }
     }, 
     "llvm": {
         "remote": {
             "id": "apple/swift-llvm"
         }
     }, 
     "swift-xcode-playground-support": {
         "remote": {
             "id": "apple/swift-xcode-playground-support"
         }
     }, 
     "swift-stress-tester": {
         "remote": {
             "id": "apple/swift-stress-tester"
         }
     }, 
     "clang": {
         "remote": {
             "id": "apple/swift-clang"
         }
     }, 
     "llbuild": {
         "remote": {
             "id": "apple/swift-llbuild"
         }
     }, 
     "cmark": {
         "remote": {
             "id": "apple/swift-cmark"
         }
     }, 
     "lldb": {
         "remote": {
             "id": "apple/swift-lldb"
         }
     }, 
     "swift-syntax": {
         "remote": {
             "id": "apple/swift-syntax"
         }
     }, 
     "swift-corelibs-xctest": {
         "remote": {
             "id": "apple/swift-corelibs-xctest"
         }
     }, 
     "ninja": {
         "remote": {
             "id": "ninja-build/ninja"
         }
     }, 
     "swift-corelibs-foundation": {
         "remote": {
             "id": "apple/swift-corelibs-foundation"
         }
     }, 
     "swift-integration-tests": {
         "remote": {
             "id": "apple/swift-integration-tests"
         }
     }, 
     "swiftpm": {
         "remote": {
             "id": "apple/swift-package-manager"
         }
     }, 
     "swift": {
         "remote": {
             "id": "apple/swift"
         }
     }, 
     "swift-corelibs-libdispatch": {
         "remote": {
             "id": "apple/swift-corelibs-libdispatch"
         }
     }, 
     "icu": {
         "platforms": [
             "Linux"
         ], 
         "remote": {
             "id": "unicode-org/icu"
         }
     }
 }, 
 "branch-schemes": {
     "repro": {
         "repos": {
             "compiler-rt": "f41c0bcde48835740d6d66ae97e5d0420c8fc5e4", 
             "llvm": "dcb9eb74a749b1d0d9a93806c155c61b0f5a18ef", 
             "swift-xcode-playground-support": "6c112c2e8a4d9ecead1d37d7b6b835f5dc3d84dc", 
             "swift-corelibs-foundation": "c82398aa779dd820c0f664b17197d9ed78e7052d", 
             "clang": "95cdf7c9af0dc102a3fbda4ae1f6265026ff2e30", 
             "llbuild": "40c302a0b2ac12e99d1355986e4f16590d1fb03e", 
             "cmark": "b00c889eb66d38cb8f37847efef89aeca4e4b520", 
             "lldb": "ef92a319d68420a4065cdc991d0e350f6e9f4eb3", 
             "swift-syntax": "aa2489d6359d4ac971e815bd91b7bff875183436", 
             "swiftpm": "5b562d7e7380c46624446065bfa9b05a8b131393", 
             "swift-corelibs-xctest": "0d9d2909c1590aa598f96c6c33329a1bb97ac172", 
             "swift-stress-tester": "2fc093642df924f6adf9de9e4397c7c6fc8b5fc8", 
             "ninja": "253e94c1fa511704baeb61cf69995bbf09ba435e", 
             "swift-integration-tests": "b5f9eec316c0446e62bb3badde5d043c8bbd8cf9", 
             "swift": "e1e22edb164a50eb092f6b9716b0c46877325eb7", 
             "swift-corelibs-libdispatch": "5bcd598e5ce08327ad8b2988e62d9fdbbe96ae64", 
             "icu": "b3054b334d8cf84dcae12df7e1a4f1f5c0997445"
         }, 
         "aliases": [
             "repro"
         ]
     }
 }
}

@swift-ci
Copy link
Collaborator Author

Comment by Jonathan Leonard (JIRA)

I tried `./swift/utils/build-script -R` from the root and got the same result. Do I need to do a clean build from there (previous attempts were all inside swift/ including the one that produced all the cached symbols/objects).

@gottesmm
Copy link
Member

Is this a docker container? Can I reproduce this locally?

@swift-ci
Copy link
Collaborator Author

Comment by Jonathan Leonard (JIRA)

No, not docker. It’s one of the top two AMIs (Amazon Linux) when you are asked to pick a base image when spinning up a new EC2 instance on AWS. If you wanna try it, be sure to select a lot of memory and disk.

@swift-ci
Copy link
Collaborator Author

Comment by Jonathan Leonard (JIRA)

Actually looks like there are Docker images for this system. May not be bit for bit identical but it may still reproduce the problem:

https://hub.docker.com/_/amazonlinux/

@swift-ci
Copy link
Collaborator Author

Comment by Jonathan Leonard (JIRA)

I just tried the amazonlinux docker locally with:

docker run -it amazonlinux /bin/bash

and it behaved identically up through 83% of the llvm build (before unfortunately my host ran out of disk space). I'm 99% confident that you'd get the same result if you were to successfully complete the llvm build and then proceed with the swift one.

Also, I roughly followed this guide (which is a bit out of date):
https://gist.github.com/carsonmcdonald/8edeb5ac745a1e94699e

With the following modifications:
-. yum-config-manager isn't present in the docker image and apparently not required.
-. the clang install on line 32 has to be skipped as there's no package for it.
-. symlinks on lines 35 and 36 do not refer to any files that exist and no equivalent counterparts could be found so also skipped.
-. skipped lines 38 and 39 as it already has the triple underscore version.
-. skipped git clone'ing ninja as the update-checkout clone already does that.
-. skipped the patches from lines 55 to 122 (as these applied to the Swift codebase roughly 3 years ago).

And for the llvm build, I created a /build in the ROOT_DIR, did a

cmake -G "Unix Makefiles" ../llvm/

and then

make

That's it. And I opted for the:

./swift/utils/build-script -R

version of the build incantation.

@gottesmm
Copy link
Member

Thanks! I'll take a look.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gottesmm @swift-ci