Comment by solo turn (JIRA)

retested with e80a6b0 llvm-project, now fixed.

Comment by solo turn (JIRA)

hi theindigamer (JIRA User) i picked up the arch linux packages recently from git, as well as from release. both do not compile. the intention is to first focus on the git version, then it will trickle down to a working release over time. for this purpose i created now a daily build on github as i did not know how to integrate one in your jenkins. i am not sure how to proceed with this error.

the error is always:
/github/workspace/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/dd_interceptors.cpp:225:20: error: redefinition of 'realpath'
INTERCEPTOR(char*, realpath, const char *path, char *resolved_path) {

the build is here:

• https://github.com/soloturn/swift-aur/actions

the arch packages:

• https://aur.archlinux.org/pkgbase/swift-language/

• https://aur.archlinux.org/packages/swift-language-git/

doing a build with
LDFLAGS='-ldl -lpthread' python swift/utils/build-script --preset=buildbot_linux,no_test install_destdir="$srcdir/build" installable_package="$srcdir/swift-ar
ch-pkg.tar.gz"

soloturn (JIRA User), one possibility is that /usr/include/bits/stdlib.h is being (unexpectedly) included, causing the redefinition error. I don't know why that would be happening, but you could try using --extra-cmake-options="-DCMAKE_IGNORE_PATH=/usr/include/bits" or temporarily removing that header from /usr/include/bits to check if that fixes the issue.

Comment by solo turn (JIRA)

theindigamer (JIRA User) tried so, https://github.com/soloturn/swift-aur/runs/852851416?check_suite_focus=true#step:3:324

```

• env /usr/sbin/cmake -G Ninja -DCMAKE_C_COMPILER:PATH=/usr/sbin/clang -DCMAKE_CXX_COMPILER:PATH=/usr/sbin/clang++ -DCMAKE_LIBTOOL:PATH=/usr/sbin/libtool -DLLVM_VERSION_MAJOR:STRING=10 -DLLVM_VERSION_MINOR:STRING=0 -DLLVM_VERSION_PATCH:STRING=0 -DCLANG_VERSION_MAJOR:STRING=10 -DCLANG_VERSION_MINOR:STRING=0 -DCLANG_VERSION_PATCH:STRING=0 -DCMAKE_MAKE_PROGRAM=/usr/sbin/ninja -DCMAKE_BUILD_TYPE:STRING=Release -DCMAKE_IGNORE_PATH=/usr/include/bits /github/workspace/swift-language-git/src/cmark
  ```

but for the configure step of compiler-rt it does not do it or has no effect:
https://github.com/soloturn/swift-aur/runs/852851416?check_suite_focus=true#step:3:4892

the patch applied is here:
https://aur.archlinux.org/cgit/aur.git/commit/?h=swift-language-git&id=6d3213d1aec00aa552cfef393134ffaf49752688

will try to create an image which lacks the header file.

Comment by solo turn (JIRA)

thanks theindigamer (JIRA User), i deleted bits/stdlib.h and references which helped: soloturn/build-aur-action@48ca1a3

/usr/include/stdlib.h has:

/* Define some macros helping to catch buffer overflows. */
#if __USE_FORTIFY_LEVEL > 0 && defined __fortify_function
# include <bits/stdlib.h>
#endif

deleting the if, and the header file can be done on this docker image, but on normal operations this will not be sustainable.

arch linux has a simpler package structure than other distributions, as it does not split between runtime and -dev packages, make the swift compile work for arch should fix it for any linux. how would you recommend to fix the detection / usage of the file in the original swift build?

Sorry, I forgot to respond here. Ok, so at least we understand what is causing the issue. I'm not sure what exactly is setting the __USE_FORTIFY_LEVEL setting (which apparently seems to be required), since it isn't mentioned in the Swift or LLVM code I'm looking at. Do you know what's going on there? As a hack, it might be worthwhile experimenting with:

• Setting {{_USE_FORTIFY_LEVEL=0}} in CMake (maybe pass {{-D_USE_FORTIFY_LEVEL=0 }}to CMake?)

• Somehow undef-ing __fortify_function (again, I don't know what is defining this).

• If <bit/stdlib.h> has a header guard say BITS_STDLIB_H, try defining that header guard in CMake (-DBITS_STDLIB_H=1) so that even if the header is included, it just expands to empty.

Also, from what I can tell, this doesn't have something to do with Swift specifically... It seems like an LLVM configuration issue... might be worth asking on the llvm-dev mailing list or filing a bug report against LLVM in case anyone has run into this before. It might be worth checking how the LLVM packages are built for Arch Linux and if they have any special hacks.

Comment by solo turn (JIRA)

arch llvm-10.0.1 package is here:

https://www.archlinux.org/packages/extra/x86_64/llvm/

there is no patch to the code coming from the release tarball or git, the build script does:

cmake .. -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr \
-DLLVM_HOST_TRIPLE=$CHOST \
-DLLVM_BUILD_LLVM_DYLIB=ON \
-DLLVM_LINK_LLVM_DYLIB=ON \
-DLLVM_INSTALL_UTILS=ON \
-DLLVM_ENABLE_RTTI=ON \
-DLLVM_ENABLE_FFI=ON \
-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=AVR \
-DLLVM_BUILD_TESTS=ON \
-DLLVM_BUILD_DOCS=ON \
-DLLVM_ENABLE_SPHINX=ON \
-DLLVM_ENABLE_DOXYGEN=OFF \
-DSPHINX_WARNINGS_AS_ERRORS=OFF \
-DLLVM_BINUTILS_INCDIR=/usr/include
ninja all ocaml_doc

arch uses "makepkg" which sets in its config the host triple (e.g. x86_64-pc-linux-gnu" to "chost" which is set:

chost = run_command(cc, '-dumpmachine').stdout().strip()
carch = chost.split('-')[0]

when packaging, llvm-config.h is provided additionally to distinguish between 32 bit and 64 bit:

https://github.com/archlinux/svntogit-packages/blob/packages/llvm/trunk/llvm-config.h

#include <bits/wordsize.h>

#if __WORDSIZE == 32
#include "llvm-config-32.h"
#elif __WORDSIZE == 64
#include "llvm-config-64.h"
#else
#error "Unknown word size"
#endif

Comment by solo turn (JIRA)

then there is compiler-rt:
https://github.com/archlinux/svntogit-packages/blob/packages/compiler-rt/trunk/PKGBUILD

no patch or hack, it does:

cmake .. -G Ninja
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_INSTALL_PREFIX=/usr
ninja

and the apple build does, according to:
https://github.com/soloturn/swift-aur/runs/912654366?check_suite_focus=true

/github/workspace/swift-language-git/src/build/buildbot_linux/llvm-linux-x86_64/./bin/clang++ -I/github/workspace/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/../.. -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -Wall -std=c++14 -Wno-unused-parameter -O3 -DNDEBUG -m64 -fno-lto -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -fno-rtti -MD -MT lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o -MF lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o.d -o lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o -c /github/workspace/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/dd_interceptors.cpp

Looking at the address sanitizer code, I do see there's a macro -D_FORTIFY_SOURCE, along with

clang/test/Driver/darwin-asan-nofortify.c
1:// Make sure AddressSanitizer disables _FORTIFY_SOURCE on Darwin.
5:// CHECK: #define _FORTIFY_SOURCE 0 

I suspect that might be related to the issue you are seeing. I do see a -D_FORTIFY_SOURCE=2 in the invocation you pasted. Maybe you can try overriding that setting somehow from the CMake settings?

Comment by solo turn (JIRA)

@natecook1000 you wrote the python build-script. what makes compiling compiler-rt special in the swift case? from the command line i see it is using /src/swift-language-git/src/build/Ninja-ReleaseAssert/llvm-linux-x86_64/./bin/clang++ instead of /usr/bin/c++ as the normal package build. how could the two react different to things in includes?

Comment by solo turn (JIRA)

natesg (JIRA User), theindigamer (JIRA User), i tried to build the arch linux compiler-rt package now with clang, clang++ instead of g++, and it gives the same error. for you guys llvm is "upstream" isn't it and therefor it needs to be fixed there?

i did download the PKGBUILD file from arch: https://github.com/archlinux/svntogit-packages/blob/packages/compiler-rt/trunk/PKGBUILD

and then:
$ export CC=clang
$ export CXX=clang++
$ makepkg -s$

and it really gives the same error when compiling swift.
/usr/bin/clang++ -I../lib/tsan/dd/../.. -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -Wall -std=c++14 -Wno-unused-parameter -O3 -DNDEBUG -m64 -fno-lto -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -fno-rtti -MD -MT lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o -MF lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o.d -o lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o -c ../lib/tsan/dd/dd_interceptors.cpp
../lib/tsan/dd/dd_interceptors.cpp:225:1: warning: attribute declaration must precede definition [-Wignored-attributes]
...__NTH (realpath (const char *__restrict __name, char *__restrict __resolved))
^
../lib/tsan/dd/dd_interceptors.cpp:225:20: error: redefinition of 'realpath'
...

Comment by solo turn (JIRA)

this is upstrean in compiler-rt, will report bug there if they permit.

> i did download the PKGBUILD file from arch

> and it really gives the same error when compiling swift

Thank you. Yes, that means it should be fixed upstream, not in Swift.

Comment by solo turn (JIRA)

bug is https://bugs.llvm.org/show_bug.cgi?id=47163

Comment by Fan Jiang (JIRA)

@compnerd This one is the one I was talking about today 🙂