The link args come from utils/static-executable-args.lnk, which is copied into place by CMake. We could do some configure-time tests to figure out whether -ldl is necessary; do you think that would be sufficient? I'd rather not have the compiler call out to icu-config every time it needs to link a static executable.

@belkadan yes, I had seen that file and it would be enough to put -ldl in there but then it would even do that if your system's ICU is compiled with --disable-dyload.

But thinking about it, it probably improves the current situation because 1) it's unlikely that your ICU is compiled with --disable-dyload and 2) we can't make fully working static binaries anyway because glibc doesn't support that properly (and libFoundation.a also isn't a thing (yet)). I'll make a pull request...

@belkadan #7188

I was thinking rather than update the file directly, we would change it to use CMake's configure_file feature, which allows us to insert substitutions. I'm fine with CMake calling icu-config.

I suppose that's not great for binary distributions, though.

Sorry about that. But yes, as you say it shouldn't really be checked on the system that builds swiftc. And I'm guessing most users use the binary distribution...
Hmm, unsure what to do really.

For static binaries (using the -static-executable option) you should not link in libdl which is obviously a problem given how libicu is compiled on ubuntu.

To work around this I added a --libicu option to the swift build script which will compile up libicu using the correct configure options to produce static versions of the libraries that dont require libdl and put them in lib/swift_static/linux and shared versions in the lib/swift/linux. This means that if building a binary distribution you could provide versions of libicu suitable for static linking without worrying about how the distribution provided package has configured it.

My intention with utils/static-executable-args.lnk was that it could be overridden by a package maintainer or built from a script at build time if building on a different OS eg android or FreeBSD, the static file is just there as an initial version for ubuntu.

@spevans, I do agree that -ldl for static binaries is not quite right (but linking glibc statically is so too). What about the following: If Swift is compiled with --libicu we do not put -ldl in utils/static-executable-args.lnk but if it's not we do (as it's as you say an initial version for Ubuntu).

You are correct that if Swift is compiled with --libicu then -ldl should not be added. The easiest way to do this is probably to have a script like utils/gen-static-stdlib-link-args that does the same for static-executable-args.lnk (or expand utils/gen-static-stdlib-link-args to generate both). This is called from https://github.com/apple/swift/blob/master/lib/Driver/CMakeLists.txt#L24 and has knowledge of whether --libicu was used. This script can also be expanded in the future to handle other ELF platforms that may have different library requirements.

If the only reason to create static binaries using -ldl is to get the libicu statically linked into the executable then the -static-stdlib option should already do this on ubuntu as the static-stdlib-args.lnk file contains full paths to the ICU .a library files

The alternative is to not copy/create the static-executable-args.lnk file at all when --libicu has not been used. swiftc sees the lack of lib/swift_static/linux/static-executable-args.lnk as an indication that static linking isnt supported and gives the following error:

$ ~/swift/usr/bin/swiftc  -emit-executable -static-executable -o main.static main.swift                                                   
LLVM ERROR: -static-executable not supported on this platform

This is probably the correct action to take when libicu hasnt been built to be fully static.

Comment by jamlee (JIRA)

this problem is up to me.

After #19860 has been merged, can this issue be closed now?

No, it requires the fix in #15183 which just needs an update for PR comments but otherwise works:

$ cat test.swift
print("Hello")
$ ~/swift-install/usr/bin/swiftc -static-executable -o test test.swift
$ ./test
Hello
$ ldd test
not a dynamic executable
$ ls -l test
-rwxrwxr-x 1 spse spse 36853040 Nov  4 21:50 test

@spevans cool, thanks! Mind checking nm test | grep ' dl' ? The bug was about us needed the static version of libdl (yes, that's a thing). I added that in #7188 and it's still there:

https://github.com/apple/swift/blob/master/utils/static-executable-args.lnk#L15

So if this is truly fixed, then we should remove -ldl from utils/static-executable-args.lnk again and then we can close this issue.

Ok, firstly libICU is now compiled without use of dlopen()

$ nm ~/swift-install/usr/lib/swift_static/linux/libicuucswift.a |grep ' dl'
$ nm ~/swift-install/usr/lib/swift_static/linux/libicui18nswift.a |grep ' dl'
$ nm ~/swift-install/usr/lib/swift_static/linux/libicudataswift.a |grep ' dl'
$ cat ~/swift-install/usr/lib/swift_static/linux/static-executable-args.lnk 
-static
-lswiftCore
-lswiftImageInspectionStatic
-lpthread
-latomic
-licui18nswift
-licuucswift
-licudataswift

$ ~/swift-install/usr/bin/swiftc -v -static-executable -o static_test ~/src/test/test.swift 
Swift version 4.2-dev (LLVM 49db994493, Clang 256dc06b21, Swift b5d0def2db)
Target: x86_64-unknown-linux-gnu
/home/spse/swift-install/usr/bin/swift -frontend -c -primary-file /home/spse/src/test/test.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -color-diagnostics -module-name static_test -o /tmp/test-fc87ff.o
/home/spse/swift-install/usr/bin/swift-autolink-extract /tmp/test-fc87ff.o -o /tmp/test-f85369.autolink
/usr/bin/clang++ -fuse-ld=gold -target x86_64-unknown-linux-gnu /tmp/test-fc87ff.o @/tmp/test-f85369.autolink -L /home/spse/swift-install/usr/lib/swift_static/linux @/home/spse/swift-install/usr/lib/swift_static/linux/static-executable-args.lnk --target=x86_64-unknown-linux-gnu -v -o static_test
clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.3.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/usr/bin/ld.gold" -z relro --hash-style=gnu -m elf_x86_64 -static -o static_test /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu/crt1.o /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu/crti.o /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/crtbeginT.o -L/home/spse/swift-install/usr/lib/swift_static/linux -L/usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../.. -L/usr/lib/llvm-6.0/bin/../lib -L/lib -L/usr/lib /tmp/test-fc87ff.o -lswiftSwiftOnoneSupport -lswiftCore -lswiftCore -lswiftImageInspectionStatic -lpthread -latomic -licui18nswift -licuucswift -licudataswift -lstdc++ -lm --start-group -lgcc -lgcc_eh -lc --end-group /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/crtend.o /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu/crtn.o

Note however that glibc still has its own internal dl* functions but none of the -ldl ones

$ nm static_test |grep ' dl'
0000000000844da0 t dlclose_doit
00000000023b97a0 b dl_close_state.10955
0000000000841790 t dlinfo_doit
00000000008082b0 W dl_iterate_phdr
00000000008418e0 t dlmopen_doit
0000000000844ca0 t dlopen_doit
0000000000842a80 t dl_open_worker
0000000000844de0 t dlsym_doit
0000000000844eb0 t dlvsym_doit

@spevans awesome, thanks! Yes, the glibc ones we can't fix without dropping glibc which is a bigger task not for today. Should we then just remove -ldl from utils/static-executable-args.lnk?

Removing -ldl is done as part of #15183 which fixes up -static-executable to work correctly (its currently broken in master) and also adds a regression test to catch future breakage.

@weissi, can this be closed now that the linux toolchain no longer uses separate ICU libraries?

@weissi, can this be closed now that the linux toolchain no longer uses separate ICU libraries?

@finagolfin FoundationXML still needs libxml2 and libicu, not sure if that counts?

root@a7b17e46668f:/tmp# apt-get remove -y libicu-dev libicu74 icu-devtools
[...]
The following packages will be REMOVED:
  icu-devtools libicu-dev libicu74 libxml2 libxml2-dev
[...]
root@a7b17e46668f:/tmp# echo 'import FoundationXML' | swiftc -o test -
root@a7b17e46668f:/tmp# ./test
./test: error while loading shared libraries: libxml2.so.2: cannot open shared object file: No such file or directory

but when I install libxml2 again

root@a7b17e46668f:/tmp# apt-get -y install libxml2
[...]
The following additional packages will be installed:
  libicu74
The following NEW packages will be installed:
  libicu74 libxml2
[...]

which then makes ./test work.

tl;dr: The toolchains still requires libicu in certain situations. Not sure about FoundationInternationalization.

FoundationXML still needs libxml2 and libicu

OK, but this issue isn't whether FoundationXML needs libicu, but whether building on linux generally works. For that niche case of linking FoundationXML dynamically, you say it's working, whereas I don't think we support building FoundationXML with static executables.

So where's the problem now?

FoundationXML still needs libxml2 and libicu

OK, but this issue isn't whether FoundationXML needs libicu, but whether building on linux generally works. For that niche case of linking FoundationXML dynamically, you say it's working, whereas I don't think we support building FoundationXML with static executables.

So where's the problem now?

That's fair. Let's close.