Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-100] Red-hat - Support - lldb build #4593

Open
swift-ci opened this issue Dec 7, 2015 · 40 comments
Open

[SR-100] Red-hat - Support - lldb build #4593

swift-ci opened this issue Dec 7, 2015 · 40 comments

Comments

@swift-ci
Copy link

swift-ci commented Dec 7, 2015

Previous ID SR-100
Radar None
Original Reporter thawkins (JIRA User)
Type New Feature

Attachment: Download

Additional Detail from JIRA
Votes 2
Component/s Compiler, LLDB for Swift
Labels New Feature
Assignee None
Priority Medium

md5: 8e846ffa24d7a9a16bf9b2eda3207878

Issue Description:

1'm positing a few bugs for redhat support issues, whilst it is not an official supported distro, there are only a couple of small blockers for easy integration

when building swift-lldb on redhat os's, the Python support code is placed in ./lib64 and not /lib

all other libs are placed in /lib

This appears to be an upstream lldb issue.

The only place which impacts swift is the building of the install package, which expects everything to be in ./lib, the symlinks inside /lib64/Python2.7 even reffer to the ./lib directory

There are several possible fixes for this

1. Get upstream to fix the lldb bug
2. Patch the lldb build script to copy or move the Python2.7 directory from ./lib64 to ./lib post build.
3. Patch the package install file builder to look in both locations ie ./lib64 after ./lib

@trfiala
Copy link
Mannequin

trfiala mannequin commented Dec 9, 2015

Can you post more info related to this in the upstream llvm.org bugzilla tracker? I've done some work in the past to get llvm.org LLDB working on RHEL 7 and had solved some of the lib64/lib issues for Python on x86_64. It would be good to look at what is different with the configuration where you're seeing the issue.

Thanks!

@swift-ci
Copy link
Author

Comment by Timothy S Hawkins (JIRA)

Noted, i will try and setup a vanila build of lldb on f23 and check if it fails in the same way, and document.

Do you know if the llvm/lldb project has a dockerfile for ubuntu i can hack to create a clean test setup for f23.

@swift-ci
Copy link
Author

Comment by Timothy S Hawkins (JIRA)

I have built a script to automate the f23 build, see below.

https://github.com/thawkins/fedora-swift

@swift-ci
Copy link
Author

Comment by Timothy S Hawkins (JIRA)

This bug has already been reported on lldb upstream (including somebody else hitting it on the swift build)

https://llvm.org/bugs/show_bug.cgi?id=18957

@swift-ci
Copy link
Author

swift-ci commented Jan 6, 2016

Comment by Jeremy Fergason (JIRA)

thawkins (JIRA User) were you able to determine if the base lldb built successfully on fc23? I've put together a specfile and just currently patch lldb. There's a bugzilla ticket for inclusion of the swift RPM's in Fedora:

https://bugzilla.redhat.com/show_bug.cgi?id=1295115

@alblue
Copy link

alblue commented Jan 18, 2017

The ability to specify a suffix for lldb was added here:

apple/swift-lldb@35bb23f

Specifically, it added an LLVM_LIBDIR_SUFFIX flag that could be used to have /lib/ or /lib64/ as appropriate.

  • set(LLVM_LIBRARY_OUTPUT_INTDIR ${CMAKE_BINARY_DIR}/${CMAKE_CFG_INTDIR}/lib)
    + set(LLVM_LIBRARY_OUTPUT_INTDIR ${CMAKE_BINARY_DIR}/${CMAKE_CFG_INTDIR}/lib${LLVM_LIBDIR_SUFFIX})

If you set LLVM_LIBDIR_SUFFIX to empty, it should be installed into /lib/ as expected. Does this not work for you?

@swift-ci
Copy link
Author

swift-ci commented Feb 4, 2017

Comment by Timothy S Hawkins (JIRA)

@alex, this problem is still happening

Im not sure how to apply your fix,

Im setting up a new project on github to build swift on fedora25, it will support using a docker container.

But this is the only test that fails, and with a few workarounds by running the build-script with tests disabled, and then hacking the build result a bit, then running it again with tests enabled I get a fully functional version of 3.1, works inside my CLion IDE including interactive debugging, so i assume everything is ok. It just fails tests because of this issue this issue.

I will have the scripts and instructions up in the next 24 hours, the config should run on ubuntu or macosx, under docker.

https://github.com/FedoraSwift/fedora-swift2

@swift-ci
Copy link
Author

swift-ci commented Feb 4, 2017

Comment by Timothy S Hawkins (JIRA)

{{this is the one test that fails

******************* TEST 'Swift(linux-x86_64) :: Runtime/linux-fatal-backtrace.swift' FAILED ********************
Script:
--
rm -rf /root/swift-source/build/buildbot_linux_fc25/swift-linux-x86_64/test-linux-x86_64/Runtime/Output/linux-fatal-backtrace.swift.tmp
mkdir -p /root/swift-source/build/buildbot_linux_fc25/swift-linux-x86_64/test-linux-x86_64/Runtime/Output/linux-fatal-backtrace.swift.tmp !Screenshot from 2017-02-04 22-12-33.png|thumbnail! 
/root/swift-source/build/buildbot_linux_fc25/swift-linux-x86_64/bin/swiftc -target x86_64-unknown-linux-gnu  -module-cache-path '/tmp/swift-testsuite-clang-module-cacheXzZsiC' -swift-version 3  /root/swift-source/swift/test/Runtime/linux-fatal-backtrace.swift -o /root/swift-source/build/buildbot_linux_fc25/swift-linux-x86_64/test-linux-x86_64/Runtime/Output/linux-fatal-backtrace.swift.tmp/a.out
not --crash /root/swift-source/build/buildbot_linux_fc25/swift-linux-x86_64/test-linux-x86_64/Runtime/Output/linux-fatal-backtrace.swift.tmp/a.out 2>&1 | PYTHONPATH=/root/swift-source/build/buildbot_linux_fc25/lldb-linux-x86_64/lib/python2.7/site-packages /root/swift-source/swift/utils/symbolicate-linux-fatal /root/swift-source/build/buildbot_linux_fc25/swift-linux-x86_64/test-linux-x86_64/Runtime/Output/linux-fatal-backtrace.swift.tmp/a.out - | /root/swift-source/swift/utils/backtrace-check -u
--
Exit Code: 1

Command Output (stderr):
--
Traceback (most recent call last):
File "/root/swift-source/swift/utils/symbolicate-linux-fatal", line 30, in <module>
 import lldb
File "/root/swift-source/build/buildbot_linux_fc25/lldb-linux-x86_64/lib/python2.7/site-packages/lldb/__init__.py", line 43, in <module>
 _lldb = swig_import_helper()
File "/root/swift-source/build/buildbot_linux_fc25/lldb-linux-x86_64/lib/python2.7/site-packages/lldb/__init__.py", line 42, in swig_import_helper
 return importlib.import_module('_lldb')
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
 __import__(name)
ImportError: No module named _lldb
Traceback (most recent call last):
File "/root/swift-source/swift/utils/backtrace-check", line 81, in <module>
 main()
File "/root/swift-source/swift/utils/backtrace-check", line 78, in main
 assert(found_stack_trace_entry)
AssertionError

--

********************
}}

@alblue
Copy link

alblue commented Feb 6, 2017

Will investigate further regarding this test. If you have an easy way of reproducing it with a docker container, that would be very helpful.

Fortunately it's testing a standalone tool that helps symbolicate crash references, rather than being part of the core Swift runtime, so if this is the only test that fails then you should be good to go.

@swift-ci
Copy link
Author

swift-ci commented Feb 7, 2017

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

I tried to reproduce the Fedora 24 compilation & symbolicate-linux-fatal failure using https://github.com/FedoraSwift/fedora-swift and fedora:24 docker image. I hit number of missing packages not provisioned by swiftbuild.sh setup. So far I added the following:

dnf install autoconf sudo git automake libtool make
# The below are for Fedora 22, not sure if they are suitable for 24?)
https://copr-be.cloud.fedoraproject.org/results/lebauce/Darling/fedora-22-x86_64/libkqueue-2.0.1-1/libkqueue-devel-2.0.1-1.x86_64.rpm
https://copr-be.cloud.fedoraproject.org/results/lebauce/Darling/fedora-22-x86_64/libkqueue-2.0.1-1/libkqueue-2.0.1-1.x86_64.rpm

That's still not enough, current failure is:

[89/320] CompileC: closure/data.c
FAILED: ../build/buildbot_linux/foundation-linux-x86_64/Foundation/closure/data.c.o
mkdir -p `dirname ../build/buildbot_linux/foundation-linux-x86_64/Foundation/closure/data.c.o`; /root/tmp/swiftbuild/build/buildbot_linux/llvm-linux-x86_64/bin/clang -fcolor-diagnostics -fdollars-in-identifiers -fblocks -fobjc-runtime=macosx-10.11 -fintegrated-as -fPIC --target=x86_64-linux-gnu -O2   -Ibootstrap/common/usr/include -Ibootstrap/common/usr/local/include   -Ibootstrap/x86_64-linux-gnu/usr/include -Ibootstrap/x86_64-linux-gnu/usr/local/include  -I../build/buildbot_linux/foundation-linux-x86_64/Foundation -I../build/buildbot_linux/foundation-linux-x86_64 -I../build/buildbot_linux/foundation-linux-x86_64/Foundation/usr//lib/swift -I../build/buildbot_linux/foundation-linux-x86_64/Foundation/usr//lib/swift/CoreFoundation -I../build/buildbot_linux/foundation-linux-x86_64/Foundation/usr//lib/swift/CoreFoundation -I../build/buildbot_linux/foundation-linux-x86_64/Foundation/usr//lib/swift/CoreFoundation -DDEPLOYMENT_TARGET_LINUX -D_GNU_SOURCE -DCF_CHARACTERSET_DATA_DIR="CoreFoundation/CharacterSets"-DU_SHOW_DRAFT_API -DCF_BUILDING_CF -DDEPLOYMENT_RUNTIME_SWIFT -fconstant-cfstrings -fexceptions -Wno-shorten-64-to-32 -Wno-deprecated-declarations -Wno-unreachable-code -Wno-conditional-uninitialized -Wno-unused-variable -Wno-int-conversion -Wno-unused-function -I/usr/include/libxml2 -I/usr/include/curl -I./ -DDEPLOYMENT_ENABLE_LIBDISPATCH -I/root/tmp/swiftbuild/swift-corelibs-libdispatch -I/root/tmp/swiftbuild/build/buildbot_linux/libdispatch-linux-x86_64/tests -include CoreFoundation/Base.subproj/CoreFoundation_Prefix.h  -c closure/data.c -o ../build/buildbot_linux/foundation-linux-x86_64/Foundation/closure/data.c.o
In file included from <built-in>:1:
In file included from ./CoreFoundation/Base.subproj/CoreFoundation_Prefix.h:30:
../build/buildbot_linux/foundation-linux-x86_64/Foundation/usr//lib/swift/CoreFoundation/CFBase.h:72:10: fatal error: 'Block.h' file not found
#include <Block.h>
         ^~~~~~~~~
1 error generated.

I'll try https://github.com/FedoraSwift/fedora-swift2 with f24, or https://github.com/FedoraSwift/fedora-swift with centos7.2. I don't fully understand the reasons & differences between the two repos.

@swift-ci
Copy link
Author

swift-ci commented Feb 7, 2017

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

Tried building fedora-swift2 with the following buildswift config:

TAG=""
BRANCH="master"
FEDORA_VERSION=25

However, this fails rather quickly with:

+ swiftutils/utils/update-checkout --scheme master --clone --tag --reset-to-remote
usage: update-checkout [-h] [--clone] [--clone-with-ssh] [--skip-history]
                       [--skip-repository DIRECTORY] [--scheme BRANCH-SCHEME]
                       [--reset-to-remote] [--clean] [--config CONFIG]
                       [--github-comment GITHUB-COMMENT] [--dump-hashes]
                       [--dump-hashes-config BRANCH-SCHEME-NAME]
                       [--tag TAG-NAME] [-j N_PROCESSES]
update-checkout: error: argument --tag: expected one argument

Seems like the following comment is invalid:
https://github.com/FedoraSwift/fedora-swift2/blame/1154b5e67fedc5db67cffb4d6ee54213c0680277/README.md#L76

I'll do a bit more work to progress, but would appreciate better build script to repro the problem.

@swift-ci
Copy link
Author

swift-ci commented Feb 7, 2017

Comment by Timothy S Hawkins (JIRA)

hmmm, i though i fixed this, i will test timorrow, i just build a cooy on fedora 25 of swift-3.1-branch which worked fine.

@swift-ci
Copy link
Author

swift-ci commented Feb 7, 2017

Comment by Timothy S Hawkins (JIRA)

Note, even getting past this problem, there are a bunch of issues in the module-import side of the compiler, where it makes a bunch of assumptiins about versiins and layout of gcc that are only true on ubuntu, fedora has gcc6 ubuntu is still running gcc5 . But at least we can get this one sorted.

@swift-ci
Copy link
Author

swift-ci commented Feb 7, 2017

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

I tried fedora-swift2 with the following bulidswift:

TAG="swift-3.0.2-RELEASE"
BRANCH="master"
FEDORA_VERSION=25

That progressed much further, but still failed. This time with:

[4/548] Building CXX object lib/Basic/CMakeFiles/swiftBasic.dir/SourceLoc.cpp.o
FAILED: lib/Basic/CMakeFiles/swiftBasic.dir/SourceLoc.cpp.o
/usr/bin/clang++   -DGTEST_HAS_RTTI=0 -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib/Basic -I/root/swift-source/swift/lib/Basic -I/root/swift-source/swift/include -Iinclude -I/root/swift-source/llvm/include -I/root/swift-source/build/buildbot_linux_swift-3.0.2-RELEASE_fc25/llvm-linux-x86_64/include -I/root/swift-source/build/buildbot_linux_swift-3.0.2-RELEASE_fc25/llvm-linux-x86_64/tools/clang/include -I/root/swift-source/llvm/tools/clang/include -I/root/swift-source/cmark/src -I/root/swift-source/build/buildbot_linux_swift-3.0.2-RELEASE_fc25/cmark-linux-x86_64/src -fno-stack-protector -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Werror=date-time -std=c++11 -fcolor-diagnostics -ffunction-sections -fdata-sections -Wdocumentation -Wimplicit-fallthrough -Wunreachable-code -Woverloaded-virtual -O2    -UNDEBUG  -fno-exceptions -fno-rtti -I/usr/include -target x86_64-unknown-linux-gnu -O2 -momit-leaf-frame-pointer -g0 -UNDEBUG -MD -MT lib/Basic/CMakeFiles/swiftBasic.dir/SourceLoc.cpp.o -MF lib/Basic/CMakeFiles/swiftBasic.dir/SourceLoc.cpp.o.d -o lib/Basic/CMakeFiles/swiftBasic.dir/SourceLoc.cpp.o -c /root/swift-source/swift/lib/Basic/SourceLoc.cpp
/root/swift-source/swift/lib/Basic/SourceLoc.cpp:85:15: error: use of overloaded operator '=' is ambiguous (with operand types 'std::pair<const char *, const VirtualFile *>' and 'void')
  CachedVFile = {};
  ~~~~~~~~~~~ ^ ~~
/usr/bin/../lib/gcc/x86_64-redhat-linux/6.3.1/../../../../include/c++/6.3.1/bits/stl_pair.h:376:7: note: candidate function
      operator=(typename conditional<
      ^
/usr/bin/../lib/gcc/x86_64-redhat-linux/6.3.1/../../../../include/c++/6.3.1/bits/stl_pair.h:359:7: note: candidate function
      operator=(typename conditional<
      ^
/usr/bin/../lib/gcc/x86_64-redhat-linux/6.3.1/../../../../include/c++/6.3.1/bits/stl_pair.h:370:7: note: candidate function has been explicitly deleted
      operator=(typename conditional<
      ^
/root/swift-source/swift/lib/Basic/SourceLoc.cpp:102:15: error: use of overloaded operator '=' is ambiguous (with operand types 'std::pair<const char *, const VirtualFile *>' and 'void')
  CachedVFile = {};
  ~~~~~~~~~~~ ^ ~~
/usr/bin/../lib/gcc/x86_64-redhat-linux/6.3.1/../../../../include/c++/6.3.1/bits/stl_pair.h:376:7: note: candidate function
      operator=(typename conditional<
      ^
/usr/bin/../lib/gcc/x86_64-redhat-linux/6.3.1/../../../../include/c++/6.3.1/bits/stl_pair.h:359:7: note: candidate function
      operator=(typename conditional<
      ^
/usr/bin/../lib/gcc/x86_64-redhat-linux/6.3.1/../../../../include/c++/6.3.1/bits/stl_pair.h:370:7: note: candidate function has been explicitly deleted
      operator=(typename conditional<
      ^
2 errors generated.

Like before, thawkins (JIRA User), if you can provide me with a reliable repro of the symbolicate-linux-fatal, I'll try fixing that issue.

@swift-ci
Copy link
Author

swift-ci commented Feb 7, 2017

Comment by Timothy S Hawkins (JIRA)

use:

on Fc25

BRANCH="swift-3.1-branch"
TAG=""
That runs to completion. Refresh your scripts, as i just found an issue with detecting empty tags.

Im not sure what happened with

BRANCH="master"
TAG="swift-3.0.2-RELEASE"

It was working a few days ago,

I will investigate further

@swift-ci
Copy link
Author

swift-ci commented Feb 7, 2017

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

thawkins (JIRA User) Thanks, I pulled and kicked off another build.

I'm assuming this will not repro the symbolicate-linux-fatal error, thanks to the worked-around here:
https://github.com/FedoraSwift/fedora-swift2/blob/1154b5e67fedc5db67cffb4d6ee54213c0680277/buildswift#L180-L197

I had a quick look at the presets, and it seems like I should be able to repro the issue if I kill all of this:
https://github.com/FedoraSwift/fedora-swift2/blob/1154b5e67fedc5db67cffb4d6ee54213c0680277/buildswift#L171-L197

Correct?

@swift-ci
Copy link
Author

swift-ci commented Feb 7, 2017

Comment by Timothy S Hawkins (JIRA)

I found this on:
https://github.com/linux-on-ibm-z/docs/wiki/Building-Swift

Additional Notes (For SLES 12-SP1 & RHEL 7.1):

Currently, the build will stop during the installation phase due to a known issue in LLDB's CMake files.

This is a LLDB bug (https://llvm.org/bugs/show_bug.cgi?id=23785), preventing the installation of REPL/LLDB on SLES and RHEL:

CMake Error at scripts/cmake_install.cmake:36 (file):
file INSTALL cannot find
"/localbox/bryanpkc/swift/build/Ninja-RelWithDebInfoAssert/lldb-linux-s390x/lib/python2.7".
Call Stack (most recent call first):
cmake_install.cmake:42 (include)
According to SR-39 (https://bugs.swift.org/browse/SR-39), this can be worked around by adjusting the generated Ninja scripts to use lib64/python2.7 instead of lib/python2.7. Two files need to be changed:

build/Ninja-RelWithDebInfoAssert/lldb-linux-s390x/scripts/cmake_install.cmake

Replace all occurrences of /lib with /lib64, i.e. (in vim)

:%s#/lib#/lib64#g

build/Ninja-RelWithDebInfoAssert/lldb-linux-s390x/scripts/Python/modules/readline/cmake_install.cmake

Replace all occurrences of lib/python2.7 with lib64/python2.7, i.e.

:%s#/lib/python2.7#/lib64/python2.7#g

After editing the cmake_install.cmake files as described, the (incremental, i.e. no -c) build-script command can be re-issued and it will pick from where it left off and finish the installation. The corrected files will continue to work for future incremental builds unless the build is reconfigured with CMake.

@swift-ci
Copy link
Author

swift-ci commented Feb 7, 2017

Comment by Timothy S Hawkins (JIRA)

@gregor, yes, the double build was my workaround, it runs the build without tests first and then hacks the image in build/directory to copy all in lldb/lib to lldb/lib64 and vice versa so that both /lib and /lib64 end up being the same, and contain both the contents of /lib and /lib64

I also had to add a link to liblldb.so.3 as an entrypoint in that folder, as the Jetbrains Clion IDE looks for that to invoke the debugger,

@swift-ci
Copy link
Author

swift-ci commented Feb 7, 2017

Comment by Timothy S Hawkins (JIRA)

TAG=""
BRANCH="master"

builds ok too if you want to live on the edge.

@swift-ci
Copy link
Author

swift-ci commented Feb 8, 2017

Comment by Timothy S Hawkins (JIRA)

Hi Gregor

I have upgraded the build scripts again,

1. Deal with the issue in the source/swift/lib/Basic/SourceLoc.cpp that stops 3.0.2 from building
2. Add --privilege=true to container invocation, so that REPL and lldb get the kernel privileges required to work inside the container
3. Set 3.0.2 on f25 as default as that is the current stable release.

3.0.2 may be a better tag to work with as it wont be a moving target.

@swift-ci
Copy link
Author

swift-ci commented Feb 9, 2017

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

thawkins (JIRA User) I have managed to build successfully, so I'm set up properly. I'll try look at fixing the symbolicate tool tests asap (unfortunately away for hols for a week starting tomorrow).

Separately, I wanted to build CentOS. Do you know if those scripts work OK?

@swift-ci
Copy link
Author

swift-ci commented Feb 9, 2017

Comment by Timothy S Hawkins (JIRA)

Im pretty certain, it wont work on centos7 as is, i was going to apply some effort to getting it to work, its going to need a different set of package installs etc. And the gcc version is different again, but iys a lot of demand, so it is worth the effort. If you can fix the lldb issue, i suspect it will also work on rhel/centos.

Im having to spend some of my time on continuing my learning of swift, im writting a 3d printing slicer and i wanted to use swift to do it. This project was just a step on the way to that end.

@swift-ci
Copy link
Author

swift-ci commented Feb 9, 2017

Comment by Vladislav Dembskiy (JIRA)

On Linux From Scratch with fresh checkout I have got the same error as Gregor reported two days ago:

CoreFoundation/Collections.subproj/CFBasicHash.c:14:10: fatal error: 'Block.h' file not found
#include <Block.h>

@swift-ci
Copy link
Author

Comment by Timothy S Hawkins (JIRA)

Vladislav

execute this in the build directory root, between update-checkout and build-script

sed -i '/#include <Block.h>/d' swift-corelibs-foundation/CoreFoundation/Base.subproj/CFBase.h
sed -i 's/#include <Block.h>/#include <closure\/Block.h>/g' swift-corelibs-foundation/CoreFoundation/Collections.subproj/CFBasicHash.c
sed -i 's/#include <Block.h>/#include <closure\/Block.h>/g' swift-corelibs-foundation/CoreFoundation/RunLoop.subproj/CFRunLoop.c
sed -i 's/CachedVFile = {};/CachedVFile = {nullptr, nullptr};/g' swift/lib/Basic/SourceLoc.cpp

@swift-ci
Copy link
Author

Comment by Vladislav Dembskiy (JIRA)

Thank you Timothy! It works but not everywhere. I did fresh checkout, applied your hint and got error at another place:

/mnt/swift/swift-source/swift/tools/SourceKit/lib/Support/Concurrency-libdispatch.cpp:20:10: fatal error: 'Block.h' file not found
#include <Block.h>
^
1 error generated.

Probably we need more sed commands There is a workaround. Just create a symlink from compiler-rt/lib/BlocksRuntime/Block.h to /usr/Include but it is not good.

@swift-ci
Copy link
Author

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

For those interested, I fixed the symbolicate tool failure in the following PR:
apple/swift#7685

thawkins (JIRA User) However, I don't think this is going to be enough to complete the build without the lib/lib64 workaround. It looks like cmake install is misconfigured and looks at lib rather lib64 dir. I started looking at how to best fixing that, but if you have a solution already, please let me know

@swift-ci
Copy link
Author

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

The install issue may be fixed by the following patch (applied in the lldb repo):

diff --git a/scripts/CMakeLists.txt b/scripts/CMakeLists.txt
index cd4a8e1..e490529 100644
--- a/scripts/CMakeLists.txt
+++ b/scripts/CMakeLists.txt
@@ -12,13 +12,17 @@ set(SWIG_HEADERS
 include(FindPythonInterp)

 if (NOT CMAKE_SYSTEM_NAME MATCHES "Windows")
-  set(SWIG_PYTHON_DIR
-    ${CMAKE_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX}/python${PYTHON_VERSION_MAJOR}.${PYTHON_VERSION_MINOR})
+    execute_process(
+        COMMAND
+            ${PYTHON_EXECUTABLE}
+                "-c"
+                "from distutils.sysconfig import get_python_lib; print(get_python_lib(True, False, \"${CMAKE_BINARY_DIR}\"))"
+        OUTPUT_VARIABLE SWIG_PYTHON_DIR)
 else()
   set(SWIG_PYTHON_DIR ${CMAKE_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX}/site-packages)
 endif()

-set(SWIG_INSTALL_DIR lib${LLVM_LIBDIR_SUFFIX})
+string(REGEX MATCH "lib/|lib64/" SWIG_INSTALL_DIR ${SWIG_PYTHON_DIR})

 # Generating the LLDB framework correctly is a bit complicated because the
 # framework depends on the swig output.

I'll run local tests first to see whether it doesn't explode + helps.

@swift-ci
Copy link
Author

Comment by Timothy S Hawkins (JIRA)

This is great gregor, is this something that can be commited back to tbe lldb repo, and potentialy back to upstream as they have the same problem I belive.

Thank you for your hard work, we are definatly gettIng closer to having swift "just build" on fedora/centos/redhat. This was one of the major hurdles.

With this fixed i will be able to dump the double build and reenable all the tests.

@alblue
Copy link

alblue commented Feb 24, 2017

Swift maintains its own fork of lldb which this will need to be reviewed for first of all, and then subsequently the change can be merged or cherry-picked back up to the master lldb repository. So it can be done but might take a few cycles to go through all of the processes to make it happen, as well as responding to reviews that may come up as a result of the suggested fix.

@swift-ci
Copy link
Author

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

thawkins (JIRA User), yes, it really belongs in upstream lldb, not in Swift fork of lldb repo. However, it may be simpler for us to test it out in the Swift repo first. My aim is to have Fedora build without the lib64/lib unification workaround. I don't think we are there yet. The lldb stuff is one thing, there were also some sourcekitd issues I hit, which for now I just worked around by disabling it.

@swift-ci
Copy link
Author

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

Fixed some new-line mistake in the patch attached below, and pushed it as a PR:
https://github.com/apple/swift-lldb/pull/153/files

Restarting the full build now to see what other issues crop up.

@swift-ci
Copy link
Author

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

thawkins (JIRA User) I started looking at the Block.h issues. It looks to me like the patch you're making to corelibs-foundation is effectively a bug there. Would you mind submitting a PR for it (or I can, if you'd prefer).

@swift-ci
Copy link
Author

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

thawkins (JIRA User) spamming you further. The build completes successfully for me with the following changes:

diff --git a/buildswift b/buildswift
index 5bbc46e..dad351c 100755
--- a/buildswift
+++ b/buildswift
@@ -31,6 +32,7 @@ $SUDO dnf install -y --best --allowerasing git \
     libedit-devel \
     libxml2-devel \
     libsqlite3x-devel \
+    libatomic \
     curl \
     curl-devel \
     file \

Could you apply the above to your repo pls?

The other one, in swift repo, disables sourcekitd (it fails on BlocksRuntime):

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 2afb3dc..e39bad6 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -363,7 +363,7 @@ if("${CMAKE_SYSTEM_NAME}" STREQUAL "Darwin")
   set(SWIFT_BUILD_SOURCEKIT_default TRUE)
 elseif("${CMAKE_SYSTEM_NAME}" STREQUAL "Linux")
   if(EXISTS ${SWIFT_PATH_TO_LIBDISPATCH_SOURCE})
-    set(SWIFT_BUILD_SOURCEKIT_default TRUE)
+    set(SWIFT_BUILD_SOURCEKIT_default FALSE)
   else()
     set(SWIFT_BUILD_SOURCEKIT_default FALSE)
   endif()

@swift-ci
Copy link
Author

Comment by Timothy S Hawkins (JIRA)

I will try and get this done tomorrow

@swift-ci
Copy link
Author

Comment by Vladislav Dembskiy (JIRA)

Sorry for interrupting you but would you be so kind to explain why building standalone libBlocksRuntime in unacceptable solution? libblocksruntime-dev is listed within requirements in swift README file. Unfortunately not every distribution has this library.
I have build the library on Linux From Scratch and on Gentoo. And sourcekit compiled without errors.
Please, see the bug https://bugs.swift.org/browse/SR-3998

@swift-ci
Copy link
Author

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

Vladislav (JIRA User) the sourcekitd build disabling was/is a workaround. The ideal solution would be to get the relevant distros to package up libBlocksRuntime. Less ideal would be to build and distribute it with Swift (the canonical sources for blocks runtime is: https://github.com/apple/swift-compiler-rt/tree/58b2838add7be5e265ea1f8c30bebef9cf078e56/lib/BlocksRuntime). First once would require working with distros + new release (likely a while), second one requires a bit of work on the build system (from what I could see there isn't a cmake set up to build those BlocksRuntime yet). Yo'r approach of using external git repo that packages up BlocksRuntime works, but wouldn't work if you were to include the platform into ci.swift.org (depends if you want a one-off build, or a supported platform).

@swift-ci
Copy link
Author

swift-ci commented Mar 3, 2017

Comment by Gregor Milos (Grzegorz Miłoś) (JIRA)

thawkins (JIRA User) hey, I haven't seen corelibs-foundation PR from you yet. Just a friendly reminder, I'm happy to push that in if you prefer.

@swift-ci
Copy link
Author

swift-ci commented Mar 3, 2017

Comment by Timothy S Hawkins (JIRA)

Im a bit tied up and overwelmed at the moment, please go ahead.

@swift-ci
Copy link
Author

Comment by Timothy S Hawkins (JIRA)

Gregor

Have you managed to get the lldb fix commited, im still getting this test failure in my swift-3.1 builds.

FAIL: Swift(linux-x86_64) :: Runtime/linux-fatal-backtrace.swift (8140 of 9137)
******************** TEST 'Swift(linux-x86_64) :: Runtime/linux-fatal-backtrace.swift' FAILED ********************
Script:
--
rm -rf /root/swift-source/build/buildbot_linux_swift-3.1-branch_fc25/swift-linux-x86_64/test-linux-x86_64/Runtime/Output/linux-fatal-backtrace.swift.tmp
mkdir -p /root/swift-source/build/buildbot_linux_swift-3.1-branch_fc25/swift-linux-x86_64/test-linux-x86_64/Runtime/Output/linux-fatal-backtrace.swift.tmp
/root/swift-source/build/buildbot_linux_swift-3.1-branch_fc25/swift-linux-x86_64/bin/swiftc -target x86_64-unknown-linux-gnu  -module-cache-path '/tmp/swift-testsuite-clang-module-cache7nZJP9' -swift-version 3  /root/swift-source/swift/test/Runtime/linux-fatal-backtrace.swift -o /root/swift-source/build/buildbot_linux_swift-3.1-branch_fc25/swift-linux-x86_64/test-linux-x86_64/Runtime/Output/linux-fatal-backtrace.swift.tmp/a.out
not --crash /root/swift-source/build/buildbot_linux_swift-3.1-branch_fc25/swift-linux-x86_64/test-linux-x86_64/Runtime/Output/linux-fatal-backtrace.swift.tmp/a.out 2>&1 | PYTHONPATH=/root/swift-source/build/buildbot_linux_swift-3.1-branch_fc25/lldb-linux-x86_64/lib/python2.7/site-packages /root/swift-source/swift/utils/symbolicate-linux-fatal /root/swift-source/build/buildbot_linux_swift-3.1-branch_fc25/swift-linux-x86_64/test-linux-x86_64/Runtime/Output/linux-fatal-backtrace.swift.tmp/a.out - | /root/swift-source/swift/utils/backtrace-check -u
--
Exit Code: 1

Command Output (stderr):
--
Traceback (most recent call last):
  File "/root/swift-source/swift/utils/symbolicate-linux-fatal", line 30, in <module>
    import lldb
  File "/root/swift-source/build/buildbot_linux_swift-3.1-branch_fc25/lldb-linux-x86_64/lib/python2.7/site-packages/lldb/__init__.py", line 43, in <module>
    _lldb = swig_import_helper()
  File "/root/swift-source/build/buildbot_linux_swift-3.1-branch_fc25/lldb-linux-x86_64/lib/python2.7/site-packages/lldb/__init__.py", line 42, in swig_import_helper
    return importlib.import_module('_lldb')
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named _lldb
Traceback (most recent call last):
  File "/root/swift-source/swift/utils/backtrace-check", line 81, in <module>
    main()
  File "/root/swift-source/swift/utils/backtrace-check", line 78, in main
    assert(found_stack_trace_entry)
AssertionError

--

********************

@alblue
Copy link

alblue commented Mar 13, 2017

LLVM 4.0 has been tagged, which meant it was closed for upstream contributions. I think that since the PR for the swift-llvm branch wasn't seen as Swift specific it should be punted upstream, but I don't know when this will happen.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants