Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SR-6084] libdispatch missing from Trunk Development snapshots #675

Closed
swift-ci opened this issue Oct 7, 2017 · 21 comments
Closed

[SR-6084] libdispatch missing from Trunk Development snapshots #675

swift-ci opened this issue Oct 7, 2017 · 21 comments
Labels
bug libdispatch

Comments

@swift-ci
Copy link

swift-ci commented Oct 7, 2017

Previous ID SR-6084
Radar rdar://problem/35040697
Original Reporter dowobeha (JIRA User)
Type Bug
Status Resolved
Resolution Done
Additional Detail from JIRA
Votes 1
Component/s libdispatch
Labels Bug
Assignee dgrove-oss (JIRA)
Priority Medium

Watchers: @shahmishal

md5: 5685fbff17fef7e5a99d2d3c5371f970

Issue Description:

In all trunk development releases for Ubuntu 16.04 currently listed on the Swift downloads page, libdispatch is absent.

#!/bin/bash                                                                                         

dates="2017-09-22 2017-09-23 2017-09-24 2017-09-25 2017-09-26 2017-09-27 2017-09-28 2017-09-29 2017-09-30 2017-10-01 2017-10-06"

for date in ${dates}; do

    echo "Downloading swift-DEVELOPMENT-SNAPSHOT-${date}-a-ubuntu16.04.tar.gz..."
    wget https://swift.org/builds/development/ubuntu1604/swift-DEVELOPMENT-SNAPSHOT-${date}-a/swift-DEVELOPMENT-SNAPSHOT-${date}-a-ubuntu16.04.tar.gz &> /dev/null

    tar xfvz swift-DEVELOPMENT-SNAPSHOT-${date}-a-ubuntu16.04.tar.gz &> /dev/null

done

echo "Searching for libdispatch..."
find swift-DEVELOPMENT-SNAPSHOT-* -name "*dispatch*"
@belkadan
Copy link

belkadan commented Oct 9, 2017

Yep, we've been reworking the build system, and keep hoping it'll be fixed any day now.

cc @compnerd, das (JIRA User), mww (JIRA User), dgrove-oss (JIRA User)

@compnerd
Copy link
Collaborator

compnerd commented Oct 9, 2017

Hmm, the install rules should install it now. I fixed the last of the install issues recently. Can someone test if this still is a problem?

@belkadan
Copy link

belkadan commented Oct 9, 2017

Looks like it's still missing from today's snapshot, but maybe it's tomorrow's that'll have all the necessary changes?

@compnerd
Copy link
Collaborator

compnerd commented Oct 9, 2017

Hmm, do we have the logs for the build? I think that the changes went in on Friday, so I would expect the snapshot to contain it. Maybe the build script needs more changes?

@belkadan
Copy link

belkadan commented Oct 9, 2017

Should be this build, I think: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-16_04/450/

@compnerd
Copy link
Collaborator

compnerd commented Oct 9, 2017

      • Installing libdispatch —
        [1/2] Install the project...
        – Install configuration: "Release"
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/base.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/block.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/data.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/dispatch.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/group.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/introspection.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/io.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/object.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/once.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/queue.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/semaphore.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/source.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/time.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/module.modulemap
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/os/object.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/os/linux_base.h
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/lib/libdispatch.so
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/lib/swift/linux/x86_64/Dispatch.swiftmodule
        – Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/lib/swift/linux/x86_64/Dispatch.swiftdoc

The installation is definitely occurring. The question is, why is that not getting included in the tar ball?

pushd /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/
tar -c -z -f /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-DEVELOPMENT-SNAPSHOT-2017-10-08-a-ubuntu16.04.tar.gz --owner=0 --group=0 usr/

So, this should really pick that up AFAICT. What am I missing?

@belkadan
Copy link

belkadan commented Oct 9, 2017

Whoops, well, the build log I showed is for October 9, and the most recent snapshot on the site is October 8, so that might not be the same.

@ankitspd
Copy link
Member

This completely breaks the package manager. Is the CI not running swift-integration-tests?
/cc @shahmishal

@shahmishal
Copy link
Member

apple/swift#12383

@swift-ci
Copy link
Author

Comment by David Grove (JIRA)

fwiw, I did a local build and I think the installation path is not right for the libdispatch install. I'm invoking build-script with install_destdir=/home/dgrove/swift/swift-local-install.

That install_destdir path is getting doubly copied. Ie, libdispatch.so is getting put into

/home/dgrove/swift/swift-local-install/home/dgrove/swift/swift-local-instal/usr/bin/libdispatch.so.

It should be located at /home/dgrove/swift/swift-local-instal/usr/bin/libdispatch.so.

Perhaps this stanza in src/CMakelists.txt is wrong?

install(TARGETS
{{ dispatch}}
{{ DESTINATION}}
{{ "${CMAKE_INSTALL_FULL_LIBDIR}")}}

@swift-ci
Copy link
Author

Comment by David Grove (JIRA)

I think this might do it: #313

It at least puts the files in the right place for me when building manually.

@swift-ci
Copy link
Author

Comment by Oliver Chick (JIRA)

It looks like libdispatch is still missing from the nightlies:

➜  DEVELOPMENT-SNAPSHOT-2017-10-16-a git:(master) ✗ pwd
/home/oliver/.swiftenv/versions/DEVELOPMENT-SNAPSHOT-2017-10-16-a
➜  DEVELOPMENT-SNAPSHOT-2017-10-16-a git:(master) ✗ find . -name libdispatch.a
➜  DEVELOPMENT-SNAPSHOT-2017-10-16-a git:(master) ✗ find . -name libdispatch.so

@swift-ci
Copy link
Author

Comment by David Grove (JIRA)

ok. I've taken a look at the build logs from the job that produced 2017-10-16a. I'm not sure if this is a problem or not, but the first time libdispatch is built and installed it is being given an install path that is derived from SourceKit.

Here's a snippet of the log:

[1/2] cd /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/libdispatch-linux-x86_64 && /usr/bin/cmake -E create_symlink /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-corelibs-libdispatch/dispatch/generic/module.modulemap /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-corelibs-libdispatch/dispatch/module.modulemap && /usr/bin/cmake -E create_symlink /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-corelibs-libdispatch/private/generic/module.modulemap /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-corelibs-libdispatch/private/module.modulemap
[1/2] Install the project...
-- Install configuration: ""
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/base.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/block.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/data.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/dispatch.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/group.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/introspection.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/io.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/object.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/once.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/queue.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/semaphore.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/source.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/time.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/dispatch/module.modulemap
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/os/object.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/include/os/linux_base.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/lib/libdispatch.so
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/lib/swift/linux/x86_64/Dispatch.swiftmodule
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/tools/SourceKit/libdispatch-prefix/lib/swift/linux/x86_64/Dispatch.swiftdoc
[890/898] Completed 'libdispatch'
[891/898] Linking CXX shared library lib/libsourcekitdInProc.so

Much later in the log, libdispatch is installed again, but still has the duplicated path fragment that I thought was fixed by libdispatch PR 313. I don't understand why this is happening.

-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/bin/swift-build-tool
<div class="collapseHeader">Installing libdispatch<div class="collapseAction"><p onClick="doToggle(this)">Hide Details</p></div></div><div class="expanded">--- Installing libdispatch ---
</div>+ env DESTDIR=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/ /usr/bin/cmake --build /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/libdispatch-linux-x86_64 -- install
[1/2] cd /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/libdispatch-linux-x86_64 &amp;&amp; /usr/bin/cmake -E create_symlink /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-corelibs-libdispatch/dispatch/generic/module.modulemap /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-corelibs-libdispatch/dispatch/module.modulemap &amp;&amp; /usr/bin/cmake -E create_symlink /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-corelibs-libdispatch/private/generic/module.modulemap /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-corelibs-libdispatch/private/module.modulemap
[1/2] Install the project...
-- Install configuration: "Release"
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/base.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/block.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/data.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/dispatch.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/group.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/introspection.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/io.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/object.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/once.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/queue.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/semaphore.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/source.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/time.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/module.modulemap
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/os/object.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/os/linux_base.h
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/lib/libdispatch.so
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/lib/swift/linux/x86_64/Dispatch.swiftmodule
-- Installing: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/lib/swift/linux/x86_64/Dispatch.swiftdoc
<div class="collapseHeader">Installing foundation<div class="collapseAction"><p onClick="doToggle(this)">Hide Details</p></div></div><div class="expanded">--- Installing foundation ---
</div>+ pushd /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-corelibs-foundation
~/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-corelibs-foundation ~/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04
+ /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/ninja-build/ninja install
[1/1] mkdir -p "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install//usr//lib/swift/linux"; cp "../build/buildbot_linux/foundation-linux-x86_64/Foundation/libFoundation.so" "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install//usr//lib/swift/linux"; mkdir -p "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install//usr//lib/swift_static/linux"; cp "../build/buildbot_linux/foundation-linux-x86_64/Foundation/libFoundation.a" "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install//usr//lib/swift_static/linux"; mkdir -p "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install//usr//lib/swift/linux/x86_64"; cp "../build/buildbot_linux/foundation-linux-x86_64/Foundation/Foundation.swiftmodule" "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install//usr//lib/swift/linux/x86_64/"; cp "../build/buildbot_linux/foundation-linux-x86_64/Foundation/Foundation.swiftdoc" "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install//usr//lib/swift/linux/x86_64/"; mkdir -p "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install//usr//local/include"; rsync -a "../build/buildbot_linux/foundation-linux-x86_64/Foundation//usr//lib/swift/CoreFoundation" "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install//usr//lib/swift/"
+ popd

Note the problem path:
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/swift-nightly-install/usr/include/dispatch/base.h

@alblue
Copy link

alblue commented Oct 17, 2017

So the reason why libdiapatch and sourcekit are related is due to making sourcekit available on Linux.

SourceKit is in the swift project and built with make, but it depends on dispatch at the implementation level. On Darwin, this comes from the system, so it works.

On Linux we have to build dispatch first. But the libdispatch project has a build that compiles both the native library and also the swift representations. We can’t build the swift representations before swift itself is built, so we have a circular dependency.

The solution at the time was to build the C based libdispatch first and privately for sourcekit, then blow it away and recompile it again for swifts usage later. That way SourceKit could be built for Linux and made available in the build.

I suspect that if both SourceKit and Dispatch could be converted to use Cmake, then the dependencies for SourceKit -> libdispatch and Swift Dispatch -> libdispatch would be easier to resolve automatically. But on Darwin this circular dependency is sidestepped because we have SourceKit -> systemdispatch and Swift separately.

@alblue
Copy link

alblue commented Oct 17, 2017

NB the last successful build which contained dispatch was 18 September, which was much later than the SourceKit/dispatch changes (SR-1676). Builds from 19 September don’t have dispatch, so that’s when the regression was introduced.

@swift-ci
Copy link
Author

Comment by David Grove (JIRA)

I'm pretty sure the regression matches to switching to CMake to drive the build of dispatch.

The SourceKit snippet I quoted might be fine, but I honestly don't understand how the dependencies are being managed within CMake to ensure that the second build of libdispatch starts from a "clean enough" slate. So I grabbed the log snippets from both installs of libdispatch out of paranoia.

@ddunbar
Copy link
Member

ddunbar commented Oct 17, 2017

@swift-ci create

@ankitspd
Copy link
Member

apple/swift#12502
#314

@belkadan
Copy link

This happened, right?

@swift-ci
Copy link
Author

Comment by David Grove (JIRA)

Yep. Fixed. Just verified on latest 16.04 snapshot build.

dgrove$ find . -name "dispatch"
./swift-DEVELOPMENT-SNAPSHOT-2018-02-26-a-ubuntu16.04/usr/lib/swift/dispatch
./swift-DEVELOPMENT-SNAPSHOT-2018-02-26-a-ubuntu16.04/usr/lib/swift/dispatch/dispatch.h
./swift-DEVELOPMENT-SNAPSHOT-2018-02-26-a-ubuntu16.04/usr/lib/swift/linux/libdispatch.so

@compnerd
Copy link
Collaborator

I believe that this is fixed now.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 5, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug libdispatch
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@compnerd @alblue @ddunbar @belkadan @shahmishal @ankitspd @swift-ci