Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-116] Provide/host pre-built Swift binary packages for different Linux distributions #42738

Open
swiftix mannequin opened this issue Dec 7, 2015 · 13 comments
Open
Labels
feature A feature request or implementation transfer candidate The issue may belong in another repository † website This issue was supposed to belong in the swift-org-website repository

Comments

@swiftix
Copy link
Mannequin

swiftix mannequin commented Dec 7, 2015

Previous ID SR-116
Radar None
Original Reporter @swiftix
Type New Feature
Additional Detail from JIRA
Votes 6
Component/s
Labels New Feature
Assignee None
Priority Medium

md5: d9fc07dd916c03f0648cc2c7263878e5

Issue Description:

The swift.org site provides pre-built binaries for Ubuntu already, but their are tarballs and not in the DEB format, which is the preferred way to install packages on Debian-based systems.
RPM-based Linux distributions would need packages in the RPM format.

It would be nice if someone would take care and prepare/host pre-built Swift packages for different Linux distributions, so that they can be installed in a regular way used by a given Linux distribution. In the ideal case, it would be nice to have something like stable packages and nightly builds.

For Ubuntu, one would probably need to create and register a package on Launchpad. For other Linux distributions similar steps would be required.

@swift-ci
Copy link
Collaborator

Comment by Ryan Lovelett (JIRA)

For those interested in a package for Arch Linux (Pacman) I have a PKGBUILD script that I am using to build Pacman packages. Once I have this settled down I'd be happy to start making a Debian package too.

Arch Linux PKGBUILD

@swift-ci
Copy link
Collaborator

Comment by Joe (JIRA)

Likewise, for those interested in Ubuntu packages for both Trusty and Wily, I'm hosting them on S3 with instructions for obtaining here: http://dev.iachieved.it/iachievedit/ubuntu-packages-for-open-source-swift/. While I do sign the packages I am not using a Launchpad PPA due to all the extra red tape it entails. There are also alpha-level ARM packages available.

@swift-ci
Copy link
Collaborator

swift-ci commented Jan 2, 2016

Comment by Jeremy Fergason (JIRA)

I've also started the process to get Swift into Fedora 23 and eventually into EPEL for RHEL 7.

You can track the progress for this via Fedora's Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1295115

Or you can download the packages before they are officially accepted:

Main Package: https://s3-us-west-2.amazonaws.com/swift-rpm/swift-lang-2.2-0.1.snapshot20151231.fc23.x86_64.rpm
Package Manager: https://s3-us-west-2.amazonaws.com/swift-rpm/swift-lang-package-manager-2.2-0.1.snapshot20151231.fc23.x86_64.rpm
Docs: https://s3-us-west-2.amazonaws.com/swift-rpm/swift-lang-docs-2.2-0.1.snapshot20151231.fc23.x86_64.rpm

SRPM: https://s3-us-west-2.amazonaws.com/swift-rpm/swift-lang-2.2-0.1.snapshot20151231.fc23.src.rpm

@swift-ci
Copy link
Collaborator

swift-ci commented Jan 4, 2016

Comment by Ryan Lovelett (JIRA)

I think it might be a good idea to try and coordinate as much as possible and to the extent that it makes sense, the types and number of packages that will eventually be made and included in the package managers upstream in the distros. I have a few reasons for this. The first one is that we could then lobby/provide MRs for specific build presets that make it easier to build for packaging in Linux distros much the same way the core team has done for OS X.

For instance, jdfergason (JIRA User) I noticed your providing three packages. I cannot help but think that is a really good idea. In fact, looking at the available build parameters of `build-script` it sort of makes me wonder. Would it be better to provide a few more? Specifically breaking out a swift-lldb package?

Another thing that Arch's packaging system (and I'm sure others do as well) provide are multiple steps (e.g., patch, build, test, package). Right now I wonder how, if at all other packagers manage this? Do you not worry about it?

Side note, to jdfergason (JIRA User), would you mind sharing your arguments to build-script to build each of those packages you've made.

@swift-ci
Copy link
Collaborator

swift-ci commented Jan 5, 2016

Comment by Jeremy Fergason (JIRA)

rlovelett (JIRA User) I totally agree about coordinating the way things are packaged across distributions where ever possible.

I agree that maybe there should be some more packages. Considering build-script, perhaps:

  • swift-lang (swift compiler command --llbuild)

  • swift-lang-repl (swift repl --lldb)

  • swift-lang-package-manager (swift package manager --swiftpm)

  • swift-lang-xctest (xctest --xctest)

  • swift-lang-foundation (foundation library --foundation)

  • swift-docs (generated documentation)

In RPM there are also separate stages (prep, build, check, install, ...). Sounds similar to Arches steps.

To build the different packages in RPM I build everything and then select specific installed files for each package. Here's I call build-script multiple times depending on what stage you are in.

Build:

utils/build-script --assertions --no-swift-stdlib-assertions --llbuild --swiftpm --xctest --build-subdir=rpm --lldb --release --foundation -- --swift-enable-ast-verifier=0 --install-prefix=/usr '--swift-install-components=compiler;clang-builtin-headers;stdlib;sdk-overlay;license' --build-swift-static-stdlib=1 --skip-test-lldb=1 --reconfigure

Check:

utils/build-script --assertions --no-swift-stdlib-assertions --llbuild --swiftpm --xctest --build-subdir=rpm --lldb --release --test --validation-test --foundation -- --swift-enable-ast-verifier=0 --install-prefix=/usr '--swift-install-components=compiler;clang-builtin-headers;stdlib;sdk-overlay;license' --build-swift-static-stdlib=1 --skip-test-lldb=1 --reconfigure 

Install:

utils/build-script --assertions --no-swift-stdlib-assertions --llbuild --swiftpm --xctest --build-subdir=rpm --lldb --release --foundation -- --swift-enable-ast-verifier=0 --install-swift --install-lldb --install-llbuild --install-swiftpm --install-xctest --install-prefix=/usr '--swift-install-components=compiler;clang-builtin-headers;stdlib;sdk-overlay;license' --build-swift-static-stdlib=1 --skip-test-lldb=1 --install-destdir=%{buildroot} --install-foundation --reconfigure

@swift-ci
Copy link
Collaborator

swift-ci commented Jan 5, 2016

Comment by Jeremy Fergason (JIRA)

As I've thought about it a bit more maybe it should be called swift-lang-repl but swift-lang-lldb as originally suggested.

@swift-ci
Copy link
Collaborator

swift-ci commented Jan 5, 2016

Comment by Joe (JIRA)

What is the rationale behind so many packages? I would look at what is the minimal set of packages to actually do real work without triggering dependencies pulling in other packages anyway. Do folks really want to install swift without the swift package manager if everyone is using the swift package manager? Is including the REPL with every package really adding that much overhead (I haven't checked perhaps it is). Perhaps on smaller systems (RaspPi and a 4G SD card) splitting out things you really don't need makes more sense.

Like I said I'm currently packaging for Ubuntu and while there is definitely some automation to it I'm bundling things up for trusty, wily, jessie, on x86_64 and arm. And that's one package. Exploding it out is just to going to create more variations to manage and test.

@swift-ci
Copy link
Collaborator

swift-ci commented Jan 5, 2016

Comment by Ryan Lovelett (JIRA)

I think the impetus would be preventing package conflicts and providing a small package that can be the base runtime dependency for 3rd-party Swift packages (i.e., libswiftCore.so). Allow me to explain.

Let us say that for the sake of discussion I made a great Swift program and now I want to distribute it as a package for my distribution. I built it using Swift package manager and looking at the dependencies of the final binary (e.g., `readelf`) the application depends on 5 shared objects: libswiftCore.so, libstc++.so, libm.so, libgcc_s.so, and libc.so. Should I, the packager of the this new hypothetical project, have to depend on the Swift REPL (read Swift LLDB) when the only thing my prospective end-user will need from Swift is libswiftCore.so?

I specifically called out Swift LLDB because I would imagine that executable has a conflict with the standard LLDB package (on Debian/Ubuntu). Or put another way, I'm pretty sure the package you just described conflicts with the main LLDB package such that an end-user can only simultaneously have one installed.

Returning to my hypothetical package, which has only one Swift dependency (libswiftCore.so), given the above conflict can now only be installed on a machine that the user does not have LLDB installed on. This seems like a heavy handed dependency possibly leading to a "dependency hell" situation considering the end-user will never use Swift LLDB. For instance, a C developer who wants nothing to do with developing Swift and just wants to run this hypothetical Swift application. Though I am completely willing to concede for some people this might not be a wide spread problem. Of course it does nothing to ease the conflict of a C developer who wants to also develop Swift; but that is a different problem.

To that end, I know that Ubuntu/Debian packages typically have the concept of the library/executable package and then the developer/header packages; usually denoted by "libfoo" and "libfoo-dev". One could argue that the more pragmatic, and certainly one with precedence in existing distribution packages, would be to combine the more developer oriented parts into a package of its own with the runtime in a package of its own.

To what I assume to be your deeper question: how much separation is too much? I will concede that some of the packages listed by jdfergason (JIRA User) might not achieve the ends described above. But generally I am of the opinion that more than a single package is necessary regardless of maintenance complexity.

My goal stated plainly is to have as many distributions as humanly possible support the Swift language with main line Swift packages. With the eventual goal of making it easy for developers to package and distribute their software so that anyone can install them; not just other Swift developers. Therefore, in my opinion to do that they need to be general purpose, read: not just targeting for other Swift developers, and do the least amount of "collateral damage".

joeiachievedit (JIRA User) parenthetically, do you test installing the mainline LLDB with the Swift packages you build? Do they conflict and if so have properly mark your packages as having conflicts with LLDB?

@swift-ci
Copy link
Collaborator

swift-ci commented Jan 6, 2016

Comment by Jeremy Fergason (JIRA)

The conflict with LLDB is definitely an issue for the Fedora / RHEL / CentOS family of distributions. Additionally, it's a convention within these distributions to place docs and devel into separate packages which is why I broke those out. Regarding the other ones I was just listing which packages could be installed separately using build-script.

I think it makes sense to distribute the swift stdlib, xctest, lldb, and the package manager as separate packages because they really are distinct, albeit related tools.

@lhoward
Copy link
Contributor

lhoward commented Jan 6, 2016

I only took a quick look at the Fedora spec file so correct me if I'm wrong but – might be worth giving some thought to shared library versioning for libswiftCore.so etc, otherwise it's going to be an ABI compatibility nightmare.

@swift-ci
Copy link
Collaborator

swift-ci commented Jan 6, 2016

Comment by Joe (JIRA)

rlovelett (JIRA User) I see your point regarding creating a distribution package that depends on a few Swift dependencies. Yes, it would not make sense to pull in the REPL if you really only depend on a handful of libraries. And you are correct, I did not take into consideration lldb when I packaged the monolithic swift-2.2 I am creating for Ubuntu; it will conflict with a system lldb. Interesting enough, no one has emailed, tweeted, or left a comment regarding that until now, but that is irrelevant; I was just lucky.

So in principle we're all in agreement that we should break things up, and yes, to the deeper question my concern is how much and what those divisions are. Looking at the presets is a starting point but I'm not sure its the ending point, I liken those to build targets and not necessarily package targets. Perhaps we should take the conversation offline with jdfergason (JIRA User) and Howard (JIRA User) via e-mail or Skype and come up with a proposal to submit with all of the rationale?

PS - great catch on the lldb :sheepish grin:

@swift-ci
Copy link
Collaborator

swift-ci commented Feb 1, 2017

Comment by Timothy S Hawkins (JIRA)

+1 for this, swift seems to have failed to make it to the redhat/centos/fedora world, which is a shame as that is very much the target audience for swift/linux being the bulk of the installed server base.

@swift-ci
Copy link
Collaborator

Comment by Lane Schwartz (JIRA)

+1 I hope this is being actively worked on. As a professor, one of the only things holding me back from teaching Swift is its low platform availability (Linux support only on Ubuntu).

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@AnthonyLatsis AnthonyLatsis added feature A feature request or implementation transfer candidate The issue may belong in another repository † website This issue was supposed to belong in the swift-org-website repository and removed new feature labels Apr 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A feature request or implementation transfer candidate The issue may belong in another repository † website This issue was supposed to belong in the swift-org-website repository
Projects
None yet
Development

No branches or pull requests

3 participants