[SR-14008] "Library not loaded: /usr/lib/swift/libswift_Differentiation.dylib" issue for swift test
#4454
Labels
Comments
|
Potentially related PR: apple/swift#34216 The PR changes the install name directory ( |
|
With SIP enabled, `DYLD_*` variables will be scrubbed, which means that `DYLD_LIBRARY_PATH` would not make it to the process AIUI. A quick sanity check: $ codesign --display --entitlements :- /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Xcode/Agents/xctest
Executable=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Xcode/Agents/xctest
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.get-task-allow</key>
<true/>
</dict>
</plist>This seems to not have the required |
|
Comment by Alexey Novakov (JIRA) it seems my issue is related to this: DYLD_LIBRARY_PATH=/Users/Alexey_Novakov/Library/Developer/Toolchains/swift-5.4-DEVELOPMENT-SNAPSHOT-2021-03-25-a.xctoolchain/usr/lib/swift/macosx xcrun --toolchain "Swift Development Snapshot" swift run
[1/1] Planning build
* Build Completed!dyld: lazy symbol binding failed: Symbol not found: _swift_autoDiffCreateLinearMapContext
Referenced from: /Users/Alexey_Novakov/dev/git/swift-linear-regression/.build/x86_64-apple-macosx/debug/swift-linear-regression
Expected in: /usr/lib/swift/libswiftCore.dylib
dyld: Symbol not found: _swift_autoDiffCreateLinearMapContext
Referenced from: /Users/Alexey_Novakov/dev/git/swift-linear-regression/.build/x86_64-apple-macosx/debug/swift-linear-regression
Expected in: /usr/lib/swift/libswiftCore.dylib
zsh: abort DYLD_LIBRARY_PATH= xcrun --toolchain "Swift Development Snapshot" swift runotool paths: % otool -L .build/x86_64-apple-macosx/debug/swift-linear-regression
.build/x86_64-apple-macosx/debug/swift-linear-regression:
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1770.255.0)
/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
@rpath/libswiftCore.dylib (compatibility version 1.0.0, current version 0.0.0)
@rpath/libswiftCoreFoundation.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
@rpath/libswiftCoreGraphics.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
@rpath/libswiftDarwin.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
@rpath/libswiftDispatch.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
@rpath/libswiftFoundation.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
@rpath/libswiftIOKit.dylib (compatibility version 1.0.0, current version 1.0.0, weak)
@rpath/libswiftObjectiveC.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
@rpath/libswiftSwiftOnoneSupport.dylib (compatibility version 1.0.0, current version 0.0.0)
@rpath/libswiftXPC.dylib (compatibility version 1.0.0, current version 1.1.0, weak)
@rpath/libswift_Differentiation.dylib (compatibility version 1.0.0, current version 0.0.0) |
|
I have a reproducer for this. The Swift package tests usually run just fine, but encountering this edge case stops them from happening. To reproduce:
import _Differentiation
extension Array.DifferentiableView: RandomAccessCollection
where Element: Differentiable {
public typealias Element = Array<Element>.Element
public typealias Index = Array<Element>.Index
public typealias SubSequence = Array<Element>.SubSequence
public subscript(position: Array<Element>.Index) -> Element {
get { fatalError() }
set { fatalError() }
}
public subscript(bounds: Range<Array<Element>.Index>) -> SubSequence {
get { fatalError() }
set { fatalError() }
}
public var startIndex: Index { fatalError() }
public var endIndex: Index { fatalError() }
}
@novakov-alexey I wasn't able to reproduce your compile failure from #4454 (comment). From your recollection, did it involve something reminiscent of my reproducer? |
|
Related: tensorflow/swift-models#727 Note as a result of this, the build process for testing s4tf/s4tf (formerly swift-apis) becomes greatly more complex. But it's still possible (this is an example of a workaround). # prepare for building S4TF
export TOOLCHAINS="org.swift.57202205041a"
tensorflow_version="2.8.0"
DESTDIR=${PWD}/Library/tensorflow-${tensorflow_version}
# build S4TF
cd s4tf
swift build -Xswiftc -DTENSORFLOW_USE_STANDARD_TOOLCHAIN \
-Xcc -I${DESTDIR}/usr/include -Xlinker -L${DESTDIR}/usr/lib
cp ${DESTDIR}/usr/lib/libx10.dylib .build/$(uname -m)-apple-macosx/debug/libx10.dylib
# fails, but that's okay
swift test -Xswiftc -DTENSORFLOW_USE_STANDARD_TOOLCHAIN \
-Xcc -I${DESTDIR}/usr/include -Xlinker -L${DESTDIR}/usr/lib
export DYLD_LIBRARY_PATH="/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-05-04-a.xctoolchain/usr/lib/swift/macosx"
# actually runs the tests
/Applications/Xcode.app/Contents/Developer/usr/bin/xctest \
.build/$(uname -m)-apple-macosx/debug/TensorFlowPackageTests.xctest |
|
Perhaps I could remove this crash from S4TF by taking the Would my idea be permissible? |
|
@abertelrud @neonichu @lorentey - this echoes some of our discussion, correct? |
|
The contents of the (Beware: while The version of dyld in macOS Monterey (and later) does not support using DYLD_LIBRARY_PATH to override libraries loaded via (If this dyld limitation is causing you trouble, please do file a feedback report describing the problem -- such reports do help prioritizing the issue.) |
|
|
Also beware -- custom builds of the Swift Standard Library (whether they were produced by manually building the stdlib from the apple/swift repo, or by downloading one of the toolchain snapshots from swift.org) aren't drop-in replacements for the stdlib that ships in any OS version. Such stdlib builds are not guaranteed to interoperate with OS components that load the Swift stdlib, including Foundation. |
So if I set |
What I call the "core stdlib" lives in the The code you've linked to is part of a separate, standalone module called |
Yes; I was asking if it's okay to make a PR to apple/swift, where I add this extension to code in the |
Setting a higher minimum deployment target will cause the compiler to use absolute |
I don't see why not. Changes to private parts of the stdlib merely require an approval from a stdlib code owner -- the only challenge there is to find someone who has time to understand & review the addition. This is a much lower bar than going through Swift Evolution, but it's not entirely free: new entry points come with a code size & maintenance cost. |
|
How do you set the minimum deployment target when compiling with SwiftPM? I tried changing |
|
Did adding the version number cause the resulting binary to use |
On closer look, isn't the extension already in the |
Where did you find the extension? I didn't see anything that conforms |
It's the same before Terminal outputAnd after Terminal outputAdding |
|
Also, here's my
|
|
Ah thanks! What I missed is that you want to add this conformance to this existing type. I don't have an opinion on whether that is a desirable thing -- I am not familiar with this module -- however, I doubt moving the conformance to the Swift repository will resolve the root cause of the loader error. These messages indicate that the library The most likely cause is that the executable invoked here has an The first order of business would be to figure out which binary is doing this and fixing it. (It may well be a binary that ships in the downloadable toolchain -- I know we have such issues there, we've seen something similar last week. 😞) This implies that the loader tried to load the non-public library As the
Oh I see, so Alexey's comment above is outdated. That's great -- so the dyld issue does not apply: you only need to figure out how to get DYLD_LIBRARY_PATH to apply to the launched process. |
Most dylibs whose install names are under these directories do not actually exist there as separate files; instead, the dynamic loader picks them up from the system-wide dyld shared cache. |
That error is present on trunk snapshots, but not on 5.7 branch snapshots. For example, the 2022-05-04 and 2022-06-08 trunk snapshots produce the concurrency error. The 2022-06-04 v5.7 and 2022-06-13 v5.7 snapshots do not. I was building the package using the June 13, 2022 v5.7 Development Snapshot in #4454 (comment). Same behavior on the June 8 trunk snapshot though. Terminal output (before, then after adding platforms to Package.swift) |
Would that mean I can't access one of Apple's API frameworks like Metal if |
|
On Apple platforms, the Swift Standard Library is an integral part of the OS, and while its public ABI is stable, its binary interface with the rest of the operating system isn't. Therefore, we can only support loading higher-level OS frameworks (incl. Foundation) with the specific version of the Standard Library that ships within the same OS release. In particular, some system integration entry points that are shipping in the OS may not be present in the open source toolchains, or they may have different expected behavior across OS releases. Allowing the system entry points to change is crucial to improving and/or maintaining system performance across OS versions. Replacing the OS stdlib with alternative builds may or may not work. In general, I would expect things to mostly work when loading fresh stdlib builds on the macOS version that we have installed on ci.swift.org -- but our test suite is limited to exercising parts of Foundation (some of the bridging implementations, to be specific). It is not testing other frameworks like Metal, and it doesn't even cover all parts of Foundation. (We try to avoid intentionally breaking things, but since we're not actively testing interoperability (or even designing for it), things may be in various stages of brokenness at any given point. The more frameworks you import and the farther you get from the macOS version that's on CI, the more likely you'll run into subtle (or not so subtle) issues.) |
Let's test that assumption, because if it happens to fix the error, that's good news for S4TF. And if it doesn't fix the error, we learn that we should not add that extension to the Stdlib yet (maybe hold off on DifferentiableCollection too). |
|
I got S4TF compiling on a release tool chain (Xcode 14 beta, Swift 5.7 release). The test error still happened, even though AutoDiff is imported as a Swift package (philipturner/differentiation). This means we can narrow down the crash without building a custom toolchain. This is great news! |
Definitely. Things just got a lot wierder... I linked SR-14008 to a compiler crash that happens I did get S4TF to compile on a development toolchain + philipturner/differentiation (against my expectations), and I only had to disable the code causing this crash. Attempting to override the standard library's Reproducer (this could be narrowed down further by shrinking philipturner/differentiation): Make an empty Swift package via // swift-tools-version: 5.6
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
name: "AutoDiffPackage2",
products: [
.library(
name: "AutoDiffPackage2",
targets: ["AutoDiffPackage2"]),
],
dependencies: [
.package(url: "https://github.com/philipturner/differentiation", branch: "main"),
],
targets: [
.target(
name: "AutoDiffPackage2",
dependencies: [
.product(name: "_Differentiation", package: "differentiation"),
]),
.testTarget(
name: "AutoDiffPackage2Tests",
dependencies: ["AutoDiffPackage2"]),
]
)In the only source file within import Differentiation
extension Array.DifferentiableView: RandomAccessCollection
where Element: Differentiable {
// All of these properties must be public.
public typealias Element = Array<Element>.Element
public typealias Index = Array<Element>.Index
public typealias SubSequence = Array<Element>.SubSequence
public subscript(position: Array<Element>.Index) -> Element {
get { fatalError() }
set { fatalError() }
}
public subscript(bounds: Range<Array<Element>.Index>) -> SubSequence {
get { fatalError() }
set { fatalError() }
}
public var startIndex: Index { fatalError() }
public var endIndex: Index { fatalError() }
}
struct Tensor: Differentiable & AdditiveArithmetic {
typealias TangentVector = Self
static func == (lhs: Self, rhs: Self) {
fatalError()
}
static func + (lhs: Self, rhs: Self) {
fatalError()
}
}
@differentiable(reverse)
func Tensor_stack(_ lhs: Tensor) -> Tensor {
Tensor_init(stacking: [lhs])
}
@differentiable(reverse)
func Tensor_init(stacking tensors: [Tensor]) -> Tensor {
fatalError()
}
@derivative(of: Tensor_init(stacking:))
func _vjpStacking(
stacking tensors: [Tensor]
) -> (value: Tensor, pullback: (Tensor) -> Array<Tensor>.TangentVector) {
fatalError()
}Error message on release toolchains:
Google Colab (Swift 5.5 Release)Google Colab (Swift 5.6 Release)Xcode 13.4.1 Toolchain (Swift 5.6.1 Release)Error message on development toolchains:
April 18, 2021 (v5.5-dev)November 12, 2021 Trunk Development Snapshot (v5.6-dev)June 13, 2022 v5.7 Branch Development SnapshotAlso, here's the branch I'm currently working on that compiles S4TF on the Xcode 14 beta toolchain: https://github.com/philipturner/s4tf/tree/compile-on-release-toolchain (PR: s4tf/s4tf#17) |
|
Repository: https://github.com/philipturner/s4tf-crasher-1 Colab Notebook: https://colab.research.google.com/drive/1Mq9FdUTfQTyfOkTXZvCFR7mVIaf-ShS0?usp=sharing Based on the data above, the earliest someone can reproduce this bug is late 2020/early 2021. That's around the same time SR-14008 was filed: December 2020. Furthermore, both bugs require conforming Update: I'm not so sure this is the same bug as SR-14008. It is reported in apple/swift#59876. |
|
I have a different narrowed reproducer. Substitute the reproducer in #4454 (comment) with this. // Import not used in this file.
import _Differentiation
fileprivate struct Array_DifferentiableView<Element> {}
extension Array_DifferentiableView: AdditiveArithmetic {
fileprivate static func + (lhs: Self, rhs: Self) -> Self { fatalError() }
fileprivate static func - (lhs: Self, rhs: Self) -> Self { fatalError() }
}This produces the crash as usual: But something interesting happens when you change the visibility of |
Error diagnostic (1) Compiler crash (2) HereError diagnostic (3) The compiler crash with This crash can be narrowed down further. Make a Swift file with the contents below and run // Import not used in this file.
import _Differentiation
fileprivate struct Array_DifferentiableView<Element> {}
extension Array_DifferentiableView: AdditiveArithmetic {
fileprivate static func + (lhs: Self, rhs: Self) -> Self { fatalError() }
private static func - (lhs: Self, rhs: Self) -> Self { fatalError() }
}The crash changes some time between 2021-04-18 and 2021-07-07. On the 2021-04-18 toolchain, it doesn't show an error when both methods are On the January 29, 2020 toolchain, the error diagnostic is this: My original reproducer with conforming Using Colab, the The above information makes it seem like a problem with scoped declarations. Swift automatically synthesizes an internal/fileprivate-access form of |
|
This bug does not actually impact S4TF on release toolchains. I encountered an error while running tests, but did not realize that the error was just the X10 error. I bypassed this by manually copying libx10 into After suppressing the X10 error on v5.7 release, My vision for S4TF is something that even beginners to Swift can use. You can compile, import, and contribute to it, immediately. No one needs to wait 20 minutes to download a massive dev toolchain on their computer. No one needs to wait over an hour for Bazel to compile tensorflow/tensorflow. No one needs to learn how to use Bash or the command line. You can run it on any platform - even iOS - because the new dependency chain is written entirely in Swift (philipturner/differentiation, swift-reflection-mirror, swift-opencl, s4tf/metal, s4tf/opencl). These are the accessibility barriers I faced when Google sponsored S4TF, and I'm glad they will be lifted upon the release of Swift 5.7. With that, I am not going to investigate SR-14008 further. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Attachment: Download
Environment
https://swift.org/builds/development/xcode/swift-DEVELOPMENT-SNAPSHOT-2020-12-23-a/swift-DEVELOPMENT-SNAPSHOT-2020-12-23-a-osx.pkg
Apple Swift version 5.3-dev (LLVM bcee1b98f3b26c5, Swift dbcf7fe1bf76226)
Target: x86_64-apple-darwin19.6.0
Additional Detail from JIRA
md5: 4ecdf44eee77ebefd73035c76ce61be5
Issue Description:
With development snapshot toolchains,
swift build/swift runfor executable/library targets importing_Differentiationhas no problem.However, running
swift testfor test targets importing_Differentiationfails with a dynamic linker issue:Setting
DYLD_LIBRARY_PATHto the toolchain library directory containinglibswift_Differentiation.dylibdoes not help:However, manually invoking the
xctestcommand fromswift test -vwithDYLD_LIBRARY_PATHdoes work:Maybe this is because
swift testdoes not pass theDYLD_LIBRARY_PATHenvironment variable to thexctestinvocation?The text was updated successfully, but these errors were encountered: