Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-9741] [AD] Deserialization failure of AD-generated struct types #52172

Closed
rxwei opened this issue Jan 24, 2019 · 9 comments
Closed

[SR-9741] [AD] Deserialization failure of AD-generated struct types #52172

rxwei opened this issue Jan 24, 2019 · 9 comments
Assignees
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. swift for tensorflow

Comments

@rxwei
Copy link
Member

rxwei commented Jan 24, 2019

Previous ID SR-9741
Radar None
Original Reporter @rxwei
Type Bug
Status Resolved
Resolution Done
Additional Detail from JIRA
Votes 0
Component/s Swift for TensorFlow
Labels Bug
Assignee @marcrasi
Priority Medium

md5: b1b015f4fa75ca65cf786d107843db57

Issue Description:

import TensorFlow
public struct Conv2D<Scalar>: Layer
    where Scalar : FloatingPoint & Differentiable & TensorFlowScalar {
    public var filter: Tensor<Scalar>
    @noDerivative public let strides: (Int32, Int32)
    @noDerivative public let padding: Padding

    @differentiable(wrt: (self, input))
    public func applied(to input: Tensor<Scalar>) -> Tensor<Scalar> {
        return input.convolved2D(withFilter: filter,
                                 strides: (1, strides.0, strides.1, 1),
                                 padding: padding)
    }
}
*** DESERIALIZATION FAILURE (please include this section in any bug report) ***
top-level value not found
Cross-reference to module 'DeepLearning'
... AD__$s12DeepLearning6Conv2DV6filter10TensorFlow0E0VyxGvg__Type__src_0_wrt_0

Stack dump:
0.  Program arguments: /usr/local/src/swift-build/swift/swift-nightly-install/Library/Developer/Toolchains/swift-tensorflow-DEVELOPMENT-2019-01-24-a.xctoolchain/usr/bin/swift -frontend -merge-modules -emit-module /Users/rxwei/Development/swift-apis/.build/x86_64-apple-macosx/debug/DeepLearning.build/Helpers~partial.swiftmodule /Users/rxwei/Development/swift-apis/.build/x86_64-apple-macosx/debug/DeepLearning.build/Layer~partial.swiftmodule /Users/rxwei/Development/swift-apis/.build/x86_64-apple-macosx/debug/DeepLearning.build/Loss~partial.swiftmodule /Users/rxwei/Development/swift-apis/.build/x86_64-apple-macosx/debug/DeepLearning.build/Optimizer~partial.swiftmodule -parse-as-library -sil-merge-partial-modules -disable-diagnostic-passes -disable-sil-perf-optzns -target x86_64-apple-macosx10.10 -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -I /Users/rxwei/Development/swift-apis/.build/x86_64-apple-macosx/debug -F /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -enable-testing -g -module-cache-path /Users/rxwei/Development/swift-apis/.build/x86_64-apple-macosx/debug/ModuleCache -swift-version 4.2 -Onone -D SWIFT_PACKAGE -D DEBUG -color-diagnostics -emit-module-doc-path /Users/rxwei/Development/swift-apis/.build/x86_64-apple-macosx/debug/DeepLearning.swiftdoc -module-name DeepLearning -o /Users/rxwei/Development/swift-apis/.build/x86_64-apple-macosx/debug/DeepLearning.swiftmodule
1.  While deserializing SIL function "AD__$s12DeepLearning6Conv2DV6filter10TensorFlow0E0VyxGvg__vjp_src_0_wrt_0"
0  swift                    0x000000010861b8f8 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1  swift                    0x000000010861ab75 llvm::sys::RunSignalHandlers() + 85
2  swift                    0x000000010861bf02 SignalHandler(int) + 258
3  libsystem_platform.dylib 0x00007fff76edcb3d _sigtramp + 29
4  libsystem_platform.dylib 0x000000010becfbc6 _sigtramp + 2499752102
5  libsystem_c.dylib        0x00007fff76d9b1c9 abort + 127
6  swift                    0x0000000105a92483 swift::ModuleFile::fatal(llvm::Error) + 675
7  swift                    0x0000000105a934af swift::ModuleFile::getType(llvm::PointerEmbeddedInt<unsigned int, 31>) + 111
8  swift                    0x0000000105abb429 swift::SILDeserializer::readSILInstruction(swift::SILFunction*, swift::SILBasicBlock*, swift::SILBuilder&, unsigned int, llvm::SmallVectorImpl<unsigned long long>&) + 7881
9  swift                    0x0000000105ab7be2 swift::SILDeserializer::readSILFunctionChecked(llvm::PointerEmbeddedInt<unsigned int, 31>, swift::SILFunction*, llvm::StringRef, bool, bool) + 7826
10 swift                    0x0000000105acc19a swift::SILDeserializer::getAllSILFunctions() + 586
11 swift                    0x0000000105b3f0ff swift::SerializedSILLoader::getAllForModule(swift::Identifier, swift::FileUnit*) + 127
12 swift                    0x00000001049304b2 performCompile(swift::CompilerInstance&, swift::CompilerInvocation&, llvm::ArrayRef<char const*>, int&, swift::FrontendObserver*, swift::UnifiedStatsReporter*) + 11346
13 swift                    0x000000010492c7cd swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 3021
14 swift                    0x00000001048de3be main + 686
15 libdyld.dylib            0x00007fff76cf308d start + 1
16 libdyld.dylib            0x0000000000000027 start + 2301677467
<unknown>:0: error: unable to execute command: Abort trap: 6

<unknown>:0: error: merge-module command failed due to signal 6 (use -v to see invocation)
@marcrasi
Copy link
Mannequin

marcrasi mannequin commented Jan 24, 2019

Do I need some extra stuff that's not in head to reproduce this? When I try to compile it, I get

workspace/mergefun.swift:2:31: error: use of undeclared type 'Layer'
public struct Conv2D<Scalar>: Layer
                              ^~~~~

This is the same error message that Gogul found and fixed in lldb, though Gogul only made lldb changes to fix that, so non-lldb invocations should not be affected by the problem or the fix.

If this isn't too difficult, it's probably less disruptive to fix-forward rather than reverting the merge, because the merge has been in for a few days and people have been working off of it.

@rxwei
Copy link
Member Author

rxwei commented Jan 24, 2019

Latest HEAD should have `Layer` defined already. That said, you can also try compiling https://github.com/rxwei/DeepLearning using `swift build`. That would fail as well.

@marcrasi
Copy link
Mannequin

marcrasi mannequin commented Jan 24, 2019

I may not have the absolute latest head. I'll try pulling that and rebuilding and seeing if I can reproduce, and then I'll see if I can figure out a fix-forward.

@dan-zheng says he's working on a revert of the merge in the meantime.

@swift-ci
Copy link
Collaborator

Comment by Gogul Balakrishnan (JIRA)

@rxwei & @dan-zheng, I was not able to reproduce this on my linux box.

@marcrasi
Copy link
Mannequin

marcrasi mannequin commented Jan 24, 2019

I finally finished a clean build at head. I can also compile the code posted in this bug successfully. (On my gLinux)

@rxwei
Copy link
Member Author

rxwei commented Jan 24, 2019

Have you tried `swift build`? I think running swiftc doesn't deserialize things.

@marcrasi
Copy link
Mannequin

marcrasi mannequin commented Jan 24, 2019

Oh, running `swift build` in https://github.com/rxwei/DeepLearning (at commit 731ce402ce0bd7459b898b112deb89379bbda893) does indeed trigger this error message for me.

@rxwei
Copy link
Member Author

rxwei commented Jan 24, 2019

We did some further digging and verified that it's not caused by the merge. It may actually be related to AD generics support #22023 We haven't tested `swift build` between that point and the merge.

@rxwei
Copy link
Member Author

rxwei commented Jan 25, 2019

It's not caused by the merge, but caused by a bug in AD where primal value structs are generated.
@marcrasi sent #22102 that fixed this.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. swift for tensorflow
Projects
None yet
Development

No branches or pull requests

2 participants