Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-9661] [AD] Simple layer regression #52105

Closed
swift-ci opened this issue Jan 14, 2019 · 1 comment
Closed

[SR-9661] [AD] Simple layer regression #52105

swift-ci opened this issue Jan 14, 2019 · 1 comment
Assignees
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. swift for tensorflow

Comments

@swift-ci
Copy link
Collaborator

Previous ID SR-9661
Radar None
Original Reporter jekbradbury (JIRA User)
Type Bug
Status Resolved
Resolution Done
Additional Detail from JIRA
Votes 0
Component/s Swift for TensorFlow
Labels Bug
Assignee @dan-zheng
Priority Medium

md5: c6c94612e055c3b0b954a131ae9687b9

Issue Description:

This appears to have broken yesterday/today (diagnostic is that "property weight is not differentiable"), apparently because Differentiable.CotangentVector no longer has @FieldwiseProductSpace:

struct DenseLayer: Differentiable {
    var weight: Tensor<Float>
    var bias: Tensor<Float>

    func applied(to input: Tensor<Float>) -> Tensor<Float> {
        return matmul(input, weight) + bias
    }
}

let model = DenseLayer(weight: Tensor<Float>(zeros: [5, 5]),
                       bias: Tensor<Float>(zeros: [5]))
let x = Tensor<Float>(zeros: [4, 5])
let grad = model.gradient { m in m.applied(to: x) }

As a note, if @differentiable(wrt: (self)) is added to the properties, the snippet compiles but gets into an infinite loop at runtime between a synthetic primal and a synthetic VJP for DenseLayer.weight.

@swift-ci
Copy link
Collaborator Author

Comment by James Bradbury (JIRA)

I don't yet understand why, but the declaration @_fieldwiseProductSpace typealias CotangentVector = DenseLayer.AllDifferentiableVariables (with the attribute present in -emit-silgen) is simply typealias CotangentVector = DenseLayer.AllDifferentiableVariables by the time of the differentiation pass.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. swift for tensorflow
Projects
None yet
Development

No branches or pull requests

1 participant