Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-9665] [AD] Memory leaks in AD-synthesized code #52109

Closed
swift-ci opened this issue Jan 15, 2019 · 1 comment
Closed

[SR-9665] [AD] Memory leaks in AD-synthesized code #52109

swift-ci opened this issue Jan 15, 2019 · 1 comment
Assignees
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. swift for tensorflow

Comments

@swift-ci
Copy link
Collaborator

Previous ID SR-9665
Radar None
Original Reporter jekbradbury (JIRA User)
Type Bug
Status Resolved
Resolution Done
Additional Detail from JIRA
Votes 0
Component/s Swift for TensorFlow
Labels Bug
Assignee @rxwei
Priority Medium

md5: 52bd64279af763b4c0920c828b77f588

Issue Description:

I believe at least checkpoints + some adjoint intermediates aren't being freed.

For instance, this snippet hits a GPU OOM (on a card with 8GiB VRAM) after 313 steps:

struct Add: Differentiable {
    var bias: Tensor<Float>
    func applied(to input: Tensor<Float>) -> Tensor<Float> {
        return (input + bias).sum()
    }
}

let model = Add(bias: Tensor<Float>(zeros: [512]))
let x = Tensor<Float>(zeros: [4096, 512])
for i in 1...1000 {
  print(i)
  let grad = model.gradient { m in m.applied(to: x) }
}
@rxwei
Copy link
Member

rxwei commented Jan 28, 2019

#22169

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. swift for tensorflow
Projects
None yet
Development

No branches or pull requests

2 participants