New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SR-14115] SIL: derivative type calculation for coroutines #54404
Comments
@rxwei @dan-zheng @BradLarson Likely you already discussed this issue and maybe even came to some conclusion. Below are my thoughts about the issue. For the sake of clarify I will consider the following trivial code in the examples: struct S: Differentiable {
private var _x : Float
var x: Float {
get{_x}
set(newValue) { _x = newValue }
_modify { yield &_x }
}
init(_ x : Float) {
self._x = x
}
} Here, the We need to specify:
VJP typeIn the simple case the VJP type is the same as the Differentiable-constrained original function type with the addition of one additional result: the pullback (for the sake of simplicity I am ignoring some additional cases like reabstraction thunks, etc.). Assume for a moment that we'd already defined a pullback type for Then if
In reality I think we'd need 2 pullbacks or some clever combination of them. See below. PullbackPullback propagates derivative information from active results back to parameters. In the normal SIL world functions have single entries and single returns. Therefore, in general, for a function Essentially, we can think about co-routines as a pair of functions: one for the first part until One could think about using a co-routine as a pullback. If we'd try to execute the co-routine in reverse, then the output of reversed- It looks like the only solution is to pass these adjoints indirect, storing the values to adjoint buffers after co-routine return. So, in our example the co-routine pullback type would be An alternative to this would be some kind of pair of pullbacks: one for the part of the function before Both approaches have the following assumptions:
Are there any other ideas? I'm slightly leaning towards co-routine pullback that is returned indirect. Is there anything I missed? |
Some status update: I'm having a proof-of-concept implementation that seems to generate correct SIL code for things like: struct S: Differentiable {
private var _x : Float
func _endMutation() {
// do something
}
var x: Float {
get{_x}
set(newValue) { _x = newValue }
_modify {
defer { _endMutation() }
if (x > 0) { // invoke getter to have some nested function calls
yield &_x
} else {
yield &_x
}
}
}
init(_ x : Float) {
self._x = x
}
} Implementation notes:
Still, it seems there are some quite fundamental obstacles that might require some language extensions / changes. I summarized the caveats in the following forum post: https://forums.swift.org/t/accessor-coroutines-poor-children/67061 |
Additional Detail from JIRA
md5: fe00df47fa9d2ad8d02ae88725898756
Parent-Task:
_read
and_modify
accessor differentiationIssue Description:
SIL has dedicated coroutine function types: https://github.com/apple/swift/blob/master/docs/SIL.rst#coroutine-types
Figure out derivative type calculation for coroutines.
The text was updated successfully, but these errors were encountered: