Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SR-11262] Array.subscript.read / Collection.subscript.read allocate #53663

Open
weissi opened this issue Aug 6, 2019 · 3 comments
Open

[SR-11262] Array.subscript.read / Collection.subscript.read allocate #53663

weissi opened this issue Aug 6, 2019 · 3 comments
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. compiler The Swift compiler in itself performance

Comments

@weissi
Copy link
Member

weissi commented Aug 6, 2019

Previous ID SR-11262
Radar rdar://problem/53996328
Original Reporter @weissi
Type Bug

Attachment: Download

Additional Detail from JIRA
Votes 1
Component/s Compiler
Labels Bug, Performance
Assignee None
Priority Medium

md5: f042d194a7f9a1490b93803f8a1659e2

Issue Description:

description

The following program (attached as a SwiftPM project) allocates 40,000 times, all of the allocations calling malloc directly from protocol witness for Collection.subscript.read and Array.subscript.read. That's very surprising and smells a bit like a coroutine allocation (similar to SR-10604 and SR-11231).

This is the dtrace trace for the mallocs

$ sudo dtrace -n 'pid$target::malloc:entry { @mallocs[ustack(5)] = count(); } ::END { printa(@mallocs); }' -c .build/release/repro40k | tail -n 40
dtrace: description 'pid$target::malloc:entry ' matched 5 probes
dtrace: pid 12880 has exited
              libsystem_malloc.dylib`malloc
              libswiftCore.dylib`protocol witness for Collection.subscript.read in conformance [A]+0x1e
              libswiftCore.dylib`protocol witness for IteratorProtocol.next() in conformance IndexingIterator<A>+0x1cc
              libswiftCore.dylib`EnumeratedSequence.Iterator.next()+0x130
              repro40k`static EventLoopFuture._reduceSuccesses0<A>(_:_:_:onValue:)+0x46c
             9999

              libsystem_malloc.dylib`malloc
              libswiftCore.dylib`protocol witness for Collection.subscript.read in conformance [A]+0x1e
              libswiftCore.dylib`Collection.map<A>(_:)+0x297
              repro40k`closure #&#8203;3 in static EventLoopFuture.whenAllSucceed(_:on:)+0x85
              repro40k`partial apply for thunk for @escaping @callee_guaranteed () -> (@owned [Result<A, Error>])+0x14
            10000

              libsystem_malloc.dylib`malloc
              libswiftCore.dylib`Array.subscript.read+0x35
              libswiftCore.dylib`protocol witness for Collection.subscript.read in conformance [A]+0x38
              libswiftCore.dylib`Collection.map<A>(_:)+0x297
              repro40k`closure #&#8203;3 in static EventLoopFuture.whenAllSucceed(_:on:)+0x85
            10000

              libsystem_malloc.dylib`malloc
              libswiftCore.dylib`Array.subscript.read+0x35
              libswiftCore.dylib`protocol witness for Collection.subscript.read in conformance [A]+0x38
              libswiftCore.dylib`protocol witness for IteratorProtocol.next() in conformance IndexingIterator<A>+0x1cc
              libswiftCore.dylib`EnumeratedSequence.Iterator.next()+0x130
            10000

source code

the program code is here:

import NIO


func doTheThing<T>(_ array: [EventLoopFuture<T>]) throws -> [T] {
    return try EventLoopFuture<T>.whenAllSucceed(array, on: array.first!.eventLoop).wait()
}

let eventLoop = EmbeddedEventLoop()


var lots: [EventLoopFuture<Int>] = Array(repeatElement(eventLoop.makeSucceededFuture(1), count: 10_000))
print(try! doTheThing(lots).count)

repro

tar xf repro40k.tar.gz
cd repro40k
swift build -c release
sudo dtrace -n 'pid$target::malloc:entry { @mallocs[ustack(5)] = count(); } ::END { printa(@mallocs); }' -c .build/release/repro40k

versions

Apple Swift version 5.1 (swiftlang-1100.0.255 clang-1100.0.30.2)

and

swift-DEVELOPMENT-SNAPSHOT-2019-08-03-a.xctoolchain
@weissi
Copy link
Member Author

weissi commented Aug 6, 2019

@swift-ci create

@weissi
Copy link
Member Author

weissi commented Aug 6, 2019

CC @rjmccall

@eeckstein
Copy link
Member

The problem here is that there is an alloc_stack in Array.subscript.read (holding the element), which is implemented as an malloc, because it spans over the yield.
We can solve or improve this on multiple levels:

  • try to get rid of the alloc_stack+copy_addr in the SIL optimizer (which is not so trivial)

  • improve coroutine lowering: don't malloc if the value fits in the scratch buffer

  • somehow enable specialisation + devirtualization + inlining of that coroutine. This could be done e.g. with cross-module optimizations.

None of this solutions is trivial. In the meantime, I suggest to workaround the problem by not using Array.enumerated if the array is generic and cannot be specialized. Use something like:

  for idx in (0..<arr.count) {
    let elem = arr[idx]
    ...
  }

Note that currently even iterating over an array with for..in has the same problem (in a generic context). But this is fixed with #27088

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. compiler The Swift compiler in itself performance
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@weissi @eeckstein