Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-10693] String initialized from UTF16 leaks memory #3416

Open
swift-ci opened this issue May 15, 2019 · 14 comments
Open

[SR-10693] String initialized from UTF16 leaks memory #3416

swift-ci opened this issue May 15, 2019 · 14 comments

Comments

@swift-ci
Copy link
Contributor

Previous ID SR-10693
Radar None
Original Reporter plinth666 (JIRA User)
Type Bug
Additional Detail from JIRA
Votes 0
Component/s Foundation, Standard Library
Labels Bug
Assignee None
Priority Medium

md5: 3416773356bc3de4bde432670dab6a2f

Issue Description:

The following code sample creates a string and prints it. There are two variants, only one of which can be compiled in at a time: one built from UTF8 data, the other from UTF16 data.

When invoked with

leaks -atEnd – ./myExecutable

the UTF8 version reports no leaks. The UTF16 one reports a leak in CFString.

import Foundation{{ }}

//typealias AChar = CChar

typealias AChar = UInt16{{ }}

let a:[AChar] = [0x53, 0x79, 0x73, 0x74, 0x65, 0x6d, 0x2e, 0x41, 0x72, 0x67, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x45, 0x78, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0]{{ }}

let aptr = UnsafeMutablePointer <AChar>.allocate(capacity: a.count){{ }}

var bptr = aptr

for i in 0 ..< a.count {{{ }}

{{ bptr.initialize (to: a[i])}}{{ }}

{{ bptr = bptr.advanced(by: 1)}}

{{}}}

aptr.deallocate(){{ }}

//var str = String (utf8String: aptr)!

var str = String (utf16CodeUnits: aptr, count: a.count){{ }}

print (str)

@belkadan
Copy link

I think this is because this initializer comes from Foundation and therefore uses autorelease in its implementation. You can work around it by wrapping your logic in a call to autoreleasepool.

@milseman, @millenomi, this kind of thing keeps hitting people. Is there anything we can do about it?

@swift-ci
Copy link
Contributor Author

Comment by Steve Hawley (JIRA)

Thanks for the quick response. I'll look at your suggestion.

For my particular issue, I have text that is in UTF16 from memory owned by someone else and I need it in a String with memory owned by swift that goes away when ARC says it should with minimal memory allocations in between. In the lifetime of an app, there could be millions of them.

@millenomi
Copy link
Contributor

Honestly I don't see what. This is why the ObjC Command-Line Tool project template in Xcode has an @autoreleasepool { … } block in it; it makes me question whether the Swift one should, too.

If you invoke API provided by ObjC Foundation, you can get autoreleased objects; it is a non-goal to prevent that from happening, just as it is a non-goal to not have leaks in application code that doesn't set up an autorelease pool, as it is a non-goal to attempt to deallocate objects in a dying process. We document as such here: https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/MemoryMgmt/Articles/mmRules.html#//apple_ref/doc/uid/20000994-SW1

When an application terminates, objects may not be sent a dealloc message. Because the process’s memory is automatically cleared on exit, it is more efficient simply to allow the operating system to clean up resources than to invoke all the memory management methods.

In an AppKit or UIKit application, the autorelease pool will automatically be managed by NSRunLoop and NSApplication/UIApplication working in concert.

@millenomi
Copy link
Contributor

plinth666 (JIRA User) Does the issue go away if you add an autoreleasepool { … } block around the entire thing?

@belkadan
Copy link

Sorry, I didn't mean "avoid autoreleases showing up as leaks". I meant "avoid use of autorelease in Foundation String APIs". (That's probably only possible if we have the autorelease-pool-steal thing working in MRR code; can't remember if that landed.)

@milseman
Copy link
Mannequin

milseman mannequin commented May 16, 2019

CC @Catfish-Man

@Catfish-Man
Copy link
Member

The most obvious thing that comes to mind is having String(utf16CodeUnits:, count🙂 call String.decodeCString instead of NSString(characters:,length🙂

@Catfish-Man
Copy link
Member

I guess that requires null termination, so we might have to add a bytes+length version for this or something. Still, in principle, this should be straightforward I think.

@milseman
Copy link
Mannequin

milseman mannequin commented May 16, 2019

If you want to make a native Swift string, you use the `String(decoding:as🙂` initializer. The C string ones are for only for C strings, that is nul-terminated.

@Catfish-Man
Copy link
Member

There we go, that's the one I was thinking of. Thanks Michael.

@swift-ci
Copy link
Contributor Author

Comment by Steve Hawley (JIRA)

Thanks for the help. I've tentatively switched over to the decoding:as initializer.

In my environment, I can't (conveniently) use an autorelease pool, but yes, that does take care of the leak.

From a pragmatic standpoint, the utf16CodeUnits variant violates the principal of least astonishment in that the behavior is outwardly surprising. As a client of this API, I would expect one of two things as an outcome: the underlying representation should behave consistently with similar strings constructed in a similar manner OR the documentation should be clear that using this particular constructor demands an autorelease pool and the consequences of not doing so.

In theory, the underlying representation of the String shouldn't matter to the consumer, but in practice if the underlying representation affects the consumer, as it affected me, then it matters very much.

To be clear, I understand the challenges of working on a large codebase and appreciate that you can't always have ideal documentation or consistent behaviors. I hope I don't sound preachy or condescending - I really appreciate the attention given to this.

@millenomi
Copy link
Contributor

It's something that I'm not quite sure how to handle. On one hand, I'm glad there is an alternative that fits your needs. On the other, If you import Foundation, or any other ObjC module, then any composition of this code with other code that happens to use Foundation API or any other ObjC module's API will be incorrect if there isn't an autorelease pool in place.

Can you stop importing Foundation in your code?

  1> String.init(utf16CodeUnits:count:) 
error: repl.swift:1:1: error: type 'String' has no member 'init(utf16CodeUnits:count:)'
String.init(utf16CodeUnits:count:)
^~~~~~ ~~~~


  1> import Foundation
  2> String.init(utf16CodeUnits:count:) 
$R0: (UnsafePointer<unichar>, Int) -> String = 0x00000001001642e0 $__lldb_expr5`partial apply forwarder for (extension in Foundation):Swift.String.init(utf16CodeUnits: Swift.UnsafePointer<Swift.UInt16>, count: Swift.Int) -> Swift.String at <compiler-generated>

@swift-ci
Copy link
Contributor Author

Comment by Steve Hawley (JIRA)

The answer to can I avoid using Foundation is 'yes' and 'no'.

I'm not at liberty to explain in detail the circumstances, but I can say that I'm writing code that writes lots and lots of swift and where that gets used depends entirely on what my clients choose to do. I can say with confidence that 90-95% of all my clients will be using the code in a context in which there will be an autorelease pool in place, but in general I can neither make nor enforce that and that's how this was turned up in the first place. I can, however, guarantee that if my customers are using anything from Foundation, there will always be an autorelease pool.

Since Strings are not part of Foundation (although bridging, foreign strings, blah blah), I have to assume that they're going to be used from not Foundation as well.

@millenomi
Copy link
Contributor

I want to challenge the answer a little. If you would prefer no dependency on Foundation, it would probably be better to isolate importing Foundation to a separate module.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants