Attachment: Download

Issue Description:

I tracked down a memory leak in our production code to an unbalanced retain when operating on an indirect enum.

I'm attaching SwiftPM package with a minimal repro. Decompress and run with:

swift build

.build/debug/ContextMemoryLeakRepro
Successful run should show Canary being deinited, but it doesn't. The leak is caused by an unbalanced retain in List.find. Sil shows %1 being retained but never released:
// List.find(key:)
sil hidden @_T04main4ListO4findypycSgSi3key_tF : $@convention(method) (Int, @guaranteed List) -> @owned Optional<@callee_owned () -> @out Any> {
// %0 // users: %37, %50, %2
// %1 // users: %5, %4, %3
bb0(%0 : $Int, %1 : $List):
debug_value %0 : $Int, let, name "key", argno 1 // id: %2
debug_value %1 : $List, let, name "self", argno 2 // id: %3
retain_value %1 : $List // id: %4
switch_enum %1 : $List, case #List.empty!enumelt: bb1, case #List.node!enumelt.1: bb5 // id: %5
...
...
...
bb8(%55 : $Optional<@callee_owned () -> @out Any>): // Preds: bb4 bb7 bb6
return %55 : $Optional<@callee_owned () -> @out Any> // id: %56
} // end sil function '_T04main4ListO4findypycSgSi3key_tF'
Additionally (happy to file a separate report, if that's in fact a different bug), an attempt to make the List generic triggers a compiler crash:
swift build && .build/debug/ContextMemoryLeakRepro
Compile Swift Module 'ContextMemoryLeakRepro' (2 sources)
Assertion failed: ((options.contains(SubstFlags::AllowLoweredTypes) || !isa(type)) && "should not be doing AST type-substitution on a lowered SIL type;" "use SILType::subst"), function operator(), file /Users/buildnode/jenkins/workspace/oss-swift-4.0-package-osx/swift/lib/AST/Type.cpp, line 3068.
0 swift 0x000000010ca3cff8 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1 swift 0x000000010ca3bf56 llvm::sys::RunSignalHandlers() + 86
2 swift 0x000000010ca3d5be SignalHandler(int) + 366
3 libsystem_platform.dylib 0x00007fff7cc61f5a _sigtramp + 26
4 libsystem_platform.dylib 0x3432000000000003 _sigtramp + 2201608387
5 libsystem_c.dylib 0x00007fff7ca8d32a abort + 127
6 libsystem_c.dylib 0x00007fff7ca55380 basename_r + 0
7 swift 0x000000010a9e1e6a llvm::Optionalswift::Type llvm::function_ref<llvm::Optionalswift::Type (swift::TypeBase*)>::callback_fn<substType(swift::Type, llvm::function_ref<swift::Type (swift::SubstitutableType*)>, llvm::function_ref<llvm::Optionalswift::ProtocolConformanceRef (swift::CanType, swift::Type, swift::ProtocolType*)>, swift::SubstOptions)::$_18>(long, swift::TypeBase*) + 2298
8 swift 0x000000010a9de478 swift::Type::transformRec(llvm::function_ref<llvm::Optionalswift::Type (swift::TypeBase*)>) const + 168
9 swift 0x000000010a9de6fe swift::Type::transformRec(llvm::function_ref<llvm::Optionalswift::Type (swift::TypeBase*)>) const + 814
10 swift 0x000000010a9def96 swift::Type::transformRec(llvm::function_ref<llvm::Optionalswift::Type (swift::TypeBase*)>) const + 3014
11 swift 0x000000010a9def96 swift::Type::transformRec(llvm::function_ref<llvm::Optionalswift::Type (swift::TypeBase*)>) const + 3014
12 swift 0x000000010a9dcde4 swift::Type::subst(llvm::function_ref<swift::Type (swift::SubstitutableType*)>, llvm::function_ref<llvm::Optionalswift::ProtocolConformanceRef (swift::CanType, swift::Type, swift::ProtocolType*)>, swift::SubstOptions) const + 196
13 swift 0x000000010a9d13f3 swift::SubstitutionMap::subst(llvm::function_ref<swift::Type (swift::SubstitutableType*)>, llvm::function_ref<llvm::Optionalswift::ProtocolConformanceRef (swift::CanType, swift::Type, swift::ProtocolType*)>) const + 323
14 swift 0x000000010a9e164d llvm::Optionalswift::Type llvm::function_ref<llvm::Optionalswift::Type (swift::TypeBase*)>::callback_fn<substType(swift::Type, llvm::function_ref<swift::Type (swift::SubstitutableType*)>, llvm::function_ref<llvm::Optionalswift::ProtocolConformanceRef (swift::CanType, swift::Type, swift::ProtocolType*)>, swift::SubstOptions)::$_18>(long, swift::TypeBase*) + 221
15 swift 0x000000010a9de478 swift::Type::transformRec(llvm::function_ref<llvm::Optionalswift::Type (swift::TypeBase*)>) const + 168
16 swift 0x000000010a9dcde4 swift::Type::subst(llvm::function_ref<swift::Type (swift::SubstitutableType*)>, llvm::function_ref<llvm::Optionalswift::ProtocolConformanceRef (swift::CanType, swift::Type, swift::ProtocolType*)>, swift::SubstOptions) const + 196
17 swift 0x0000000109af2da5 swift::irgen::TypeConverter::getExemplarType(swift::CanType) + 133
18 swift 0x0000000109af28e9 swift::irgen::TypeConverter::getTypeEntry(swift::CanType) + 409
19 swift 0x0000000109af237e swift::irgen::IRGenModule::getTypeInfo(swift::SILType) + 46
20 swift 0x0000000109af234c swift::irgen::IRGenFunction::getTypeInfo(swift::SILType) + 12
21 swift 0x0000000109b29f3e (anonymous namespace)::IRGenSILFunction::emitSILFunction() + 1134
22 swift 0x0000000109b295de swift::irgen::IRGenModule::emitSILFunction(swift::SILFunction*) + 1358
23 swift 0x0000000109a4478b swift::irgen::IRGenerator::emitGlobalTopLevel() + 459
24 swift 0x0000000109b06f77 performIRGeneration(swift::IRGenOptions&, swift::ModuleDecl*, std::__1::unique_ptr<swift::SILModule, std::__1::default_deleteswift::SILModule >, llvm::StringRef, llvm::LLVMContext&, swift::SourceFile*, llvm::GlobalVariable**, unsigned int) + 1159
25 swift 0x0000000109b075a6 swift::performIRGeneration(swift::IRGenOptions&, swift::SourceFile&, std::__1::unique_ptr<swift::SILModule, std::__1::default_deleteswift::SILModule >, llvm::StringRef, llvm::LLVMContext&, unsigned int, llvm::GlobalVariable**) + 86
26 swift 0x00000001099cb2d5 performCompile(swift::CompilerInstance&, swift::CompilerInvocation&, llvm::ArrayRef<char const*>, int&, swift::FrontendObserver*, swift::UnifiedStatsReporter*) + 12533
27 swift 0x00000001099c72f5 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 3493
28 swift 0x0000000109989400 main + 3312
29 libdyld.dylib 0x00007fff7c9e1145 start + 1
Stack dump:
0. Program arguments: /Library/Developer/Toolchains/swift-4.0.2-RELEASE.xctoolchain/usr/bin/swift -frontend -c /Users/gmilos/Repos/ContextMemoryLeakRepro3/Sources/ContextMemoryLeakRepro/Context.swift -primary-file /Users/gmilos/Repos/ContextMemoryLeakRepro3/Sources/ContextMemoryLeakRepro/main.swift -target x86_64-apple-macosx10.10 -enable-objc-interop -sdk /Applications/Xcode.real.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -I /Users/gmilos/Repos/ContextMemoryLeakRepro3/.build/x86_64-apple-macosx10.10/debug -F /Applications/Xcode.real.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -enable-testing -g -module-cache-path /Users/gmilos/Repos/ContextMemoryLeakRepro3/.build/x86_64-apple-macosx10.10/debug/ModuleCache -swift-version 4 -D SWIFT_PACKAGE -emit-module-doc-path /Users/gmilos/Repos/ContextMemoryLeakRepro3/.build/x86_64-apple-macosx10.10/debug/ContextMemoryLeakRepro.build/mainpartial.swiftdoc -Onone -module-name ContextMemoryLeakRepro -emit-module-path /Users/gmilos/Repos/ContextMemoryLeakRepro3/.build/x86_64-apple-macosx10.10/debug/ContextMemoryLeakRepro.build/mainpartial.swiftmodule -emit-dependencies-path /Users/gmilos/Repos/ContextMemoryLeakRepro3/.build/x86_64-apple-macosx10.10/debug/ContextMemoryLeakRepro.build/main.d -emit-reference-dependencies-path /Users/gmilos/Repos/ContextMemoryLeakRepro3/.build/x86_64-apple-macosx10.10/debug/ContextMemoryLeakRepro.build/main.swiftdeps -num-threads 8 -o /Users/gmilos/Repos/ContextMemoryLeakRepro3/.build/x86_64-apple-macosx10.10/debug/ContextMemoryLeakRepro.build/main.swift.o

1. While emitting IR SIL function "@_T022ContextMemoryLeakRepro4ListO4findxycSgSi3key_tF".
   for 'find(key:)' at /Users/gmilos/Repos/ContextMemoryLeakRepro3/Sources/ContextMemoryLeakRepro/main.swift:7:5
   I've attached the repro for the same tarball too.