[SR-8211] Support a mechanism for zeroing an UnsafeMutable{,Raw}BufferPointer #50743

Lukasa · 2018-07-09T14:58:25Z


Previous ID	SR-8211
Radar	None
Original Reporter	@Lukasa
Type	New Feature

Additional Detail from JIRA


Votes	0
Component/s	Standard Library
Labels	New Feature, StarterProposal
Assignee	None
Priority	Medium

md5: 3a2dea04d2903bbfde833b4186202d1f

Issue Description:

Currently Swift provides no mechanism to zero a buffer of memory that is safe from the compiler "helpfully" optimising it away. The natural thing to try to do, {{ pointer.initialize(repeating: 0) }} can be elided by the compiler, as demonstrated by this short Swift program:

let myBufferPtr = UnsafeMutableBufferPointer<UInt8>.allocate(capacity: 1000)
myBufferPtr.initialize(from: 0...255)
myBufferPtr.initialize(repeating: 0)
myBufferPtr.deallocate()

When compiled with optimisations turned on, the following SIL is emitted for the main function:

let myBufferPtr: UnsafeMutableBufferPointer<UInt8>

// myBufferPtr
sil_global hidden [let] @$S4test11myBufferPtrSrys5UInt8VGvp : $UnsafeMutableBufferPointer<UInt8>

// _swiftEmptyArrayStorage
sil_global @_swiftEmptyArrayStorage : $_SwiftEmptyArrayStorage

// main
sil @main : $@convention(c) (Int32, UnsafeMutablePointer<Optional<UnsafeMutablePointer<Int8>>>) -> Int32 {
bb0(%0 : $Int32, %1 : $UnsafeMutablePointer<Optional<UnsafeMutablePointer<Int8>>>):
  alloc_global @$S4test11myBufferPtrSrys5UInt8VGvp // id: %2
  %3 = global_addr @$S4test11myBufferPtrSrys5UInt8VGvp : $*UnsafeMutableBufferPointer<UInt8> // user: %22
  %4 = integer_literal $Builtin.Int64, 1000       // users: %42, %10, %5
  %5 = struct $Int (%4 : $Builtin.Int64)          // user: %21
  %6 = metatype $@thick UInt8.Type                // users: %15, %7
  %7 = builtin "strideof"<UInt8>(%6 : $@thick UInt8.Type) : $Builtin.Word // user: %8
  %8 = builtin "zextOrBitCast_Word_Int64"(%7 : $Builtin.Word) : $Builtin.Int64 // user: %10
  %9 = integer_literal $Builtin.Int1, -1          // users: %37, %10
  %10 = builtin "smul_with_overflow_Int64"(%8 : $Builtin.Int64, %4 : $Builtin.Int64, %9 : $Builtin.Int1) : $(Builtin.Int64, Builtin.Int1) // users: %12, %11
  %11 = tuple_extract %10 : $(Builtin.Int64, Builtin.Int1), 0 // user: %14
  %12 = tuple_extract %10 : $(Builtin.Int64, Builtin.Int1), 1 // user: %13
  cond_fail %12 : $Builtin.Int1                   // id: %13
  %14 = builtin "truncOrBitCast_Int64_Word"(%11 : $Builtin.Int64) : $Builtin.Word // user: %16
  %15 = builtin "alignof"<UInt8>(%6 : $@thick UInt8.Type) : $Builtin.Word // user: %16
  %16 = builtin "allocRaw"(%14 : $Builtin.Word, %15 : $Builtin.Word) : $Builtin.RawPointer // users: %46, %34, %19, %18
  %17 = integer_literal $Builtin.Word, 1000       // user: %18
  bind_memory %16 : $Builtin.RawPointer, %17 : $Builtin.Word to $*UInt8 // id: %18
  %19 = struct $UnsafeMutablePointer<UInt8> (%16 : $Builtin.RawPointer) // user: %20
  %20 = enum $Optional<UnsafeMutablePointer<UInt8>>, #Optional.some!enumelt.1, %19 : $UnsafeMutablePointer<UInt8> // user: %21
  %21 = struct $UnsafeMutableBufferPointer<UInt8> (%20 : $Optional<UnsafeMutablePointer<UInt8>>, %5 : $Int) // users: %30, %22
  store %21 to %3 : $*UnsafeMutableBufferPointer<UInt8> // id: %22
  %23 = alloc_stack $IndexingIterator<ClosedRange<UInt8>> // users: %30, %31
  %24 = integer_literal $Builtin.Int8, 0          // user: %27
  %25 = integer_literal $Builtin.Int8, -1         // user: %28
  // function_ref specialized UnsafeMutableBufferPointer.initialize<A>(from:)
  %26 = function_ref @$SSr10initialize4from8IteratorQyd___Sitqd___t7ElementQyd__Rszs8SequenceRd__lFs5UInt8V_SNyAIGTg5 : $@convention(method) (ClosedRange<UInt8>, UnsafeMutableBufferPointer<UInt8>) -> (@out IndexingIterator<ClosedRange<UInt8>>, Int) // user: %30
  %27 = struct $UInt8 (%24 : $Builtin.Int8)       // users: %41, %29
  %28 = struct $UInt8 (%25 : $Builtin.Int8)       // user: %29
  %29 = struct $ClosedRange<UInt8> (%27 : $UInt8, %28 : $UInt8) // user: %30
  %30 = apply %26(%23, %29, %21) : $@convention(method) (ClosedRange<UInt8>, UnsafeMutableBufferPointer<UInt8>) -> (@out IndexingIterator<ClosedRange<UInt8>>, Int)
  dealloc_stack %23 : $*IndexingIterator<ClosedRange<UInt8>> // id: %31
  %32 = integer_literal $Builtin.Int64, 0         // user: %35
  %33 = integer_literal $Builtin.Int64, 1         // user: %37
  %34 = pointer_to_address %16 : $Builtin.RawPointer to [strict] $*UInt8 // user: %40
  br bb1(%32 : $Builtin.Int64)                    // id: %35

// %36                                            // users: %39, %37
bb1(%36 : $Builtin.Int64):                        // Preds: bb2 bb0
  %37 = builtin "sadd_with_overflow_Int64"(%36 : $Builtin.Int64, %33 : $Builtin.Int64, %9 : $Builtin.Int1) : $(Builtin.Int64, Builtin.Int1) // user: %38
  %38 = tuple_extract %37 : $(Builtin.Int64, Builtin.Int1), 0 // users: %44, %42
  %39 = builtin "truncOrBitCast_Int64_Word"(%36 : $Builtin.Int64) : $Builtin.Word // user: %40
  %40 = index_addr %34 : $*UInt8, %39 : $Builtin.Word // user: %41
  store %27 to %40 : $*UInt8                      // id: %41
  %42 = builtin "cmp_eq_Int64"(%38 : $Builtin.Int64, %4 : $Builtin.Int64) : $Builtin.Int1 // user: %43
  cond_br %42, bb3, bb2                           // id: %43

bb2:                                              // Preds: bb1
  br bb1(%38 : $Builtin.Int64)                    // id: %44

bb3:                                              // Preds: bb1
  %45 = integer_literal $Builtin.Word, -1         // users: %46, %46
  %46 = builtin "deallocRaw"(%16 : $Builtin.RawPointer, %45 : $Builtin.Word, %45 : $Builtin.Word) : $()
  %47 = integer_literal $Builtin.Int32, 0         // user: %48
  %48 = struct $Int32 (%47 : $Builtin.Int32)      // user: %49
  return %48 : $Int32                             // id: %49
} // end sil function 'main'

Note that we never call the initializer with zero. Instead we see only the initialisation with the range, and then the deallocation.

There should be some supported Swift way to securely overwrite a buffer of memory that does not involve making a call through libc. This is useful with system programs that are handling sensitive data (e.g. passphrases).

belkadan · 2018-07-09T17:16:14Z

memset_s will work in the meantime, but this is a good request.

belkadan · 2018-07-09T17:18:05Z

@lorentey, think this would be a good StarterProposal?

Lukasa · 2018-07-09T17:59:58Z

As a purely informational note, memset_s is not available in glibc so is not usable on Linux.

lorentey · 2018-07-09T19:30:14Z

@belkadan I think so!

Although, to be honest, I'm quite surprised at the current behavior. It seems that changing the second, partially duplicate initialization to myBufferPtr.assign(repeating: 0) keeps both loops in place, which is more in line with what I expected to happen.

belkadan · 2018-07-09T19:33:59Z

LLVM is always allowed to remove writes that it determines are unused, so just because we don't do that optimization now doesn't mean it won't change in the future. That's why things like memset_s exist.

obvgab · 2023-08-07T17:31:28Z

Hey y'all! New to Swift open source. I was trying to replicate this issue, and I found a few things that were interesting.
It seems that, yes, the function doesn't exist in the main function in SIL:

// main
sil @main : $@convention(c) (Int32, UnsafeMutablePointer<Optional<UnsafeMutablePointer<Int8>>>) -> Int32 {
[%1: noescape **]
[global: read,write,copy,destroy,allocate,deinit_barrier]
bb0(%0 : $Int32, %1 : $UnsafeMutablePointer<Optional<UnsafeMutablePointer<Int8>>>):
  alloc_global @$s4safe7pointerSrys5UInt8VGvp     // id: %2
  %3 = global_addr @$s4safe7pointerSrys5UInt8VGvp : $*UnsafeMutableBufferPointer<UInt8> // user: %14
  %4 = integer_literal $Builtin.Int64, 1000       // users: %34, %12
  %5 = integer_literal $Builtin.Int1, -1          // user: %29
  %6 = integer_literal $Builtin.Word, 0           // users: %38, %8
  %7 = integer_literal $Builtin.Word, 1000        // users: %9, %8
  %8 = builtin "allocRaw"(%7 : $Builtin.Word, %6 : $Builtin.Word) : $Builtin.RawPointer // users: %26, %38, %10, %9
  %9 = bind_memory %8 : $Builtin.RawPointer, %7 : $Builtin.Word to $*UInt8
  %10 = struct $UnsafeMutablePointer<UInt8> (%8 : $Builtin.RawPointer) // user: %11
  %11 = enum $Optional<UnsafeMutablePointer<UInt8>>, #Optional.some!enumelt, %10 : $UnsafeMutablePointer<UInt8> // user: %13
  %12 = struct $Int (%4 : $Builtin.Int64)         // user: %13
  %13 = struct $UnsafeMutableBufferPointer<UInt8> (%11 : $Optional<UnsafeMutablePointer<UInt8>>, %12 : $Int) // users: %22, %14
  store %13 to %3 : $*UnsafeMutableBufferPointer<UInt8> // id: %14
  %15 = alloc_stack $IndexingIterator<ClosedRange<UInt8>> // users: %22, %23
  %16 = integer_literal $Builtin.Int8, 0          // user: %17
  %17 = struct $UInt8 (%16 : $Builtin.Int8)       // users: %33, %21
  %18 = integer_literal $Builtin.Int8, -1         // user: %19
  %19 = struct $UInt8 (%18 : $Builtin.Int8)       // user: %21
  // function_ref specialized UnsafeMutableBufferPointer.initialize<A>(from:)
  %20 = function_ref @$sSr10initialize4from8IteratorQyd___Sitqd___t7ElementQyd__RszSTRd__lFs5UInt8V_SNyAHGTg5 : $@convention(method) (ClosedRange<UInt8>, UnsafeMutableBufferPointer<UInt8>) -> (@out IndexingIterator<ClosedRange<UInt8>>, Int) // user: %22
  %21 = struct $ClosedRange<UInt8> (%17 : $UInt8, %19 : $UInt8) // user: %22
  %22 = apply %20(%15, %21, %13) : $@convention(method) (ClosedRange<UInt8>, UnsafeMutableBufferPointer<UInt8>) -> (@out IndexingIterator<ClosedRange<UInt8>>, Int)
  dealloc_stack %15 : $*IndexingIterator<ClosedRange<UInt8>> // id: %23
  %24 = integer_literal $Builtin.Int64, 0         // user: %27
  %25 = integer_literal $Builtin.Int64, 1         // user: %29
  %26 = pointer_to_address %8 : $Builtin.RawPointer to [strict] $*UInt8 // user: %32
  br bb1(%24 : $Builtin.Int64)                    // id: %27

// %28                                            // users: %31, %29
bb1(%28 : $Builtin.Int64):                        // Preds: bb2 bb0
  %29 = builtin "sadd_with_overflow_Int64"(%28 : $Builtin.Int64, %25 : $Builtin.Int64, %5 : $Builtin.Int1) : $(Builtin.Int64, Builtin.Int1) // user: %30
  %30 = tuple_extract %29 : $(Builtin.Int64, Builtin.Int1), 0 // users: %36, %34
  %31 = builtin "truncOrBitCast_Int64_Word"(%28 : $Builtin.Int64) : $Builtin.Word // user: %32
  %32 = index_addr [stack_protection] %26 : $*UInt8, %31 : $Builtin.Word // user: %33
  store %17 to %32 : $*UInt8                      // id: %33
  %34 = builtin "cmp_eq_Int64"(%30 : $Builtin.Int64, %4 : $Builtin.Int64) : $Builtin.Int1 // user: %35
  cond_br %34, bb3, bb2                           // id: %35

bb2:                                              // Preds: bb1
  br bb1(%30 : $Builtin.Int64)                    // id: %36

bb3:                                              // Preds: bb1
  %37 = integer_literal $Builtin.Word, -1         // user: %38
  %38 = builtin "deallocRaw"(%8 : $Builtin.RawPointer, %37 : $Builtin.Word, %6 : $Builtin.Word) : $()
  %39 = integer_literal $Builtin.Int32, 0         // user: %40
  %40 = struct $Int32 (%39 : $Builtin.Int32)      // user: %41
  return %40 : $Int32                             // id: %41
} // end sil function 'main'

However, when using -emit-ir to get LLVM's IR, the main function's call to zero the pointer still seems to exist:

define i32 @main(i32 %0, ptr nocapture readnone %1) #0 {
entry:
  %2 = tail call noalias ptr @swift_slowAlloc(i64 1000, i64 -1) #1
  %3 = ptrtoint ptr %2 to i64
  store i64 %3, ptr @"$s4safe7pointerSrys5UInt8VGvp", align 8
  store i64 1000, ptr getelementptr inbounds (%TSrys5UInt8VG, ptr @"$s4safe7pointerSrys5UInt8VGvp", i64 0, i32 1), align 8
  %4 = icmp eq ptr %2, null
  br i1 %4, label %"$sSr10initialize4from8IteratorQyd___Sitqd___t7ElementQyd__RszSTRd__lFs5UInt8V_SNyAHGTg5.exit", label %vector.body
  ... ; <- omitted for brevity, this is where vector.body is declared to fill 0...255
  br label %"$sSr10initialize4from8IteratorQyd___Sitqd___t7ElementQyd__RszSTRd__lFs5UInt8V_SNyAHGTg5.exit"

"$sSr10initialize4from8IteratorQyd___Sitqd___t7ElementQyd__RszSTRd__lFs5UInt8V_SNyAHGTg5.exit": ; preds = %vector.body, %entry
  tail call void @llvm.memset.p0.i64(ptr noundef nonnull align 1 dereferenceable(1000) %2, i8 0, i64 1000, i1 false)
  tail call void @swift_slowDealloc(ptr nonnull %2, i64 -1, i64 -1) #1
  ret i32 0
}

llvm.memset is used to zero it out. This can presumably be optimized away by LLVM since volatile is set to false.
Decompiling after running swiftc, though, shows the function call still present to some extent:

Now, I'm not entirely fluent in IR, SIL, or even decompilation--so take my investigation with a grain of salt. Maybe the .initialize(repeating:) call is snug somewhere in the rest of the generated SIL.
For a safe initialization, should we use llvm's volatile parameter? Would love some guidance here

glessard · 2023-09-05T23:31:35Z

rdar://95723264

obvgab · 2023-09-06T02:27:28Z

I don't see that on OpenRadar, so I presume it's private. Keep me posted!

glessard · 2023-09-06T15:15:29Z

Posting the radar just communicates that it's tracked by Apple. The likeliest thing is that there's no extra information attached beyond what's in this issue.

swift-ci transferred this issue from apple/swift-issues Apr 25, 2022

AnthonyLatsis added good first issue Good for newcomers swift evolution proposal needed Flag → feature: A feature that warrants a Swift evolution proposal feature A feature request or implementation and removed StarterProposal new feature labels Nov 11, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SR-8211] Support a mechanism for zeroing an UnsafeMutable{,Raw}BufferPointer #50743

[SR-8211] Support a mechanism for zeroing an UnsafeMutable{,Raw}BufferPointer #50743

Lukasa commented Jul 9, 2018

belkadan commented Jul 9, 2018

belkadan commented Jul 9, 2018

Lukasa commented Jul 9, 2018

lorentey commented Jul 9, 2018

belkadan commented Jul 9, 2018

obvgab commented Aug 7, 2023

glessard commented Sep 5, 2023

obvgab commented Sep 6, 2023

glessard commented Sep 6, 2023

[SR-8211] Support a mechanism for zeroing an UnsafeMutable{,Raw}BufferPointer #50743

[SR-8211] Support a mechanism for zeroing an UnsafeMutable{,Raw}BufferPointer #50743

Comments

Lukasa commented Jul 9, 2018

belkadan commented Jul 9, 2018

belkadan commented Jul 9, 2018

Lukasa commented Jul 9, 2018

lorentey commented Jul 9, 2018

belkadan commented Jul 9, 2018

obvgab commented Aug 7, 2023

glessard commented Sep 5, 2023

obvgab commented Sep 6, 2023

glessard commented Sep 6, 2023