cc jingham@apple.com (JIRA User)

I can reproduce the problem if I change the key type from String to Int or to Int64.

However, if I change the key type to Int32, the problem disappears.

Let's consider the Int64 case.

NativeHashedStorageHandler has a few instance variables of interest:

• m_key_stride is set to the byte stride of the key type: in this case, 8 for Int64.

• m_value_stride is set to the byte stride of the value type: in this case, 20 for Decimal.

• m_element_type is set to the Swift tuple type (key: Int64, value: Foundation.Decimal).

• m_key_stride_padded is set to the byte stride of m_element_type (which is 32) minus m_value_stride: in this case, 32 - 20 = 12.

So there are four bytes of padding somewhere in the tuple's storage.

NativeHashedStorageHandler::GetElementAtIndex creates a DataBufferHeap of size m_key_stride_padded + m_value_stride = 12 + 20 = 32 = the stride of the tuple. It fills 8 bytes of the buffer, starting at offset 0, with the Int64 key. It fills 20 bytes of the buffer, starting at offset m_key_stride_padded = 12, with the Decimal value. The “middle” 4 bytes are left set to zero (which is how they were initialized), so these must be the padding.

Later, ValueObjectConstResultImpl::CreateChildAtIndex is tasked with creating the
ValueObject representing child 1 (the Decimal) of the tuple stored in the buffer. This method ultimately drills down and (I believe) uses the target's Swift runtime to get the offset of child 1 (the value member of the tuple). The Swift runtime reports that the offset is 8. This does not match the offset used by NativeHashedStorageHandler when it filled the buffer holding the tuple.

Checking the actual bytes of an (Int64, Decimal) tuple in the Swift REPL shows that indeed the padding is at the end, not in the middle:

  8> var t3 = ([0x123456789abcdef0: 7] as [Int64: Decimal]).first!
t3: (key: Int64, value: Decimal) = {
  key = 1311768467463790320
  value = [00 21 00 00 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00]
}
  9> withUnsafeBytes(of: &t3) { Array($0) }.map({ String(format: "%02x", $0) }).joined(separator: " ") 
$R4: String = "f0 de bc 9a 78 56 34 12 00 21 00 00 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
               ^--------- key -------^ ^-------------------- value ------------------^ ^-- pad --^

So the bug apparently stems from the disagreement between NativeHashedStorageContainer and the Swift runtime regarding the layout of the tuple.

The bug doesn't happen with an Int32 key because in that case, the tuple storage has no padding.

It also doesn't happen if I use a Dictionary<Decimal, Int64>. In that case, the padding bytes are indeed stored in the middle of the tuple. We can check in the REPL:

 10> var t4 = ([7: 0x123456789abcdef0] as [Decimal: Int64]).first! 
t4: (key: Decimal, value: Int64) = {
  key = [00 21 00 00 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00]
  value = 1311768467463790320
}
 11> withUnsafeBytes(of: &t4) { Array($0) }.map({ String(format: "%02x", $0) }).joined(separator: " ") 
$R5: String = "00 21 00 00 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f0 de bc 9a 78 56 34 12"
               ^---------------------- key ------------------------------^ ^-- pad --^ ^------ value --------^

I have no idea how Swift chooses where to put the padding…

Anyway, it seems to me that NativeHashedStorageContainer ought to compute the layout the same way CreateChildAtIndex does—by asking the Swift runtime—instead of assuming that the padding is in the middle.

cc @lorentey

Thanks for tracking this down! I agree, NativeHashedStorageContainer should go through the Swift runtime for tuple layout.

@swift-ci create

Please consider trying on top-of-tree, where this is fixed. I fixed this recently.
Thank you a lot for your report and your detailed analysis!

dtdebugger:bin davide$ ./lldb --repl
Welcome to Swift version 5.0-dev (LLVM 2007ecfbec, Clang 9dad98a2d2, Swift fb8b9e143d).
Type :help for assistance.
  1> :command script import decimal.py 
registering Decimal type summaries
  1> import Foundation 
  2> let dec: Decimal = 7 
dec: Decimal = [00 21 00 00 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00]
  3> let dict: [String: Decimal] = ["x": dec]
dict: [String : Decimal] = 1 key/value pair {
  [0] = {
    key = "x"
    value = [00 21 00 00 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00]
  }
}
  4>  

Fixed.

verified in swift-DEVELOPMENT-SNAPSHOT-2018-12-04-a.xctoolchain

Thanks Davide!