[SR-8999] NSData contentsOf expects files to never shrink and to always report st_size == number of readable bytes #3616

weissi · 2018-10-15T15:07:04Z


Previous ID	SR-8999
Radar	rdar://problem/45369000
Original Reporter	@weissi
Type	Bug
Status	Resolved
Resolution	Done

Additional Detail from JIRA


Votes	0
Component/s	Foundation
Labels	Bug
Assignee	@millenomi
Priority	Medium

md5: fd2ac0a4981a8b2036f67f99bf55ad26

Issue Description:

There's a number of bugs in NSData's file reading code: [Its code](https://github.com/apple/swift-corelibs-foundation/blob/43ecec27e47099e49aa2e668bdb6b00cb642850c/Foundation/NSData.swift#L457-L466) relies on two things that both are not true:

files don't shrink
`st_size` is equal to the number of readable bytes

How does this code rely on those untrue conditions?

        var remaining = Int(info.st_size)
        var total = 0
        while remaining > 0 {
            let amt = read(fd, data.advanced(by: total), remaining)
            if amt < 0 {
                break
            }
            remaining -= amt
            total += amt
        }

This will spin in a loop until it has read `remaining` bytes in total which might never happen: For one the file might have shrunk between that `fstat` and the `read` calls and on top of that some pseudo-files in Linux (especially in `/sys`) return a large `st_size` and you can't actually read any bytes. For example on my machine:

$ ls -la /sys/devices/pnp0/00\:00/uevent
-rw-r--r-- 1 root root 4096 Oct 15 13:54 /sys/devices/pnp0/00:00/uevent
$ stat /sys/devices/pnp0/00\:00/uevent
  File: '/sys/devices/pnp0/00:00/uevent'
  Size: 4096        Blocks: 0          IO Block: 4096   regular file
Device: 77h/119d    Inode: 2450        Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2018-10-15 14:01:25.211472722 +0000
Modify: 2018-10-15 14:01:25.211472722 +0000
Change: 2018-10-15 14:01:25.211472722 +0000
 Birth: -

so the file has `st_size = 4096` but

$ cat /sys/devices/pnp0/00\:00/uevent | wc -c 
0

so we can't actually read anything from it... Testing this with `Foundation` shows this problem "nicely"

  1> import Foundation
  2> String(contentsOfFile: "/sys/devices/pnp0/00:00/uevent")
[... hangs forever ...]

weissi · 2018-10-18T13:57:19Z

@swift-ci create

millenomi · 2018-10-19T20:47:39Z

Per discussion with @spevans —

Simon Evans [15:19]
I think we should unify `NSData.readBytesFromFileWithExtendedAttributes()` and `FileHandle._readDataOfLength()` since they both duplicate each other as part of the fix for SR-8999

Lily Vulcano [15:26]
good call
perhaps NSData can just form a FileHandle and read to the end of the file.
we don’t know the length, which is the problem.

Simon Evans [15:29]
`FileHandle.readDataToEndOfFile()` should probably work

spevans · 2018-10-20T15:12:49Z

#1732

spevans · 2018-11-14T16:30:57Z

This has been fixed in swift-DEVELOPMENT-SNAPSHOT-2018-11-13

$ cat sr_8999.swift
import Foundation
let s = try String(contentsOfFile: "/sys/devices/pnp0/00:00/uevent")
print(s)
$ ~/swift-DEVELOPMENT-SNAPSHOT-2018-11-13-a-ubuntu18.04/usr/bin/swift sr_8999.swift 
DRIVER=system

swift-ci transferred this issue from apple/swift-issues Apr 25, 2022

shahmishal transferred this issue from apple/swift May 5, 2022

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SR-8999] NSData contentsOf expects files to never shrink and to always report st_size == number of readable bytes #3616

[SR-8999] NSData contentsOf expects files to never shrink and to always report st_size == number of readable bytes #3616

weissi commented Oct 15, 2018

weissi commented Oct 18, 2018

millenomi commented Oct 19, 2018

spevans commented Oct 20, 2018

spevans commented Nov 14, 2018

[SR-8999] NSData contentsOf expects files to never shrink and to always report st_size == number of readable bytes #3616

[SR-8999] NSData contentsOf expects files to never shrink and to always report st_size == number of readable bytes #3616

Comments

weissi commented Oct 15, 2018

weissi commented Oct 18, 2018

millenomi commented Oct 19, 2018

spevans commented Oct 20, 2018

spevans commented Nov 14, 2018