Uploaded image for project: 'Swift'
  1. Swift
  2. SR-11557

URLSession closing unrelated file descriptors

    XMLWordPrintable

    Details

      Description

      This is a very serious issue where in Swift 5.1, corelibs-foundation's URLSession is closing unrelated file descriptors, leading to arbitrary undefined behaviour. Swift 5.0.x do not seem to have this issue.

      The attached (single file but depending on SwiftNIO to have a web server) SwiftPM project uses URLSession to continuously make requests to the web server that the projects itself spawns too.

      A successful run should look like

      === ROUND 1 ===
      ........................................................................................................................................................................................................done
      === ROUND 2 ===
      ........................................................................................................................................................................................................done
      === ROUND 3 ===
      ........................................................................................................................................................................................................done
      === ROUND 4 ===
      ........................................................................................................................................................................................................done
      === ROUND 5 ===
      ........................................................................................................................................................................................................done
      === ROUND 6 ===
      ........................................................................................................................................................................................................done
      [ continues forever ]
      

      which it does on macOS (as well as on 5.0.3 on Linux).

      On Linux however, the program usually gets stuck but arbitrary bad things can happen. The problem is best seen when started as

      swift build && strace -ff .build/debug/URLSessionRandoCloseRepro 2>&1 | grep --line-buffered -B5 EBADF
      

      which greps for syscalls returning EBADF (which should never happen).

      Running it however leads to:

      [pid  4474] <... dup resumed> )         = 59
      [pid  4482] <... clock_gettime resumed> {tv_sec=48178, tv_nsec=154579697}) = 0
      [pid  4474] close(7 <unfinished ...>
      [pid  4482] sendto(7, "GET / HTTP/1.1\r\nHost: localhost:"..., 218, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
      [pid  4474] <... close resumed> )       = 0
      [pid  4482] <... sendto resumed> )      = -1 EBADF (Bad file descriptor)
      --
      [pid  4474] <... dup resumed> )         = 10
      [pid  4481] <... clock_gettime resumed> {tv_sec=48182, tv_nsec=195487097}) = 0
      [pid  4474] dup(1 <unfinished ...>
      [pid  4481] sendto(44, "GET / HTTP/1.1\r\nHost: localhost:"..., 218, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
      [pid  4474] <... dup resumed> )         = 12
      [pid  4481] <... sendto resumed> )      = -1 EBADF (Bad file descriptor)
      --
      [pid  4474] close(50 <unfinished ...>
      [pid  4476] <... clock_gettime resumed> {tv_sec=48188, tv_nsec=404958097}) = 0
      [pid  4474] <... close resumed> )       = 0
      [pid  4476] sendto(23, "GET / HTTP/1.1\r\nHost: localhost:"..., 218, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
      [pid  4474] close(34)                   = 0
      [pid  4476] <... sendto resumed> )      = -1 EBADF (Bad file descriptor)
      --
      [pid  4485] futex(0x7fcc61109260, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1569966418, tv_nsec=881007200}, 0xffffffff <unfinished ...>
      [pid  4474] close(37 <unfinished ...>
      [pid  4479] fstat(9,  <unfinished ...>
      [pid  4474] <... close resumed> )       = 0
      [pid  4475] <... timerfd_settime resumed> NULL) = 0
      [pid  4479] <... fstat resumed> 0x7fcc54974ae8) = -1 EBADF (Bad file descriptor)
      --
      [pid  4491] <... openat resumed> )      = 7
      [pid  4474] <... close resumed> )       = 0
      [pid  4491] fstat(7,  <unfinished ...>
      [pid  4474] close(9 <unfinished ...>
      [pid  4491] <... fstat resumed> {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
      [pid  4474] <... close resumed> )       = -1 EBADF (Bad file descriptor)
      --
      [pid  4474] close(12 <unfinished ...>
      [pid  4482] <... clock_gettime resumed> {tv_sec=48267, tv_nsec=918388397}) = 0
      [pid  4474] <... close resumed> )       = 0
      [pid  4482] sendto(7, "GET / HTTP/1.1\r\nHost: localhost:"..., 218, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
      [pid  4474] close(14 <unfinished ...>
      [pid  4482] <... sendto resumed> )      = -1 EBADF (Bad file descriptor)
      --
      [pid  4489] <... clock_gettime resumed> {tv_sec=48268, tv_nsec=26907697}) = 0
      [pid  4474] <... close resumed> )       = 0
      [pid  4474] close(7 <unfinished ...>
      [pid  4489] sendto(7, "GET / HTTP/1.1\r\nHost: localhost:"..., 218, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
      [pid  4474] <... close resumed> )       = 0
      [pid  4489] <... sendto resumed> )      = -1 EBADF (Bad file descriptor)
      --
      [pid  4474] <... close resumed> )       = 0
      [pid  4489] <... clock_gettime resumed> {tv_sec=48268, tv_nsec=433895397}) = 0
      [pid  4474] close(7 <unfinished ...>
      [pid  4489] sendto(7, "GET / HTTP/1.1\r\nHost: localhost:"..., 218, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
      [pid  4474] <... close resumed> )       = 0
      [pid  4489] <... sendto resumed> )      = -1 EBADF (Bad file descriptor)
      

      which shows severe file descriptor mismanagement in many cases.

      One line repro from macOS:

      tar xf URLSessionRandoCloseRepro.tar.gz
      cd URLSessionRandoCloseRepro
      docker run -it --rm --privileged -v "$PWD:$PWD" -w "$PWD" swift:5.1 bash -c 'apt-get -yy update && apt-get -yy install strace && swift build && strace -ff .build/debug/URLSessionRandoCloseRepro 2>&1 | grep --line-buffered -B5 EBADF'
      

      Many thanks to Jordane Belanger who first reported the issue on the Vapor Discord.

        Attachments

          Activity

            People

            • Assignee:
              millenomi Lily Vulcano
              Reporter:
              jw Johannes Weiss
            • Votes:
              3 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: