Comment by Ted Goddard (JIRA)

Attempted to debug with

MallocStackLogging=1 MallocCheckHeapStart=100 MallocCheckHeapEach=100 MallocPreScribble=1 MallocScribble=1

but these did not seem to affect the runtime on linux.

Comment by Ted Goddard (JIRA)

Both URLSession variants (as commented out) crash.

Comment by Ted Goddard (JIRA)

Verified crash occurs with swift-4.0-RELEASE

Comment by Konrad Feiler (JIRA)

I'm encountering the same problem, while trying to update a Server-Side-Swift API-validation-tool from Swift3 to Swift4.

Everything is fine on macOS, but opening >1 dataTask's concurrently on linux will crash with the above double-free.

Further testing showed that even when creating separate `urlSession` for each dataTask it still crashes: (Example code I used)

    private let lanes: [(session: URLSession, sema: DispatchSemaphore)] = {
        return (0..<maxConcurrency).map({ _ -> (URLSession, DispatchSemaphore) in
            return (URLSession(configuration: .default), DispatchSemaphore(value: 1))
        })
    }()

    func load(url, completion: @escaping (Result) -> ()) {
        let lane = lanes[laneCtr]
        laneCtr = (laneCtr + 1) % maxConcurrency
        lane.sema.wait()
        let dataTask = lane.session.dataTask(with: url) { (data,response, error) in
            defer { lane.sema.signal() }
        // ...

Comment by Ted Goddard (JIRA)

It almost seems like it is not a concurrency issue, since the crash still occurs with a semaphore guarding it (perhaps similar to Konrad's approach above):

import Foundation
import Dispatch

let urlString = CommandLine.arguments[1]

let urlSession = URLSession(configuration: URLSessionConfiguration.default)

let semaphore = DispatchSemaphore(value: 1)

func fetch() {
    _ = semaphore.wait(timeout: .now() + .seconds(10))
    print("waited, will fetch")
    if let url = URL(string: urlString) {
        let urlRequest = URLRequest(url: url)
        let task = urlSession.dataTask(with: urlRequest) { (data, response, error) in
            print(error)
            if let data = data {
                print("\(data.count) bytes")
            }
            print("    done fetch")
            semaphore.signal()

        }
        task.resume()
    }
}

func randomTime() -> Double {
    return Double(random()) / Double(UInt32.max)
}

func delayFetch() {
    let delay = randomTime()
    DispatchQueue.main.asyncAfter(deadline: .now() + delay) {
        //print("delay \(delay)")
        fetch()
        delayFetch()
    }
}

delayFetch()
delayFetch()
delayFetch()
delayFetch()

RunLoop.main.run()

The logging appears perfectly interleaved, yet the crash still occurs.

Comment by Konrad Feiler (JIRA)

tedgoddard (JIRA User)

Try building and running this in linux: https://gist.github.com/Bersaelor/393925f7e420cf9696153fcd7c5982f8

If I set `maxConcurrency = 1` it will not crash, for >1 it will.
For `1` it will only ever do one URLRequest at a time, only starting the next one once the previous one is back. That doesn't really help with my use case though [testing of rather slow API responses].

Comment by Ted Goddard (JIRA)

On my machine, the above gist crashes as well with similar errors.

Linux carbon 4.10.0-35-generic #39-Ubuntu SMP Wed Sep 13 07:46:59 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

I wonder if the root cause of this bug is the transfer of the network bytes from the release pool of the calling queue to the callback queue. Perhaps other Foundation functions that use system calls and multiple queues are similarly affected.

Comment by Ted Goddard (JIRA)

Comment in SR-5972 indicates that fix is in Swift 4.0.1. I will test. Swift master Ubuntu 16.04 is not functional due to missing libdispatch.so.

Comment by Ted Goddard (JIRA)

Crash is fixed or certainly much less frequent with

Swift version 4.0.1-dev (LLVM 2dedb62a0b, Clang b9d76a314c, Swift cf73eadf04)

Target: x86_64-unknown-linux-gnu

(I have not observed the crash with this version.)

Yes, I'm unable to reproduce it with the latest master. Some concurrency fixes (from @weissi) did go in in the last two weeks. They fixed this, possibly.

Comment by George Leontiev (JIRA)

I am able to reproduce this on
swift-DEVELOPMENT-SNAPSHOT-2017-10-28-a using a large number of concurrent requests.

Tested with high concurrency with both 4.2 and 5.1.5 on ubuntu18.04 and no crashes observed anymore.