Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SR-12292] SwiftPM: file descriptors open in swift-build are inherited into the children #4584

Open
weissi opened this issue Feb 27, 2020 · 3 comments
Open

[SR-12292] SwiftPM: file descriptors open in swift-build are inherited into the children #4584

weissi opened this issue Feb 27, 2020 · 3 comments
Labels
bug

Comments

@weissi
Copy link
Member

weissi commented Feb 27, 2020

Previous ID SR-12292
Radar rdar://problem/59862469
Original Reporter @weissi
Type Bug
Additional Detail from JIRA
Votes 1
Component/s Package Manager
Labels Bug
Assignee None
Priority Medium

md5: cc250847758272db72425962ef116676

Issue Description:

Just like Foundation used to do (apple/swift-corelibs-foundation#2608 SwiftPM fails to close the file descriptors before spawning child processes. This means that some file descriptors of swift-build are also available in the child processes, usually compilers.

Repro:

git clone https://github.com/apple/swift-nio && cd swift-nio && swift build -c release -j 1

then in a different terminal do

lsof -p $(pgrep swift-build)

to see the file descriptors open in swift-build. For me, this shows:

swift-bui 682 root    0u   CHR  136,0      0t0      3 /dev/pts/0
swift-bui 682 root    1u   CHR  136,0      0t0      3 /dev/pts/0
swift-bui 682 root    2u   CHR  136,0      0t0      3 /dev/pts/0
swift-bui 682 root    3r  FIFO   0,12      0t0  46206 pipe
swift-bui 682 root    4w  FIFO   0,12      0t0  46206 pipe
swift-bui 682 root    5uw  REG   0,90    24576 919901 /Users/johannes/devel/swift-nio/.build/build.db
swift-bui 682 root    6r  FIFO   0,12      0t0  47138 pipe
swift-bui 682 root    7u   REG   0,90    21032 919903 /Users/johannes/devel/swift-nio/.build/build.db-journal
swift-bui 682 root    9r  FIFO   0,12      0t0  47139 pipe

Now, let's see who has access to for example the pipe with node 46206: {{lsof | grep 46206}

root@321d462c7b65:/Users/johannes/devel/swift-nio# lsof | grep 46206
swift-bui 682     root    3r  FIFO   0,12       0t0  46206 pipe
swift-bui 682     root    4w  FIFO   0,12       0t0  46206 pipe
bash      682 683 root    3r  FIFO   0,12       0t0  46206 pipe
bash      682 683 root    4w  FIFO   0,12       0t0  46206 pipe
swift-bui 682 688 root    3r  FIFO   0,12       0t0  46206 pipe
swift-bui 682 688 root    4w  FIFO   0,12       0t0  46206 pipe
swift-bui 682 689 root    3r  FIFO   0,12       0t0  46206 pipe
swift-bui 682 689 root    4w  FIFO   0,12       0t0  46206 pipe
swiftc    718     root    3r  FIFO   0,12       0t0  46206 pipe
swiftc    718     root    4w  FIFO   0,12       0t0  46206 pipe
swift     719     root    3r  FIFO   0,12       0t0  46206 pipe
swift     719     root    4w  FIFO   0,12       0t0  46206 pipe

which seems a bit excessive that this file descriptor is shared between so many processes. Also note how it's file descriptor 3 and 4 for every process which is probably due to the fact that the fds are inherited through posix_spawn.

--> https://twitter.com/krzyzanowskim/status/1232979379237638144
@weissi
Copy link
Member Author

weissi commented Feb 27, 2020

@swift-ci create

@weissi
Copy link
Member Author

weissi commented Apr 30, 2020

CC @aciidb0mb3r

@weissi
Copy link
Member Author

weissi commented May 1, 2020

This probably also makes https://bugs.swift.org/browse/SR-11929 worse

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tomerd @weissi