Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-1141] [llbuild] Add support for understanding Swift parseable output #810

Open
ddunbar opened this issue Apr 4, 2016 · 2 comments
Open

Comments

@ddunbar
Copy link
Member

ddunbar commented Apr 4, 2016

Previous ID SR-1141
Radar rdar://problem/22640310
Original Reporter @ddunbar
Type New Feature
Additional Detail from JIRA
Votes 1
Component/s llbuild
Labels New Feature, StarterBug
Assignee None
Priority Medium

md5: fc8862bc1e8ba85bdbc5a4da5be2b191

relates to:

  • SR-1143 [llbuild] Support for scheduling multithreaded tasks

Issue Description:

We should have support for understanding swiftc's "parseable output".

This will allow us to report improved status about compilation of large projects, because we can report on the status of individual files as they are compiled by swiftc.

This is also important for building additional features on (like better multithreaded scheduling) that rely on being able to get richer status information from the compiler.

@ddunbar
Copy link
Member Author

ddunbar commented Apr 13, 2016

This is a somewhat involved project because it is going to require changing a number of internal interfaces.

I believe the high level tasks, independent, tasks here are as follows:

1. We need a way for the execution queue to be able to notify clients immediately when data is available from a subprocess (so we can process individual swiftc command messages as soon as they arrive). This probably requires some kind of delegate or callback interfaces on `ExecutionQueue::executeProcess()`.

2. `SwiftCompilerShellCommand` needs to implement those interfaces, and then decode individual messages as they arrive, and arrange them into whatever interesting internal structure is necessary to make use of them (e.g., knowing how many files are outstanding).

3. We need to extend the status reporting APIs so that the high-level client has a sensible API for seeing the status of commands which themselves are broken down into individual steps. This can be done independently of 1 & 2 (e.g., it should be possible to implement a command which runs two subprocesses, and reports separate status for each).

4. `SwiftCompilerShellCommand` needs to use the APIs in #3 to inform the client about what is actually going on.

This would complete this initial basic task, so for example now incremental compiles will show exactly what files `swiftc` is rebuilding.

The next thing to do after that is to then make the scheduler aware of when a subprocess is using multiple threads, so that it can schedule appropriately. This will require using the information from `swiftc` (and maybe even adding new information messages) to know how many threads it is currently using, and also new internal APIs to interact with the scheduler.

@belkadan
Copy link

Resetting assignee on all Starter Bugs not modified since 2018.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants