Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SR-9713] Adding a new benchmark should be easier and less error-prone #52151

Open
swift-ci opened this issue Jan 20, 2019 · 4 comments
Open

[SR-9713] Adding a new benchmark should be easier and less error-prone #52151

swift-ci opened this issue Jan 20, 2019 · 4 comments
Labels
improvement standard library Area: Standard library umbrella

Comments

@swift-ci
Copy link
Collaborator

Previous ID SR-9713
Radar None
Original Reporter BalestraPatrick (JIRA User)
Type Improvement
Additional Detail from JIRA
Votes 0
Component/s Standard Library
Labels Improvement
Assignee None
Priority Medium

md5: 4c7513e1924c08dd259349acbc66153e

Issue Description:

After working on SR-8908 as part of SR-8905, I think the process of adding a new benchmark could be improved and automated. Right now, the [documentation](https://github.com/apple/swift/tree/master/benchmark#adding-new-benchmarks) mentions 3 steps in order to create and register a new single-source benchmark with the driver:

  1. Add a new Swift file (YourTestNameHere.swift), built according to the template below, to the {{single-source}}directory.

  2. Add the filename of the new Swift file to CMakeLists.txt

  3. Edit main.swift. Import and register your new Swift module.

I'd like to see this process automated with a simple script that takes the name of a new benchmark and correctly creates, sets up the files and inserts the benchmark in the correct and sorted position for example.

Would this makes sense? I have a PR ready to be opened with a WIP implementation.

cc: @milseman @palimondo
@palimondo
Copy link
Mannequin

palimondo mannequin commented Jan 21, 2019

I appreciate the effort Patrick! That’s in fact how I started contributing to Swift myself: #9193
But the script-generated approach was replaced sometime last year (can’t find the exact PR at the moment) in favor of the manual approach that’s in now. Maybe @eeckstein or @gottesmm can fill in the reasons behind that…

@swift-ci
Copy link
Collaborator Author

Comment by Patrick Balestra (JIRA)

@palimondo That's interesting. I opened a PR yesterday just to make the code visible, I am happy to close if it is not needed or it's not in the direction you want to go. #22016

@palimondo
Copy link
Mannequin

palimondo mannequin commented Jan 21, 2019

Personally I find your PR helpful, I just meant to warn you against automating too much as the previous scripting has been curbed.

@eeckstein
Copy link
Member

In the long term we want to switch to swiftpm for building the benchmarks, which limits the possibility of generating sources, etc.
But your script can still be useful, because it's a manual process and not invoked from the build.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
improvement standard library Area: Standard library umbrella
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@eeckstein @swift-ci