I'm a little concerned that we'll lose what little coverage we have for the interpreter; right now there's a (non-buildbot) test mode that turns simple compile-and-run invocations into interpreter invocations. StdlibUnittest currently doesn't support that because it runs all tests out of process by default.

I should probably file a bug for this.

Filed SR-1125 and SR-1126 (neither of which should block this effort).

Comment by Kyle Yoon (JIRA)

Is this task mostly changing

// RUN: %target-run %t/ArrayBridge > %t.txt
// RUN: FileCheck %s < %t.txt

into

// RUN: %target-run-stdlib-swift | FileCheck %s

for the listed files?
Also, does it require you to re-build the toolchain and test every time?

Sorry for all the questions, again!

No need to apologize for clarifying the task! On the contrary, thanks for these follow-up questions.

The ideal end state of the listed tests would be the following:

// RUN: %target-run-simple-swift

A good example is test/1_stdlib/NSEnumeratorAPI.swift. Notice that that test does not use FileCheck at all. Instead, it uses the TestSuite construct from StdlibUnittest. This library is capable of making assertions that are not based on textual output, like FileCheck is. We want to avoid text output-based assertions.

Comment by Sammy Gutierrez (JIRA)

Here is the work I have for this ticket.

I was hoping someone could take a quick look over it before submitting the PR (should I ask for help in the forums instead?). I attempted to remove FileCheck from any files that I could in /test/stdlib but there were a few places where I did not know what to do:

• Reflection_jit.swift - Could not get Reflection.swift tests to run with JIT (deleted this file for now)

• SwiftObjectNSObject.swift - Calls SwiftObjectNSObject.m and checks for output.

cc @airspeedswift

Resetting assignee for all Starter Bugs not modified since 2018.

I think we lost this one since it wasn't a full PR. In the future, please feel free to make a PR even if it's still WIP.