You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
swift-corelibs-xctest includes a functional test suite that uses the lit test runner and a Python program that matches output line by line. Here's an example.
Notice that the parentheses in the CHECK lines need to be escaped. This is because each line is interpreted as a regular expression. This makes writing CHECK lines very cumbersome and error-prone.
The Python program used to match output in corelibs-xctest should use a similar mechanism: regexes should delineated, instead of having the entire line be interpreted as a regex. For example, the CHECK lines in the test linked above should become:
// CHECK: Test Case 'SingleFailingTestCase.test_fails' started.// CHECK: {{.*}}/Tests/Functional/SingleFailingTestCase/main.swift:24: error: SingleFailingTestCase.test_fails : XCTAssertTrue failed - // CHECK: Test Case 'SingleFailingTestCase.test_fails' failed ({{\d+}}.{{\d+}} seconds).// CHECK: Executed 1 test, with 1 failure (0 unexpected) in {{\d+}}.{{\d+}} ({{\d+}}.{{\d+}}) seconds// CHECK: Total executed 1 test, with 1 failure (0 unexpected) in {{\d+}}.{{\d+}} ({{\d+}}.{{\d+}}) seconds
(Of course, even better would be to have corelibs-xctest use FileCheck directly, but that program is only available as part of a full LLVM build, and we don't want to introduce such a dependency.)
The text was updated successfully, but these errors were encountered:
Hi @modocache, I'm seeing that a lot of the starter tasks for XCTest (SR-1046, SR-1045, SR-1048 and this one) require some knowledge of LLVM and Python. I'm very new to both but am willing to study them. Do you recommend picking up any of these stories for someone with not much experience in this area? If so, which one would be the easiest to go with?
Hi Kyle, sorry for the late reply. I think this task involves knowledge of Python. If you don't know Python already, it should be fairly easy to learn, and this might be a good starter project!
That being said, based on the discussion in #94 I think we're going to move in another direction: using FileCheck directly, instead of using xctest_checker. So I think this task might be obsolete--I'll close it once mike-ferris-apple (JIRA User) weighs in.
Additional Detail from JIRA
md5: 36d197324a801db0dd6d5b0d341b6a3e
Issue Description:
swift-corelibs-xctest includes a functional test suite that uses the
lit
test runner and a Python program that matches output line by line. Here's an example.Notice that the parentheses in the
CHECK
lines need to be escaped. This is because each line is interpreted as a regular expression. This makes writingCHECK
lines very cumbersome and error-prone.On the other hand,
FileCheck
, a matching program used by most of the LLVM and Swift test suites, does not interpretCHECK
lines as regexes. Instead, parts of the line may be interpreted as a regex using a special pattern, such as{{.*}}
.The Python program used to match output in corelibs-xctest should use a similar mechanism: regexes should delineated, instead of having the entire line be interpreted as a regex. For example, the
CHECK
lines in the test linked above should become:(Of course, even better would be to have corelibs-xctest use
FileCheck
directly, but that program is only available as part of a full LLVM build, and we don't want to introduce such a dependency.)The text was updated successfully, but these errors were encountered: