Uploaded image for project: 'Swift'
  1. Swift
  2. SR-875

Make swift-corelibs-xctest Functional tests regex matching more like FileCheck

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Medium
    • Resolution: Won't Do
    • Component/s: XCTest
    • Labels:

      Description

      swift-corelibs-xctest includes a functional test suite that uses the lit test runner and a Python program that matches output line by line. Here's an example.

      Notice that the parentheses in the CHECK lines need to be escaped. This is because each line is interpreted as a regular expression. This makes writing CHECK lines very cumbersome and error-prone.

      On the other hand, FileCheck, a matching program used by most of the LLVM and Swift test suites, does not interpret CHECK lines as regexes. Instead, parts of the line may be interpreted as a regex using a special pattern, such as {{.*}}.

      The Python program used to match output in corelibs-xctest should use a similar mechanism: regexes should delineated, instead of having the entire line be interpreted as a regex. For example, the CHECK lines in the test linked above should become:

      // CHECK: Test Case 'SingleFailingTestCase.test_fails' started.
      // CHECK: {{.*}}/Tests/Functional/SingleFailingTestCase/main.swift:24: error: SingleFailingTestCase.test_fails : XCTAssertTrue failed - 
      // CHECK: Test Case 'SingleFailingTestCase.test_fails' failed ({{\d+}}.{{\d+}} seconds).
      // CHECK: Executed 1 test, with 1 failure (0 unexpected) in {{\d+}}.{{\d+}} ({{\d+}}.{{\d+}}) seconds
      // CHECK: Total executed 1 test, with 1 failure (0 unexpected) in {{\d+}}.{{\d+}} ({{\d+}}.{{\d+}}) seconds
      

      (Of course, even better would be to have corelibs-xctest use FileCheck directly, but that program is only available as part of a full LLVM build, and we don't want to introduce such a dependency.)

        Attachments

          Activity

            People

            Assignee:
            modocache Brian Ivan Gesiak
            Reporter:
            modocache Brian Ivan Gesiak
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: