Uploaded image for project: 'Swift'
  1. Swift
  2. SR-11576

[Workaround in place] Flaky Driver tests

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Reopened
    • Priority: Medium
    • Resolution: Unresolved
    • Component/s: Compiler
    • Labels:

      Description

      I've occasionally run into issues with the tests Driver/linker.swift, Driver/linker-clang_rt.swift and Driver/embed-bitcode.swift being flaky.

      The following test results are with e052da7d8886fa0439677852e8f7830b20c2e1da (tag: swift-DEVELOPMENT-SNAPSHOT-2019-09-25-a) but I've seen the same issues at other points as well.

      Here's a Python runner to check this:

      #!/usr/bin/env python3
      
      import subprocess
      import sys
      
      total_runs = 200
      num_fails = 0
      
      for _ in range(total_runs):
        devnull = subprocess.DEVNULL
        done = subprocess.run(['../llvm/utils/lit/lit.py',
          '../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/test-macosx-x86_64/Driver',
          '--filter=(linker|embed-bitcode)'], stdout=devnull, stderr=devnull)
        if done.returncode != 0:
          num_fails += 1
          sys.stdout.write('-')
        else:
          sys.stdout.write('+')
        sys.stdout.flush()
      
      print("\nEncountered {} failures out of {} total runs".format(
          num_fails, total_runs))
      

      Running the script once gave me 4 failures out of 200 runs.

      N.B. The script also runs 4 other tests, not sure how those are interacting with the flaky tests, but it seems hard/impossible to get the failure if only the 2 flaky tests are run.

        Attachments

          Activity

            People

            Assignee:
            beccadax Becca Royal-Gordon
            Reporter:
            theindigamer Varun Gandhi
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: