[SR-10801] `swift test --filter` should display summary for the entire group of tests run, not per-test - Swift

Details

Type: Improvement
Status: Open
Priority: Medium
Resolution: Unresolved
Component/s: Package Manager
Labels:
- StarterBug

Radar URL:
rdar://problem/45138914

Description

When running `swift test` with no --filter, the summary at the bottom is useful to see how many tests failed:When running `swift test` with no --filter, the summary at the bottom is useful to see how many tests failed:


...
Test Case '-[IOTests.TypesTests testPacker]' passed (0.000 seconds).
Test Case '-[IOTests.TypesTests testStruct]' started.
Test Case '-[IOTests.TypesTests testStruct]' passed (0.000 seconds).
Test Suite 'TypesTests' passed at 2018-10-09 13:48:23.339.
	 Executed 2 tests, with 0 failures (0 unexpected) in 0.000 (0.001) seconds
Test Suite 'IOPackageTests.xctest' failed at 2018-10-09 13:48:23.339.
	 Executed 21 tests, with 2 failures (0 unexpected) in 0.116 (0.117) seconds
Test Suite 'All tests' failed at 2018-10-09 13:48:23.339.
	 Executed 21 tests, with 2 failures (0 unexpected) in 0.116 (0.117) seconds

With the --filter it seems to run each matched test individually as "selected tests", so the summary is printed per test meaning you can't see at a glance how many tests failed by looking at the last line:

Compile Swift Module 'IOTests' (8 sources)
Linking ./.build/x86_64-apple-macosx10.10/debug/IOPackageTests.xctest/Contents/MacOS/IOPackageTests
Test Suite 'Selected tests' started at 2018-10-09 13:50:42.838
Test Suite 'IOPackageTests.xctest' started at 2018-10-09 13:50:42.838
Test Suite 'MemoryTests' started at 2018-10-09 13:50:42.838
Test Case '-[IOTests.MemoryTests testPageList]' started.
Test Case '-[IOTests.MemoryTests testPageList]' passed (0.077 seconds).
Test Suite 'MemoryTests' passed at 2018-10-09 13:50:42.915.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.077 (0.077) seconds
Test Suite 'IOPackageTests.xctest' passed at 2018-10-09 13:50:42.915.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.077 (0.077) seconds
Test Suite 'Selected tests' passed at 2018-10-09 13:50:42.915.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.077 (0.077) seconds
Test Suite 'Selected tests' started at 2018-10-09 13:50:42.962
Test Suite 'IOPackageTests.xctest' started at 2018-10-09 13:50:42.962
Test Suite 'MemoryTests' started at 2018-10-09 13:50:42.962
Test Case '-[IOTests.MemoryTests testRead16]' started.
Test Case '-[IOTests.MemoryTests testRead16]' passed (0.075 seconds).
Test Suite 'MemoryTests' passed at 2018-10-09 13:50:43.038.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.075 (0.076) seconds
Test Suite 'IOPackageTests.xctest' passed at 2018-10-09 13:50:43.038.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.075 (0.076) seconds
Test Suite 'Selected tests' passed at 2018-10-09 13:50:43.038.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.075 (0.076) seconds
Test Suite 'Selected tests' started at 2018-10-09 13:50:43.085
Test Suite 'IOPackageTests.xctest' started at 2018-10-09 13:50:43.085
Test Suite 'MemoryTests' started at 2018-10-09 13:50:43.085
Test Case '-[IOTests.MemoryTests testRead32]' started.
Test Case '-[IOTests.MemoryTests testRead32]' passed (0.062 seconds).
Test Suite 'MemoryTests' passed at 2018-10-09 13:50:43.147.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.062 (0.062) seconds
Test Suite 'IOPackageTests.xctest' passed at 2018-10-09 13:50:43.147.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.062 (0.062) seconds
Test Suite 'Selected tests' passed at 2018-10-09 13:50:43.147.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.062 (0.062) seconds
Test Suite 'Selected tests' started at 2018-10-09 13:50:43.191
Test Suite 'IOPackageTests.xctest' started at 2018-10-09 13:50:43.191
Test Suite 'MemoryTests' started at 2018-10-09 13:50:43.191
Test Case '-[IOTests.MemoryTests testRead64]' started.
Test Case '-[IOTests.MemoryTests testRead64]' passed (0.060 seconds).
Test Suite 'MemoryTests' passed at 2018-10-09 13:50:43.251.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.060 (0.060) seconds
Test Suite 'IOPackageTests.xctest' passed at 2018-10-09 13:50:43.251.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.060 (0.060) seconds
Test Suite 'Selected tests' passed at 2018-10-09 13:50:43.252.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.060 (0.060) seconds
Test Suite 'Selected tests' started at 2018-10-09 13:50:43.297
Test Suite 'IOPackageTests.xctest' started at 2018-10-09 13:50:43.297
Test Suite 'MemoryTests' started at 2018-10-09 13:50:43.297
Test Case '-[IOTests.MemoryTests testRead8]' started.
/Users/best/Documents/IOTests/MemoryTests.swift:23: error: -[IOTests.MemoryTests testRead8] : XCTAssertTrue failed -
Test Case '-[IOTests.MemoryTests testRead8]' failed (0.059 seconds).
Test Suite 'MemoryTests' failed at 2018-10-09 13:50:43.356.
	 Executed 1 test, with 1 failure (0 unexpected) in 0.059 (0.059) seconds
Test Suite 'IOPackageTests.xctest' failed at 2018-10-09 13:50:43.356.
	 Executed 1 test, with 1 failure (0 unexpected) in 0.059 (0.059) seconds
Test Suite 'Selected tests' failed at 2018-10-09 13:50:43.356.
	 Executed 1 test, with 1 failure (0 unexpected) in 0.059 (0.060) seconds
Test Suite 'Selected tests' started at 2018-10-09 13:50:43.403
Test Suite 'IOPackageTests.xctest' started at 2018-10-09 13:50:43.404
Test Suite 'MemoryTests' started at 2018-10-09 13:50:43.404
Test Case '-[IOTests.MemoryTests testWrite16]' started.
Test Case '-[IOTests.MemoryTests testWrite16]' passed (0.062 seconds).
Test Suite 'MemoryTests' passed at 2018-10-09 13:50:43.465.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.062 (0.062) seconds
Test Suite 'IOPackageTests.xctest' passed at 2018-10-09 13:50:43.466.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.062 (0.062) seconds
Test Suite 'Selected tests' passed at 2018-10-09 13:50:43.466.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.062 (0.062) seconds
Test Suite 'Selected tests' started at 2018-10-09 13:50:43.513
Test Suite 'IOPackageTests.xctest' started at 2018-10-09 13:50:43.514
Test Suite 'MemoryTests' started at 2018-10-09 13:50:43.514
Test Case '-[IOTests.MemoryTests testWrite32]' started.
Test Case '-[IOTests.MemoryTests testWrite32]' passed (0.071 seconds).
Test Suite 'MemoryTests' passed at 2018-10-09 13:50:43.585.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.071 (0.071) seconds
Test Suite 'IOPackageTests.xctest' passed at 2018-10-09 13:50:43.585.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.071 (0.071) seconds
Test Suite 'Selected tests' passed at 2018-10-09 13:50:43.585.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.071 (0.071) seconds
Test Suite 'Selected tests' started at 2018-10-09 13:50:43.633
Test Suite 'IOPackageTests.xctest' started at 2018-10-09 13:50:43.633
Test Suite 'MemoryTests' started at 2018-10-09 13:50:43.633
Test Case '-[IOTests.MemoryTests testWrite8]' started.
Test Case '-[IOTests.MemoryTests testWrite8]' passed (0.062 seconds).
Test Suite 'MemoryTests' passed at 2018-10-09 13:50:43.695.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.062 (0.062) seconds
Test Suite 'IOPackageTests.xctest' passed at 2018-10-09 13:50:43.695.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.062 (0.062) seconds
Test Suite 'Selected tests' passed at 2018-10-09 13:50:43.695.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.062 (0.062) seconds

`swift test --filter` should display summary for the entire group of tests run, not per-test

Details

Description

Attachments

Activity

People

Dates