Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-752] print does not print in some XCTest situations #405

Closed
drewcrawford opened this issue Feb 16, 2016 · 8 comments
Closed

[SR-752] print does not print in some XCTest situations #405

drewcrawford opened this issue Feb 16, 2016 · 8 comments
Assignees

Comments

@drewcrawford
Copy link

Previous ID SR-752
Radar None
Original Reporter @drewcrawford
Type Bug
Status Closed
Resolution Won't Do

Attachment: Download

Environment

Linux x64
swift-DEVELOPMENT-SNAPSHOT-2016-02-08-a

Additional Detail from JIRA
Votes 0
Component/s XCTest
Labels Bug
Assignee @modocache
Priority Medium

md5: d46b18027b75685e1ff1ab234eddcb5f

Issue Description:

Okay, let me fix this trainwreck of a bug report. In the attached sample project, we have a function that prints:

public func hello() {
    print("Hello world.")
}

We have a test that calls this function, and then crashes:

class FooTests: XCTestCase {
    func testFoo() {
        hello()
        fatalError("'Hello world' will not be printed.")
    }
}

On OSX / Darwin XCTest, we see the print statement (expected behavior):

Test Suite 'All tests' started at 2016-02-24 02:25:10.840
Test Suite 'XCTestRun.xctest' started at 2016-02-24 02:25:10.841
Test Suite 'FooTests' started at 2016-02-24 02:25:10.841
Test Case '-[footests.FooTests testFoo]' started.
Hello world.
fatal error: 'Hello world' will not be printed.: file src/FooTests.swift, line 6

But on Linux (corelibs-xctest), we do not:

Compiling Swift Module 'lib' (1 sources)
Linking Library:  .atllbuild/products/lib.a
Compiling Swift Module 'footests' (2 sources)
Linking executable .atllbuild/products/footests
Test Case 'FooTests.testFoo' started.
fatal error: 'Hello world' will not be printed.: file src/FooTests.swift, line 6

It is not as simple as "print isn't buffered on Linux", because a trivial example of print-then-fatalError prints as expected. So there is something to do with XCTest specifically involved here.

Implementation notes

SwiftPM does not support XCTest in the Feb 8th snapshot, so instead this sample project uses atbuild. Apologies in advance for foisting an unfamiliar build system on you. You can download it here. On OSX, to reproduce it is simply atbuild in the working directory. For Linux, I have replicated my entire environment in Docker, and you can just run docker build . on any system with Docker installed to see the Linux behavior.

On Darwin, atbuild uses Darwin XCTest, builds .xctest bundles, and runs tests with xcrun xctest /path/to/bundle.xctest. On Linux, it builds executables linked with corelibs-xctest and runs them via the system POSIX call.

@modocache
Copy link
Mannequin

modocache mannequin commented Feb 23, 2016

Thanks for the report, @drewcrawford! I'm a little confused, though--why do you think XCTLog should be defined? I'm not familiar with that function, and looking through swift-corelibs-xctest, we don't appear to define it anywhere. Apple XCTest doesn't document it in any of its public headers, either.

Apple XCTest does define a class XCTestLog:

#import <XCTest/XCTestObserver.h>

/*!
 * XCTestLog is deprecated.
 */

DEPRECATED_ATTRIBUTE
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wdeprecated-declarations"
@interface XCTestLog : XCTestObserver
#pragma clang diagnostic pop

@property (readonly, strong) NSFileHandle *logFileHandle;
- (void)testLogWithFormat:(NSString *)format, ... NS_FORMAT_FUNCTION(1,2);
- (void)testLogWithFormat:(NSString *)format arguments:(va_list)arguments NS_FORMAT_FUNCTION(1,0);

@end

Note that XCTestLog is deprecated.

In any case, could you explain what the bug is here? Sorry for not catching on!

@drewcrawford
Copy link
Author

Sorry for writing a trainwreck of a bug report. Let me explain how I got here:

1. print didn't work inside an XCTest target, on x64 Linux. Is that a bug? Well, maybe not... I know XCTest does weird XPC stuff on Darwin, maybe stdout isn't connected on Linux by design.
2. Typed in XCT and waited for autocomplete, oh look! XCTLog XCTestLog sounds like a log mechanism. Guess I should use this.
3. But not on Linux, because it's not implemented.
4. Begrudingly use fputs("(whatever)\n",stderr) everywhere, take a shot, file a bug.

Knowing what I know now:

1. Maybe the real bug is that print doesn't work?
2. Maybe the real bug is that there's no documentation that would tell me whether print is supposed to work or not?
3. Maybe the real bug is that we don't ship a logging function that works instead?
4. Maybe there's an Xcode bug in here that doesn't sound an alarm when I try to use a deprecated symbol?

![](Screen Shot 2016-02-23 at 6.26.06 PM.png)

I know that I still haven't given enough information to get any of these issues reproducible, but it would be a good start to find out which ones are expected and which are not.

@modocache
Copy link
Mannequin

modocache mannequin commented Feb 24, 2016

Aha, thanks for the detailed follow-up! 🙂

print() should most certainly work from within an XCTestCase method... I wonder what's going on here. Can you describe what you mean by "not work"? Is it not being output? Are you sure the test method is being executed? Is the method included in your allTests list?

@modocache
Copy link
Mannequin

modocache mannequin commented Mar 10, 2016

Ping @drewcrawford--did you see my comment above? Does this still happen for you?

@drewcrawford
Copy link
Author

Sorry @modocache. I decided to reply by rewriting the bug description, since my original was so bad, and was a poor introduction to anyone who might stumble onto the issue. I thought it would notify you. It seems it did not.

I am still affected by the issue. The rewritten bug description includes a full test case.

Thanks for following up.

@modocache
Copy link
Mannequin

modocache mannequin commented Apr 2, 2016

The root cause of this may be https://bugs.swift.org/browse/SR-1127. Based on the comment by Dmitri there, we may want to close this as a "won't fix".

@briancroom
Copy link
Collaborator

@drewcrawford Could you follow-up on whether this is still affecting you, and whether you think that the `print()` buffering could be the source? Does the missing output appear if you explicitly flush before the `fatalError`? I'll point out that XCTest itself does explicitly flush after each of its `print` statements: https://github.com/apple/swift-corelibs-xctest/blob/master/Sources/XCTest/PrintObserver.swift#L69

@modocache
Copy link
Mannequin

modocache mannequin commented Jul 26, 2016

I'm going to go ahead and close this due to:

  1. The fact that it's probably no longer a problem for the original reporter

  2. "Fixing" the print() behavior is a much larger problem that is better tracked in https://bugs.swift.org/browse/SR-1127

Thanks for the report, though!

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 9, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants