This happens because we add color diagnostics flag depending on whether the terminal is tty or not. We need to remove that flag from the command signature.

Comment by Andreas Wendleder (JIRA)

Would it be acceptable to remove all occurrences of Color in

swift-package-manager/Sources/Basic/TerminalController.swift?

Comment by Andreas Wendleder (JIRA)

This is still not resolved with Xcode 10.2 and Swift 5.0.

@aciidb0mb3r any idea why swiftc rebuilds in this case? Shouldn't it be up to swiftc to no-op even if swiftpm / llbuild ask for it to build again?

@belkadan might know about that.

The -color-diagnostics flag isn't a normal part of Swift's interface; it's an unstable frontend option. I guess the right fix would be to promote it to a proper option so that it can be marked DoesNotAffectIncrementalBuild and SwiftPM can stop using -Xfrontend to pass it. (I assume it's doing that because it actually captures the output before printing it, and so swiftc's tty autodetection fails?)

I've submitted apple/swift#24144 and #2099 for this

The swiftc change is merged and the swiftpm change is ready but waiting so swiftpm still works with Xcode for the time being

Looks like this is all merged through. Is it ready to be marked Resolved?

#2103 Still needs to be merged

Comment by Andreas Wendleder (JIRA)

Doesn't work in 5.1. 🙁

Comment by Andreas Wendleder (JIRA)

Doesn't work in 5.2.

andreasw (JIRA User) how are you validating if this is fixed or not? It looks to me at this point like it is, I'm testing by using `swift build -v` from a terminal, and then from a makefile. There is still a problem (which we can repurpose this bug for, or create a new one) where any change in environment variables seems to cause swiftpm to re-process the Package.swift files from all dependencies, which does still take some extra time, but it doesn't necessarily re-build anything in that case (unless the env var change actually has an affect on the output). That issue might also not be a trivial solve since Package.swift files can contain arbitrary logic around this.

Comment by Andreas Wendleder (JIRA)

Exactly as specified in the bug report; since executing make from within vim via ":m" is not a terminal everything is recompiled.

1. Type "make" on the command line - everything is compiled.

2. Type "make" again on command line - nothing is compiled since everything is already compiled.

3. Open vim and type ":m" which invokes make but not with terminal output - everything is compiled again.

When you say "everything is recompiled" how are you determining this part? Here's an example of the behavior I see now with Swift 5.2:

1. Have a makefile with `swift build -v`
2. Run `make foo` from the command line twice, note the second time no commands are run
3. Run `make foo | tee /dev/null` from the command line, note Package.swift files are re-computed, and if you pass `-Xswiftc -driver-show-incremental` you see the swiftmodule file is being rebuilt, but no source files
4. Run `:make foo` in vim after step 3 and you'll see no commands are run
5. Repeat step 2 and then 4 and you'll see commands run each time.

The fact that it's running any commands at all is 2 fold:

1. Re my comment above, any change in env vars will cause Package.swift files to be recomputed, likely these change between make, shell, and vim (unless you call `swift build` with `env -i` )
2. This code:

        if BuildParameters.isTTY {
            args += ["-color-diagnostics"]
        }

Adds the `-color-diagnostics` are only in the shell without pipe case, but since my original swiftc changes, this doesn't cause a full-recompile.

So tl;dr AFAICT things aren't being recompiled here (besides the module being re-merged) but there is more noise in the logs, and arguably more things happening, than there should be

Comment by Andreas Wendleder (JIRA)

To be precise:

A simple hello world works as expected.

Compiling my project with "-c debug" works as expected.

Compiling my project with "-c release" which has dependencies in Package.swift and options like "-Xswiftc -Ounchecked" recompiles in vim (or a make after vim :m).

bq. When you say "everything is recompiled" how are you determining this part?
It takes ages to finish. I double checked with "perf vim", here are the first lines:

7,72% swift swift [.] llvm::DenseMapBase<llvm::DenseMap<(anonymous namespace)::AliasKeyTy, swift::AliasAnalysis::AliasResult, llvm::DenseMapInfo<(anonymous namespace)::AliasKey
1,76% swift swift [.] llvm::DenseMapBase<llvm::SmallDenseMap<swift::LSLocation, unsigned int, 32u, llvm::DenseMapInfoswift::LSLocation, llvm::detail::DenseMapPair<swift::LSLo
1,67% swift libc-2.31.so [.] _int_malloc
1,36% swift swift [.] swift::ProjectionPath::computeSubSeqRelation
1,33% swift swift [.] (anonymous namespace)::SILVerifier::visitSILInstruction
1,26% swift libc-2.31.so [.] _int_free
1,26% swift swift [.] llvm::SmallBitVector::operator[]
1,09% swift swift [.] swift::ValueEnumerator<swift::ValueBase*, unsigned long>::getIndex
1,01% swift swift [.] (anonymous namespace)::AccessMarkerElimination::stripMarkers
0,90% swift libc-2.31.so [.] malloc
0,90% swift swift [.] swift::ProjectionPath::hasNonEmptySymmetricDifference
0,88% swift swift [.] llvm::PMDataManager::findAnalysisPass
0,82% swift swift [.] llvm::FoldingSetNodeID::AddInteger
0,81% swift swift [.] swift::LSBase::hasIdenticalProjectionPath
0,80% swift swift [.] swift::TypeVariableType::Implementation::getRepresentative
0,64% swift swift [.] swift::constraints::ConstraintGraph::lookupNode
0,62% swift swift [.] swift::GenericSignatureBuilder::maybeResolveEquivalenceClass
0,56% swift swift [.] swift::AliasAnalysis::alias
0,55% swift swift [.] llvm::SmallPtrSetImplswift::constraints::Constraint\*::insert

Besides, verbose output swift without recompilation looks like this

lsb_release -r
pkg-config --variable pc_path pkg-config
whereas in or after changing to vim it looks like this

lsb_release -r
/home/gonsolo/downloads/swift-5.2.4-RELEASE-ubuntu20.04/usr/bin/swiftc -L /home/gonsolo/downloads/swift-5.2.4-RELEASE-ubuntu20.04/usr/lib/swift/pm/4_2 -lPackageDescription -Xlinker -rpath -Xlinker /home/gonsolo/downloads/swift-5.2.4-RELEASE-ubuntu20.04/usr/lib/swift/pm/4_2 -swift-version 5 -I /home/gonsolo/downloads/swift-5.2.4-RELEASE-ubuntu20.04/usr/lib/swift/pm/4_2 -package-description-version 5.1.0 /home/gonsolo/work/gonzales/Package.swift -o /tmp/TemporaryDirectory.bCAcO8/gonzales-manifest
/tmp/TemporaryDirectory.bCAcO8/gonzales-manifest -fileno 1
/home/gonsolo/downloads/swift-5.2.4-RELEASE-ubuntu20.04/usr/bin/swiftc -L /home/gonsolo/downloads/swift-5.2.4-RELEASE-ubuntu20.04/usr/lib/swift/pm/4_2 -lPackageDescription -Xlinker -rpath -Xlinker /home/gonsolo/downloads/swift-5.2.4-RELEASE-ubuntu20.04/usr/lib/swift/pm/4_2 -swift-version 4.2 -I /home/gonsolo/downloads/swift-5.2.4-RELEASE-ubuntu20.04/usr/lib/swift/pm/4_2 -package-description-version 4.2.0 /home/gonsolo/work/gonzales/.build/checkouts/png/Package.swift -o /tmp/TemporaryDirectory.arvohb/png-manifest
/tmp/TemporaryDirectory.arvohb/png-manifest -fileno 1
pkg-config --variable pc_path pkg-config
/home/gonsolo/downloads/swift-5.2.4-RELEASE-ubuntu20.04/usr/bin/swiftc -module-name PNG -incremental -emit-dependencies -emit-module -emit-module-path /home/gonsolo/work/gonzales/.build/x86_64-unknown-linux-gnu/release/PNG.swiftmodule -output-file-map /home/gonsolo/work/gonzales/.build/x86_64-unknown-linux-gnu/release/PNG.build/output-file-map.json -parse-as-library -whole-module-optimization -num-threads 4 -c /home/gonsolo/work/gonzales/.build/checkouts/png/sources/png/4.2.swift /home/gonsolo/work/gonzales/.build/checkouts/png/sources/png/benchmarkreference.swift /home/gonsolo/work/gonzales/.build/checkouts/png/sources/png/math.swift /home/gonsolo/work/gonzales/.build/checkouts/png/sources/png/png.swift /home/gonsolo/work/gonzales/.build/checkouts/png/sources/png/z.swift -I /home/gonsolo/work/gonzales/.build/x86_64-unknown-linux-gnu/release -target x86_64-unknown-linux-gnu -swift-version 4.2 -O -g -j4 -DSWIFT_PACKAGE -Xcc -fmodule-map-file=/home/gonsolo/work/gonzales/.build/checkouts/png/sources/zlib/module.modulemap -module-cache-path /home/gonsolo/work/gonzales/.build/x86_64-unknown-linux-gnu/release/ModuleCache -parseable-output -color-diagnostics
/home/gonsolo/downloads/swift-5.2.4-RELEASE-ubu... (lots more of this)

Interesting, I wonder if this reproduces differently on macOS (where I was testing) and linux.

Comment by Andreas Wendleder (JIRA)

I have a medium sized project with one dependency. I have a Makefile which basically does a "swift build". I often type "make|head" to see the first error. Every time the dependency is merged and often the whole project is recompiled. This is very annoying.

In addition, typing "make" (without the pipe) recompiles everything always as does compiling within vim with ":make".

Comment by Andreas Wendleder (JIRA)

Even more annoying: I have a "test" goal in my Makefile which just runs my app and which depends on the build. Even if i just typed "make" to compile and then "make test" to see the output of my app, everything is recompiled!