The test and validation-test directories in the Swift project define tests and validation tests. The lit testing tool is used to run the tests. lit allows test suites to be configured via a file named lit.cfg.
Both test/lit.cfg and validation-test/lit.cfg use parameters that are defined as part of the Swift CMake build scripts. The CMake portion of the build system is aware of a lot of information that is useful to the tests, such as the location where LLVM build products will be placed--something like /path/to/build/llvm-macosx-x86_64/bin. CMake communicates these values to lit via the files test/lit.site.cfg.in and validation-test/lit.site.cfg.in. You can see the exact lines that CMake find-and-replaces variables like @LLVM_BINARY_DIR@ here.
I was recently bit by this system: I had added a new variable, @SWIFT_ANDROID_NDK_PATH@, to test/lit.site.cfg.in, but *not* to validation-test/lit.site.cfg.in. I was confused why my tests failed to be configured with the correct path sometimes.
It would be nice if I only had to add variables like @SWIFT_ANDROID_NDK_PATH@ to a single file, as opposed to several.
Accomplishing this requires basic knowledge of how the Swift test suite is run, as well some CMake and Python syntax. This task is a good opportunity to learn more about the Swift build system, its test suite, CMake, and lit.