[SR-6932] libSyntax: Some trivia are not so trivial #49480
Labels
bug
A deviation from expected or documented behavior. Also: expected but undesirable behavior.
compiler
The Swift compiler in itself
† libswiftSyntax
† Area → compiler: the once-integrated C++ Syntax library succeeded by SwiftSyntax
stale
Resolution: No longer relevant, stale
Additional Detail from JIRA
md5: 99576744d57298235b0f4e2fea0f3023
Issue Description:
We mostly think of trivia as things that could be dropped without changing semantics, but some are crucial. For example, the space in “
let x=1
” prevents tokenization as “letx
,=
,1
.” Also, the newlines between statements, where there are no semicolons, are required. A formatter/pretty-printer that normalizes whitespace needs to be able to find-and-preserve or ensure-it-recreates nontrivial whitespace.The current libSyntax API makes that especially difficult when multiple identifiers and/or keywords appear together, because the rules about what characters can be part of an identifier are complex. One could special-case all the known token combinations, but when the language grows a new token these rules are very likely to break down until the formatter is updated and rebuilt with a new libSyntax. It should be possible to handle this in an efficient way that is very likely to be resilient to future language changes.
The text was updated successfully, but these errors were encountered: