Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SR-7061] libSyntax built from source contains nulls in simple file definition #49609

Closed
keith opened this issue Feb 22, 2018 · 9 comments
Closed

[SR-7061] libSyntax built from source contains nulls in simple file definition #49609

keith opened this issue Feb 22, 2018 · 9 comments
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. compiler The Swift compiler in itself † libswiftSyntax † Area → compiler: the once-integrated C++ Syntax library succeeded by SwiftSyntax source tooling Area: IDE support, SourceKit, and other source tooling stale Resolution: No longer relevant, stale unexpected behavior Bug: Unexpected behavior or incorrect output

Comments

@keith
Copy link
Collaborator

keith commented Feb 22, 2018 •
edited by AnthonyLatsis

Previous ID SR-7061
Radar None
Original Reporter @keith
Type Bug
Additional Detail from JIRA
Votes 0
Component/s Source Tooling
Labels Bug, Syntax
Assignee None
Priority Medium

md5: 60ecd6763a9bc9f8e4fae102fde25ade

Issue Description:

If you build Swift (as of 0ae61f6) and attempt to fetch the syntax of this file:

import Foundation

Using:

/path/to/swift/build/Ninja-DebugAssert/swift-macosx-x86_64/bin/swiftc -frontend -emit-syntax test.swift

You end up with this result:

{
  "kind": "SourceFile",
  "layout": [
    {
      "kind": "CodeBlockItemList",
      "layout": [
        {
          "kind": "CodeBlockItem",
          "layout": [
            {
              "kind": "ImportDecl",
              "layout": [
                null,
                {
                  "tokenKind": {
                    "kind": "kw_import"
                  },
                  "leadingTrivia": [],
                  "trailingTrivia": [
                    {
                      "kind": "Space",
                      "value": 1
                    }
                  ],
                  "presence": "Present"
                },
                null,
                {
                  "kind": "AccessPath",
                  "layout": [
                    {
                      "kind": "AccessPathComponent",
                      "layout": [
                        {
                          "tokenKind": {
                            "kind": "identifier",
                            "text": "Foundation"
                          },
                          "leadingTrivia": [],
                          "trailingTrivia": [],
                          "presence": "Present"
                        },
                        null
                      ],
                      "presence": "Present"
                    }
                  ],
                  "presence": "Present"
                }
              ],
              "presence": "Present"
            },
            null
          ],
          "presence": "Present"
        }
      ],
      "presence": "Present"
    },
    {
      "tokenKind": {
        "kind": "eof"
      },
      "leadingTrivia": [
        {
          "kind": "Newline",
          "value": 1
        }
      ],
      "trailingTrivia": [],
      "presence": "Present"
    }
  ],
  "presence": "Present"
}

Note the nulls scattered throughout the output. This does seem to not have the same problem as #49608
@belkadan
Copy link
Contributor

cc @nkcsgexi

@nkcsgexi
Copy link
Member

this is not a problem per se since the Json is just internal representation of the source file. We expect users to use typed Swift syntax apis that can be serialized from this format.

Could you clarify what do you try to achieve?

@keith
Copy link
Collaborator Author

keith commented Feb 22, 2018

I guess I was expecting there to not be any nulls here. I'm currently testing out https://github.com/prettier/plugin-swift which uses libSyntax. But if it just needs to be updated to handle nulls that's fine, I was just surprised to see them added.

@nkcsgexi
Copy link
Member

I think we intentionally use "null" here to reduce the serialization size. @rintaro is that right? probably we should rename it to something else to avoid such confusion.

@rintaro
Copy link
Mannequin

rintaro mannequin commented Feb 22, 2018

I intentionally introduced null to represent missing node as a part of performance optimization.
For instance, every IdentifierExpr have optional DeclNameArguments (e.g. (x:y:)) node. Allocating it for every identifier expressions is definitely a waste of time and resource.

Keith (JIRA User), please do not use JSON representation, RawSyntax or SyntaxData directly. They are really implementation detail.

@swift-ci
Copy link
Collaborator

Comment by Orta Therox (JIRA)

Is there a chance you can explain what the supported way would be to get the AST syntax out?

For example, do we need to build our own CLI to link with libSyntax and pull out the AST and export it in our own JSON format?

Ideally we'd want a somewhat reliable source that is available with the default tools like the above example. If it changes between swift versions, that's feasible to handle with enough eyes and hands on the problem.

@keith
Copy link
Collaborator Author

keith commented Jul 25, 2018

@rintaro any thoughts on orta.therox@gmail.com (JIRA User)'s comment?

@rintaro
Copy link
Mannequin

rintaro mannequin commented Jul 27, 2018

orta.therox@gmail.com (JIRA User) Sorry for the late.

For example, do we need to build our own CLI to link with libSyntax and pull out the AST and export it in our own JSON format?

The current primary way is to use SwiftSyntax library. Although it's still work in progress, and it's not included in Xcode toolchain, SwiftSyntax API is currently the only supported way for handling syntax tree.

@nkcsgexi Any comments?

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@AnthonyLatsis
Copy link
Collaborator

libSyntax was removed in #62145 and succeeded by swift-syntax.

@AnthonyLatsis AnthonyLatsis closed this as not planned Won't fix, can't repro, duplicate, stale Apr 5, 2023
@AnthonyLatsis AnthonyLatsis added compiler The Swift compiler in itself stale Resolution: No longer relevant, stale source tooling Area: IDE support, SourceKit, and other source tooling unexpected behavior Bug: Unexpected behavior or incorrect output labels Apr 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. compiler The Swift compiler in itself † libswiftSyntax † Area → compiler: the once-integrated C++ Syntax library succeeded by SwiftSyntax source tooling Area: IDE support, SourceKit, and other source tooling stale Resolution: No longer relevant, stale unexpected behavior Bug: Unexpected behavior or incorrect output
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@keith @belkadan @nkcsgexi @swift-ci @AnthonyLatsis