Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-7109] JSONDecoder fails to pop codingPath entry after exception #4252

Open
swift-ci opened this issue Mar 2, 2018 · 2 comments
Open
Assignees

Comments

@swift-ci
Copy link
Contributor

swift-ci commented Mar 2, 2018

Previous ID SR-7109
Radar None
Original Reporter pdwilson12 (JIRA User)
Type Bug
Additional Detail from JIRA
Votes 0
Component/s Foundation
Labels Bug
Assignee @itaiferber
Priority Medium

md5: cbee189340585f8ddc888fbf34a8951f

Issue Description:

When decoding a structured JSON file in which a particular field can be either an array or a dictionary, I detect which is the case by asking the decoder for the unkeyed container and then if it generates an exception, try the keyed container:

{{ for key in keyedContainer.allKeys {}}
{{ switch key {}}
{{ case .items:}}
{{ var nested = try? keyedContainer.nestedUnkeyedContainer(forKey: key)}}
{{ if nested == nil {}}
{{ let singleItem = try? keyedContainer.decode(JSRefWrapper.self, forKey: key)}}

After this code runs and hits an exception in the first "try?", the data returned from the decoder's codingPath incorrectly lists the "items" element twice.

Looking at the implementation of JSONDecoder, I found this code multiple times in JSONEncoder.swift:

{{ fileprivate func with<T>(pushedKey key: CodingKey, _ work: () throws -> T) rethrows -> T {}}
{{ self.codingPath.append(key)}}
{{ let ret: T = try work()}}
{{ self.codingPath.removeLast()}}
{{ return ret}}
{{ }}}

If the work() block throws an exception (as is done by nestedUnkeyedContainer()), the removeLast() call will not be executed.

I think removeLast should be put into a 'defer' block, which will cause it to be executed even on an exception...

{{ fileprivate func with<T>(pushedKey key: CodingKey, _ work: () throws -> T) rethrows -> T {}}
{{ self.codingPath.append(key)}}
{{ defer { self.codingPath.removeLast()}}
{{ let ret: T = try work()}}
{{ return ret}}
{{ }}}

@swift-ci
Copy link
Contributor Author

swift-ci commented Mar 5, 2018

Comment by Peter Wilson (JIRA)

To attempt to workaround this issue, I reversed the order in which I try to decode the object vs array of objects (the object being the common case), resulting in this updated code:

    for key in keyedContainer.allKeys {

        switch key {

        case .items:

            if let singleItem = try? keyedContainer.decode(JSRefWrapper.self, forKey: key) {

                items = \[singleItem\]

            } else if var nested = try? keyedContainer.nestedUnkeyedContainer(forKey: key) {

                items = \[\]

                while !nested.isAtEnd {

                    items!.append(try nested.decode(JSRefWrapper.self))

                }

            } else {

                throw JSError.Failure("Unable to parse items")

            }

        }

    }

    try super.init(from: decoder)

When it hits the array-of-objects case (ie, exception raised/ignored in first if test) in a unit test parsing this JSON:

            "properties": {

                "feet": {"type": "array", "items": \[ { "type": "number" } \] },

                "toes": {"type": "array", "items": { "type": "number" } },

            },

it crashes during the super.init(from🙂 call (which is the default implementation for an object) as if it is trying to parse the current object as an array, which is probably due to the same state-restoration issue.

fatal error: 'try!' expression unexpectedly raised an error: Swift.DecodingError.typeMismatch(Swift.Dictionary<Swift.String, Any>, Swift.DecodingError.Context(codingPath: [schemas.JSObjectSchema.ObjectCodingKeys.properties, Swift._DictionaryCodingKey(stringValue: "feet", intValue: nil), schemas.JSArraySchema.ArrayCodingKeys.items], debugDescription: "Expected to decode Dictionary<String, Any> but found an array instead.", underlyingError: nil)): file /home/buildnode/jenkins/workspace/oss-swift-4.0-package-linux-ubuntu-14_04/swift/stdlib/public/core/ErrorType.swift, line 181

Current stack trace:

0 libswiftCore.so 0x00007f71d593ce50 _swift_stdlib_reportFatalErrorInFile + 221

1 libswiftCore.so 0x00007f71d56481b0 closure #1 in closure #1 in closure #1 in assertionFailure(:_:file:line:flags🙂 + 284

2 libswiftCore.so 0x00007f71d58e6fd1 <unavailable> + 4149201

3 libswiftCore.so 0x00007f71d58fd689 <unavailable> + 4241033

4 libswiftCore.so 0x00007f71d5647a8b <unavailable> + 1399435

5 libswiftCore.so 0x00007f71d5860059 <unavailable> + 3596377

6 libswiftCore.so 0x00007f71d589de3f <unavailable> + 3849791

7 libswiftCore.so 0x00007f71d5647a8b <unavailable> + 1399435

8 libswiftCore.so 0x00007f71d57e0140 specialized assertionFailure(:_:file:line:flags🙂 + 144

9 libswiftCore.so 0x00007f71d56955d0 swift_unexpectedError + 371

10 openapi_indexerPackageTests.xctest 0x000000000051849d <unavailable> + 1148061

11 openapi_indexerPackageTests.xctest 0x0000000000505484 <unavailable> + 1070212

12 openapi_indexerPackageTests.xctest 0x0000000000504c66 <unavailable> + 1068134

13 openapi_indexerPackageTests.xctest 0x000000000050a56a <unavailable> + 1090922

14 openapi_indexerPackageTests.xctest 0x0000000000509ce4 <unavailable> + 1088740

15 openapi_indexerPackageTests.xctest 0x000000000050b41c <unavailable> + 1094684

16 libFoundation.so 0x00007f71d5148120 JSONDecoder.decode<A>(_:from🙂 + 1185

17 openapi_indexerPackageTests.xctest 0x000000000040f5f0 <unavailable> + 62960

18 openapi_indexerPackageTests.xctest 0x00000000004298b2 <unavailable> + 170162

19 openapi_indexerPackageTests.xctest 0x000000000040a056 <unavailable> + 41046

20 openapi_indexerPackageTests.xctest 0x000000000042983b <unavailable> + 170043

21 libXCTest.so 0x00007f71d5c42c7f <unavailable> + 203903

22 libXCTest.so 0x00007f71d5c40721 <unavailable> + 194337

23 libXCTest.so 0x00007f71d5c4296f <unavailable> + 203119

24 libXCTest.so 0x00007f71d5c428ab <unavailable> + 202923

25 libXCTest.so 0x00007f71d5c44119 <unavailable> + 209177

26 libXCTest.so 0x00007f71d5c435c0 <unavailable> + 206272

27 libXCTest.so 0x00007f71d5c26800 XCTestCase.invokeTest() + 102

28 libXCTest.so 0x00007f71d5c36ea6 <unavailable> + 155302

29 libXCTest.so 0x00007f71d5c26740 XCTestCase.perform(_🙂 + 14

30 libXCTest.so 0x00007f71d5c20f20 XCTest.run() + 662

31 libXCTest.so 0x00007f71d5c3777b <unavailable> + 157563

32 libXCTest.so 0x00007f71d5c376b0 <unavailable> + 157360

33 libXCTest.so 0x00007f71d5c37a84 <unavailable> + 158340

34 libXCTest.so 0x00007f71d5c233b0 XCTMain(_🙂 + 3770

35 openapi_indexerPackageTests.xctest 0x000000000040991a <unavailable> + 39194

36 libc.so.6 0x00007f71d3801e50 __libc_start_main + 245

37 openapi_indexerPackageTests.xctest 0x0000000000409049 <unavailable> + 36937

@swift-ci
Copy link
Contributor Author

Comment by Peter Wilson (JIRA)

I tested the code with Swift 5.3.3 and it no longer generates an error. I don't see the problem code in the Foundation JSON handling code, either. Looks like it has been totally rewritten to eliminate modifying `self.codingPath` directly.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant