Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-6631] Swift JSONEncoder is dependent to the locale decimal point policy encoding #4487

Open
swift-ci opened this issue Dec 18, 2017 · 5 comments

Comments

@swift-ci
Copy link
Contributor

Previous ID SR-6631
Radar None
Original Reporter b (JIRA User)
Type Bug

Attachment: Download

Environment

The bug is confirmed on Darwin (but should affect Linux too).

Additional Detail from JIRA
Votes 1
Component/s Foundation
Labels Bug
Assignee None
Priority Medium

md5: 3eaf41e50aaf89f052741cedf5637529

is duplicated by:

  • SR-6642 JSONEncoder on Linux should not produce incorrect JSON when setlocale() is invoked

Issue Description:

When using a locale that does not use "." as decimal separator, the JSONEncoder & JSONSerialization encodes the Double value with the localized decimal separator.

The bug is easy to reproduce sample code :

import Foundation

// Required to call setLocale
import Darwin

// Let's set to french
setlocale(LC_ALL,"fr_FR")
 
struct Shot:Codable{
    let seconds:Double
}
 
let shot = Shot(seconds: 1.1)
 
do{
    let data = try JSONEncoder().encode(shot)
    if let json =  String(data:data, encoding:.utf8){
        // the result is : "{"seconds":1,1000000000000001}"
        // should be : "{"seconds":1.1000000000000001}"
        // The decimal separator should not be "," but "."
        print(json)
    }
}catch{
    print("\(error)")
}
 
exit(EX_OK)
 

Here is my proposed fix in `JSONSerialization.swift` line 308 . Specifying `kCFNumberFormatterDecimalSeparator` is required. `

    private lazy var _numberformatter: CFNumberFormatter = {
        let formatter: CFNumberFormatter
        formatter = CFNumberFormatterCreate(nil, CFLocaleCopyCurrent(), kCFNumberFormatterNoStyle)
        CFNumberFormatterSetProperty(formatter, kCFNumberFormatterMaxFractionDigits, NSNumber(value: 15))
       CFNumberFormatterSetProperty(formatter, kCFNumberFormatterDecimalSeparator,"."._cfObject)
        CFNumberFormatterSetFormat(formatter, "0.###############"._cfObject)
        return formatter
    }()

Gonna try to submit this fix.

@swift-ci
Copy link
Contributor Author

Comment by Benoit Pereira da Silva (JIRA)

The problem occurs when calling JSONSerialization.data(withJSONObject: dictionary)

//  main.swift
//  doubleEncoding
//
//  Created by Benoit Pereira da silva on 02/12/2017.
//  Copyright © 2017 Pereira da Silva. All rights reserved.
//

import Foundation

// Let's set to french
setlocale(LC_ALL,"fr_FR")

do{
    let dictionary = ["decimalValue":1.1]
    let data = try JSONSerialization.data(withJSONObject: dictionary)
    if let json =  String(data:data, encoding:.utf8){
        // the result is : "{"decimalValue":1,1000000000000001}"
        // The decimal separator should not be "," but "."
        print(json)
    }
}catch{
    print("\(error)")
}

@swift-ci
Copy link
Contributor Author

Comment by Benoit Pereira da Silva (JIRA)

BUT The problem does not occur within swift-corelibs-foundation test when adding the following test `swift-corelibs-foundation/testJSONSerialization.swift`

 func test_serializeFloatingPointWithANonDotDecimalSeparatorLocale() {
        // We use a locale that is using "," as decimal separator.
        setlocale(LC_ALL,"fr_FR")

        let dictionary = ["decimalValue":1.1]
        do{
            let data = try JSONSerialization.data(withJSONObject: dictionary)
            if let json =  String(data:data, encoding:.utf8){
                XCTAssert(!json.contains(","), "Expected result: \"{\"decimalValue\":1.1000000000000001}\" found: \"\(json)\"" )
            }else{
                XCTFail("Failed during string encoding")
            }
        }catch{
            XCTFail("Failed during serialization")
        }
    }

@gparker42
Copy link
Mannequin

gparker42 mannequin commented Dec 19, 2017

That CFLocaleCopyCurrent() looks suspicious. I would expect the number formatter used by JSON parsing to start with an explicit parse-friendly locale such as CFLocalCreate(nil, "en_US_POSIX").

@swift-ci
Copy link
Contributor Author

Comment by Benoit Pereira da Silva (JIRA)

Greg

1- This problem seems not to be in swift-corelibs-foundation I ve been adding a test in `swift-corelibs-foundation/testJSONSerialization.swift` that is not failing, while my CLI sample fails.

Philippe Hausler has written on swift-corelibs-dev@swift.org : " Building a CLI tool will use the objective-c Foundation in the system so that wont use your freshly built swift-corelibs-foundation. So that means we have a bug in the objc side if this is actually happening on Darwin (which is a completely different issue…) That should be a radar against Foundation and I definitely think that may very well be a bug… "

2- That's why i've been opening a radar https://bugreport.apple.com/web/?problemID=36107307

@swift-ci
Copy link
Contributor Author

Comment by Benoit Pereira da Silva (JIRA)

Philippe Hausler emitted the hypothesis that the bug https://bugs.swift.org/browse/SR-6631 was a bug in Darwin Objc Foundations. He was right!

The following Objc Implementation is faulting:

 #import <Foundation/Foundation.h>

int main(int argc, const char * argv[]) {
    @autoreleasepool {
        // Let's set to french
        setlocale(LC_ALL,"fr_FR");
        NSDictionary *dictionary = @{@"decimalValue":@1.1};
        NSError* error = nil;
        NSData* data = [NSJSONSerialization dataWithJSONObject:dictionary options:kNilOptions error:&error];
        NSString * json = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
        NSLog(@"%@",json);
    }
    return 0;
}

Result :

2017-12-19 09:08:49.657316+0100 doubleEncodingObjc[2016:82301]{"decimalValue":1,1000000000000001}

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant