cc @phausler

This is not a swift failure:

#import <Foundation/Foundation.h>

int main(int argc, const char * argv[]) {
    @autoreleasepool {
        // insert code here...
        assert([NSCharacterSet.URLHostAllowedCharacterSet characterIsMember: ':']);
//        CharacterSet.urlHostAllowed.contains(":") // true
        NSString *res = [@":" stringByAddingPercentEncodingWithAllowedCharacters:NSCharacterSet.URLHostAllowedCharacterSet];
        assert([res isEqualToString:@"%3A"]);
//        ":".addingPercentEncoding(withAllowedCharacters: CharacterSet.urlHostAllowed) // "%3A"
//        // actual result: "%3A"
//        // expected result: ":"
        
        NSString *res2 = [@":" stringByAddingPercentEncodingWithAllowedCharacters:[NSCharacterSet characterSetWithCharactersInString:@":"]];
        assert([res2 isEqualToString:@":"]);
//        ":".addingPercentEncoding(withAllowedCharacters: CharacterSet(charactersIn: ":")) // ":"
//        // actual result: ":"
//        // expected result: ":"
    }
    return 0;
}

My guess is that URL hosts are defined as the part after the slashes and before a port

http://host:port

So that would mean that the host having a ':' character would mean that it would early encode to a breaking character for a port. So perhaps ':' should not be allowed in a host? Not certain there...

Comment by Marc Knaup (JIRA)

IPv6 addresses contain ":" so it's legit and shouldn't be encoded.

It used to work as expected in iOS 10. Good catch that ObjC is also affected. Should I file a radar for this instead?

yep radar is the way to get this fixed

Comment by Marc Knaup (JIRA)

Filed as rdar://33216827

Ok, I followed up on the radar and this is the response according to the current maintainer of NSURL/CFURLRef (long short is that it behaves correctly):

String.addingPercentEncoding has to special case the single characters sets, URLHostAllowedCharacterSet and URLPathAllowedCharacterSet. (If you make copies of the character sets URLHostAllowedCharacterSet or URLPathAllowedCharacterSet, you'll get the behavior you expected in the bug description.)

URLHostAllowedCharacterSet has to be special-cased because the host component of a URL can be an IP-literal (e.g. an IPv6 address), an IPv4address, or a reg-name (registered name). The form IPv4address requires everything to be decimal digits or periods, so nothing will be percent-encoded. reg-names can be any character and invalid characters will be percent-encoded. IP-literals always start with a '[' and end with a ']'. So, when URLHostAllowedCharacterSet is passed, String.addingPercentEncoding will percent-encode ':' characters UNLESS the string is identified as a IP-literal.

URLPathAllowedCharacterSet has to be special-cased for two reasons:
(1) String.addingPercentEncoding when passed URLHostAllowedCharacterSet percent-encodes ';' characters (even though they are legal in the path) for backwards compatibility with CFURL and NSURL where there was a deprecated parameters URL component.
(2) String.addingPercentEncoding when passed URLHostAllowedCharacterSet percent-encodes any ':' characters found before the first '/' character in the string to ensure the path will work when the URL has no scheme.

Here's some code to illustrate:

https://gist.github.com/phausler/e945f1b9c6486ff9a990487261fdeab4

Comment by Marc Knaup (JIRA)

wow, that's quite unexpected and should be documented. Esp. since the behavior has changed since the last iOS version.

Thanks for the insight and examples. That's really helpful to understand what's going on and it's implications when using/modifying CharacterSet.urlHostAllowed.