Fractal Softworks Forum

Starsector => Bug Reports & Support => Topic started by: TJJ on July 02, 2015, 06:52:42 AM

Title: Can't have '#' inside json Strings
Post by: TJJ on July 02, 2015, 06:52:42 AM
The way comments are stripped from json files doesn't respect whether the comment character ('#') appears inside quotes.
This makes it impossible to include the '#' character in any text body loaded from a json file.

e.g. "displayName":"Luddic #Church"

Will cause the game to crash on load.

I'd suggest patching json.jar so that its tokenizer treats '#' characters as comments.
That'd both fix this issue and remove the need for the rather inefficient 2-step parsing that reading json files currently involves.
Title: Re: Can't have '#' inside json Strings
Post by: Alex on July 02, 2015, 08:33:19 AM
IIRC you should be able to escape it with a \. Can you give that a quick try?

Edit: actually, never mind, that doesn't work. Was thinking something else.

Edit2: right, you can use this: \u0023
Title: Re: Can't have '#' inside json Strings
Post by: TJJ on July 02, 2015, 09:18:57 AM
Ah, nice workaround.

Though I wasn't actually having problems with embedding '#' in json Strings, rather I was having problems using the included json.jar API to parse Starsector's data files. :)
I was passing the json straight to the loader, without 1st stripping the comments (as your LoadingUtils class does).

While investigating the cause, I spotted the above mentioned '#' limitation.

Rather than duplicating your 2-stage parsing, I just patched overrode JSONTokenizer.nextClean() thus:
(disclaimer: I've not yet exhaustively tested it; there might be places where placing a '#' still causes a failure)

Code
package org.tjj.starsector.ssme;

import java.io.Reader;

import org.json.JSONException;
import org.json.JSONTokener;

/**
 * Fixes the json tokenizer so it respects '#' comments,
 * so you don't need to strip them out of Starsector's data files before parsing them.
 *
 * @author TehJumpingJawa
 *
 */
public class ImprovedJSONTokener extends JSONTokener {

public ImprovedJSONTokener(Reader arg0) {
super(arg0);
}

public ImprovedJSONTokener(String arg0) {
super(arg0);
}

public char nextClean() throws JSONException {
boolean inComment = false;
for (;;) {
char c = this.next();
if (c == 0) {
// eof
return c;
} else {
if (inComment) {
if (c == '\n') {
inComment = false;
}
} else {
if (c == '#') {
inComment = true;
} else if (c > ' ') {
return c;
}
}
}
}
}
}
Title: Re: Can't have '#' inside json Strings
Post by: Alex on July 02, 2015, 09:58:51 AM
Gotcha. Yeah, that looks like it should work. At least at first glance and without knowing exactly how nextClean() is used :)