Fractal Softworks Forum

Please login or register.

Login with username, password and session length

Author Topic: Can't have '#' inside json Strings  (Read 1660 times)

TJJ

  • Admiral
  • *****
  • Posts: 1905
    • View Profile
Can't have '#' inside json Strings
« on: July 02, 2015, 06:52:42 AM »

The way comments are stripped from json files doesn't respect whether the comment character ('#') appears inside quotes.
This makes it impossible to include the '#' character in any text body loaded from a json file.

e.g. "displayName":"Luddic #Church"

Will cause the game to crash on load.

I'd suggest patching json.jar so that its tokenizer treats '#' characters as comments.
That'd both fix this issue and remove the need for the rather inefficient 2-step parsing that reading json files currently involves.
Logged

Alex

  • Administrator
  • Admiral
  • *****
  • Posts: 24111
    • View Profile
Re: Can't have '#' inside json Strings
« Reply #1 on: July 02, 2015, 08:33:19 AM »

IIRC you should be able to escape it with a \. Can you give that a quick try?

Edit: actually, never mind, that doesn't work. Was thinking something else.

Edit2: right, you can use this: \u0023
« Last Edit: July 02, 2015, 08:53:16 AM by Alex »
Logged

TJJ

  • Admiral
  • *****
  • Posts: 1905
    • View Profile
Re: Can't have '#' inside json Strings
« Reply #2 on: July 02, 2015, 09:18:57 AM »

Ah, nice workaround.

Though I wasn't actually having problems with embedding '#' in json Strings, rather I was having problems using the included json.jar API to parse Starsector's data files. :)
I was passing the json straight to the loader, without 1st stripping the comments (as your LoadingUtils class does).

While investigating the cause, I spotted the above mentioned '#' limitation.

Rather than duplicating your 2-stage parsing, I just patched overrode JSONTokenizer.nextClean() thus:
(disclaimer: I've not yet exhaustively tested it; there might be places where placing a '#' still causes a failure)

Code
package org.tjj.starsector.ssme;

import java.io.Reader;

import org.json.JSONException;
import org.json.JSONTokener;

/**
 * Fixes the json tokenizer so it respects '#' comments,
 * so you don't need to strip them out of Starsector's data files before parsing them.
 *
 * @author TehJumpingJawa
 *
 */
public class ImprovedJSONTokener extends JSONTokener {

public ImprovedJSONTokener(Reader arg0) {
super(arg0);
}

public ImprovedJSONTokener(String arg0) {
super(arg0);
}

public char nextClean() throws JSONException {
boolean inComment = false;
for (;;) {
char c = this.next();
if (c == 0) {
// eof
return c;
} else {
if (inComment) {
if (c == '\n') {
inComment = false;
}
} else {
if (c == '#') {
inComment = true;
} else if (c > ' ') {
return c;
}
}
}
}
}
}
« Last Edit: August 14, 2015, 03:48:55 AM by TJJ »
Logged

Alex

  • Administrator
  • Admiral
  • *****
  • Posts: 24111
    • View Profile
Re: Can't have '#' inside json Strings
« Reply #3 on: July 02, 2015, 09:58:51 AM »

Gotcha. Yeah, that looks like it should work. At least at first glance and without knowing exactly how nextClean() is used :)
Logged