-
Notifications
You must be signed in to change notification settings - Fork 0
io_nats_jparse_parser_functable

The JsonParserFunctions class is a collection of functions that are utilized by the JsonFuncParser to parse JSON.
public static boolean parseKeyNoQuote(final CharSource source, final TokenList tokens) {
int ch = source.nextSkipWhiteSpace();
final int startIndex = source.getIndex() - 1;
final int tokenListIndex = tokens.getIndex();
tokens.placeHolder();
boolean found = false;
switch(ch) {
case ParseConstants.STRING_START_TOKEN:
final int strStartIndex = startIndex + 1;
final int strEndIndex = source.findEndOfEncodedString();
tokens.add(new Token(strStartIndex + 1, strEndIndex, TokenTypes.STRING_TOKEN));
found = true;
break;
case ParseConstants.OBJECT_END_TOKEN:
tokens.undoPlaceholder();
return true;
default:
if (Character.isAlphabetic(ch)) {
final int start = source.getIndex();
final int end = source.findAttributeEnd();
tokens.add(new Token(start, end, TokenTypes.STRING_TOKEN));
found = true;
} else {
throw new UnexpectedCharacterException("Parsing key", "Unexpected character found", source);
}
}
boolean done = source.findObjectEndOrAttributeSep();
if (!done && found) {
tokens.set(tokenListIndex, new Token(startIndex + 1, source.getIndex(), TokenTypes.ATTRIBUTE_KEY_TOKEN));
} else if (found && done) {
throw new UnexpectedCharacterException("Parsing key", "Not found", source);
}
return done;
}The method parseKeyNoQuote in the JsonParserFunctions class takes two parameters: source of type CharSource and tokens of type TokenList. It returns a boolean value.
Here is a step-by-step description of what the method does based on its body:
- It retrieves the next character from the
sourceand skips any leading white spaces. - It saves the current index of the
sourceminus 1 asstartIndex. - It saves the current index of the
tokensastokenListIndex. - It adds a placeholder token to the
tokenslist. - It initializes a boolean variable
foundas false. - It enters a switch statement based on the value of the character
ch.- If
chis equal toParseConstants.STRING_START_TOKEN:- It calculates the start index of the string as
strStartIndexby adding 1 tostartIndex. - It calculates the end index of the string as
strEndIndexby calling thefindEndOfEncodedStringmethod on thesourceobject. - It adds a new
Tokenobject to thetokenslist with the start index, end index, and typeTokenTypes.STRING_TOKEN. - It sets
foundto true.
- It calculates the start index of the string as
- If
chis equal toParseConstants.OBJECT_END_TOKEN:- It undoes the placeholder token added to the
tokenslist. - It returns true.
- It undoes the placeholder token added to the
- If
chis neitherParseConstants.STRING_START_TOKENnorParseConstants.OBJECT_END_TOKEN:- If
chis an alphabetic character:- It saves the current index of the
sourceasstart. - It finds the end index of the attribute by calling the
findAttributeEndmethod on thesourceobject and saves it asend. - It adds a new
Tokenobject to thetokenslist with the start index, end index, and typeTokenTypes.STRING_TOKEN. - It sets
foundto true.
- It saves the current index of the
- If
chis not an alphabetic character:- It throws an
UnexpectedCharacterExceptionwith the message "Unexpected character found" and thesourceobject as arguments.
- It throws an
- If
- If
- It checks if it needs to find the end of the object or the attribute separator by calling the
findObjectEndOrAttributeSepmethod on thesourceobject. The result is saved in thedonevariable. - If
doneis false andfoundis true:- It updates the placeholder token in the
tokenslist with the start index asstartIndex + 1and the current index of thesourceas the end index, and the typeTokenTypes.ATTRIBUTE_KEY_TOKEN.
- It updates the placeholder token in the
- If
foundis true anddoneis true:- It throws an
UnexpectedCharacterExceptionwith the message "Not found" and thesourceobject as arguments.
- It throws an
- It returns the value of
done.
public static boolean parseKeyWithEncode(final CharSource source, final TokenList tokens) {
int ch = source.nextSkipWhiteSpace();
final int startIndex = source.getIndex() - 1;
final int tokenListIndex = tokens.getIndex();
tokens.placeHolder();
boolean found = false;
switch(ch) {
case ParseConstants.STRING_START_TOKEN:
final int strStartIndex = startIndex + 1;
final int strEndIndex = source.findEndOfEncodedString();
tokens.add(new Token(strStartIndex + 1, strEndIndex, TokenTypes.STRING_TOKEN));
found = true;
break;
case ParseConstants.OBJECT_END_TOKEN:
tokens.undoPlaceholder();
return true;
default:
throw new UnexpectedCharacterException("Parsing key", "Unexpected character found", source);
}
boolean done = source.findObjectEndOrAttributeSep();
if (!done && found) {
tokens.set(tokenListIndex, new Token(startIndex + 1, source.getIndex(), TokenTypes.ATTRIBUTE_KEY_TOKEN));
} else if (found && done) {
throw new UnexpectedCharacterException("Parsing key", "Not found", source);
}
return done;
}The parseKeyWithEncode method is defined in the io.nats.jparse.parser.functable.JsonParserFunctions class. It takes two parameters: source of type CharSource and tokens of type TokenList. The method has a return type of boolean.
The purpose of this method is to parse a key while encoding it, given a character source (source) and a list of tokens (tokens).
Below is a step-by-step description of what the method does:
-
The method starts by retrieving the next character from the
source, skipping any white space characters. -
It initializes two variables:
startIndexto hold the index of the character in thesourceminus 1 andtokenListIndexto hold the current index of thetokenslist. -
A placeholder token is added to the
tokenslist. -
The method enters a
switchstatement based on the value of the retrieved character. -
If the retrieved character is equal to the constant
ParseConstants.STRING_START_TOKEN, the method proceeds to parse a string. It initializes two variables:strStartIndexto hold the index of the start of the string plus 1 andstrEndIndexto hold the index of the end of the encoded string. A newTokenobject is created with the start and end indexes, and a token type ofTokenTypes.STRING_TOKEN. The newly created token is added to thetokenslist. Thefoundvariable is set totrue. -
If the retrieved character is equal to the constant
ParseConstants.OBJECT_END_TOKEN, the method undoes the placeholder token and returnstrue. -
If none of the above cases match, an
UnexpectedCharacterExceptionis thrown, indicating that an unexpected character was found while parsing the key. -
The method then tries to find the end of the object or the attribute separator. The
donevariable is set totrueif the end is found. -
If
doneisfalseandfoundistrue, the method updates the token at thetokenListIndexin thetokenslist with a newTokenobject that has the updated start and end indexes, and a token type ofTokenTypes.ATTRIBUTE_KEY_TOKEN. -
If
foundistrueanddoneistrue, anUnexpectedCharacterExceptionis thrown, indicating that the key was not found. -
Finally, the method returns the value of the
donevariable.
This method is used to parse a key while encoding it, and it handles different cases based on the character read from the source. If the key is successfully parsed, the method updates the tokens list accordingly. However, if an unexpected character is encountered or the key is not found, an exception is thrown.

public static boolean parseKeyNoEncode(final CharSource source, final TokenList tokens) {
int ch = source.nextSkipWhiteSpace();
final int startIndex = source.getIndex() - 1;
final int tokenListIndex = tokens.getIndex();
tokens.placeHolder();
boolean found = false;
switch(ch) {
case ParseConstants.STRING_START_TOKEN:
final int strStartIndex = startIndex + 1;
final int strEndIndex = source.findEndString();
tokens.add(new Token(strStartIndex + 1, strEndIndex, TokenTypes.STRING_TOKEN));
found = true;
break;
case ParseConstants.OBJECT_END_TOKEN:
tokens.undoPlaceholder();
return true;
default:
throw new UnexpectedCharacterException("Parsing key", "Unexpected character found", source);
}
boolean done = source.findObjectEndOrAttributeSep();
if (!done && found) {
tokens.set(tokenListIndex, new Token(startIndex + 1, source.getIndex(), TokenTypes.ATTRIBUTE_KEY_TOKEN));
} else if (found && done) {
throw new UnexpectedCharacterException("Parsing key", "Not found", source);
}
return done;
}
The JsonFuncParser class is an implementation of the JsonParser interface. It uses a function table to define the behavior of the parser. The function table, which is an array of ParseFunction objects, determines which function to call when parsing a specific character. The function table can be customized by subclassing this class and overriding the initFuncTable method. Additionally, the function table can be configured using the JsonParserBuilder.
private void doParse(final CharSource source, final TokenList tokens, final int ch) {
if (ch < 256) {
funcTable[ch].parse(source, tokens);
} else {
defaultFunc.parse(source, tokens);
}
}The method doParse defined in class io.nats.jparse.parser.functable.JsonFuncParser is a private method with the following step-by-step description:
- Accepts three parameters:
sourceof typeCharSource,tokensof typeTokenList, andchof typeint. - Checks if the value of
chis less than 256. - If
chis less than 256: a. Retrieves the appropriate parser from thefuncTablearray at indexch. b. Invokes theparsemethod on the retrieved parser, passingsourceandtokensas arguments. - If
chis not less than 256: a. Invokes theparsemethod on thedefaultFunc, passingsourceandtokensas arguments.
This method essentially determines the appropriate parser to use based on the value of ch and delegates the parsing task to the selected parser using the parse method.

private void parseArray(final CharSource source, final TokenList tokens) {
final int startSourceIndex = source.getIndex();
final int tokenListIndex = tokens.getIndex();
tokens.placeHolder();
boolean done = false;
while (!done) {
done = parseArrayItem(source, tokens);
}
final Token arrayToken = new Token(startSourceIndex, source.getIndex(), TokenTypes.ARRAY_TOKEN);
tokens.set(tokenListIndex, arrayToken);
}The method parseArray is defined in the class io.nats.jparse.parser.functable.JsonFuncParser. It takes two parameters: source of type CharSource and tokens of type TokenList.
Here is a step-by-step description of what the method does:
- It stores the current index of the source in a variable called
startSourceIndex. - It also stores the current index of the
tokensin a variable calledtokenListIndex. - It inserts a placeholder token in the
tokenslist using theplaceHolder()method. - It initializes a boolean variable called
doneto false. - It enters a while loop that continues until
donebecomes true. - Inside the while loop, it calls the
parseArrayItem()method passingsourceandtokensas parameters. TheparseArrayItem()method is responsible for parsing each item in the array. - After
parseArrayItem()returns, it checks the value ofdoneand updates it accordingly. - Once all the items in the array have been parsed, it creates a new
Tokenobject calledarrayTokenusing thestartSourceIndex, current index of thesource, and the typeTokenTypes.ARRAY_TOKEN. - Finally, it updates the token at
tokenListIndexin thetokenslist with the newly createdarrayToken.
That's the step-by-step description of what the parseArray method does based on its body.

private boolean parseArrayItem(CharSource source, TokenList tokens) {
char ch = (char) source.nextSkipWhiteSpace();
forLoop: for (; ch != ETX; ch = (char) source.nextSkipWhiteSpace()) {
switch(ch) {
case ARRAY_END_TOKEN:
source.next();
return true;
case ARRAY_SEP:
source.next();
return false;
default:
doParse(source, tokens, ch);
break forLoop;
}
}
if (source.getCurrentChar() == ARRAY_END_TOKEN) {
source.next();
return true;
}
return false;
}The parseArrayItem method is defined in the JsonFuncParser class, which is located in the package io.nats.jparse.parser.functable.
This method takes two parameters:
-
sourceof typeCharSource, which represents the source of characters to parse. -
tokensof typeTokenList, which represents the list of tokens that have been parsed so far.
Here is a step-by-step description of what the parseArrayItem method does based on its body:
- Retrieve the next character from the
sourceand assign it to the variablech. Skip any whitespace characters. - Start a
forloop that iterates until the characterETX(end of text) is encountered. - Within the
forloop, check the value ofchusing aswitchstatement. - If
chis equal toARRAY_END_TOKEN(representing the end of the array), move the source to the next character and returntrueto indicate that the array item has been successfully parsed. - If
chis equal toARRAY_SEP(representing the separator between array items), move the source to the next character and returnfalseto indicate that more array items need to be parsed. - If none of the above cases match (i.e.,
chis any other character), call thedoParsemethod to further parse the item using the current characterch. Then, break out of theforloop to stop parsing the current array item and move on to the next one. - After the
forloop ends, check if the current character in thesourceis equal toARRAY_END_TOKEN. - If it is, move the source to the next character and return
trueto indicate that the array item has been successfully parsed. - If the current character in the
sourceis not equal toARRAY_END_TOKEN, returnfalseto indicate that the array item has not been fully parsed yet.
That's the step-by-step description of what the parseArrayItem method does based on its body.

private boolean parseValue(final CharSource source, TokenList tokens) {
int ch = source.nextSkipWhiteSpace();
final int startIndex = source.getIndex();
final int tokenListIndex = tokens.getIndex();
tokens.placeHolder();
doParse(source, tokens, ch);
ch = source.skipWhiteSpace();
switch(ch) {
case OBJECT_END_TOKEN:
if (source.getIndex() == tokenListIndex) {
throw new UnexpectedCharacterException("Parsing Value", "Key separator before value", source);
}
tokens.set(tokenListIndex, new Token(startIndex, source.getIndex(), TokenTypes.ATTRIBUTE_VALUE_TOKEN));
return true;
case OBJECT_ATTRIBUTE_SEP:
if (source.getIndex() == tokenListIndex) {
throw new UnexpectedCharacterException("Parsing Value", "Key separator before value", source);
}
tokens.set(tokenListIndex, new Token(startIndex, source.getIndex(), TokenTypes.ATTRIBUTE_VALUE_TOKEN));
return false;
default:
throw new UnexpectedCharacterException("Parsing Value", "Unexpected character", source, source.getCurrentChar());
}
}The parseValue method in the JsonFuncParser class is used to parse a JSON value from a given source and add it to a list of tokens. Here is a step-by-step description of the method's functionality:
- Get the next character from the source, skipping any white spaces.
- Store the current index of the source as the starting index of the value.
- Store the current index of the tokens list.
- Create a placeholder token in the tokens list to represent the parsed value. This placeholder will be replaced later with the actual value token.
- Call the
doParsemethod to parse the value and update the tokens list. - Get the next character from the source after parsing the value, skipping any white spaces.
- Check the value of the next character using a switch statement.
- If the next character is an object end token, check that the source index is equal to the token list index. If it is not, throw an
UnexpectedCharacterExceptionindicating that there was a key separator before the value. Otherwise, update the placeholder token in the token list with the correct start and end indexes and set its type asATTRIBUTE_VALUE_TOKEN. Finally, returntrueto indicate that the parsing is successful. - If the next character is an object attribute separator, perform the same check as in step 8. If the check fails, throw an
UnexpectedCharacterExceptionindicating that there was a key separator before the value. Otherwise, update the placeholder token in the token list with the correct start and end indexes and set its type asATTRIBUTE_VALUE_TOKEN. Finally, returnfalseto indicate that the parsing is not yet complete. - If none of the above cases match, throw an
UnexpectedCharacterExceptionindicating that there is an unexpected character in the value, along with the current character from the source.
This method is responsible for parsing a JSON value from a source and adding it to the tokens list, while also handling different scenarios such as object end token and object attribute separator.

private void parseObject(final CharSource source, TokenList tokens) {
final int startSourceIndex = source.getIndex();
final int tokenListIndex = tokens.getIndex();
tokens.placeHolder();
boolean done = false;
while (!done) {
done = parseKey.parse(source, tokens);
if (!done)
done = parseValue(source, tokens);
}
source.next();
tokens.set(tokenListIndex, new Token(startSourceIndex, source.getIndex(), TokenTypes.OBJECT_TOKEN));
}The parseObject() method in the class io.nats.jparse.parser.functable.JsonFuncParser is responsible for parsing a JSON object. Here is a step-by-step description of what this method is doing based on its body:
-
The method begins by creating two local variables:
startSourceIndexandtokenListIndex. These variables are assigned the current index of the source and tokens. -
The
tokens.placeHolder()method is called to create a placeholder token in thetokenslist. -
A boolean variable named
doneis declared and initialized asfalse. This variable is used to indicate whether the parsing of the object is complete. -
A while loop is started with the condition
!done. This loop continues until the parsing of the object is complete. -
The
parseKey.parse(source, tokens)method is called to parse a key from the source. If the parsing is successful and returnstrue, the value ofdoneis set totrue, indicating that the parsing of the object is complete. If the parsing of the key is not successful, the methodparseValue(source, tokens)is called to parse a value. -
Once the parsing of the object is complete, the
source.next()method is called to move the index of the source to the next character. -
The
tokens.set(tokenListIndex, new Token(startSourceIndex, source.getIndex(), TokenTypes.OBJECT_TOKEN))method is called to set the token attokenListIndexto a newTokenobject representing the parsed JSON object. ThestartSourceIndexis the starting index of the object, and the current index of the source is the ending index of the object. TheTokenTypes.OBJECT_TOKENindicates that the token represents an object.
In summary, the parseObject() method parses a JSON object by repeatedly calling methods to parse keys and values until the entire object is parsed. It then updates the tokens list with a new token representing the parsed object.

- Java Open AI Client
- Using ChatGpt embeddings and hyde to improve search results
- Anthropics Claude Chatbot Gets Upgrade
- Elon Musks XAi new frontier for artificial intelligence
- Using Mockito to test JAI Java Open AI Client
- Fine tuning journey with Open AI API
- Using Open AI to create callback functions, the basis for plugins
- Using Java Open AI Client Async
- Fastest Java JSON Parser
- Java Open AI API Client on Github
- Medium: Introducing Java Open AI Client
- Medium: Using ChatGPT, Embeddings, and HyDE to Improve Search Results