Tokenize Large String process
The Tokenize Large String process segments a string on a specified delimiter. As compared with the Tokenize String process, the Tokenize Large String process overcomes the limitation of the traditional tokenizer against large inputs by setting up a loop that detects the size of the input and parses it in 10,000-character segments, appending the segments together to create a fully-parsed result.
The following figure displays the Tokenize String process and the Tokenize Large String process in BMC Atrium Orchestrator Development Studio.
Tokenize String process
Tokenize Large String process
The following table describes the process inputs for the Tokenize Large String request process.
Input | Description | Required |
|---|---|---|
string | Specifies the string to be parsed on a specified character | Yes |
delimiter | Specifies the character on which the specified string is parsed | No |
The following table describes the process output parameters for the Tokenize Large String process.
Tokenize String process output
Output | Description |
|---|---|
tokens | Specifies an XML document with a root element, <tokens>, and child elements, <token>, containing the resulting string segments |