How to define a custom separator in TLex
Article Metadata
TLex does not provide support for user defined separators, hence a possible workaround could be using the MarkedToken(), Mark(), Get(), and Peek() methods for achieving the desired results.
// As an example taking "," as a separator.
_LIT8(KSomeConstString, "first, second, third, fourth,");
TLex8 lex(KSomeConstString);
TChar ch;
TBuf8<50> token;
while((ch = lex.Get()) != 0 )
{
while ((ch = lex.Peek()) != ',')
lex.Inc();
token.Copy(lex.MarkedToken());
/* Now we have the string as the token,
* do something.. */
lex.Inc();
lex.Mark();
}
30 Sep
2009
By default TLex uses space as a separator when parsing. However, in many scenarios you might have to use your own custom separator. This article provides a work around for setting custom separator in TLex using the MarkedToken() function.