std::list< std::pair< std::string, std::string > > WordTokenizerResults
Results of the tokenizer; each element is a pair where the first string represents a possible space a...
Definition: wordtokenizer.h:26
static void tokenize(const std::string &s, WordTokenizerResults &results)
Tokenizes the passed string and stores the results.
Definition: wordtokenizer.cpp:27