lucene::analysis::WhitespaceTokenizer Class Reference
A WhitespaceTokenizer is a tokenizer that divides text at whitespace. More...
#include <Analyzers.h>
Inheritance diagram for lucene::analysis::WhitespaceTokenizer:
Public Member Functions | |
| WhitespaceTokenizer (lucene::util::Reader *in) | |
| Construct a new WhitespaceTokenizer. | |
| ~WhitespaceTokenizer () | |
Protected Member Functions | |
| bool | isTokenChar (const TCHAR c) const |
| Collects only characters which do not satisfy _istspace. | |
Detailed Description
A WhitespaceTokenizer is a tokenizer that divides text at whitespace.
Adjacent sequences of non-Whitespace characters form tokens.
Constructor & Destructor Documentation
| lucene::analysis::WhitespaceTokenizer::WhitespaceTokenizer | ( | lucene::util::Reader * | in | ) | [inline] |
Construct a new WhitespaceTokenizer.
| lucene::analysis::WhitespaceTokenizer::~WhitespaceTokenizer | ( | ) | [inline] |
Member Function Documentation
| bool lucene::analysis::WhitespaceTokenizer::isTokenChar | ( | const TCHAR | c | ) | const [protected, virtual] |
Collects only characters which do not satisfy _istspace.
Implements lucene::analysis::CharTokenizer.
The documentation for this class was generated from the following file: