presage  0.8.7
Public Member Functions
ReverseTokenizer Class Reference

#include <reverseTokenizer.h>

Inheritance diagram for ReverseTokenizer:
Inheritance graph
[legend]
Collaboration diagram for ReverseTokenizer:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 ReverseTokenizer (std::istream &stream, const std::string blanks, const std::string separs)
virtual ~ReverseTokenizer ()
virtual int countTokens ()
virtual bool hasMoreTokens () const
virtual std::string nextToken ()
virtual double progress () const

Detailed Description

Tokenizes a stream from the end to the beginning. _ _ _ _ _ _ _ |_|_|_|_|_|_|_| | | | | | offend | offset offbeg

Definition at line 38 of file reverseTokenizer.h.


Constructor & Destructor Documentation

ReverseTokenizer::ReverseTokenizer ( std::istream &  stream,
const std::string  blanks,
const std::string  separs 
)

Definition at line 27 of file reverseTokenizer.cpp.

References Tokenizer::offend, and Tokenizer::offset.

Definition at line 44 of file reverseTokenizer.cpp.


Member Function Documentation

int ReverseTokenizer::countTokens ( ) [virtual]

Returns the number of tokens left.

Implements Tokenizer.

Definition at line 47 of file reverseTokenizer.cpp.

References hasMoreTokens(), nextToken(), Tokenizer::offend, Tokenizer::offset, and Tokenizer::stream.

Here is the call graph for this function:

bool ReverseTokenizer::hasMoreTokens ( ) const [virtual]

Tests if there are more tokens.

Implements Tokenizer.

Definition at line 69 of file reverseTokenizer.cpp.

References Tokenizer::offbeg, and Tokenizer::offset.

Referenced by countTokens().

Here is the caller graph for this function:

std::string ReverseTokenizer::nextToken ( ) [virtual]

Returns the next token.

Implements Tokenizer.

Definition at line 80 of file reverseTokenizer.cpp.

References Tokenizer::isBlankspace(), Tokenizer::isSeparator(), Tokenizer::lowercaseMode(), Tokenizer::offbeg, Tokenizer::offend, Tokenizer::offset, and Tokenizer::stream.

Referenced by countTokens(), and ContextTracker::update().

Here is the call graph for this function:

Here is the caller graph for this function:

double ReverseTokenizer::progress ( ) const [virtual]

Returns progress percentage.

Implements Tokenizer.

Definition at line 174 of file reverseTokenizer.cpp.

References Tokenizer::offbeg, Tokenizer::offend, and Tokenizer::offset.


The documentation for this class was generated from the following files: