Classes | Namespaces
tokenizer.h File Reference
#include <zorba/config.h>
#include <zorba/locale.h>
#include <zorba/internal/unique_ptr.h>
#include <zorba/internal/ztd.h>
Include dependency graph for tokenizer.h:

Go to the source code of this file.

Classes

class  zorba::Tokenizer::Callback
 A Callback is called once per token. More...
struct  zorba::Tokenizer::Numbers
 A Numbers contains the current token, sentence, and paragraph numbers. More...
class  zorba::Tokenizer
 A Tokenizer breaks a string into a stream of word tokens. More...
class  zorba::TokenizerProvider
 A TokenizerProvider provides a Tokenizer for a given language. More...

Namespaces

namespace  zorba
 

Copyright 2006-2011 The FLWOR Foundation.


blog comments powered by Disqus