tokenizer.rb

Path: lib/scoped_search/query_language/tokenizer.rb
Last Update: Tue Dec 04 07:25:24 +0000 2012

The Tokenizer module adds methods to the query language compiler that transforms a query string into a stream of tokens, which are more appropriate for parsing a query string.

[Validate]