Package | Description |
---|---|
org.simmetrics | |
org.simmetrics.tokenizers | |
org.simmetrics.utils |
Modifier and Type | Method and Description |
---|---|
static StringMetric |
StringMetrics.createForListMetric(Metric<List<String>> metric,
Simplifier simplifier,
Tokenizer tokenizer)
Creates a new composite string metric.The tokenizer is used to tokenize
the simplified strings.
|
static StringMetric |
StringMetrics.createForListMetric(Metric<List<String>> metric,
Tokenizer tokenizer)
Creates a new composite string metric.
|
static StringMetric |
StringMetrics.createForSetMetric(Metric<Set<String>> metric,
Simplifier simplifier,
Tokenizer tokenizer)
Creates a new composite string metric.The tokenizer is used to tokenize
the simplified strings.
|
static StringMetric |
StringMetrics.createForSetMetric(Metric<Set<String>> metric,
Tokenizer tokenizer)
Creates a new composite string metric.
|
StringMetricBuilder.CollectionMetricTokenizerStep<T> |
StringMetricBuilder.CollectionMetricInitialSimplifierStep.tokenize(Tokenizer tokenizer)
Adds a tokenization step to the metric.
|
StringMetricBuilder.CollectionMetricTokenizerStep<T> |
StringMetricBuilder.CollectionMetricSimplifierStep.tokenize(Tokenizer tokenizer)
Adds a tokenization step to the metric.
|
StringMetricBuilder.CollectionMetricTokenizerStep<T> |
StringMetricBuilder.CollectionMetricInitialTokenizerStep.tokenize(Tokenizer tokenizer)
Adds a tokenization step to the metric.
|
StringMetricBuilder.CollectionMetricTokenizerStep<T> |
StringMetricBuilder.CollectionMetricTokenizerStep.tokenize(Tokenizer tokenizer)
Adds a tokenization step to the metric.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractTokenizer
Convenience tokenizer.
|
class |
QGram
Deprecated.
Use
Tokenizers.qGram(int) instead |
class |
QGramExtended
Deprecated.
|
class |
Whitespace
Deprecated.
use
Tokenizers.whitespace() instead |
Modifier and Type | Method and Description |
---|---|
static Tokenizer |
Tokenizers.chain(List<Tokenizer> tokenizers)
Chains tokenizers together.
|
static Tokenizer |
Tokenizers.chain(Tokenizer tokenizer,
Tokenizer... tokenizers)
Chains tokenizers together.
|
static Tokenizer |
Tokenizers.filter(Tokenizer tokenizer,
com.google.common.base.Predicate<String> predicate)
Constructs a new filtering tokenizer.
|
static Tokenizer |
Tokenizers.pattern(Pattern pattern)
Returns a tokenizer that splits a string into tokens around the pattern
as if calling
pattern.split(input,-1) . |
static Tokenizer |
Tokenizers.pattern(String regex)
Returns a tokenizer that splits a string into tokens around the pattern
as if calling
Pattern.compile(regex).split(input,-1) . |
static Tokenizer |
Tokenizers.qGram(int q)
Returns a basic q-gram tokenizer for a variable q.
|
static Tokenizer |
Tokenizers.qGramWithFilter(int q)
Returns a basic q-gram tokenizer for a variable q.
|
static Tokenizer |
Tokenizers.qGramWithPadding(int q)
Returns a basic q-gram tokenizer for a variable q.The input is padded
with q-1 special characters before being tokenized.
|
static Tokenizer |
Tokenizers.qGramWithPadding(int q,
String padding)
Returns a basic q-gram tokenizer for a variable Q.The Q-Gram is extended
beyond the length of the string with padding.
|
static Tokenizer |
Tokenizers.qGramWithPadding(int q,
String startPadding,
String endPadding)
Returns a basic q-gram tokenizer for a variable Q.The Q-Gram is extended
beyond the length of the string with padding.
|
static Tokenizer |
Tokenizers.transform(Tokenizer tokenizer,
com.google.common.base.Function<String,String> function)
Constructs a new transforming tokenizer.
|
static Tokenizer |
Tokenizers.whitespace()
Returns a tokenizer that splits a string into tokens around whitespace.
|
Modifier and Type | Method and Description |
---|---|
static Tokenizer |
Tokenizers.chain(Tokenizer tokenizer,
Tokenizer... tokenizers)
Chains tokenizers together.
|
static Tokenizer |
Tokenizers.chain(Tokenizer tokenizer,
Tokenizer... tokenizers)
Chains tokenizers together.
|
static Tokenizer |
Tokenizers.filter(Tokenizer tokenizer,
com.google.common.base.Predicate<String> predicate)
Constructs a new filtering tokenizer.
|
static Tokenizer |
Tokenizers.transform(Tokenizer tokenizer,
com.google.common.base.Function<String,String> function)
Constructs a new transforming tokenizer.
|
Modifier and Type | Method and Description |
---|---|
static Tokenizer |
Tokenizers.chain(List<Tokenizer> tokenizers)
Chains tokenizers together.
|
Modifier and Type | Interface and Description |
---|---|
interface |
TokenizingTokenizer
Deprecated.
|
Modifier and Type | Class and Description |
---|---|
class |
CachingTokenizer
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
Tokenizer |
Tokenizing.getTokenizer()
Deprecated.
Gets the tokenizer.
|
Tokenizer |
CachingTokenizer.getTokenizer()
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
void |
Tokenizing.setTokenizer(Tokenizer tokenizer)
Deprecated.
Sets the tokenizer.
|
void |
CachingTokenizer.setTokenizer(Tokenizer tokenizer)
Deprecated.
|
Constructor and Description |
---|
CachingTokenizer(int initialCapacity,
int maximumSize,
Tokenizer tokenizer)
Deprecated.
Creates a caching tokenizer with
initialCapacity and
maximumSize . |
Copyright © 2014–2018. All rights reserved.