Package org.apache.lucene.analysis.pt
Class RSLPStemmerBase
- java.lang.Object
-
- org.apache.lucene.analysis.pt.RSLPStemmerBase
-
- Direct Known Subclasses:
GalicianMinimalStemmer
,GalicianStemmer
,PortugueseMinimalStemmer
,PortugueseStemmer
public abstract class RSLPStemmerBase extends java.lang.Object
Base class for stemmers that use a set of RSLP-like stemming steps.RSLP (Removedor de Sufixos da Lingua Portuguesa) is an algorithm designed originally for stemming the Portuguese language, described in the paper A Stemming Algorithm for the Portuguese Language, Orengo et. al.
Since this time a plural-only modification (RSLP-S) as well as a modification for the Galician language have been implemented. This class parses a configuration file that describes
RSLPStemmerBase.Step
s, where each Step contains a set ofRSLPStemmerBase.Rule
s.The general rule format is:
{ "suffix", N, "replacement", { "exception1", "exception2", ...}}
where:suffix
is the suffix to be removed (such as "inho").N
is the min stem size, where stem is defined as the candidate stem after removing the suffix (but before appending the replacement!)replacement
is an optimal string to append after removing the suffix. This can be the empty string.exceptions
is an optional list of exceptions, patterns that should not be stemmed. These patterns can be specified as whole word or suffix (ends-with) patterns, depending upon the exceptions format flag in the step header.
A step is an ordered list of rules, with a structure in this format:
{ "name", N, B, { "cond1", "cond2", ... } ... rules ... };
where:name
is a name for the step (such as "Plural").N
is the min word size. Words that are less than this length bypass the step completely, as an optimization. Note: N can be zero, in this case this implementation will automatically calculate the appropriate value from the underlying rules.B
is a "boolean" flag specifying how exceptions in the rules are matched. A value of 1 indicates whole-word pattern matching, a value of 0 indicates that exceptions are actually suffixes and should be matched with ends-with.conds
are an optional list of conditions to enter the step at all. If the list is non-empty, then a word must end with one of these conditions or it will bypass the step completely as an optimization.
- See Also:
- RSLP description
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected static class
RSLPStemmerBase.Rule
A basic rule, with no exceptions.protected static class
RSLPStemmerBase.RuleWithSetExceptions
A rule with a set of whole-word exceptions.protected static class
RSLPStemmerBase.RuleWithSuffixExceptions
A rule with a set of exceptional suffixes.protected static class
RSLPStemmerBase.Step
A step containing a list of rules.
-
Field Summary
Fields Modifier and Type Field Description private static java.util.regex.Pattern
excPattern
private static java.util.regex.Pattern
headerPattern
private static java.util.regex.Pattern
repPattern
private static java.util.regex.Pattern
stripPattern
-
Constructor Summary
Constructors Constructor Description RSLPStemmerBase()
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description protected static java.util.Map<java.lang.String,RSLPStemmerBase.Step>
parse(java.lang.Class<? extends RSLPStemmerBase> clazz, java.lang.String resource)
Parse a resource file into an RSLP stemmer description.private static java.lang.String[]
parseList(java.lang.String s)
private static RSLPStemmerBase.Rule[]
parseRules(java.io.LineNumberReader r, int type)
private static RSLPStemmerBase.Step
parseStep(java.io.LineNumberReader r, java.lang.String header)
private static java.lang.String
parseString(java.lang.String s)
private static java.lang.String
readLine(java.io.LineNumberReader r)
-
-
-
Method Detail
-
parse
protected static java.util.Map<java.lang.String,RSLPStemmerBase.Step> parse(java.lang.Class<? extends RSLPStemmerBase> clazz, java.lang.String resource)
Parse a resource file into an RSLP stemmer description.- Returns:
- a Map containing the named Steps in this description.
-
parseStep
private static RSLPStemmerBase.Step parseStep(java.io.LineNumberReader r, java.lang.String header) throws java.io.IOException
- Throws:
java.io.IOException
-
parseRules
private static RSLPStemmerBase.Rule[] parseRules(java.io.LineNumberReader r, int type) throws java.io.IOException
- Throws:
java.io.IOException
-
parseList
private static java.lang.String[] parseList(java.lang.String s)
-
parseString
private static java.lang.String parseString(java.lang.String s)
-
readLine
private static java.lang.String readLine(java.io.LineNumberReader r) throws java.io.IOException
- Throws:
java.io.IOException
-
-