|
Java example source code file (CustomStemmingPreprocessor.java)
The CustomStemmingPreprocessor.java Java example source codepackage org.deeplearning4j.text.tokenization.tokenizer.preprocessor; import lombok.NonNull; import org.tartarus.snowball.SnowballProgram; /** * This is StemmingPreprocessor compatible with different StemmingProcessors defined as lucene/tartarus SnowballProgram * Like, but not limited to: RussianStemmer, DutchStemmer, FrenchStemmer etc * * PLEASE NOTE: This preprocessor is NOT thread-safe. * * @author raver119@gmail.com */ public class CustomStemmingPreprocessor extends CommonPreprocessor { private SnowballProgram stemmer; public CustomStemmingPreprocessor(@NonNull SnowballProgram stemmer) { this.stemmer = stemmer; } @Override public String preProcess(String token) { String prep = super.preProcess(token); stemmer.setCurrent(prep); stemmer.stem(); return stemmer.getCurrent(); } } Other Java examples (source code examples)Here is a short list of links related to this Java CustomStemmingPreprocessor.java source code file: |
... this post is sponsored by my books ... | |
#1 New Release! |
FP Best Seller |
Copyright 1998-2021 Alvin Alexander, alvinalexander.com
All Rights Reserved.
A percentage of advertising revenue from
pages under the /java/jwarehouse
URI on this website is
paid back to open source projects.