Uses of Interface
opennlp.tools.tokenize.TokenContextGenerator
Packages that use TokenContextGenerator
Package
Description
Contains classes related to finding token or words in a string.
-
Uses of TokenContextGenerator in opennlp.tools.tokenize
Classes in opennlp.tools.tokenize that implement TokenContextGeneratorModifier and TypeClassDescriptionclassA defaultTokenContextGeneratorwhich produces events for maxent decisions for tokenization.Methods in opennlp.tools.tokenize that return TokenContextGeneratorConstructors in opennlp.tools.tokenize with parameters of type TokenContextGeneratorModifierConstructorDescriptionTokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator.TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator. -
Uses of TokenContextGenerator in opennlp.tools.tokenize.lang
Methods in opennlp.tools.tokenize.lang that return TokenContextGeneratorModifier and TypeMethodDescriptionFactory.createTokenContextGenerator(String languageCode, Set<String> abbreviations) Initializes a customizedTokenContextGeneratorvia a set ofabbreviations.