Package org.apache.storm.hdfs.bolt
Class AvroGenericRecordBolt
java.lang.Object
org.apache.storm.topology.base.BaseComponent
org.apache.storm.topology.base.BaseRichBolt
org.apache.storm.hdfs.bolt.AbstractHdfsBolt
org.apache.storm.hdfs.bolt.AvroGenericRecordBolt
- All Implemented Interfaces:
Serializable,IBolt,IComponent,IRichBolt
- See Also:
-
Field Summary
Fields inherited from class org.apache.storm.hdfs.bolt.AbstractHdfsBolt
collector, configKey, fileNameFormat, fileRetryCount, fs, fsUrl, hdfsConfig, maxOpenFiles, offset, partitioner, rotationActions, rotationCounterMap, rotationPolicy, rotationTimer, syncPolicy, tickTupleInterval, writeLock, writers -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionaddRotationAction(RotationAction action) protected voiddoPrepare(Map<String, Object> conf, TopologyContext topologyContext, OutputCollector collector) protected StringgetWriterKey(Tuple tuple) AvroGenericRecordBolt must override this method because messages with different schemas cannot be written to the same file.protected AbstractHDFSWritermakeNewWriter(org.apache.hadoop.fs.Path path, Tuple tuple) withConfigKey(String configKey) withFileNameFormat(FileNameFormat fileNameFormat) withMaxOpenFiles(int maxOpenFiles) withPartitioner(Partitioner partitioner) withRotationPolicy(FileRotationPolicy rotationPolicy) withSyncPolicy(SyncPolicy syncPolicy) withTickTupleIntervalSeconds(int interval) Methods inherited from class org.apache.storm.hdfs.bolt.AbstractHdfsBolt
cleanup, declareOutputFields, execute, getBasePathForNextFile, getComponentConfiguration, prepare, rotateOutputFile
-
Constructor Details
-
AvroGenericRecordBolt
public AvroGenericRecordBolt()
-
-
Method Details
-
withFsUrl
-
withConfigKey
-
withFileNameFormat
-
withSyncPolicy
-
withRotationPolicy
-
addRotationAction
-
withTickTupleIntervalSeconds
-
withMaxOpenFiles
-
withPartitioner
-
doPrepare
protected void doPrepare(Map<String, Object> conf, TopologyContext topologyContext, OutputCollector collector) throws IOException- Specified by:
doPreparein classAbstractHdfsBolt- Throws:
IOException
-
getWriterKey
AvroGenericRecordBolt must override this method because messages with different schemas cannot be written to the same file. By treating the complete schema as the "key" AbstractHdfsBolt will associate a different writer for every distinct schema.- Specified by:
getWriterKeyin classAbstractHdfsBolt
-
makeNewWriter
protected AbstractHDFSWriter makeNewWriter(org.apache.hadoop.fs.Path path, Tuple tuple) throws IOException - Specified by:
makeNewWriterin classAbstractHdfsBolt- Throws:
IOException
-