T
- the type of the materialized recordspublic class ParquetRecordWriter<T> extends org.apache.hadoop.mapreduce.RecordWriter<Void,T>
ParquetOutputFormat
Constructor and Description |
---|
ParquetRecordWriter(ParquetFileWriter w,
WriteSupport<T> writeSupport,
MessageType schema,
Map<String,String> extraMetaData,
int blockSize,
int pageSize,
CodecFactory.BytesCompressor compressor,
int dictionaryPageSize,
boolean enableDictionary,
boolean validating,
ParquetProperties.WriterVersion writerVersion) |
Modifier and Type | Method and Description |
---|---|
void |
close(org.apache.hadoop.mapreduce.TaskAttemptContext context) |
void |
write(Void key,
T value) |
public ParquetRecordWriter(ParquetFileWriter w, WriteSupport<T> writeSupport, MessageType schema, Map<String,String> extraMetaData, int blockSize, int pageSize, CodecFactory.BytesCompressor compressor, int dictionaryPageSize, boolean enableDictionary, boolean validating, ParquetProperties.WriterVersion writerVersion)
w
- the file to write towriteSupport
- the class to convert incoming recordsschema
- the schema of the recordsextraMetaData
- extra meta data to write in the footer of the fileblockSize
- the size of a block in the file (this will be approximate)compressor
- the compressor used to compress the pagesdictionaryPageSize
- the threshold for dictionary sizeenableDictionary
- to enable the dictionaryvalidating
- if schema validation should be turned onpublic void close(org.apache.hadoop.mapreduce.TaskAttemptContext context) throws IOException, InterruptedException
close
in class org.apache.hadoop.mapreduce.RecordWriter<Void,T>
IOException
InterruptedException
public void write(Void key, T value) throws IOException, InterruptedException
write
in class org.apache.hadoop.mapreduce.RecordWriter<Void,T>
IOException
InterruptedException
Copyright © 2015. All rights reserved.