Class PartitionedBlock<T extends CacheBlock>
- java.lang.Object
-
- org.apache.sysds.runtime.instructions.spark.data.PartitionedBlock<T>
-
- All Implemented Interfaces:
Externalizable,Serializable
public class PartitionedBlock<T extends CacheBlock> extends Object implements Externalizable
This class is for partitioned matrix/frame blocks, to be used as broadcasts. Distributed tasks require block-partitioned broadcasts but a lazy partitioning per task would create instance-local copies and hence replicate broadcast variables which are shared by all tasks within an executor.- See Also:
- Serialized Form
-
-
Constructor Summary
Constructors Constructor Description PartitionedBlock()PartitionedBlock(int rlen, int clen, int blen)PartitionedBlock(T block, int blen)PartitionedBlock(T block, long[] dims, int blen)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidclearBlocks()PartitionedBlock<T>createPartition(int offset, int numBlks)TgetBlock(int[] ix)TgetBlock(int rowIndex, int colIndex)longgetBlocksize()longgetDim(int i)longgetExactSerializedSize()longgetInMemorySize()longgetNumCols()intgetNumColumnBlocks()intgetNumDimBlocks(int dim)intgetNumRowBlocks()longgetNumRows()voidreadExternal(ObjectInput is)Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast deserialization.voidsetBlock(int rowIndex, int colIndex, T block)voidwriteExternal(ObjectOutput os)Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast serialization.
-
-
-
Method Detail
-
createPartition
public PartitionedBlock<T> createPartition(int offset, int numBlks)
-
getNumRows
public long getNumRows()
-
getNumCols
public long getNumCols()
-
getDim
public long getDim(int i)
-
getBlocksize
public long getBlocksize()
-
getNumRowBlocks
public int getNumRowBlocks()
-
getNumColumnBlocks
public int getNumColumnBlocks()
-
getNumDimBlocks
public int getNumDimBlocks(int dim)
-
getBlock
public T getBlock(int rowIndex, int colIndex)
-
getBlock
public T getBlock(int[] ix)
-
setBlock
public void setBlock(int rowIndex, int colIndex, T block)
-
getInMemorySize
public long getInMemorySize()
-
getExactSerializedSize
public long getExactSerializedSize()
-
clearBlocks
public void clearBlocks()
-
readExternal
public void readExternal(ObjectInput is) throws IOException
Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast deserialization.- Specified by:
readExternalin interfaceExternalizable- Parameters:
is- object input- Throws:
IOException- if IOException occurs
-
writeExternal
public void writeExternal(ObjectOutput os) throws IOException
Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast serialization.- Specified by:
writeExternalin interfaceExternalizable- Parameters:
os- object output- Throws:
IOException- if IOException occurs
-
-