Function dslab_mr::placement_strategies::common::collect_all_input

source · 
pub fn collect_all_input(
    input: &[DataItem],
    dfs: &DistributedFileSystem
) -> Vec<(DataItem, Vec<Id>)>
Expand description

Helper function to collect all inputs along with the location of each one.

  • DataItem::Replicated is split into individual chunks.
  • DataItem::Local is split into parts with the size of chunk_size except at most one.
  • DataItem::Chunk remains unchanged.