Struct dslab_mr::placement_strategies::packing_scheduler::PackingScheduler

pub struct PackingScheduler { /* private fields */ }

Implementation of a packing scheduler.

It first splits all input data uniformly among all tasks while trying to group them by host. Then it puts all tasks in a queue and when there is at least one host with enough resources to run next task it chooses the best host according to scalar product of resource needs for a task and available resources on a host, multiplied by locality score. Locality score is 1 when all input is on the target host, other_host_input_penalty if all the data is on the same rack, and other_host_input_penalty * other_rack_input_penalty if some inputs are on a different rack.

Implementations

impl PackingScheduler

pub fn new(other_rack_input_penalty: f64, other_host_input_penalty: f64) -> Self

Creates new scheduler.

Trait Implementations

impl DynamicPlacementStrategy for PackingScheduler

fn on_stage_ready( &mut self, dag_id: usize, stage_id: usize, graph: &Dag, input_data: &BTreeMap<usize, Vec<DataItem>>, _input_data_shuffled: &BTreeMap<usize, Vec<Vec<DataItem>>>, dfs: &DistributedFileSystem, compute_host_info: &BTreeMap<Id, ComputeHost>, network: &Network ) -> Vec<StageActions>

Callback when a stage of some DAG is ready.

fn on_task_completed( &mut self, _dag_id: usize, _stage_id: usize, _task: usize, _graph: &Dag, _input_data: &BTreeMap<usize, Vec<DataItem>>, _input_data_shuffled: &BTreeMap<usize, Vec<Vec<DataItem>>>, _dfs: &DistributedFileSystem, compute_host_info: &BTreeMap<Id, ComputeHost>, _network: &Network ) -> Vec<StageActions>

Callback when some task is completed.

fn on_stage_completed( &mut self, _dag_id: usize, _stage_id: usize, _graph: &Dag, _input_data: &BTreeMap<usize, Vec<DataItem>>, _input_data_shuffled: &BTreeMap<usize, Vec<Vec<DataItem>>>, _dfs: &DistributedFileSystem, compute_host_info: &BTreeMap<Id, ComputeHost>, _network: &Network ) -> Vec<StageActions>

Callback when some stage is completed.

fn register_dag(&mut self, _dag_id: usize, _graph: &Dag)

Callback when new NewDag is received by Runner.