public class ProportionalCapacityPreemptionPolicy extends Object implements SchedulingEditPolicy
SchedulingEditPolicy
that is designed to be
paired with the CapacityScheduler
. At every invocation of editSchedule()
it computes the ideal amount of resources assigned to each
queue (for each queue in the hierarchy), and determines whether preemption
is needed. Overcapacity is distributed among queues in a weighted fair manner,
where the weight is the amount of guaranteed capacity for the queue.
Based on this ideal assignment it determines whether preemption is required
and select a set of containers from each application that would be killed if
the corresponding amount of resources is not freed up by the application.
If not in observeOnly
mode, it triggers preemption requests via a
ContainerPreemptEvent
that the ResourceManager
will ensure
to deliver to the application (or to execute).
If the deficit of resources is persistent over a long enough period of time
this policy will trigger forced termination of containers (again by generating
ContainerPreemptEvent
).Modifier and Type | Field and Description |
---|---|
EventHandler<ContainerPreemptEvent> |
dispatcher |
static String |
MAX_IGNORED_OVER_CAPACITY
Maximum amount of resources above the target capacity ignored for
preemption.
|
static String |
MONITORING_INTERVAL
Time in milliseconds between invocations of this policy
|
static String |
NATURAL_TERMINATION_FACTOR
Given a computed preemption target, account for containers naturally
expiring and preempt only this percentage of the delta.
|
static String |
OBSERVE_ONLY
If true, run the policy but do not affect the cluster with preemption and
kill events.
|
static String |
TOTAL_PREEMPTION_PER_ROUND
Maximum percentage of resources preempted in a single round.
|
static String |
WAIT_TIME_BEFORE_KILL
Time in milliseconds between requesting a preemption from an application
and killing the container.
|
Constructor and Description |
---|
ProportionalCapacityPreemptionPolicy() |
ProportionalCapacityPreemptionPolicy(Configuration config,
EventHandler<ContainerPreemptEvent> dispatcher,
CapacityScheduler scheduler) |
ProportionalCapacityPreemptionPolicy(Configuration config,
EventHandler<ContainerPreemptEvent> dispatcher,
CapacityScheduler scheduler,
Clock clock) |
Modifier and Type | Method and Description |
---|---|
void |
editSchedule()
This method is invoked at regular intervals.
|
long |
getMonitoringInterval() |
String |
getPolicyName() |
void |
init(Configuration config,
EventHandler<ContainerPreemptEvent> disp,
PreemptableResourceScheduler sched) |
public static final String OBSERVE_ONLY
public static final String MONITORING_INTERVAL
public static final String WAIT_TIME_BEFORE_KILL
public static final String TOTAL_PREEMPTION_PER_ROUND
public static final String MAX_IGNORED_OVER_CAPACITY
public static final String NATURAL_TERMINATION_FACTOR
MAX_IGNORED_OVER_CAPACITY
). For example, a termination factor of 0.5
will reclaim almost 95% of resources within 5 * WAIT_TIME_BEFORE_KILL
, even absent natural termination.public EventHandler<ContainerPreemptEvent> dispatcher
public ProportionalCapacityPreemptionPolicy()
public ProportionalCapacityPreemptionPolicy(Configuration config, EventHandler<ContainerPreemptEvent> dispatcher, CapacityScheduler scheduler)
public ProportionalCapacityPreemptionPolicy(Configuration config, EventHandler<ContainerPreemptEvent> dispatcher, CapacityScheduler scheduler, Clock clock)
public void init(Configuration config, EventHandler<ContainerPreemptEvent> disp, PreemptableResourceScheduler sched)
init
in interface SchedulingEditPolicy
public void editSchedule()
SchedulingEditPolicy
editSchedule
in interface SchedulingEditPolicy
public long getMonitoringInterval()
getMonitoringInterval
in interface SchedulingEditPolicy
public String getPolicyName()
getPolicyName
in interface SchedulingEditPolicy
Copyright © 2013 Apache Software Foundation. All rights reserved.