Package org.apache.storm.scheduler

# Interface ISchedulingState

All Known Implementing Classes:
  `Cluster`, `SingleTopologyCluster`

public interface ISchedulingState
An interface that provides access to the current scheduling state. The scheduling state is not guaranteed to be thread safe.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `NormalizedResourceRequest`
  `getAllScheduledResourcesForNode(String nodeId)`
  Get all scheduled resources for node.
  `Set<Integer>`
  `getAssignablePorts(SupervisorDetails supervisor)`
  Get the ports that are not blacklisted.
  `List<WorkerSlot>`
  `getAssignableSlots()`
  Get all non-blacklisted slots in the cluster.
  `List<WorkerSlot>`
  `getAssignableSlots(SupervisorDetails supervisor)`
  Return all non-blacklisted slots on this supervisor.
  `int`
  `getAssignedNumWorkers(TopologyDetails topology)`
  Get the number of workers assigned to a topology.
  `default Set<String>`
  `getAssignedRacks(String... topologyIds)`
  Determine the list of racks on which topologyIds have been assigned.
  `SchedulerAssignment`
  `getAssignmentById(String topologyId)`
  get the current assignment for the topology.
  `Map<String,SchedulerAssignment>`
  `getAssignments()`
  Get all the assignments.
  `Set<Integer>`
  `getAvailablePorts(SupervisorDetails supervisor)`
  Return the available ports of this supervisor.
  `NormalizedResourceOffer`
  `getAvailableResources(SupervisorDetails sd)`
  Get the resources on the supervisor that are available to be scheduled.
  `List<WorkerSlot>`
  `getAvailableSlots()`
  Get all the available worker slots in the cluster.
  `List<WorkerSlot>`
  `getAvailableSlots(SupervisorDetails supervisor)`
  Return all the available slots on this supervisor.
  `Set<String>`
  `getBlacklistedHosts()`
  Get all of the hosts that are blacklisted.
  `double`
  `getClusterTotalCpuResource()`
  Get the total amount of CPU resources in cluster.
  `Map<String,Double>`
  `getClusterTotalGenericResources()`
  Get the total amount of generic resources (excluding CPU and memory) in cluster.
  `double`
  `getClusterTotalMemoryResource()`
  Get the total amount of memory resources in cluster.
  `Map<String,Object>`
  `getConf()`
  Get the nimbus configuration.
  `String`
  `getHost(String supervisorId)`
  Map a supervisor to a given host.
  `default Map<String,String>`
  `getHostToRack()`
  Get host -> rack map - the inverse of networkTopography.
  `Map<String,List<ExecutorDetails>>`
  `getNeedsSchedulingComponentToExecutors(TopologyDetails topology)`
  Get the component name to executor list for executors that need to be scheduled.
  `Map<ExecutorDetails,String>`
  `getNeedsSchedulingExecutorToComponents(TopologyDetails topology)`
  Get the executor to component name map for executors that need to be scheduled.
  `Map<String,List<String>>`
  `getNetworkTopography()`
  Get the network topography (rackId -> nodes in the rack).
  `List<WorkerSlot>`
  `getNonBlacklistedAvailableSlots(List<String> blacklistedSupervisorIds)`
  Get all the available worker slots in the cluster, that are not blacklisted.
  `NormalizedResourceOffer`
  `getNonBlacklistedClusterAvailableResources(Collection<String> blacklistedSupervisorIds)`
  Get the resources in the cluster that are available for scheduling.
  `double`
  `getScheduledCpuForNode(String nodeId)`
  Get the total cpu currently scheduled on a node.
  `double`
  `getScheduledMemoryForNode(String nodeId)`
  Get the total memory currently scheduled on a node.
  `Map<String,String>`
  `getStatusMap()`
  Get all topology scheduler statuses.
  `SupervisorDetails`
  `getSupervisorById(String nodeId)`
  Get a specific supervisor with the `nodeId`.
  `Map<String,SupervisorDetails>`
  `getSupervisors()`
  Get all the supervisors.
  `List<SupervisorDetails>`
  `getSupervisorsByHost(String host)`
  Get all the supervisors on the specified `host`.
  `Map<String,SupervisorResources>`
  `getSupervisorsResourcesMap()`
  Get the amount of used and free resources on a supervisor.
  `Topologies`
  `getTopologies()`
  Get all of the topologies.
  `Map<String,TopologyResources>`
  `getTopologyResourcesMap()`
  Get the amount of resources used by topologies.
  `Collection<ExecutorDetails>`
  `getUnassignedExecutors(TopologyDetails topology)`
  get the unassigned executors of the topology.
  `Set<Integer>`
  `getUsedPorts(SupervisorDetails supervisor)`
  Get all the used ports of this supervisor.
  `Collection<WorkerSlot>`
  `getUsedSlots()`
  Get all currently occupied slots.
  `Collection<WorkerSlot>`
  `getUsedSlotsByTopologyId(String topologyId)`
  get slots used by a topology.
  `WorkerResources`
  `getWorkerResources(WorkerSlot ws)`
  Get the resources for a given slot.
  `Map<String,Map<WorkerSlot,WorkerResources>>`
  `getWorkerResourcesMap()`
  Gets the reference to the full topology->worker resource map.
  `boolean`
  `isBlackListed(String supervisorId)`
  Check is a given supervisor is on a blacklisted host.
  `boolean`
  `isBlacklistedHost(String host)`
  Check if a given host is blacklisted.
  `boolean`
  `isSlotOccupied(WorkerSlot slot)`
  Check if a slot is occupied or not.
  `boolean`
  `needsScheduling(TopologyDetails topology)`
  Does the topology need scheduling.
  `boolean`
  `needsSchedulingRas(TopologyDetails topology)`
  Like `needsScheduling(TopologyDetails)` but does not take into account the number of workers requested.
  `List<TopologyDetails>`
  `needsSchedulingTopologies()`
  Get all of the topologies that need scheduling.
  `boolean`
  `wouldFit(WorkerSlot ws, ExecutorDetails exec, TopologyDetails td, NormalizedResourceOffer resourcesAvailable, double maxHeap)`
  Would scheduling exec on ws fit? With a heap <= maxHeap total memory added <= memoryAvailable and cpu added <= cpuAvailable.

- ## Method Details

  - ### getTopologies

    Topologies getTopologies()
    Get all of the topologies.
    Returns:
      all of the topologies that are a part of the cluster.

  - ### needsSchedulingTopologies

    List<TopologyDetails> needsSchedulingTopologies()
    Get all of the topologies that need scheduling.
    Returns:
      all of the topologies that are not fully scheduled.

  - ### needsScheduling

    boolean needsScheduling(TopologyDetails topology)
    Does the topology need scheduling.

    A topology needs scheduling if one of the following conditions holds:

    - Although the topology is assigned slots, but is squeezed. i.e. the topology is assigned less slots than desired.
    - There are unassigned executors in this topology

  - ### needsSchedulingRas

    boolean needsSchedulingRas(TopologyDetails topology)
    Like `needsScheduling(TopologyDetails)` but does not take into account the number of workers requested. This is because the number of workers is ignored in RAS
    Parameters:
      `topology` - the topology to check
    Returns:
      true if the topology needs scheduling else false.

  - ### getBlacklistedHosts

    Set<String> getBlacklistedHosts()
    Get all of the hosts that are blacklisted.
    Returns:
      all of the hosts that are blacklisted

  - ### isBlackListed

    boolean isBlackListed(String supervisorId)
    Check is a given supervisor is on a blacklisted host.
    Parameters:
      `supervisorId` - the id of the supervisor
    Returns:
      true if it is else false

  - ### isBlacklistedHost

    boolean isBlacklistedHost(String host)
    Check if a given host is blacklisted.
    Parameters:
      `host` - the name of the host
    Returns:
      true if it is else false.

  - ### getHost

    String getHost(String supervisorId)
    Map a supervisor to a given host.
    Parameters:
      `supervisorId` - the id of the supervisor
    Returns:
      the actual host name the supervisor is on

  - ### getUnassignedExecutors

    Collection<ExecutorDetails> getUnassignedExecutors(TopologyDetails topology)
    get the unassigned executors of the topology.
    Parameters:
      `topology` - the topology to check
    Returns:
      the unassigned executors of the topology.

  - ### getNeedsSchedulingExecutorToComponents

    Map<ExecutorDetails,String> getNeedsSchedulingExecutorToComponents(TopologyDetails topology)
    Get the executor to component name map for executors that need to be scheduled.
    Parameters:
      `topology` - the topology this is for
    Returns:
      a executor -> component-id map which needs scheduling in this topology.

  - ### getNeedsSchedulingComponentToExecutors

    Map<String,List<ExecutorDetails>> getNeedsSchedulingComponentToExecutors(TopologyDetails topology)
    Get the component name to executor list for executors that need to be scheduled.
    Parameters:
      `topology` - the topology this is for
    Returns:
      a component-id -> executors map which needs scheduling in this topology.

  - ### getUsedPorts

    Set<Integer> getUsedPorts(SupervisorDetails supervisor)
    Get all the used ports of this supervisor.

  - ### getAvailablePorts

    Set<Integer> getAvailablePorts(SupervisorDetails supervisor)
    Return the available ports of this supervisor.

  - ### getAssignablePorts

    Set<Integer> getAssignablePorts(SupervisorDetails supervisor)
    Get the ports that are not blacklisted.
    Parameters:
      `supervisor` - the supervisor
    Returns:
      the ports that are not blacklisted

  - ### getAvailableSlots

    List<WorkerSlot> getAvailableSlots(SupervisorDetails supervisor)
    Return all the available slots on this supervisor.

  - ### getAvailableSlots

    List<WorkerSlot> getAvailableSlots()
    Get all the available worker slots in the cluster.

  - ### getNonBlacklistedAvailableSlots

    List<WorkerSlot> getNonBlacklistedAvailableSlots(List<String> blacklistedSupervisorIds)
    Get all the available worker slots in the cluster, that are not blacklisted.
    Parameters:
      `blacklistedSupervisorIds` - list of supervisor ids that should also be considered blacklisted.

  - ### getAssignableSlots

    List<WorkerSlot> getAssignableSlots(SupervisorDetails supervisor)
    Return all non-blacklisted slots on this supervisor.
    Parameters:
      `supervisor` - the supervisor
    Returns:
      the non-blacklisted slots

  - ### getAssignableSlots

    List<WorkerSlot> getAssignableSlots()
    Get all non-blacklisted slots in the cluster.

  - ### getUsedSlots

    Collection<WorkerSlot> getUsedSlots()
    Get all currently occupied slots.

  - ### isSlotOccupied

    boolean isSlotOccupied(WorkerSlot slot)
    Check if a slot is occupied or not.
    Parameters:
      `slot` - the slot be to checked.
    Returns:
      true if the specified slot is occupied.

  - ### getAssignedNumWorkers

    int getAssignedNumWorkers(TopologyDetails topology)
    Get the number of workers assigned to a topology.
    Parameters:
      `topology` - the topology this is for
    Returns:
      the number of workers assigned to this topology.

  - ### getAvailableResources

    NormalizedResourceOffer getAvailableResources(SupervisorDetails sd)
    Get the resources on the supervisor that are available to be scheduled.
    Parameters:
      `sd` - the supervisor.
    Returns:
      the resources available to be scheduled.

  - ### wouldFit

    boolean wouldFit(WorkerSlot ws, ExecutorDetails exec, TopologyDetails td, NormalizedResourceOffer resourcesAvailable, double maxHeap)
    Would scheduling exec on ws fit? With a heap <= maxHeap total memory added <= memoryAvailable and cpu added <= cpuAvailable.
    Parameters:
      `ws` - the slot to put it in
      `exec` - the executor to investigate
      `td` - the topology detains for this executor
      `resourcesAvailable` - all the available resources
      `maxHeap` - the maximum heap size for ws
    Returns:
      true it fits else false

  - ### getAssignmentById

    SchedulerAssignment getAssignmentById(String topologyId)
    get the current assignment for the topology.

  - ### getUsedSlotsByTopologyId

    Collection<WorkerSlot> getUsedSlotsByTopologyId(String topologyId)
    get slots used by a topology.

  - ### getSupervisorById

    SupervisorDetails getSupervisorById(String nodeId)
    Get a specific supervisor with the `nodeId`.

  - ### getSupervisorsByHost

    List<SupervisorDetails> getSupervisorsByHost(String host)
    Get all the supervisors on the specified `host`.
    Parameters:
      `host` - hostname of the supervisor
    Returns:
      the `SupervisorDetails` object.

  - ### getAssignments

    Map<String,SchedulerAssignment> getAssignments()
    Get all the assignments.

  - ### getSupervisors

    Map<String,SupervisorDetails> getSupervisors()
    Get all the supervisors.

  - ### getAllScheduledResourcesForNode

    NormalizedResourceRequest getAllScheduledResourcesForNode(String nodeId)
    Get all scheduled resources for node.

  - ### getNonBlacklistedClusterAvailableResources

    NormalizedResourceOffer getNonBlacklistedClusterAvailableResources(Collection<String> blacklistedSupervisorIds)
    Get the resources in the cluster that are available for scheduling.
    Parameters:
      `blacklistedSupervisorIds` - other ids that are tentatively blacklisted.

  - ### getClusterTotalCpuResource

    double getClusterTotalCpuResource()
    Get the total amount of CPU resources in cluster.

  - ### getClusterTotalMemoryResource

    double getClusterTotalMemoryResource()
    Get the total amount of memory resources in cluster.

  - ### getClusterTotalGenericResources

    Map<String,Double> getClusterTotalGenericResources()
    Get the total amount of generic resources (excluding CPU and memory) in cluster.

  - ### getNetworkTopography

    Map<String,List<String>> getNetworkTopography()
    Get the network topography (rackId -> nodes in the rack).

  - ### getHostToRack

    default Map<String,String> getHostToRack()
    Get host -> rack map - the inverse of networkTopography.

  - ### getStatusMap

    Map<String,String> getStatusMap()
    Get all topology scheduler statuses.

  - ### getTopologyResourcesMap

    Map<String,TopologyResources> getTopologyResourcesMap()
    Get the amount of resources used by topologies. Used for displaying resource information on the UI.
    Returns:
      a map that contains multiple topologies and the resources the topology requested and assigned. Key: topology id Value: an array that describes the resources the topology requested and assigned in the following format: {requestedMemOnHeap, requestedMemOffHeap, requestedCpu, assignedMemOnHeap, assignedMemOffHeap, assignedCpu}

  - ### getSupervisorsResourcesMap

    Map<String,SupervisorResources> getSupervisorsResourcesMap()
    Get the amount of used and free resources on a supervisor. Used for displaying resource information on the UI
    Returns:
      a map where the key is the supervisor id and the value is a map that represents resource usage for a supervisor in the following format: {totalMem, totalCpu, usedMem, usedCpu}

  - ### getWorkerResourcesMap

    Map<String,Map<WorkerSlot,WorkerResources>> getWorkerResourcesMap()
    Gets the reference to the full topology->worker resource map.
    Returns:
      map of topology -> map of worker slot ->resources for that worker

  - ### getWorkerResources

    WorkerResources getWorkerResources(WorkerSlot ws)
    Get the resources for a given slot.
    Parameters:
      `ws` - the slot
    Returns:
      the resources currently assigned

  - ### getScheduledMemoryForNode

    double getScheduledMemoryForNode(String nodeId)
    Get the total memory currently scheduled on a node.
    Parameters:
      `nodeId` - the id of the node
    Returns:
      the total memory currently scheduled on the node

  - ### getScheduledCpuForNode

    double getScheduledCpuForNode(String nodeId)
    Get the total cpu currently scheduled on a node.
    Parameters:
      `nodeId` - the id of the node
    Returns:
      the total cpu currently scheduled on the node

  - ### getConf

    Map<String,Object> getConf()
    Get the nimbus configuration.

  - ### getAssignedRacks

    default Set<String> getAssignedRacks(String... topologyIds)
    Determine the list of racks on which topologyIds have been assigned. Note that the returned set may contain `DNSToSwitchMapping.DEFAULT_RACK` if `getHostToRack()` is null or does not contain the assigned host.
    Parameters:
      `topologyIds` - for which assignments are examined.
    Returns:
      set of racks on which assignments have been made.