Package | Description |
---|---|
org.apache.hadoop.hdfs.protocol | |
org.apache.hadoop.hdfs.server.blockmanagement | |
org.apache.hadoop.net |
Network-related classes.
|
org.apache.hadoop.yarn.server.resourcemanager | |
org.apache.hadoop.yarn.server.resourcemanager.rmnode | |
org.apache.hadoop.yarn.util |
Modifier and Type | Class and Description |
---|---|
class |
DatanodeInfo
This class extends the primary identifier of a Datanode with ephemeral
state, eg usage information, current administrative state, and the
network location that is communicated to clients.
|
Modifier and Type | Method and Description |
---|---|
Node |
DatanodeInfo.getParent()
Return this node's parent
|
Modifier and Type | Method and Description |
---|---|
void |
DatanodeInfo.setParent(Node parent) |
Modifier and Type | Class and Description |
---|---|
class |
DatanodeDescriptor
This class extends the DatanodeInfo class with ephemeral information (eg
health, capacity, what blocks are associated with the Datanode) that is
private to the Namenode, ie this class is not exposed to clients.
|
Modifier and Type | Method and Description |
---|---|
protected void |
BlockPlacementPolicyDefault.adjustExcludedNodes(HashMap<Node,Node> excludedNodes,
Node chosenNode)
After choosing a node to place replica, adjust excluded nodes accordingly.
|
protected void |
BlockPlacementPolicyWithNodeGroup.adjustExcludedNodes(HashMap<Node,Node> excludedNodes,
Node chosenNode)
After choosing a node to place replica, adjust excluded nodes accordingly.
|
Modifier and Type | Method and Description |
---|---|
protected int |
BlockPlacementPolicyDefault.addToExcludedNodes(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes)
Add localMachine and related nodes to excludedNodes
for next replica choosing.
|
protected int |
BlockPlacementPolicyDefault.addToExcludedNodes(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes)
Add localMachine and related nodes to excludedNodes
for next replica choosing.
|
protected int |
BlockPlacementPolicyWithNodeGroup.addToExcludedNodes(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes)
Find other nodes in the same nodegroup of localMachine and add them
into excludeNodes as replica should not be duplicated for nodes
within the same nodegroup
|
protected int |
BlockPlacementPolicyWithNodeGroup.addToExcludedNodes(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes)
Find other nodes in the same nodegroup of localMachine and add them
into excludeNodes as replica should not be duplicated for nodes
within the same nodegroup
|
protected void |
BlockPlacementPolicyDefault.adjustExcludedNodes(HashMap<Node,Node> excludedNodes,
Node chosenNode)
After choosing a node to place replica, adjust excluded nodes accordingly.
|
protected void |
BlockPlacementPolicyDefault.adjustExcludedNodes(HashMap<Node,Node> excludedNodes,
Node chosenNode)
After choosing a node to place replica, adjust excluded nodes accordingly.
|
protected void |
BlockPlacementPolicyWithNodeGroup.adjustExcludedNodes(HashMap<Node,Node> excludedNodes,
Node chosenNode)
After choosing a node to place replica, adjust excluded nodes accordingly.
|
protected void |
BlockPlacementPolicyWithNodeGroup.adjustExcludedNodes(HashMap<Node,Node> excludedNodes,
Node chosenNode)
After choosing a node to place replica, adjust excluded nodes accordingly.
|
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes)
choose local node of localMachine as the target.
|
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes)
choose local node of localMachine as the target.
|
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyDefault.chooseRandom(int numOfReplicas,
String nodes,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyDefault.chooseRandom(int numOfReplicas,
String nodes,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseRandom(String nodes,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseRandom(String nodes,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyDefault.chooseRemoteRack(int numOfReplicas,
DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxReplicasPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyDefault.chooseRemoteRack(int numOfReplicas,
DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxReplicasPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyWithNodeGroup.chooseRemoteRack(int numOfReplicas,
DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxReplicasPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyWithNodeGroup.chooseRemoteRack(int numOfReplicas,
DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxReplicasPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
DatanodeDescriptor[] |
BlockManager.chooseTarget(String src,
int numOfReplicas,
DatanodeDescriptor client,
HashMap<Node,Node> excludedNodes,
long blocksize,
List<String> favoredNodes)
Choose target datanodes according to the replication policy.
|
DatanodeDescriptor[] |
BlockManager.chooseTarget(String src,
int numOfReplicas,
DatanodeDescriptor client,
HashMap<Node,Node> excludedNodes,
long blocksize,
List<String> favoredNodes)
Choose target datanodes according to the replication policy.
|
DatanodeDescriptor[] |
BlockPlacementPolicyDefault.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
boolean returnChosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize) |
DatanodeDescriptor[] |
BlockPlacementPolicyDefault.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
boolean returnChosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize) |
abstract DatanodeDescriptor[] |
BlockPlacementPolicy.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
boolean returnChosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize)
choose numOfReplicas data nodes for writer
to re-replicate a block with size blocksize
If not, return as many as we can.
|
abstract DatanodeDescriptor[] |
BlockPlacementPolicy.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
boolean returnChosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize)
choose numOfReplicas data nodes for writer
to re-replicate a block with size blocksize
If not, return as many as we can.
|
Modifier and Type | Class and Description |
---|---|
class |
NodeBase
A base class that implements interface Node
|
Modifier and Type | Field and Description |
---|---|
protected Node |
NodeBase.parent |
Modifier and Type | Method and Description |
---|---|
Node |
NetworkTopology.chooseRandom(String scope)
randomly choose one node from scope
if scope starts with ~, choose one from the all nodes except for the
ones in scope; otherwise, choose one from scope
|
Node |
NetworkTopology.getNode(String loc)
Given a string representation of a node, return its reference
|
protected Node |
NetworkTopologyWithNodeGroup.getNodeForNetworkLocation(Node node) |
protected Node |
NetworkTopology.getNodeForNetworkLocation(Node node)
Return a reference to the node given its string representation.
|
Node |
NodeBase.getParent() |
Node |
Node.getParent() |
Modifier and Type | Method and Description |
---|---|
List<Node> |
NetworkTopology.getDatanodesInRack(String loc)
Given a string representation of a rack, return its children
|
List<Node> |
NetworkTopology.getLeaves(String scope)
return leaves in scope
|
Modifier and Type | Method and Description |
---|---|
void |
NetworkTopologyWithNodeGroup.add(Node node)
Add a leaf node
Update node counter & rack counter if necessary
|
void |
NetworkTopology.add(Node node)
Add a leaf node
Update node counter & rack counter if necessary
|
boolean |
NetworkTopology.contains(Node node)
Check if the tree contains node node
|
int |
NetworkTopology.getDistance(Node node1,
Node node2)
Return the distance between two nodes
It is assumed that the distance from one node to its parent is 1
The distance between two nodes is calculated by summing up their distances
to their closest common ancestor.
|
protected Node |
NetworkTopologyWithNodeGroup.getNodeForNetworkLocation(Node node) |
protected Node |
NetworkTopology.getNodeForNetworkLocation(Node node)
Return a reference to the node given its string representation.
|
static String |
NodeBase.getPath(Node node)
Get the path of a node
|
boolean |
NetworkTopologyWithNodeGroup.isOnSameNodeGroup(Node node1,
Node node2)
Check if two nodes are on the same node group (hypervisor) The
assumption here is: each nodes are leaf nodes.
|
boolean |
NetworkTopology.isOnSameNodeGroup(Node node1,
Node node2)
Return false directly as not aware of NodeGroup, to be override in sub-class
|
boolean |
NetworkTopologyWithNodeGroup.isOnSameRack(Node node1,
Node node2) |
boolean |
NetworkTopology.isOnSameRack(Node node1,
Node node2)
Check if two nodes are on the same rack
|
protected boolean |
NetworkTopology.isSameParents(Node node1,
Node node2)
Compare the parents of each node for equality
|
void |
NetworkTopologyWithNodeGroup.pseudoSortByDistance(Node reader,
Node[] nodes)
Sort nodes array by their distances to reader
It linearly scans the array, if a local node is found, swap it with
the first element of the array.
|
void |
NetworkTopologyWithNodeGroup.pseudoSortByDistance(Node reader,
Node[] nodes)
Sort nodes array by their distances to reader
It linearly scans the array, if a local node is found, swap it with
the first element of the array.
|
void |
NetworkTopology.pseudoSortByDistance(Node reader,
Node[] nodes)
Sort nodes array by their distances to reader
It linearly scans the array, if a local node is found, swap it with
the first element of the array.
|
void |
NetworkTopology.pseudoSortByDistance(Node reader,
Node[] nodes)
Sort nodes array by their distances to reader
It linearly scans the array, if a local node is found, swap it with
the first element of the array.
|
void |
NetworkTopologyWithNodeGroup.remove(Node node)
Remove a node
Update node counter and rack counter if necessary
|
void |
NetworkTopology.remove(Node node)
Remove a node
Update node counter and rack counter if necessary
|
void |
NodeBase.setParent(Node parent)
Set this node's parent
|
void |
Node.setParent(Node parent)
Set this node's parent
|
protected static void |
NetworkTopology.swap(Node[] nodes,
int i,
int j)
swap two array items
|
Modifier and Type | Method and Description |
---|---|
int |
NetworkTopology.countNumOfAvailableNodes(String scope,
Collection<Node> excludedNodes)
return the number of leaves in scope but not in excludedNodes
if scope starts with ~, return the number of nodes that are not
in scope and excludedNodes;
|
Constructor and Description |
---|
NodeBase(String name,
String location,
Node parent,
int level)
Construct a node from its name and its location
|
Modifier and Type | Method and Description |
---|---|
static Node |
ResourceTrackerService.resolve(String hostName)
resolving the network topology.
|
Modifier and Type | Method and Description |
---|---|
Node |
RMNodeImpl.getNode() |
Node |
RMNode.getNode()
the
Node information for this node. |
Constructor and Description |
---|
RMNodeImpl(NodeId nodeId,
RMContext context,
String hostName,
int cmPort,
int httpPort,
Node node,
Resource capability) |
Modifier and Type | Method and Description |
---|---|
static Node |
RackResolver.resolve(Configuration conf,
String hostName)
Utility method for getting a hostname resolved to a node in the
network topology.
|
static Node |
RackResolver.resolve(String hostName)
Utility method for getting a hostname resolved to a node in the
network topology.
|
Copyright © 2013 Apache Software Foundation. All rights reserved.