Modifier and Type | Field and Description |
---|---|
DatanodeDescriptor[] |
DatanodeDescriptor.BlockTargetPair.targets |
Modifier and Type | Method and Description |
---|---|
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes)
choose local node of localMachine as the target.
|
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseRandom(String nodes,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseReplicaToDelete(BlockCollection bc,
Block block,
short replicationFactor,
Collection<DatanodeDescriptor> first,
Collection<DatanodeDescriptor> second) |
abstract DatanodeDescriptor |
BlockPlacementPolicy.chooseReplicaToDelete(BlockCollection srcBC,
Block block,
short replicationFactor,
Collection<DatanodeDescriptor> existingReplicas,
Collection<DatanodeDescriptor> moreExistingReplicas)
Decide whether deleting the specified replica of the block still makes
the block conform to the configured block placement policy.
|
DatanodeDescriptor[] |
BlockManager.chooseTarget(String src,
int numOfReplicas,
DatanodeDescriptor client,
HashMap<Node,Node> excludedNodes,
long blocksize,
List<String> favoredNodes)
Choose target datanodes according to the replication policy.
|
DatanodeDescriptor[] |
BlockPlacementPolicyDefault.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
boolean returnChosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize) |
abstract DatanodeDescriptor[] |
BlockPlacementPolicy.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
boolean returnChosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize)
choose numOfReplicas data nodes for writer
to re-replicate a block with size blocksize
If not, return as many as we can.
|
DatanodeDescriptor[] |
BlockPlacementPolicyDefault.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
long blocksize) |
DatanodeDescriptor |
DatanodeManager.getDatanode(DatanodeID nodeID)
Get data node by storage ID.
|
DatanodeDescriptor |
DatanodeManager.getDatanodeByHost(String host) |
DatanodeDescriptor |
DatanodeManager.getDatanodeByXferAddr(String host,
int xferPort) |
DatanodeDescriptor[] |
BlockInfoUnderConstruction.getExpectedLocations()
Create array of expected replica locations
(as has been assigned by chooseTargets()).
|
DatanodeDescriptor[] |
BlockManager.getNodes(BlockInfo block) |
Modifier and Type | Method and Description |
---|---|
Iterator<DatanodeDescriptor> |
BlockManager.datanodeIterator(Block block) |
List<DatanodeDescriptor> |
DatanodeManager.getDatanodeListForReport(HdfsConstants.DatanodeReportType type)
For generating datanode reports
|
List<DatanodeDescriptor> |
DatanodeManager.getDecommissioningNodes() |
protected Iterator<DatanodeDescriptor> |
BlockPlacementPolicyDefault.pickupReplicaSet(Collection<DatanodeDescriptor> first,
Collection<DatanodeDescriptor> second)
Pick up replica node set for deleting replica as over-replicated.
|
Iterator<DatanodeDescriptor> |
BlockPlacementPolicyWithNodeGroup.pickupReplicaSet(Collection<DatanodeDescriptor> first,
Collection<DatanodeDescriptor> second)
Pick up replica node set for deleting replica as over-replicated.
|
Modifier and Type | Method and Description |
---|---|
boolean |
BlockInfo.addNode(DatanodeDescriptor node)
Add data-node this block belongs to.
|
void |
CorruptReplicasMap.addToCorruptReplicasMap(Block blk,
DatanodeDescriptor dn,
String reason)
Mark the block belonging to datanode as corrupt.
|
protected int |
BlockPlacementPolicyDefault.addToExcludedNodes(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes)
Add localMachine and related nodes to excludedNodes
for next replica choosing.
|
protected int |
BlockPlacementPolicyWithNodeGroup.addToExcludedNodes(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes)
Find other nodes in the same nodegroup of localMachine and add them
into excludeNodes as replica should not be duplicated for nodes
within the same nodegroup
|
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes)
choose local node of localMachine as the target.
|
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyDefault.chooseRemoteRack(int numOfReplicas,
DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxReplicasPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyWithNodeGroup.chooseRemoteRack(int numOfReplicas,
DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxReplicasPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
DatanodeDescriptor[] |
BlockManager.chooseTarget(String src,
int numOfReplicas,
DatanodeDescriptor client,
HashMap<Node,Node> excludedNodes,
long blocksize,
List<String> favoredNodes)
Choose target datanodes according to the replication policy.
|
DatanodeDescriptor[] |
BlockPlacementPolicyDefault.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
boolean returnChosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize) |
abstract DatanodeDescriptor[] |
BlockPlacementPolicy.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
boolean returnChosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize)
choose numOfReplicas data nodes for writer
to re-replicate a block with size blocksize
If not, return as many as we can.
|
DatanodeDescriptor[] |
BlockPlacementPolicyDefault.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
long blocksize) |
BlockInfoUnderConstruction |
BlockInfo.convertToBlockUnderConstruction(HdfsServerConstants.BlockUCState s,
DatanodeDescriptor[] targets)
Convert a complete block to an under construction block.
|
protected boolean |
BlockPlacementPolicyDefault.isGoodTarget(DatanodeDescriptor node,
long blockSize,
int maxTargetPerRack,
boolean considerLoad,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes)
Determine if a node is a good target.
|
BlockInfo |
BlockInfo.listInsert(BlockInfo head,
DatanodeDescriptor dn)
Insert this block into the head of the list of blocks
related to the specified DatanodeDescriptor.
|
BlockInfo |
BlockInfo.listRemove(BlockInfo head,
DatanodeDescriptor dn)
Remove this block from the list of blocks
related to the specified DatanodeDescriptor.
|
BlockInfo |
BlockInfo.moveBlockToHead(BlockInfo head,
DatanodeDescriptor dn,
int curIndex,
int headIndex)
Remove this block from the list of blocks related to the specified
DatanodeDescriptor.
|
boolean |
BlockInfo.removeNode(DatanodeDescriptor node)
Remove data-node from the block.
|
void |
BlockManager.removeStoredBlock(Block block,
DatanodeDescriptor node)
Modify (block-->datanode) map.
|
void |
BlockInfoUnderConstruction.setExpectedLocations(DatanodeDescriptor[] targets)
Set expected locations
|
BlockInfoUnderConstruction |
MutableBlockCollection.setLastBlock(BlockInfo lastBlock,
DatanodeDescriptor[] locations)
Convert the last block of the collection to an under-construction block
and set the locations.
|
Modifier and Type | Method and Description |
---|---|
void |
BlockPlacementPolicy.adjustSetsWithChosenReplica(Map<String,List<DatanodeDescriptor>> rackMap,
List<DatanodeDescriptor> moreThanOne,
List<DatanodeDescriptor> exactlyOne,
DatanodeInfo cur)
Adjust rackmap, moreThanOne, and exactlyOne after removing replica on cur.
|
void |
BlockPlacementPolicy.adjustSetsWithChosenReplica(Map<String,List<DatanodeDescriptor>> rackMap,
List<DatanodeDescriptor> moreThanOne,
List<DatanodeDescriptor> exactlyOne,
DatanodeInfo cur)
Adjust rackmap, moreThanOne, and exactlyOne after removing replica on cur.
|
void |
BlockPlacementPolicy.adjustSetsWithChosenReplica(Map<String,List<DatanodeDescriptor>> rackMap,
List<DatanodeDescriptor> moreThanOne,
List<DatanodeDescriptor> exactlyOne,
DatanodeInfo cur)
Adjust rackmap, moreThanOne, and exactlyOne after removing replica on cur.
|
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes)
choose local node of localMachine as the target.
|
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyDefault.chooseRandom(int numOfReplicas,
String nodes,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseRandom(String nodes,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyDefault.chooseRemoteRack(int numOfReplicas,
DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxReplicasPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyWithNodeGroup.chooseRemoteRack(int numOfReplicas,
DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxReplicasPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseReplicaToDelete(BlockCollection bc,
Block block,
short replicationFactor,
Collection<DatanodeDescriptor> first,
Collection<DatanodeDescriptor> second) |
DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseReplicaToDelete(BlockCollection bc,
Block block,
short replicationFactor,
Collection<DatanodeDescriptor> first,
Collection<DatanodeDescriptor> second) |
abstract DatanodeDescriptor |
BlockPlacementPolicy.chooseReplicaToDelete(BlockCollection srcBC,
Block block,
short replicationFactor,
Collection<DatanodeDescriptor> existingReplicas,
Collection<DatanodeDescriptor> moreExistingReplicas)
Decide whether deleting the specified replica of the block still makes
the block conform to the configured block placement policy.
|
abstract DatanodeDescriptor |
BlockPlacementPolicy.chooseReplicaToDelete(BlockCollection srcBC,
Block block,
short replicationFactor,
Collection<DatanodeDescriptor> existingReplicas,
Collection<DatanodeDescriptor> moreExistingReplicas)
Decide whether deleting the specified replica of the block still makes
the block conform to the configured block placement policy.
|
DatanodeDescriptor[] |
BlockPlacementPolicyDefault.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
boolean returnChosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize) |
abstract DatanodeDescriptor[] |
BlockPlacementPolicy.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
boolean returnChosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize)
choose numOfReplicas data nodes for writer
to re-replicate a block with size blocksize
If not, return as many as we can.
|
DatanodeDescriptor[] |
BlockPlacementPolicyDefault.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
long blocksize) |
void |
DatanodeManager.fetchDatanodes(List<DatanodeDescriptor> live,
List<DatanodeDescriptor> dead,
boolean removeDecommissionNode)
Fetch live and dead datanodes.
|
void |
DatanodeManager.fetchDatanodes(List<DatanodeDescriptor> live,
List<DatanodeDescriptor> dead,
boolean removeDecommissionNode)
Fetch live and dead datanodes.
|
protected boolean |
BlockPlacementPolicyDefault.isGoodTarget(DatanodeDescriptor node,
long blockSize,
int maxTargetPerRack,
boolean considerLoad,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes)
Determine if a node is a good target.
|
protected Iterator<DatanodeDescriptor> |
BlockPlacementPolicyDefault.pickupReplicaSet(Collection<DatanodeDescriptor> first,
Collection<DatanodeDescriptor> second)
Pick up replica node set for deleting replica as over-replicated.
|
protected Iterator<DatanodeDescriptor> |
BlockPlacementPolicyDefault.pickupReplicaSet(Collection<DatanodeDescriptor> first,
Collection<DatanodeDescriptor> second)
Pick up replica node set for deleting replica as over-replicated.
|
Iterator<DatanodeDescriptor> |
BlockPlacementPolicyWithNodeGroup.pickupReplicaSet(Collection<DatanodeDescriptor> first,
Collection<DatanodeDescriptor> second)
Pick up replica node set for deleting replica as over-replicated.
|
Iterator<DatanodeDescriptor> |
BlockPlacementPolicyWithNodeGroup.pickupReplicaSet(Collection<DatanodeDescriptor> first,
Collection<DatanodeDescriptor> second)
Pick up replica node set for deleting replica as over-replicated.
|
void |
BlockPlacementPolicy.splitNodesWithRack(Collection<DatanodeDescriptor> dataNodes,
Map<String,List<DatanodeDescriptor>> rackMap,
List<DatanodeDescriptor> moreThanOne,
List<DatanodeDescriptor> exactlyOne)
Split data nodes into two sets, one set includes nodes on rack with
more than one replica, the other set contains the remaining nodes.
|
void |
BlockPlacementPolicy.splitNodesWithRack(Collection<DatanodeDescriptor> dataNodes,
Map<String,List<DatanodeDescriptor>> rackMap,
List<DatanodeDescriptor> moreThanOne,
List<DatanodeDescriptor> exactlyOne)
Split data nodes into two sets, one set includes nodes on rack with
more than one replica, the other set contains the remaining nodes.
|
void |
BlockPlacementPolicy.splitNodesWithRack(Collection<DatanodeDescriptor> dataNodes,
Map<String,List<DatanodeDescriptor>> rackMap,
List<DatanodeDescriptor> moreThanOne,
List<DatanodeDescriptor> exactlyOne)
Split data nodes into two sets, one set includes nodes on rack with
more than one replica, the other set contains the remaining nodes.
|
void |
BlockPlacementPolicy.splitNodesWithRack(Collection<DatanodeDescriptor> dataNodes,
Map<String,List<DatanodeDescriptor>> rackMap,
List<DatanodeDescriptor> moreThanOne,
List<DatanodeDescriptor> exactlyOne)
Split data nodes into two sets, one set includes nodes on rack with
more than one replica, the other set contains the remaining nodes.
|
Constructor and Description |
---|
BlockInfoUnderConstruction(Block blk,
int replication,
HdfsServerConstants.BlockUCState state,
DatanodeDescriptor[] targets)
Create a block that is currently being constructed.
|
Modifier and Type | Method and Description |
---|---|
static void |
JspHelper.sortNodeList(List<DatanodeDescriptor> nodes,
String field,
String order) |
Modifier and Type | Method and Description |
---|---|
DatanodeDescriptor |
INodeFileUnderConstruction.getClientNode() |
Modifier and Type | Method and Description |
---|---|
BlockInfoUnderConstruction |
INodeFileUnderConstruction.setLastBlock(BlockInfo lastBlock,
DatanodeDescriptor[] targets)
Convert the last block of the file to an under-construction block.
|
INodeFileUnderConstruction |
INodeFile.toUnderConstruction(String clientName,
String clientMachine,
DatanodeDescriptor clientNode)
Convert this file to an
INodeFileUnderConstruction . |
Constructor and Description |
---|
INodeFileUnderConstruction(INodeFile that,
String clientName,
String clientMachine,
DatanodeDescriptor clientNode) |
Modifier and Type | Method and Description |
---|---|
INodeFileUnderConstructionWithSnapshot |
INodeFileWithSnapshot.toUnderConstruction(String clientName,
String clientMachine,
DatanodeDescriptor clientNode) |
Copyright © 2013 Apache Software Foundation. All rights reserved.