HDPV2 - Services Review

A review of all Services Running

This blog discusses all the services / java process running / configured till now as part of Hadoop V2 Configuration.

First let me list out all the processes and then

[root@nn ~]# for i in $(cat /tmp/all_hosts) ; do ssh ${i} 'hostname; jps | grep -vi jps; echo' ;  done;    

nn.novalocal
4230 NameNode
4471 DFSZKFailoverController

Namenode - Runs 2 Services
1. Namenode itself
2. ZKFC Failover Controlller

snn.novalocal
3192 NameNode
3514 DFSZKFailoverController

StandbyNamenode also runs same services as Namenode

rm.novalocal
3297 Master
2870 ResourceManager
3211 JobHistoryServer

ResourceManager Runs 3 services
1. Master - This is Spark's Master (running as spark)
2. ResourceManager - Yarn Framework's ResourceManager (running as yarn)
3. JobHistoryServer - Map Reduce Job History Server (running as mapred)

d1n.novalocal
2529 QuorumPeerMain
2819 NodeManager
2598 DataNode
2986 Worker
2700 JournalNode
All 3 DataNodes are running below 5 Services
1. QuorumPeerMain - Zookeeper's Qurom service (running as zkfc)
2. NodeManager - Yarn - Node manager (running as yarn)
3. DataNode - Hdfs data Node (running as hdfs)
4. Worker - Spark worker process (running as spark)
5. JournalNode - Journal Node for HA Configuration (running as hdfs)
d2n.novalocal
2465 QuorumPeerMain
2738 NodeManager
2517 DataNode
2619 JournalNode
2907 Worker

d3n.novalocal
2513 DataNode
2901 Worker
2615 JournalNode
2730 NodeManager
2461 QuorumPeerMain