Hadoop Ubuntu Packages

Problems starting namenode with hadoop-conf-pseudo

Asked by James Page

I've installed the hadoop-config-pseudo package.

It looks like the namenode is failing to start and the below is from the namenode log. I've checked, and the directory does exist. Anything else I should check? The system is running precise.

2012-03-15 19:02:43,622 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory /var/lib/hadoop/cache/hadoop/dfs/name does not exist.
2012-03-15 19:02:43,624 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /var/lib/hadoop/cache/hadoop/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:288)
        at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:97)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:386)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:360)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)
2012-03-15 19:02:43,625 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /var/lib/hadoop/cache/hadoop/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:288)
        at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:97)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:386)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:360)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)

Question information

Language:
English Edit question
Status:
Solved
For:
Hadoop Ubuntu Packages Edit question
Assignee:
No assignee Edit question
Solved by:
James Page
Solved:
Last query:
Last reply:
Revision history for this message
James Page (james-page) said : 		#1

The hadoop-conf-pseudo package is a bit racey in that it depends on hadoop-namenode - however if the namenode has not finished deciding it can't start by the time hadoop-conf-pseudo formats the namenode filesystem the format fails and then the namenode won't start anyway.

Running:

sudo su hdfs -c "hadoop namenode -format"

Without the namenode running should workaround the issue