The Author Online Book Forums are Moving

The Author Online Book Forums will soon redirect to Manning's liveBook and liveVideo. All book forum content will migrate to liveBook's discussion forum and all video forum content will migrate to liveVideo. Log in to liveBook or liveVideo with your Manning credentials to join the discussion!

Thank you for your engagement in the AoF over the years! We look forward to offering you a more enhanced forum experience.

davidlaxer (4) [Avatar] Offline
#1
Hi,

The command 'hdfs namenode -format' doesn't run on my Mac running Maverics 10.9.3.

There's a missing class ...

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/security/RefreshUserToGroupMappingsProtocol

I tried building Hadoop 2.4.0 from sources ...

[WARNING] Failed to create parent directories for tracking file /Users/davidlaxer/.m2/repository/org/apache/maven/plugins/maven-war-plugin/2.1/maven-war-plugin-2.1.pom.lastUpdated
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [03:17 min]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [ 34.385 s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [ 18.529 s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [ 0.727 s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [01:30 min]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [ 24.267 s]
[INFO] Apache Hadoop MiniKDC ............................. SUCCESS [01:34 min]
[INFO] Apache Hadoop Auth ................................ SUCCESS [01:18 min]
[INFO] Apache Hadoop Auth Examples ....................... FAILURE [ 0.818 s]
[INFO] Apache Hadoop Common .............................. SKIPPED
[INFO] Apache Hadoop NFS ................................. SKIPPED
[INFO] Apache Hadoop Common Project ...................... SKIPPED
[INFO] Apache Hadoop HDFS ................................ SKIPPED
[INFO] Apache Hadoop HttpFS .............................. SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SKIPPED
[INFO] Apache Hadoop HDFS-NFS ............................ SKIPPED
[INFO] Apache Hadoop HDFS Project ........................ SKIPPED
[INFO] hadoop-yarn ....................................... SKIPPED
[INFO] hadoop-yarn-api ................................... SKIPPED
[INFO] hadoop-yarn-common ................................ SKIPPED
[INFO] hadoop-yarn-server ................................ SKIPPED
[INFO] hadoop-yarn-server-common ......................... SKIPPED
[INFO] hadoop-yarn-server-nodemanager .................... SKIPPED
[INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
[INFO] hadoop-yarn-server-applicationhistoryservice ...... SKIPPED
[INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
[INFO] hadoop-yarn-server-tests .......................... SKIPPED
[INFO] hadoop-yarn-client ................................ SKIPPED
[INFO] hadoop-yarn-applications .......................... SKIPPED
[INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SKIPPED
[INFO] hadoop-yarn-site .................................. SKIPPED
[INFO] hadoop-yarn-project ............................... SKIPPED
[INFO] hadoop-mapreduce-client ........................... SKIPPED
[INFO] hadoop-mapreduce-client-core ...................... SKIPPED
[INFO] hadoop-mapreduce-client-common .................... SKIPPED
[INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
[INFO] hadoop-mapreduce-client-app ....................... SKIPPED
[INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
[INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
[INFO] hadoop-mapreduce-client-hs-plugins ................ SKIPPED
[INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
[INFO] hadoop-mapreduce .................................. SKIPPED
[INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
[INFO] Apache Hadoop Distributed Copy .................... SKIPPED
[INFO] Apache Hadoop Archives ............................ SKIPPED
[INFO] Apache Hadoop Rumen ............................... SKIPPED
[INFO] Apache Hadoop Gridmix ............................. SKIPPED
[INFO] Apache Hadoop Data Join ........................... SKIPPED
[INFO] Apache Hadoop Extras .............................. SKIPPED
[INFO] Apache Hadoop Pipes ............................... SKIPPED
[INFO] Apache Hadoop OpenStack support ................... SKIPPED
[INFO] Apache Hadoop Client .............................. SKIPPED
[INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
[INFO] Apache Hadoop Scheduler Load Simulator ............ SKIPPED
[INFO] Apache Hadoop Tools Dist .......................... SKIPPED
[INFO] Apache Hadoop Tools ............................... SKIPPED
[INFO] Apache Hadoop Distribution ........................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 09:27 min
[INFO] Finished at: 2014-05-22T17:04:14-08:00
[INFO] Final Memory: 67M/164M
[INFO] ------------------------------------------------------------------------
[ERROR] Plugin org.apache.maven.plugins:maven-war-plugin:2.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-war-plugin:jar:2.1: Could not transfer artifact org.apache.maven.plugins:maven-war-plugin:pom:2.1 from/to central (http://repo.maven.apache.org/maven2): Specified destination directory cannot be created: /Users/davidlaxer/.m2/r ...

Any ideas?

Thanks in advance.

-Dave
davidlaxer (4) [Avatar] Offline
#2
Re: HDFS Error
MacBook-Pro:common $ pwd
/usr/local/hadoop/share/hadoop/common

MacBook-Pro:common $ jar tf hadoop-common-2.4.0.jar | grep security.Refresh
org/apache/hadoop/security/RefreshUserMappingsProtocol.class
Alex Holmes (47) [Avatar] Offline
#3
Re: HDFS Error
There are some OSX related installation instructions here, but I've never tried them:

http://wiki.apache.org/hadoop/Running_Hadoop_On_OS_X_10.5_64-bit_(Single-Node_Cluster)

I know this doesn't solve the problem of your local Hadoop setup, but to start out it may be easiest if you use one of the VM's that has Hadoop pre-installed from Cloudera, Hortonworks or one of the other distributions.