Error: Could not find or load main class when Running Hadoop

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Error: Could not find or load main class when Running Hadoop

Or Raz

I am using Hadoop 2.9.1 standalone (the folder I am using is after successful compilation of the source code) and whenever I run a Hadoop command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of Hadoop)

$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep input output 'dfs[a-z.]+'

I get this error :

Error: Could not find or load main class org.apache.hadoop.util.RunJar

At first, I was thinking that I am missing some environment variables but are they necessary for the standalone case? Why did I get this error? (I even replaced with another example, I believe it is a classpath problem)

hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath

/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar

ITD
Reply | Threaded
Open this post in threaded view
|

Re: Error: Could not find or load main class when Running Hadoop

ITD
Class names are case-sensitive in Java, so try

$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar Grep input output 'dfs[a-z.]+'


Воскресенье, 14 октября 2018, 18:04 +03:00 от Or Raz <[hidden email]>:

I am using Hadoop 2.9.1 standalone (the folder I am using is after successful compilation of the source code) and whenever I run a Hadoop command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of Hadoop)

$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep input output 'dfs[a-z.]+'

I get this error :

Error: Could not find or load main class org.apache.hadoop.util.RunJar

At first, I was thinking that I am missing some environment variables but are they necessary for the standalone case? Why did I get this error? (I even replaced with another example, I believe it is a classpath problem)

hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath

/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar





Reply | Threaded
Open this post in threaded view
|

Re: Error: Could not find or load main class when Running Hadoop

Or Raz
I tried, I still get the same error.

On 2018/10/14 15:18:57, ITD <[hidden email]> wrote:

> Class names are case-sensitive in Java, so try
>
> $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar Grep input output 'dfs[a-z.]+'
>
> >Воскресенье, 14 октября 2018, 18:04 +03:00 от Or Raz <[hidden email]>:
> >
> >I am using Hadoop 2.9.1 standalone (the folder I am using is after successful compilation of the source code) and whenever I run a Hadoop command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of Hadoop)
> >$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep input output 'dfs[a-z.]+'
> >I get this error :
> >>Error: Could not find or load main class org.apache.hadoop.util.RunJar
> >At first, I was thinking that I am missing some environment variables but are they necessary for the standalone case? Why did I get this error? (I even replaced with another example, I believe it is a classpath problem)
> >
> >hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath
> >/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

ITD
Reply | Threaded
Open this post in threaded view
|

Re[2]: Error: Could not find or load main class when Running Hadoop

ITD
This should work:

$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar org.apache.hadoop.examples.Grep input output 'dfs[a-z.]+'


Воскресенье, 14 октября 2018, 18:26 +03:00 от [hidden email] <[hidden email]>:

I tried, I still get the same error.

On 2018/10/14 15:18:57, ITD <[hidden email]> wrote:
> Class names are case-sensitive in Java, so try
>
> $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar Grep input output 'dfs[a-z.]+'
>
> >Воскресенье, 14 октября 2018, 18:04 +03:00 от Or Raz <[hidden email]>:
> >
> >I am using Hadoop 2.9.1 standalone (the folder I am using is after successful compilation of the source code) and whenever I run a Hadoop command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of Hadoop)
> >$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep input output 'dfs[a-z.]+'
> >I get this error :
> >>Error: Could not find or load main class org.apache.hadoop.util.RunJar
> >At first, I was thinking that I am missing some environment variables but are they necessary for the standalone case? Why did I get this error? (I even replaced with another example, I believe it is a classpath problem)
> >
> >hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath
> >/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]





ITD
Reply | Threaded
Open this post in threaded view
|

Re[2]: Error: Could not find or load main class when Running Hadoop

ITD
In reply to this post by Or Raz
Try and check your classpath, it looks a bit strange:

/usp/hdfs: what's this?

Working classpath is:
/usr/local/hadoop/etc/hadoop:\
/usr/local/hadoop/share/hadoop/common/lib/*:\
/usr/local/hadoop/share/hadoop/common/*:\
/usr/local/hadoop/share/hadoop/hdfs:\
/usr/local/hadoop/share/hadoop/hdfs/lib/*:\
/usr/local/hadoop/share/hadoop/hdfs/*:\
/usr/local/hadoop/share/hadoop/mapreduce/lib/*:\
/usr/local/hadoop/share/hadoop/mapreduce/*:\
/usr/local/hadoop/share/hadoop/yarn:\
/usr/local/hadoop/share/hadoop/yarn/lib/*:\
/usr/local/hadoop/share/hadoop/yarn/*



Воскресенье, 14 октября 2018, 18:26 +03:00 от [hidden email] <[hidden email]>:

I tried, I still get the same error.

On 2018/10/14 15:18:57, ITD <[hidden email]> wrote:
> Class names are case-sensitive in Java, so try
>
> $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar Grep input output 'dfs[a-z.]+'
>
> >Воскресенье, 14 октября 2018, 18:04 +03:00 от Or Raz <[hidden email]>:
> >
> >I am using Hadoop 2.9.1 standalone (the folder I am using is after successful compilation of the source code) and whenever I run a Hadoop command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of Hadoop)
> >$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep input output 'dfs[a-z.]+'
> >I get this error :
> >>Error: Could not find or load main class org.apache.hadoop.util.RunJar
> >At first, I was thinking that I am missing some environment variables but are they necessary for the standalone case? Why did I get this error? (I even replaced with another example, I believe it is a classpath problem)
> >
> >hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath
> >/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]





Reply | Threaded
Open this post in threaded view
|

Re: Re[2]: Error: Could not find or load main class when Running Hadoop

Or Raz
So how should I configure it?
I looked over here https://stackoverflow.com/questions/28260653/where-is-the-classpath-set-for-hadoop to Siva's answer and the output of ($HADOOP+HOME/bin/hadoop classpath) is :

hadoop2@master:~$ $HADOOP_HOME/bin/hadoop classpath
/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/*:/usr/local/hadoop-2.9.1/contrib/capacity-scheduler/*.jar

But the problem stayed the same.
How should I configure the classpath to be what you suggested?

P.S
hadoop2@master:~$ $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar org.apache.hadoop.examples.Grep input output 'dfs[a-z.]+'
Error: Could not find or load main class org.apache.hadoop.util.RunJar


On 2018/10/14 15:55:06, ITD <[hidden email]> wrote:

> Try and check your classpath, it looks a bit strange:
>
> /usp/hdfs: what's this?
>
> Working classpath is:
> /usr/local/hadoop/etc/hadoop:\
> /usr/local/hadoop/share/hadoop/common/lib/*:\
> /usr/local/hadoop/share/hadoop/common/*:\
> /usr/local/hadoop/share/hadoop/hdfs:\
> /usr/local/hadoop/share/hadoop/hdfs/lib/*:\
> /usr/local/hadoop/share/hadoop/hdfs/*:\
> /usr/local/hadoop/share/hadoop/mapreduce/lib/*:\
> /usr/local/hadoop/share/hadoop/mapreduce/*:\
> /usr/local/hadoop/share/hadoop/yarn:\
> /usr/local/hadoop/share/hadoop/yarn/lib/*:\
> /usr/local/hadoop/share/hadoop/yarn/*
>
>
>
> >Воскресенье, 14 октября 2018, 18:26 +03:00 от [hidden email] <[hidden email]>:
> >
> >I tried, I still get the same error.
> >
> >On 2018/10/14 15:18:57, ITD < [hidden email] > wrote:
> >> Class names are case-sensitive in Java, so try
> >>
> >> $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar Grep input output 'dfs[a-z.]+'
> >>
> >> >Воскресенье, 14 октября 2018, 18:04 +03:00 от Or Raz < [hidden email] >:
> >> >
> >> >I am using Hadoop 2.9.1 standalone (the folder I am using is after successful compilation of the source code) and whenever I run a Hadoop command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of Hadoop)
> >> >$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep input output 'dfs[a-z.]+'
> >> >I get this error :
> >> >>Error: Could not find or load main class org.apache.hadoop.util.RunJar
> >> >At first, I was thinking that I am missing some environment variables but are they necessary for the standalone case? Why did I get this error? (I even replaced with another example, I believe it is a classpath problem)
> >> >
> >> >hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath
> >> >/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar
> >>
> >>
> >>
> >>
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail:  [hidden email]
> >For additional commands, e-mail:  [hidden email]
> >
>
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

ITD
Reply | Threaded
Open this post in threaded view
|

Re[4]: Error: Could not find or load main class when Running Hadoop

ITD
It should be possible to add directories to classpath using HADOOP_CLASSPATH, set in etc/hadoop/hadoop-env.sh.


Воскресенье, 14 октября 2018, 19:32 +03:00 от [hidden email] <[hidden email]>:

So how should I configure it?
I looked over here https://stackoverflow.com/questions/28260653/where-is-the-classpath-set-for-hadoop to Siva's answer and the output of ($HADOOP+HOME/bin/hadoop classpath) is :

hadoop2@master:~$ $HADOOP_HOME/bin/hadoop classpath
/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/*:/usr/local/hadoop-2.9.1/contrib/capacity-scheduler/*.jar

But the problem stayed the same.
How should I configure the classpath to be what you suggested?

P.S
hadoop2@master:~$ $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar org.apache.hadoop.examples.Grep input output 'dfs[a-z.]+'
Error: Could not find or load main class org.apache.hadoop.util.RunJar


On 2018/10/14 15:55:06, ITD <[hidden email]> wrote:
> Try and check your classpath, it looks a bit strange:
>
> /usp/hdfs: what's this?
>
> Working classpath is:
> /usr/local/hadoop/etc/hadoop:\
> /usr/local/hadoop/share/hadoop/common/lib/*:\
> /usr/local/hadoop/share/hadoop/common/*:\
> /usr/local/hadoop/share/hadoop/hdfs:\
> /usr/local/hadoop/share/hadoop/hdfs/lib/*:\
> /usr/local/hadoop/share/hadoop/hdfs/*:\
> /usr/local/hadoop/share/hadoop/mapreduce/lib/*:\
> /usr/local/hadoop/share/hadoop/mapreduce/*:\
> /usr/local/hadoop/share/hadoop/yarn:\
> /usr/local/hadoop/share/hadoop/yarn/lib/*:\
> /usr/local/hadoop/share/hadoop/yarn/*
>
>
>
> >Воскресенье, 14 октября 2018, 18:26 +03:00 от [hidden email] <[hidden email]>:
> >
> >I tried, I still get the same error.
> >
> >On 2018/10/14 15:18:57, ITD < [hidden email] > wrote:
> >> Class names are case-sensitive in Java, so try
> >>
> >> $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar Grep input output 'dfs[a-z.]+'
> >>
> >> >Воскресенье, 14 октября 2018, 18:04 +03:00 от Or Raz < [hidden email] >:
> >> >
> >> >I am using Hadoop 2.9.1 standalone (the folder I am using is after successful compilation of the source code) and whenever I run a Hadoop command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of Hadoop)
> >> >$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep input output 'dfs[a-z.]+'
> >> >I get this error :
> >> >>Error: Could not find or load main class org.apache.hadoop.util.RunJar
> >> >At first, I was thinking that I am missing some environment variables but are they necessary for the standalone case? Why did I get this error? (I even replaced with another example, I believe it is a classpath problem)
> >> >
> >> >hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath
> >> >/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar
> >>
> >>
> >>
> >>
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: [hidden email]
> >For additional commands, e-mail: [hidden email]
> >
>
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]





Reply | Threaded
Open this post in threaded view
|

Re: Re[4]: Error: Could not find or load main class when Running Hadoop

Or Raz
I have added the HADOOP_CLASSPATH variable in hadoop-env.sh (before it has been used in the file) and I still get the same error.
Have you got another idea?

On 2018/10/14 16:49:04, ITD <[hidden email]> wrote:

> It should be possible to add directories to classpath using HADOOP_CLASSPATH, set in etc/hadoop/hadoop-env.sh.
>
>
> >Воскресенье, 14 октября 2018, 19:32 +03:00 от [hidden email] <[hidden email]>:
> >
> >So how should I configure it?
> >I looked over here  https://stackoverflow.com/questions/28260653/where-is-the-classpath-set-for-hadoop to Siva's answer and the output of ($HADOOP+HOME/bin/hadoop classpath) is :
> >
> >hadoop2@master:~$ $HADOOP_HOME/bin/hadoop classpath
> >/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/*:/usr/local/hadoop-2.9.1/contrib/capacity-scheduler/*.jar
> >
> >But the problem stayed the same.
> >How should I configure the classpath to be what you suggested?
> >
> >P.S
> >hadoop2@master:~$ $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar org.apache.hadoop.examples.Grep input output 'dfs[a-z.]+'
> >Error: Could not find or load main class org.apache.hadoop.util.RunJar
> >
> >
> >On 2018/10/14 15:55:06, ITD < [hidden email] > wrote:
> >> Try and check your classpath, it looks a bit strange:
> >>
> >> /usp/hdfs: what's this?
> >>
> >> Working classpath is:
> >> /usr/local/hadoop/etc/hadoop:\
> >> /usr/local/hadoop/share/hadoop/common/lib/*:\
> >> /usr/local/hadoop/share/hadoop/common/*:\
> >> /usr/local/hadoop/share/hadoop/hdfs:\
> >> /usr/local/hadoop/share/hadoop/hdfs/lib/*:\
> >> /usr/local/hadoop/share/hadoop/hdfs/*:\
> >> /usr/local/hadoop/share/hadoop/mapreduce/lib/*:\
> >> /usr/local/hadoop/share/hadoop/mapreduce/*:\
> >> /usr/local/hadoop/share/hadoop/yarn:\
> >> /usr/local/hadoop/share/hadoop/yarn/lib/*:\
> >> /usr/local/hadoop/share/hadoop/yarn/*
> >>
> >>
> >>
> >> >Воскресенье, 14 октября 2018, 18:26 +03:00 от  [hidden email] < [hidden email] >:
> >> >
> >> >I tried, I still get the same error.
> >> >
> >> >On 2018/10/14 15:18:57, ITD <  [hidden email] > wrote:
> >> >> Class names are case-sensitive in Java, so try
> >> >>
> >> >> $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar Grep input output 'dfs[a-z.]+'
> >> >>
> >> >> >Воскресенье, 14 октября 2018, 18:04 +03:00 от Or Raz <  [hidden email] >:
> >> >> >
> >> >> >I am using Hadoop 2.9.1 standalone (the folder I am using is after successful compilation of the source code) and whenever I run a Hadoop command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of Hadoop)
> >> >> >$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep input output 'dfs[a-z.]+'
> >> >> >I get this error :
> >> >> >>Error: Could not find or load main class org.apache.hadoop.util.RunJar
> >> >> >At first, I was thinking that I am missing some environment variables but are they necessary for the standalone case? Why did I get this error? (I even replaced with another example, I believe it is a classpath problem)
> >> >> >
> >> >> >hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath
> >> >> >/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar
> >> >>
> >> >>
> >> >>
> >> >>
> >> >
> >> >---------------------------------------------------------------------
> >> >To unsubscribe, e-mail:  [hidden email]
> >> >For additional commands, e-mail:  [hidden email]
> >> >
> >>
> >>
> >>
> >>
> >>
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail:  [hidden email]
> >For additional commands, e-mail:  [hidden email]
> >
>
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Re[2]: Error: Could not find or load main class when Running Hadoop

Arpit Agarwal
In reply to this post by Or Raz
You don't need to configure the classpath if you just extracted the Hadoop tarball under /usr/local/.

The only required setting is JAVA_HOME, everything else should be inferred. Is it possible some directories got moved around after unpacking?


On 2018/10/14, 9:32 AM, "razo@" <post.bgu.ac.il [hidden email]> wrote:

    So how should I configure it?
    I looked over here https://stackoverflow.com/questions/28260653/where-is-the-classpath-set-for-hadoop to Siva's answer and the output of ($HADOOP+HOME/bin/hadoop classpath) is :
   
    hadoop2@master:~$ $HADOOP_HOME/bin/hadoop classpath
    /usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/*:/usr/local/hadoop-2.9.1/contrib/capacity-scheduler/*.jar
   
    But the problem stayed the same.
    How should I configure the classpath to be what you suggested?
   
    P.S
    hadoop2@master:~$ $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar org.apache.hadoop.examples.Grep input output 'dfs[a-z.]+'
    Error: Could not find or load main class org.apache.hadoop.util.RunJar
   
   
    On 2018/10/14 15:55:06, ITD <[hidden email]> wrote:
    > Try and check your classpath, it looks a bit strange:
    >
    > /usp/hdfs: what's this?
    >
    > Working classpath is:
    > /usr/local/hadoop/etc/hadoop:\
    > /usr/local/hadoop/share/hadoop/common/lib/*:\
    > /usr/local/hadoop/share/hadoop/common/*:\
    > /usr/local/hadoop/share/hadoop/hdfs:\
    > /usr/local/hadoop/share/hadoop/hdfs/lib/*:\
    > /usr/local/hadoop/share/hadoop/hdfs/*:\
    > /usr/local/hadoop/share/hadoop/mapreduce/lib/*:\
    > /usr/local/hadoop/share/hadoop/mapreduce/*:\
    > /usr/local/hadoop/share/hadoop/yarn:\
    > /usr/local/hadoop/share/hadoop/yarn/lib/*:\
    > /usr/local/hadoop/share/hadoop/yarn/*
    >
    >
    >
    > >Воскресенье, 14 октября 2018, 18:26 +03:00 от [hidden email] <[hidden email]>:
    > >
    > >I tried, I still get the same error.
    > >
    > >On 2018/10/14 15:18:57, ITD < [hidden email] > wrote:
    > >> Class names are case-sensitive in Java, so try
    > >>
    > >> $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar Grep input output 'dfs[a-z.]+'
    > >>
    > >> >Воскресенье, 14 октября 2018, 18:04 +03:00 от Or Raz < [hidden email] >:
    > >> >
    > >> >I am using Hadoop 2.9.1 standalone (the folder I am using is after successful compilation of the source code) and whenever I run a Hadoop command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of Hadoop)
    > >> >$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep input output 'dfs[a-z.]+'
    > >> >I get this error :
    > >> >>Error: Could not find or load main class org.apache.hadoop.util.RunJar
    > >> >At first, I was thinking that I am missing some environment variables but are they necessary for the standalone case? Why did I get this error? (I even replaced with another example, I believe it is a classpath problem)
    > >> >
    > >> >hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath
    > >> >/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar
    > >>
    > >>
    > >>
    > >>
    > >
    > >---------------------------------------------------------------------
    > >To unsubscribe, e-mail:  [hidden email]
    > >For additional commands, e-mail:  [hidden email]
    > >
    >
    >
    >
    >
    >
   
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: [hidden email]
    For additional commands, e-mail: [hidden email]
   
   


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Re[2]: Error: Could not find or load main class when Running Hadoop

Or Raz
The problem was with the compilation folder I got using maven command :

mvn clean package -Pdist -Psrc -DskipTests -Dtar
The right one is:

mvn clean install -Pdist -Dtar -DskipTests

P.S. even after I change the classpath of extracted folder from compiled source folder to the class path of the respected taraball the error stayed.

On 2018/10/14 18:24:56, Arpit Agarwal <[hidden email]> wrote:

> You don't need to configure the classpath if you just extracted the Hadoop tarball under /usr/local/.
>
> The only required setting is JAVA_HOME, everything else should be inferred. Is it possible some directories got moved around after unpacking?
>
>
> On 2018/10/14, 9:32 AM, "razo@" <post.bgu.ac.il [hidden email]> wrote:
>
>     So how should I configure it?
>     I looked over here https://stackoverflow.com/questions/28260653/where-is-the-classpath-set-for-hadoop to Siva's answer and the output of ($HADOOP+HOME/bin/hadoop classpath) is :
>    
>     hadoop2@master:~$ $HADOOP_HOME/bin/hadoop classpath
>     /usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/*:/usr/local/hadoop-2.9.1/contrib/capacity-scheduler/*.jar
>    
>     But the problem stayed the same.
>     How should I configure the classpath to be what you suggested?
>    
>     P.S
>     hadoop2@master:~$ $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar org.apache.hadoop.examples.Grep input output 'dfs[a-z.]+'
>     Error: Could not find or load main class org.apache.hadoop.util.RunJar
>    
>    
>     On 2018/10/14 15:55:06, ITD <[hidden email]> wrote:
>     > Try and check your classpath, it looks a bit strange:
>     >
>     > /usp/hdfs: what's this?
>     >
>     > Working classpath is:
>     > /usr/local/hadoop/etc/hadoop:\
>     > /usr/local/hadoop/share/hadoop/common/lib/*:\
>     > /usr/local/hadoop/share/hadoop/common/*:\
>     > /usr/local/hadoop/share/hadoop/hdfs:\
>     > /usr/local/hadoop/share/hadoop/hdfs/lib/*:\
>     > /usr/local/hadoop/share/hadoop/hdfs/*:\
>     > /usr/local/hadoop/share/hadoop/mapreduce/lib/*:\
>     > /usr/local/hadoop/share/hadoop/mapreduce/*:\
>     > /usr/local/hadoop/share/hadoop/yarn:\
>     > /usr/local/hadoop/share/hadoop/yarn/lib/*:\
>     > /usr/local/hadoop/share/hadoop/yarn/*
>     >
>     >
>     >
>     > >Воскресенье, 14 октября 2018, 18:26 +03:00 от [hidden email] <[hidden email]>:
>     > >
>     > >I tried, I still get the same error.
>     > >
>     > >On 2018/10/14 15:18:57, ITD < [hidden email] > wrote:
>     > >> Class names are case-sensitive in Java, so try
>     > >>
>     > >> $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar Grep input output 'dfs[a-z.]+'
>     > >>
>     > >> >Воскресенье, 14 октября 2018, 18:04 +03:00 от Or Raz < [hidden email] >:
>     > >> >
>     > >> >I am using Hadoop 2.9.1 standalone (the folder I am using is after successful compilation of the source code) and whenever I run a Hadoop command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of Hadoop)
>     > >> >$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep input output 'dfs[a-z.]+'
>     > >> >I get this error :
>     > >> >>Error: Could not find or load main class org.apache.hadoop.util.RunJar
>     > >> >At first, I was thinking that I am missing some environment variables but are they necessary for the standalone case? Why did I get this error? (I even replaced with another example, I believe it is a classpath problem)
>     > >> >
>     > >> >hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath
>     > >> >/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar
>     > >>
>     > >>
>     > >>
>     > >>
>     > >
>     > >---------------------------------------------------------------------
>     > >To unsubscribe, e-mail:  [hidden email]
>     > >For additional commands, e-mail:  [hidden email]
>     > >
>     >
>     >
>     >
>     >
>     >
>    
>     ---------------------------------------------------------------------
>     To unsubscribe, e-mail: [hidden email]
>     For additional commands, e-mail: [hidden email]
>    
>    
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]