Sqoop import Mysql to hive import Partitions

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Sqoop import Mysql to hive import Partitions

Arun Pandian
How to create sqoop import  mysql to hive partition , this is my code



sqoop import --connect jdbc:mysql://localhost/arun  --table account --username root --password hadoop -m 1  --hive-partition-key "name" --hive-partition-value "arun" --hive-database company  --create-hive-table  --hive-table account5  --target-dir /user/sqooptest21

mysql database is arun,
table is account
select * from account;
+----+-------+------+------------+---------+
| id | name  | age  | joindate   | namess  |
+----+-------+------+------------+---------+
|  1 | Arun  |   23 | 29-01-2016 | super   |
|  2 | Mani  |   22 | 30-01-2016 | superb  |
|  3 | Sana  |   25 | 20-01-2016 | superbb |
|  4 | Vicky |   24 | 02-02-2016 | supervb |
|  5 | Vis   |   23 | 30-05-2016 | super   |
+----+-------+------+------------+---------+


i need clear explanation for sqoop import mysql to hive



Thanks & Regards
Arunpandian.l
Associate Developer
Reply | Threaded
Open this post in threaded view
|

Re: Sqoop import Mysql to hive import Partitions

Ted Yu-3
The question is better suited for sqoop mailing list. 

Meanwhile, you can search for past threads / JIRAs on this subject. 
e.g.

On Feb 2, 2016, at 5:28 AM, Arun Pandian <[hidden email]> wrote:

How to create sqoop import  mysql to hive partition , this is my code



sqoop import --connect jdbc:mysql://localhost/arun  --table account --username root --password hadoop -m 1  --hive-partition-key "name" --hive-partition-value "arun" --hive-database company  --create-hive-table  --hive-table account5  --target-dir /user/sqooptest21

mysql database is arun,
table is account
select * from account;
+----+-------+------+------------+---------+
| id | name  | age  | joindate   | namess  |
+----+-------+------+------------+---------+
|  1 | Arun  |   23 | 29-01-2016 | super   |
|  2 | Mani  |   22 | 30-01-2016 | superb  |
|  3 | Sana  |   25 | 20-01-2016 | superbb |
|  4 | Vicky |   24 | 02-02-2016 | supervb |
|  5 | Vis   |   23 | 30-05-2016 | super   |
+----+-------+------+------------+---------+


i need clear explanation for sqoop import mysql to hive



Thanks & Regards
Arunpandian.l
Associate Developer
Reply | Threaded
Open this post in threaded view
|

RE: Sqoop import Mysql to hive import Partitions

Chinnappan Chandrasekaran

 

sqoop import --table tablename\

--connect jdbc:mysql:servername\dbname

--username dbuser --password pw \

--fields-terminated-by "\t"

-- hive-partition-key 

-- hive-import

 

 

/--hive-partition-key is Name of a hive field to partition are sharded on

 

 

Thanks & Regards

Chandrasekaran

Technical Consultant

Business Solutions Group

 

Jardine OneSolution (2001) Pte Ltd

Tel +65 6551 9608 | Mobile +65 8138 4761 | Email [hidden email]

55, Ubi Avenue 1 #03-15,Singapore 408935

 

Description: 150828 - JOS email signature_grey-02

 

From: Ted Yu [mailto:[hidden email]]
Sent: Tuesday, 2 February, 2016 10:38 PM
To: Arun Pandian
Cc: [hidden email]
Subject: Re: Sqoop import Mysql to hive import Partitions

 

The question is better suited for sqoop mailing list. 

 

Meanwhile, you can search for past threads / JIRAs on this subject. 

e.g.

<a href="http://search-hadoop.com/m/C63T91ojE6B14UByZ&amp;subj=sqoop&#43;1&#43;4&#43;4&#43;hive&#43;import&#43;error">http://search-hadoop.com/m/C63T91ojE6B14UByZ&subj=sqoop+1+4+4+hive+import+error


On Feb 2, 2016, at 5:28 AM, Arun Pandian <[hidden email]> wrote:

How to create sqoop import  mysql to hive partition , this is my code

 

sqoop import --connect jdbc:mysql://localhost/arun  --table account --username root --password hadoop -m 1  --hive-partition-key "name" --hive-partition-value "arun" --hive-database company  --create-hive-table  --hive-table account5  --target-dir /user/sqooptest21

mysql database is arun,

table is account
select * from account;
+----+-------+------+------------+---------+
| id | name  | age  | joindate   | namess  |
+----+-------+------+------------+---------+
|  1 | Arun  |   23 | 29-01-2016 | super   |
|  2 | Mani  |   22 | 30-01-2016 | superb  |
|  3 | Sana  |   25 | 20-01-2016 | superbb |
|  4 | Vicky |   24 | 02-02-2016 | supervb |
|  5 | Vis   |   23 | 30-05-2016 | super   |
+----+-------+------+------------+---------+

i need clear explanation for sqoop import mysql to hive




Thanks & Regards

Arunpandian.l

Associate Developer


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

For general enquiries, please contact us at JOS Enquiry Email: [hidden email] Hotline: (+65) 6551 9611

For JOS Support, please contact us at JOS Services Email: [hidden email] Hotline: (+65) 6484 2302

 

A member of the Jardine Matheson Group, Jardine OneSolution is one of Asia’s leading providers of integrated IT services and solutions with offices in Singapore, Malaysia, Hong Kong and China. Find out more about JOS at www.jos.com


Confidentiality Notice and Disclaimer:
This email (including any attachment to it) is confidential and intended only for the use of the individual or entity named above and may contain information that is privileged. If you are not the intended recipient, you are notified that any dissemination, distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify us immediately by return email or telephone and destroy the original message (including any attachment to it). Thank you.

 

 


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
Confidentiality Notice and Disclaimer: This email (including any attachment to it) is confidential and intended only for the use of the individual or entity named above and may contain information that is privileged. If you are not the intended recipient, you are notified that any dissemination, distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify us immediately by return email or telephone and destroy the original message (including any attachment to it). Thank you.
______________________________________________________________________
Reply | Threaded
Open this post in threaded view
|

Re: Sqoop import Mysql to hive import Partitions

sivanesan sumtwo

sqoop import

--connect jdbc:mysql://servername\dbname

--username hiveuser

--query 'select id,name,salary,dept,sol  from sample5 WHERE status="Shutting" AND  $CONDITIONS'

--hive-import

--hive-partition-key status 

--hive-partition-value 'Shutting'

--fields-terminated-by "," --lines-terminated-by "\n"

--target-dir ts1234 --m 1 -P

 

/--hive-partition-key is Name of a hive field to partition are sharded on

/--hive-partition-value is Value of the hive field to partition



Thanks & Regards
L.Sivanesan.




Thanks & Regards
L.Sivanesan.

On Fri, Feb 5, 2016 at 7:23 AM, Chinnappan Chandrasekaran <[hidden email]> wrote:

 

sqoop import --table tablename\

--connect jdbc:mysql:servername\dbname

--username dbuser --password pw \

--fields-terminated-by "\t"

-- hive-partition-key 

-- hive-import

 

 

/--hive-partition-key is Name of a hive field to partition are sharded on

 

 

Thanks & Regards

Chandrasekaran

Technical Consultant

Business Solutions Group

 

Jardine OneSolution (2001) Pte Ltd

Tel +65 6551 9608 | Mobile +65 8138 4761 | Email [hidden email]

55, Ubi Avenue 1 #03-15,Singapore 408935

 

Description: 150828 - JOS email signature_grey-02

 

From: Ted Yu [mailto:[hidden email]]
Sent: Tuesday, 2 February, 2016 10:38 PM
To: Arun Pandian
Cc: [hidden email]
Subject: Re: Sqoop import Mysql to hive import Partitions

 

The question is better suited for sqoop mailing list. 

 

Meanwhile, you can search for past threads / JIRAs on this subject. 

e.g.


On Feb 2, 2016, at 5:28 AM, Arun Pandian <[hidden email]> wrote:

How to create sqoop import  mysql to hive partition , this is my code

 

sqoop import --connect jdbc:mysql://localhost/arun  --table account --username root --password hadoop -m 1  --hive-partition-key "name" --hive-partition-value "arun" --hive-database company  --create-hive-table  --hive-table account5  --target-dir /user/sqooptest21

mysql database is arun,

table is account
select * from account;
+----+-------+------+------------+---------+
| id | name  | age  | joindate   | namess  |
+----+-------+------+------------+---------+
|  1 | Arun  |   23 | 29-01-2016 | super   |
|  2 | Mani  |   22 | 30-01-2016 | superb  |
|  3 | Sana  |   25 | 20-01-2016 | superbb |
|  4 | Vicky |   24 | 02-02-2016 | supervb |
|  5 | Vis   |   23 | 30-05-2016 | super   |
+----+-------+------+------------+---------+

i need clear explanation for sqoop import mysql to hive




Thanks & Regards

Arunpandian.l

Associate Developer


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

For general enquiries, please contact us at JOS Enquiry Email: [hidden email] Hotline: (+65) 6551 9611

For JOS Support, please contact us at JOS Services Email: [hidden email] Hotline: (+65) 6484 2302

 

A member of the Jardine Matheson Group, Jardine OneSolution is one of Asia’s leading providers of integrated IT services and solutions with offices in Singapore, Malaysia, Hong Kong and China. Find out more about JOS at www.jos.com


Confidentiality Notice and Disclaimer:
This email (including any attachment to it) is confidential and intended only for the use of the individual or entity named above and may contain information that is privileged. If you are not the intended recipient, you are notified that any dissemination, distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify us immediately by return email or telephone and destroy the original message (including any attachment to it). Thank you.

 

 


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
Confidentiality Notice and Disclaimer: This email (including any attachment to it) is confidential and intended only for the use of the individual or entity named above and may contain information that is privileged. If you are not the intended recipient, you are notified that any dissemination, distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify us immediately by return email or telephone and destroy the original message (including any attachment to it). Thank you.
______________________________________________________________________