Bidirectional Data Import to Hive Using SQOOP


Authors : MD.Sirajul Huque,D.Naveen Reddy,G.Uttej,K.Rajesh Kumar,K.Ranjith Reddy

Volume/Issue : Volume 2 - 2017, Issue 4 - April

Google Scholar : https://goo.gl/qc0MEh

Scribd : https://goo.gl/61S8Lr

Thomson Reuters ResearcherID : https://goo.gl/3bkzwv

Using Hadoop for analytics and data processing requires loading data into clusters and processing it in conjunction with other data that often resides in production databases across the enterprise. Loading bulk data into Hadoop from production systems or accessing it from map reduce applications running on large clusters can be a challenging task. Users must consider details like ensuring consistency of data, the consumption of production system resources, data preparation for provisioning downstream pipeline. Transferring data using scripts is inefficient and time consuming. Directly accessing data residing on external systems from within the map reduce applications complicates applications and exposes the production system to the risk of excessive load originating from cluster nodes. This is where Apache Sqoop fits in. Apache Sqoop is currently undergoing incubation at Apache Software Foundation. More information on this project can be found at http://incubator.apache.org/sqo p. Sqoop allows easy import and export of data from structured data stores such as relational databases, enterprise data warehouses, and NoSQL systems. Using Sqoop, you can provision the data from external system on to HDFS, and populate tables in Hive and HBase.

Keywords : apache hadoop 1.2.1,apache hive, sqoop, mysql, hive.

CALL FOR PAPERS


Paper Submission Last Date
31 - December - 2020

Paper Review Notification
In 1-2 Days

Paper Publishing
In 2-3 Days

Never miss an update from Papermashup

Get notified about the latest tutorials and downloads.

Subscribe by Email

Get alerts directly into your inbox after each post and stay updated.
Subscribe
OR

Subscribe by RSS

Add our RSS to your feedreader to get regular updates from us.
Subscribe