HCatalog is a module in Apache Hive that enables non-Hive scripts to access Hive tables. You can then directly load tables with Apache Pig or MapReduce without having to worry about re-defining the input schemas, or caring about or duplicating the data’s location.
"Spark, Hadoop, Hive, and Programming Interview Questions" by Venkateswarlu Chennareddy
Showing posts with label HCatalog. Show all posts
Showing posts with label HCatalog. Show all posts
Friday, February 17, 2017
How to store Pig output into Hive table
There are two approaches explained below with 'Employee' table example to store pig output into hive table. (Prerequisite is that hive table should be already created)
A = LOAD 'EMPLOYEE.txt' USING PigStorage(',') AS(EMP_NUM:int,EMP_NAME:chararray,EMP_PHONE:int);
Approach 1: Using HCatalog
// dump pig result to Hive using Hcatalog
store A into 'Empdb.employee' using org.apache.hive.hcatalog.pig.HCatStorer();
(or)
Approach 2: Using HDFS physical location
// dump pig result to external hive warehouse location
STORE A INTO 'hdfs://<<nmhost>>:<<port>>/user/hive/warehouse/Empdb/employee/' USING PigStorage(',')
How to read data stored in Hive table using Pig
Use HCatLoader to load Hive table data using Pig
A = LOAD 'hivedb.hivetable' using org.apache.hive.hcatalog.pig.HCatLoader();
-- Load table 'sample_07'
sample_07 = LOAD 'sample_07' USING org.apache.hcatalog.pig.HCatLoader();
-- Compute the average salary of the table
salaries = GROUP sample_07 ALL;
out = FOREACH salaries GENERATE AVG(sample_07.salary);
DUMP out;
Similar to HCatLoader, use HCatStorer to update the table, e.g.:
STORE alias INTO 'sample_07' USING org.apache.hcatalog.pig.HCatStorer();
Subscribe to:
Posts (Atom)