Blog Home

COURSES »
Webinars
Blog »
Interview Questions

Pig Programming: Create Your First Apache

Pig Script
November 6, 2013  |  Big Data and Hadoop

As is the case with scripts in other programming languages such as SQL, Unix Shell, etc., Pig scripts
are used to execute a set of Apache Pig commands collectively. This helps in reducing the time and
effort invested in writing and executing each command manually while doing the Pig
programming. This blog (Pig Programming: Create Your First Apache Pig script) is a step by step
guide to help you create your first Apache Pig script.

Pig Programming: Create Your First Apache Pig Script

An Apache Pig script works in two modes:

Local Mode: In ‘local mode’, you can execute the pig script in local file system. In this case you
don’t need to store the data in Hadoop HDFS file system, instead you can work with the data stored
in local file system itself.
HDFS Mode: In ‘HDFS mode’, the data needs to be stored in HDFS file system and you can process
the data with the help of pig script.

Pig Script in HDFS Mode:

Step1: Writing a script

Open an editor (e.g. gedit) in your Cloudera Demo VM environment:

Command:  gedit sample.pig

This command will create a ‘sample.pig’ file inside the home directory of cloudera user.

Let’s write few PIG commands in the sample script.

Let us say our task is to read data from a data file and to display the required contents on the as
output.

The sample data file contains following data:

Shabbir           Khan             9314573259     Bangalore        Engineer

Manish            Sharma        8882148796     Gurgaon           Lecturer

Mahesh           Kumar          8521548932     Noida                Business

Sampath         Reddy           8547987412     Hyderabad         Engineer

Mohan            Reddy           9256458798     Hyderabad         Professor

Save the text file with the name ‘information.txt’

The sample data file contains five columns FirstName, LastName, MobileNo,

City, and Profession separated by tab key. Our task is to read the content of this file in to HDFS and
display First Name, Mobile Number and Profession of these contacts.

To process this data using Pig, this file should be present in Apache Hadoop HDFS.
Use  the following command:

Command: hadoop dfs –copyFromLocal information.txt hdfs:/

Edit the Pig script (sample.pig) to include following commands:

A = LOAD ‘/information.txt’ using PigStorage (‘\t’) as (FName: chararray, LName: chararray,

MobileNo: chararray, City: chararray, Profession: chararray);

B = FOREACH A generate FName, MobileNo, Profession;

DUMP B;

Save and close the file.

The first command loads the file ‘information.txt’ into variable A with indirect schema (FName,
LName, MobileNo, City, Profession).
The second command loads the required data from variable A to variable B.
The third line displays the content of variable B on the terminal/console.

Step 2: Execute the Pig Script

To execute the pig script in HDFS mode, run the following command:
Command: pig sample.pig

Review the result.

Congratulations on executing your first Pig script successfully!

Got a question for us? Mention them in the comments section and we will get back to you. 

Related Posts:

Get started with Big Data and Hadoop

Big Data Challenges

Why BI professionals need to be skilled in Hadoop

About Shabir Khan (3 Posts)

Like 46 Tweet 10 5