Information Integration Blog: How to Invoke REST Web services from ... http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r...

Blog about the different features of Data Integration products and offerings from IBM Analytics. Disclaimer:
The postings on this site are those of the authors and don’t necessarily represent IBM’s positions, strategies
or opinions

Fr i d a y, 1 A u g u s t 2 0 1 4 Contributors

Alekhya
How to Invoke REST Web services from DataStage
Deepa Yr
Designer
Hema Sadagopan
 Overview: Kavita Biswas

 The blog provides the information on new capability of DataStage Designer and few   Kishore Patel
  example usages Madhavi

Manish Bhide
 InfoSphere Information Server Designer provides a stage called Hierarchical Data stage  
 (which was called XML Stage in prior releases to V11.3) that has the capability to parse   Niketa Jain
 and compose the hierarchical data formats like JSON and XML. Along with that, it   Shweta R Sugurmath
  provides the capability to invoke the REST Web services.
Supraja Kakarlapudi

 It supports the REST Web services that are configured in different ways as below: Suraj Ranjan Patel
 1. That provides the responses in different formats like JSON, XML, HTML, JPEG, Audio  
Suresh Tirumalasetti
     etc.
 2. Supports different authentication mechanisms like BASIC, DIGEST, LTPA, OAUTH Vikas Mahajan
 3. Configured with SSL (Server and Client authentication) raolella
 4. Headers and Cookies
rawatrjit
 5. Different HTTP methods like GET, POST, PUT, DELETE, PATCH etc.

 Scenario:
Blog Archive
 There is so much data on the social media sites, which you want to pull, transform and 
 send for the  analytics. Here I am taking an example of the IBM Facebook page. From  2016 (3)
 which, I want to retrieve the details for how many likes does IBM page have, which  2014 (21)
 all websites talk about IBM, To fulfill the above requirement, You can use our new  August (2)
 capability called REST step from the Hierarchical Datastage stage in DataStage Designer.
How to read the data from Apache Hive
Database thr...
  Extracting data from Facebook Page
How to Invoke REST Web services from
  The Figure 1 illustrates the DataStage job which retrieves the data from Facebook using  DataStage Des...
  REST step  and parses the response data from Facebook to fetch the required data for the
 analytics. July (7)
June (12)

 Figure 1: Extract_And_Parse_DataFromFaceBookPage

The figure 2 shows the assembly design of the Hierarchical datastage. The data
is obtained from Facebook by invoking the REST API using REST step. As the 
data provided by Facebook is in format of JSON, JSON parser step is used to parse 

1 of 9 18-11-2018, 10:39:52
Information Integration Blog: How to Invoke REST Web services from ... http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r...

the data. The REST Step “GetFBPageData” and JSON Parser step called 
“ParseFBData” are added to the Assembly Outline.

Figure 2 : Assembly Editor Design

 The Rest Step “GetFBPageData“ is configured as below:

1.In the General tab: The HTTP Method “GET” is selected , URL of the IBM Facebook
   page is mentioned in the “URL” field. The URL is https://graph.facebook.com/IBM as 
   in  figure 3.
2. In the Security tab: As Facebook is configured with SSL, select the checkbox Enable the  
    the SSL and Accept the Self signed certificate as shown in figure 4.
3. In the Request tab: Facebook returns the response in different formats. Here we need in  
    the JSON format. So Specify the Content-Type as application/json under the custom 
    header. As shown in figure 5.
4. In the Response tab : Select the check box “ Pass the received body to” and a radio
    button   “A text node named body in the Output Schema” and specify text/javascript  
    under  the content type as shown in figure 6.

Figure 3: General tab of Rest Step "GetFBPageData"

 Figure 4 : Security tab of Rest Step “GetFBpageData”

 

2 of 9 18-11-2018, 10:39:52
Information Integration Blog: How to Invoke REST Web services from ... http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r...

 Figure 5 : Request tab of Rest Step “GetFBPageData”

 Figure 6 : Response tab of Rest Step “GetFBPageData”

The Output of the REST call which is invoked in the “GetFBPageData” REST step is available
in the body element in the output schema tab. The output schema of the REST step is as in
figure 7.

Figure 7 : The Output Schema of the Rest Step "GetFBPageData"

3 of 9 18-11-2018, 10:39:52
Information Integration Blog: How to Invoke REST Web services from ... http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r...

 The JSON parser step “ParseFBData” is configured as below:

1.Under the JSON Source tab : Select the String set option, and from the drop down 
   select the bodyelement coming from the REST step “GetFBPageData” as shown in the   
   figure 8.  
2.Under the Document root: Browse and select the schema which conforms with the json
   data retrieved from earlier REST step as shown in figure 9.  
3.Under Validation tab: Minimal Validation is selected by default.  

Figure 8 : JSON Source tab of JSON Parser step “ParseFBData”  

 

 Figure 9 : Document root of JSON Parser step “ParseFBData”

4 of 9 18-11-2018, 10:39:52
Information Integration Blog: How to Invoke REST Web services from ... http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r...

The Output step is configured as below: 

Under the Mappings tab, the target link “DETAILS” is mapped to the top to fetch the parsed
details of IBM Facebook page as shown in figure 10.

Figure 10 : Mappings tab of Output step

 
Compile and Execute the job to fetch the required details of the IBM Facebook page shown
in figure 11:

Figure 11 : Data from IBM facebook page.

 
 
The output describes the details like ID of the Page, what it is about , when it is founded, 
 how many likes it got, the count talking about this page, username, websites where it is  
 talked etc. This data can be used in the analytics.

Conclusion : The REST capability in the hierarchical Datastage stage can be used to fetch 

5 of 9 18-11-2018, 10:39:52
Information Integration Blog: How to Invoke REST Web services from ... http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r...

 the data like social media data from different applications like Facebook, LinkedIn, 
 Twitter etc which exposes the services using REST

We have articles published on developer works for the integration scenarios with SOFT 
Layer, Cloudant and Information Governance Catalog Glossary using DataStage.

 http://www.ibm.com/developerworks/data/library/techarticle/dm-1407governrest
/index.html 
http://www.ibm.com/developerworks/data/library/techarticle/dm-1407-softlayer-obj-
storage/index.html 
http://www.ibm.com/developerworks/data/library/techarticle/dm-1407cloudant
/index.html 

 Disclaimer: “The postings on this site are my own and don’t necessarily represent IBM’s
positions, strategies or opinions.” 

Posted by Shweta R Sugurmath at 02:04

26 comments:

Ravishankar Vellal 11 September 2014 at 04:33

How do we achieve the same REST webservice access in Version 9.1?

Reply

Replies

Shweta R Sugurmath 19 September 2014 at 02:45

Hi Ravi,

The REST Web Service is the new feature introduced in the IS Version 11.0,
hence its not possible to access REST WebServices using IS Version 9.1.

Thank you

Areeba Khan 28 June 2015 at 04:03

Hi Ravi,

Were you able to figure out how to implement this in 9.1? I see some posts
online which say that 9.1. supports it but not sure exactly how to go about it.

Please let me know if you have any inputs on this.

Thanks

Reply

indian jaguar 20 July 2015 at 23:18

Hi Ravi, Could you please send or post the JSON file sample. I was trying to parse the file,
but the unable to map the fields. Where will you find the schema for the JSON files which
you get from the facebook or twitter.
Thank you

Reply

Nirmal Kanti Sahoo 20 March 2016 at 10:36

I have one confusion, regarding fig 9 "Document root of JSON Parser step “ParseFBData”.
From where we get the JSON Schema file if I want to do the same.

Reply

6 of 9 18-11-2018, 10:39:52
Information Integration Blog: How to Invoke REST Web services from ... http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r...

Replies

Shweta R Sugurmath 12 December 2016 at 22:52

You need to used the same JSON file under document root. The file can be taken
from browser

Mark Hickok 9 February 2017 at 05:04

can you give a few simple steps as to HOW it canbe gotten from the browser?

Ahmad 18 April 2018 at 04:10

Did anyone get an answer to this?

Unknown 13 July 2018 at 04:38

call the URL from browser and then take the returned json and put it into file
.json and then you can brows from DS designer from Document root tab as in fig
9 to your file system and choose the .json file that you have just created.

Reply

Coy Currin 29 July 2016 at 01:50

Thank u for giving this information. It is saving my valuable time. Data Stage Online Training

Reply

krishna G 3 February 2017 at 10:55

Hi Sir.

The article is great sir - I am trying to replicate the same.

however regarding fig 9 "Document root of JSON Parser step “ParseFBData”.

could you please provide the JSON shema file as an attachment.

I don't know how to create the JSON schema file from the browser.

Kindly help us

Reply

Replies

Unknown 13 July 2018 at 04:36

call the URL from browser and then take the returned json and put it into file
.json and then you can brows from DS designer from Document root tab as in fig
9 to your file system and choose the .json file that you have just created.

Reply

MANI V R 1 March 2017 at 02:36

I am unable to View Figure 5 and Figure 7 in the above doc.

Could you please provide me Doc

Reply

Naseela Naz 21 June 2017 at 00:39

I really appreciate information shared above. It’s of great help. If someone want to learn
Online (Virtual) instructor lead live training in Datastage, kindly contact us
http://www.maxmunus.com/contact
MaxMunus Offer World Class Virtual Instructor led training on TECHNOLOGY. We have
industry expert trainer. We provide Training Material and Software Support. MaxMunus has
successfully conducted 100000+ trainings in India, USA, UK, Australlia, Switzerland, Qatar,
Saudi Arabia, Bangladesh, Bahrain and UAE etc.

7 of 9 18-11-2018, 10:39:52
Information Integration Blog: How to Invoke REST Web services from ... http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r...

For Demo Contact us.

Sangita Mohanty
MaxMunus
E-mail: sangita@maxmunus.com
Ph:(0) 9738075708 / 080 - 41103383
http://www.maxmunus.com/

Reply

seravina danniella 28 November 2017 at 23:02

Thank you for sharing this powerful article, your explanation is clear and very easy to
understand. Please kindly visit our site to get more information about IT solution.
Melbourne SEO Services

Reply

Unknown 16 March 2018 at 00:30

the blog is good and Interactive it is about Mulesoft API Developer it is useful for students
and Mulesoft Developers for more updates on Mulesoft mulesoft Online training

Reply

Unknown 16 March 2018 at 00:30

the blog is good and Interactive it is about Mulesoft API Developer it is useful for students
and Mulesoft Developers for more updates on Mulesoft mulesoft Online training

Reply

Unknown 16 March 2018 at 00:30

the blog is good and Interactive it is about Mulesoft API Developer it is useful for students
and Mulesoft Developers for more updates on Mulesoft mulesoft Online training

Reply

Words Doctorate 5 April 2018 at 22:13

I read this article. I think You put a lot of effort to create this article. I appreciate your
work.
thesis Writing Service

Reply

Unknown 31 May 2018 at 11:29

How to extract data from Google BigQuery useing Datastage?

Does anyone here successfully accomplish this?

Reply

Lathika Honey 13 July 2018 at 21:53

This Blog Provides Very Useful and Important Information. I just Want to share this blog with
my friends and family members. Mulesoft certification training

Reply

Unknown 23 July 2018 at 23:40

Hi Shweta,

It is a very nice blog.. Thanks for sharing the info.

Need one more help - "How to Invoke LinkedIn data from DataStage Designer" while creating
linkedin app I was unable to get all the info which I have to fill. could you please guide
me..
Find below my contact details,

Thanks
Sudipta
9903895374
ssudipta4@gmail.com

8 of 9 18-11-2018, 10:39:52
Information Integration Blog: How to Invoke REST Web services from ... http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r...

Reply

adm100388 2 October 2018 at 07:57

I'm getting this error:

"text/javascript does not match the received content type: application/json"

In the step: Figure 6 : Response tab of Rest Step “GetFBPageData”

json option doesn't there

Reply

Unknown 3 October 2018 at 01:16

Hi,
I exactly followed these steps which was a great help indeed. I ran the job and it executed
successfully but I do not get any output. Will any of you please help me. This is really
important and urgent for me.

Thanks,
Arpita (arpitasatarkar@gmail.com)

Reply

Replies

Unknown 9 October 2018 at 22:23

Did u get any help on this I am also facing the same issue.

tesfaye 16 November 2018 at 12:20

Did u get any help on this I am also facing the same issue.

Reply

Comment as:

Publish Notify me

Newer Post Home Older Post

Subscribe to: Post Comments (Atom)

Simple theme. Powered by Blogger.

9 of 9 18-11-2018, 10:39:52