Vous êtes sur la page 1sur 9

Information Integration Blog: How to Invoke REST Web services from

http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r

Blog about the different features of Data Integration

Blog about the different features of Data Integration products and offerings from IBM Analytics. Disclaimer:

The postings on this site are those of the authors and don’t necessarily represent IBM’s positions, strategies or opinions

F r i d a y ,

1

A u g u s t

2 0 1 4

How to Invoke REST Web services from DataStage Designer

Overview:

The blog provides the information on new capability of DataStage Designer and few example usages

InfoSphere Information Server Designer provides a stage called Hierarchical Data stage (which was called XML Stage in prior releases to V11.3) that has the capability to parse and compose the hierarchical data formats like JSON and XML. Along with that, it provides the capability to invoke the REST Web services.

It supports the REST Web services that are configured in different ways as below:

1. That provides the responses in different formats like JSON, XML, HTML, JPEG, Audio

etc.

2. Supports different authentication mechanisms like BASIC, DIGEST, LTPA, OAUTH

3. Configured with SSL (Server and Client authentication)

4. Headers and Cookies

5. Different HTTP methods like GET, POST, PUT, DELETE, PATCH etc.

Scenario:

There is so much data on the social media sites, which you want to pull, transform and send for the analytics. Here I am taking an example of the IBM Facebook page. From which, I want to retrieve the details for how many likes does IBM page have, which all websites talk about IBM, To fulfill the above requirement, You can use our new capability called REST step from the Hierarchical Datastage stage in DataStage Designer.

Extracting data from Facebook Page The Figure 1 illustrates the DataStage job which retrieves the data from Facebook using REST step and parses the response data from Facebook to fetch the required data for the analytics.

Figure 1: Extract_And_Parse_DataFromFaceBookPage

analytics. Figure 1: Extract_And_Parse_DataFromFaceBookPage Contributors Alekhya Deepa Yr Hema Sadagopan Kavita Biswas

Contributors

Alekhya1: Extract_And_Parse_DataFromFaceBookPage Contributors Deepa Yr Hema Sadagopan Kavita Biswas Kishore Patel Madhavi

Deepa YrExtract_And_Parse_DataFromFaceBookPage Contributors Alekhya Hema Sadagopan Kavita Biswas Kishore Patel Madhavi Manish

Hema SadagopanContributors Alekhya Deepa Yr Kavita Biswas Kishore Patel Madhavi Manish Bhide Niketa Jain

Kavita BiswasContributors Alekhya Deepa Yr Hema Sadagopan Kishore Patel Madhavi Manish Bhide Niketa Jain Shweta R

Kishore PatelContributors Alekhya Deepa Yr Hema Sadagopan Kavita Biswas Madhavi Manish Bhide Niketa Jain Shweta R Sugurmath

MadhaviAlekhya Deepa Yr Hema Sadagopan Kavita Biswas Kishore Patel Manish Bhide Niketa Jain Shweta R Sugurmath

Manish BhideDeepa Yr Hema Sadagopan Kavita Biswas Kishore Patel Madhavi Niketa Jain Shweta R Sugurmath Supraja Kakarlapudi

Niketa JainSadagopan Kavita Biswas Kishore Patel Madhavi Manish Bhide Shweta R Sugurmath Supraja Kakarlapudi Suraj Ranjan Patel

Shweta R SugurmathKavita Biswas Kishore Patel Madhavi Manish Bhide Niketa Jain Supraja Kakarlapudi Suraj Ranjan Patel Suresh Tirumalasetti

Supraja KakarlapudiPatel Madhavi Manish Bhide Niketa Jain Shweta R Sugurmath Suraj Ranjan Patel Suresh Tirumalasetti Vikas Mahajan

Suraj Ranjan PatelBhide Niketa Jain Shweta R Sugurmath Supraja Kakarlapudi Suresh Tirumalasetti Vikas Mahajan raolella rawatrjit Blog

Suresh TirumalasettiShweta R Sugurmath Supraja Kakarlapudi Suraj Ranjan Patel Vikas Mahajan raolella rawatrjit Blog Archive 2016 2014

Vikas MahajanSupraja Kakarlapudi Suraj Ranjan Patel Suresh Tirumalasetti raolella rawatrjit Blog Archive 2016 2014 (3) (21) August

raolellaSuraj Ranjan Patel Suresh Tirumalasetti Vikas Mahajan rawatrjit Blog Archive 2016 2014 (3) (21) August (2)

rawatrjitRanjan Patel Suresh Tirumalasetti Vikas Mahajan raolella Blog Archive 2016 2014 (3) (21) August (2) How

Blog Archive

2016 2014
2016
2014

(3)

(21)

August (2)

How to read the data from Apache Hive

Database thr

How to Invoke REST Web services from DataStage Des

July (7) (7)

June (12) (12)

The figure 2 shows the assembly design of the Hierarchical datastage. The data is obtained from Facebook by invoking the REST API using REST step. As the data provided by Facebook is in format of JSON, JSON parser step is used to parse

Information Integration Blog: How to Invoke REST Web services from

http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r

the data. The REST Step “GetFBPageData” and JSON Parser step called “ParseFBData” are added to
the data. The REST Step “GetFBPageData” and JSON Parser step called
“ParseFBData” are added to the Assembly Outline.
Figure 2 : Assembly Editor Design

The Rest Step “GetFBPageData“ is configured as below:

1.In the General tab: The HTTP Method “GET” is selected , URL of the IBM Facebook

page is mentioned in the “URL” field. The URL is https://graph.facebook.com/IBM as in figure 3.

2. In the Security tab: As Facebook is configured with SSL, select the checkbox Enable the the SSL and Accept the Self signed certificate as shown in figure 4.

3. In the Request tab: Facebook returns the response in different formats. Here we need in the JSON format. So Specify the Content-Type as application/json under the custom header. As shown in figure 5.

4. In the Response tab : Select the check box “ Pass the received body to” and a radio

button

under the content type as shown in figure 6.

“A text node named body in the Output Schema” and specify text/javascript

Figure 3: General tab of Rest Step "GetFBPageData"

Figure 3: General tab of Rest Step "GetFBPageData" Figure 4 : Security tab of Rest Step

Figure 4 : Security tab of Rest Step “GetFBpageData”

Information Integration Blog: How to Invoke REST Web services from

http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r

Figure 5 : Request tab of Rest Step “GetFBPageData”

Figure 5 : Request tab of Rest Step “GetFBPageData”

Figure 5 : Request tab of Rest Step “GetFBPageData” Figure 6 : Response tab of Rest

Figure 6 : Response tab of Rest Step “GetFBPageData”

Figure 6 : Response tab of Rest Step “GetFBPageData” The Output of the REST call which

The Output of the REST call which is invoked in the “GetFBPageData” REST step is available in the body element in the output schema tab. The output schema of the REST step is as in figure 7.

Figure 7 : The Output Schema of the Rest Step "GetFBPageData"

Information Integration Blog: How to Invoke REST Web services from

Integration Blog: How to Invoke REST Web services from

http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r

The JSON parser step “ParseFBData” is configured as

The JSON parser step “ParseFBData” is configured as below:

1.Under the JSON Source tab : Select the String set option, and from the drop down select the bodyelement coming from the REST step “GetFBPageData” as shown in the figure 8. 2.Under the Document root: Browse and select the schema which conforms with the json data retrieved from earlier REST step as shown in figure 9. 3.Under Validation tab: Minimal Validation is selected by default.

Validation tab: Minimal Validation is selected by default. Figure 8 : JSON Source tab of JSON

Figure 8 : JSON Source tab of JSON Parser step “ParseFBData”

8 : JSON Source tab of JSON Parser step “ParseFBData” F igure 9 : Document root

Figure 9 : Document root of JSON Parser step “ParseFBData”

Information Integration Blog: How to Invoke REST Web services from

http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r

The Output step is configured as below: Under the Mappings

The Output step is configured as below:

Under the Mappings tab, the target link “DETAILS” is mapped to the top to fetch the parsed details of IBM Facebook page as shown in figure 10.

Figure 10 : Mappings tab of Output step

shown in figure 10. Figure 10 : Mappings tab of Output step Compile and Execute the

Compile and Execute the job to fetch the required details of the IBM Facebook page shown in figure 11:

Figure 11 : Data from IBM facebook page.

shown in figure 11: Figure 11 : Data from IBM facebook page. The output describes the

The output describes the details like ID of the Page, what it is about , when it is founded, how many likes it got, the count talking about this page, username, websites where it is talked etc. This data can be used in the analytics.

Conclusion : The REST capability in the hierarchical Datastage stage can be used to fetch

Information Integration Blog: How to Invoke REST Web services from

http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r

the data like social media data from different applications like Facebook, LinkedIn, Twitter etc which exposes the services using REST

We have articles published on developer works for the integration scenarios with SOFT Layer, Cloudant and Information Governance Catalog Glossary using DataStage.

http://www.ibm.com/developerworks/data/library/techarticle/dm-1407governrest

/index.html

http://www.ibm.com/developerworks/data/library/techarticle/dm-1407-softlayer-obj-

storage/index.html

http://www.ibm.com/developerworks/data/library/techarticle/dm-1407cloudant

/index.html

Disclaimer: “The postings on this site are my own and don’t necessarily represent IBM’s positions, strategies or opinions.”

represent IBM’s positions, strategies or opinions.” Posted by Shweta R Sugurmath at 02:04 26 comments:

Posted by Shweta R Sugurmath at 02:04

or opinions.” Posted by Shweta R Sugurmath at 02:04 26 comments: Ravishankar Vellal 11 September 2014

26 comments:

Ravishankar Vellal 11 September 2014 at 04:33 Ravishankar Vellal

How do we achieve the same REST webservice access in Version 9.1?

Reply Replies
Reply
Replies

Shweta R Sugurmathsame REST webservice access in Version 9.1? Reply Replies Hi Ravi, 19 September 2014 at 02:45

Hi Ravi,

19 September 2014 at 02:45

The REST Web Service is the new feature introduced in the IS Version 11.0, hence its not possible to access REST WebServices using IS Version 9.1.

Thank you

Areeba Khan 28 June 2015 at 04:03 Areeba Khan

Hi Ravi,

Were you able to figure out how to implement this in 9.1? I see some posts online which say that 9.1. supports it but not sure exactly how to go about it.

Please let me know if you have any inputs on this.

Thanks

Reply

Hi Ravi, Could you please send or post the JSON file sample. I was trying to parse the file, but the unable to map the fields. Where will you find the schema for the JSON files which you get from the facebook or twitter. Thank youlet me know if you have any inputs on this. Thanks Reply indian jaguar 20 July

indian jaguar 20 July 2015 at 23:18

Reply

I have one confusion, regarding fig 9 "Document root of JSON Parser step “ParseFBData”. From where we get the JSON Schema file if I want to do the same.Thank you indian jaguar 20 July 2015 at 23:18 Reply Nirmal Kanti Sahoo 20 March 2016

Nirmal Kanti Sahoo 20 March 2016 at 10:36

Reply

Information Integration Blog: How to Invoke REST Web services from

http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r

Replies You need to used the same JSON file under document

RepliesYou need to used the same JSON file under document root. The

You need to used the same JSON file under document root. The file can be taken from browserReplies Shweta R Sugurmath 12 December 2016 at 22:52 Mark Hickok 9

Shweta R Sugurmath

12 December 2016 at 22:52

Mark Hickok 9 February 2017 at 05:04 Mark Hickok

can you give a few simple steps as to HOW it canbe gotten from the browser?

Ahmad 18 April 2018 at 04:10 Ahmad

Did anyone get an answer to this?

call the URL from browser and then take the returned json and put it into file .json and then you can brows from DS designer from Document root tab as in fig 9 to your file system and choose the .json file that you have just created.18 April 2018 at 04:10 Did anyone get an answer to this? Unknown 13 July 2018

Unknown 13 July 2018 at 04:38

Reply

Thank u for giving this information. It is saving my valuable time. Data Stage Online Training Data Stage Online Training

Coy Currin 29 July 2016 at 01:50

Reply

krishna G 3 February 2017 at 10:55 krishna G

Hi Sir.

The article is great sir - I am trying to replicate the same. however regarding fig 9 "Document root of JSON Parser step “ParseFBData”.

could you please provide the JSON shema file as an attachment.

I don't know how to create the JSON schema file from the browser.

Kindly help us

Reply

Repliesthe JSON schema file from the browser. Kindly help us Reply Unknown 13 July 2018 at

schema file from the browser. Kindly help us Reply Replies Unknown 13 July 2018 at 04:36

Unknown 13 July 2018 at 04:36

call the URL from browser and then take the returned json and put it into file .json and then you can brows from DS designer from Document root tab as in fig 9 to your file system and choose the .json file that you have just created.

Reply

MANI V R 1 March 2017 at 02:36 MANI V R

I

am unable to View Figure 5 and Figure 7 in the above doc.

Could you please provide me Doc

Reply

Naseela Naz 21 June 2017 at 00:39 Naseela Naz

really appreciate information shared above. It’s of great help. If someone want to learn

Online (Virtual) instructor lead live training in Datastage, kindly contact us http://www.maxmunus.com/contact MaxMunus Offer World Class Virtual Instructor led training on TECHNOLOGY. We have industry expert trainer. We provide Training Material and Software Support. MaxMunus has successfully conducted 100000+ trainings in India, USA, UK, Australlia, Switzerland, Qatar, Saudi Arabia, Bangladesh, Bahrain and UAE etc.

I

Information Integration Blog: How to Invoke REST Web services from

http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r

For Demo Contact us. Sangita Mohanty MaxMunus E-mail: sangita@maxmunus.com Ph:(0) 9738075708 / 080 - 41103383 http://www.maxmunus.com/

Reply

9738075708 / 080 - 41103383 http://www.maxmunus.com/ Reply Thank you for sharing this powerful article, your

Thank you for sharing this powerful article, your explanation is clear and very easy to understand. Please kindly visit our site to get more information about IT solution. Melbourne SEO Services Melbourne SEO Services

seravina danniella 28 November 2017 at 23:02

Reply

the blog is good and Interactive it is about Mulesoft API Developer it is useful for students and Mulesoft Developers for more updates on Mulesoft mulesoft Online training mulesoft Online training

Unknown 16 March 2018 at 00:30

Reply

the blog is good and Interactive it is about Mulesoft API Developer it is useful for students and Mulesoft Developers for more updates on Mulesoft mulesoft Online training mulesoft Online training

Unknown 16 March 2018 at 00:30

Reply

the blog is good and Interactive it is about Mulesoft API Developer it is useful for students and Mulesoft Developers for more updates on Mulesoft mulesoft Online training mulesoft Online training

Unknown 16 March 2018 at 00:30

Reply

Words Doctorate 5 April 2018 at 22:13 Words Doctorate

I read this article. I think You put a lot of effort to create this article. I appreciate your work. thesis Writing Service

Reply

How to extract data from Google BigQuery useing Datastage? Does anyone here successfully accomplish this?I appreciate your work. thesis Writing Service Reply Unknown 31 May 2018 at 11:29 Reply This

Unknown 31 May 2018 at 11:29

Reply

This Blog Provides Very Useful and Important Information. I just Want to share this blog with my friends and family members. Mulesoft certification training Mulesoft certification training

Lathika Honey 13 July 2018 at 21:53

Reply

Unknown 23 July 2018 at 23:40 Unknown

Hi Shweta,

It is a very nice blog

Thanks for sharing the info.

Need one more help - "How to Invoke LinkedIn data from DataStage Designer" while creating linkedin app I was unable to get all the info which I have to fill. could you please guide me

Find below my contact details,

Thanks

Sudipta

9903895374

ssudipta4@gmail.com

Information Integration Blog: How to Invoke REST Web services from

http://informationintegrationblog.blogspot.com/2014/08/how-to-invoke-r

Reply

adm100388 2 October 2018 at 07:57 adm100388

I'm getting this error:

"text/javascript does not match the received content type: application/json"

In the step: Figure 6 : Response tab of Rest Step “GetFBPageData” json option doesn't there

Reply

Step “GetFBPageData” json option doesn't there Reply Hi, I exactly followed these steps which was a

Hi, I exactly followed these steps which was a great help indeed. I ran the job and it executed successfully but I do not get any output. Will any of you please help me. This is really important and urgent for me.Step “GetFBPageData” json option doesn't there Reply Unknown 3 October 2018 at 01:16 Thanks, Arpita

Unknown 3 October 2018 at 01:16

Thanks, Arpita (arpitasatarkar@gmail.com)

Reply Replies
Reply
Replies

Did u get any help on this I am also facing the same issue.Thanks, Arpita (arpitasatarkar@gmail.com) Reply Replies Unknown 9 October 2018 at 22:23 tesfaye 16 November 2018

Unknown 9 October 2018 at 22:23

tesfaye 16 November 2018 at 12:20 tesfaye

Did u get any help on this I am also facing the same issue.

Reply

Comment as: Publish Notify me
Comment as: Publish Notify me

Comment as:

Comment as: Publish Notify me
Publish
Publish

Notify meComment as: Publish

Newer Post

Home

Subscribe to: Post Comments (Atom)

Older Post

Simple theme. Powered by Blogger.