BinaryReader and BinaryWriter Java Transformations

Contents
1. The BinaryReader Transformation a. Transformation Type b. Ports Overview c. Using the BinaryReader - Example 2. The BinaryWriter Transformation a. Transformation Type b. Ports Overview c. Using the BinaryWriter Example 3. How to use BinaryReader/BinaryWriter Transformations in PowerCenter a. Xml import b. Creating transformations

1. The BinaryReader Transformation

Transformation type: Passive Connected Use the BinaryReader transformation to read any binary file as a source. The BinaryReader transformations The following shows the BinaryReader Java Transformation, its Input and Output ports:

Ports Overview: Sl Name 1 FileName Description Connect a string field from a source which has the name of the file to be read and transformed into a binary field. Input & Output This is an optional field which has the ID of a file. Can be used to identify a file like a number, file id etc Output This field is of the binary datatype which can be connected to either a binary target or an instance of a BinaryWriter java transformation. Output The MD5 hash (string length 32) of the file when it is successfully read. Input/Output Input

File_ID

BinaryOut

MD5

Example: To write images into an Oracle tables BLOB field: Consider the following set of values to be loaded onto an Oracle table. The IMAGE_PATH field is a string field in this source and the actual image in the form of binary has to be loaded to the BLOB field in the target table. Source data:
SSN 987-65-4320 987-65-4321 987-65-4322 987-65-4323 FIRST_NAME LAST_NAME Lloyd Miriam Brad Kyle Warren Ruiz Moore Henderson IMAGE_PATH E:\ID_Pic01.jpg E:\ID_Pic02.jpg E:\ID_Pic03.jpg E:\ID_Pic04.jpg

Source (Flat file):

Target (Oracle):

Create a mapping using the BinaryReader Java Transformation to write the required data (images as BLOB) to the Oracle target.

2. The BinaryWriter Transformation

Transformation type: Passive Connected Use the BinaryWriter transformation to write a binary stream as a file, to file system. You can connect any binary field from a source to the BinaryWriter transformation. The following figure shows the BinaryReader Transformations Input and Output groups ports:

Ports Overview: Sl Name 1 Binary_IN Input/Output Input Description Connect a binary field from a source which has the binary data that has to be transformed into a file on the file system. The name of the file created by the BinaryWriter transformation is given by the FileName field. The FileName field is the absolute path of the file to be created. The MD5 hash of the file when it was written.

2 3

FileName OutString

Input Output

Example: To read binary (as images) from Oracle tables BLOB field: Consider the following set of values to be loaded onto an Oracle table. The IMAGE_PATH field is a string field in this source and the actual image in the form of binary has to be loaded to the BLOB field in the target table. In this example, an Expression transformation is used to modify the filenames in the format: <First_Name>_<Last_Name>.jpg The expression used in this example is:
'E:\\card_images\\' || First_Name || '_' || Last_Name || '.jpg'

The absolute path to the file must be given in the FileName field. Source (Oracle):

Create a mapping using the BinaryWriter Java Transformation to create the file on the file system.

How to use BinaryReader/BinaryWriter Transformations in PowerCenter

There are two ways in which you can use the BinaryReader and BinaryWriter Java transformations in your mappings in PowerCenter. 1. The simplest way is to import the xml files into your repository. Step 1: Go to PowerCenter Designers Repository menu-> Import Objects Step 2: Browse for the xml file, import the required object and finish the importing. Step 3: The imported transformation can be found under the Transformations folder group. Now you can use the transformation like any other transformation.

BinaryReader
BinaryReaderJavaTransformation.xml

BinaryWriter
BinaryWriterJavaTransformation.xml

2. Create the Java transformations by using the java source code. (Use this step in case the xml fails to be imported) Step 1: Open the Transformation Developer in PowerCenter. Step 2: Go to Transformation -> Create.. Step 3: Select the Java transformation type and enter the name of the transformation (Ex: BinaryReader/BinaryWriter etc). Click Create. Step 4: Select Passive type of transformation. (This CANNOT be changed later) Step 5: Create the EXACT number of ports with the EXACT names and matching Datatype and Precision/Scale. (Refer respective transformation diagrams) Step 6: Once the Step 5 is complete, navigate to the Java Code tab in the Edit Transformations dialog. Step 7: Copy the following respective blocks of code into the appropriate SubTabs of the Java Code screen:

1. Import Packages (Copy the following block of code to the Import Packages tab): BinaryReader:
import import import import import java.io.InputStream; java.io.File; java.io.FileInputStream; java.math.BigInteger; java.security.MessageDigest;

BinaryWriter:
import import import import import java.io.InputStream; java.io.ByteArrayInputStream; java.io.FileOutputStream; java.io.IOException; java.security.*;

2. Helper Code (Copy the following block of code to the Helper Code tab): BinaryReader:
static static static static static int countNullRows; Object lock = new Object(); java.io.File fileIn = null; InputStream is = null; MessageDigest digest = null;

BinaryWriter:
static int countNullRows; static Object lock = new Object(); FileOutputStream out = null; ByteArrayInputStream in = null; MessageDigest md5; byte[] digest = new byte[] {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0}; String fileMD5 = "";

3. On Input Row (Copy the following block of code to the On Input Row tab): BinaryReader:
if(!isNull("FileName")){ try { fileIn = new java.io.File(FileName); is = new FileInputStream(fileIn); digest = MessageDigest.getInstance("MD5"); } catch (Exception e) { logError("Unable to create open the file " + FileName); } long length = fileIn.length(); if (length > Integer.MAX_VALUE) { logError("File is too large"); } byte[] bytes = new byte[(int) length]; int offset = 0; int numRead = 0; try { while (offset < bytes.length && (numRead = is.read(bytes, offset, Math.min(bytes.length - offset, 512 * 1024))) >= 0) { digest.update(bytes, 0, numRead); offset += numRead; } } catch (Exception ex) { logError("Error reading file"); } finally { try { is.close(); } catch (Exception ex) { logError("Error closing file"); } } if (offset < bytes.length) { bytes = null; setNull(MD5); setNull(File_ID); logError("Could not completely read file " + fileIn.getName()); } else { byte[] md5sum = digest.digest(); BigInteger bigInt = new BigInteger(1, md5sum); BinaryOut = bytes; MD5 = bigInt.toString(16); logInfo("Successfully read file " + fileIn.getName() + ", ID: " + File_ID + ", of size " + bytes.length + " bytes, with MD5 checksum: \"" + MD5 + "\""); } }else{ synchronized(lock) { countNullRows++; } }

BinaryWriter:
if(!isNull("Binary_IN") && !isNull("FileName")) { try { String filename = FileName; in = new ByteArrayInputStream(Binary_IN); out = new FileOutputStream(filename); int c; while ((c = in.read()) != -1) { out.write(c); } fileMD5=""; try { md5 = MessageDigest.getInstance("MD5"); md5.update(Binary_IN); digest = md5.digest(); for (int i=0; i < digest.length; i++) { fileMD5 += Integer.toString( ( digest[i] & 0xff ) + 0x100, 16).substring( 1 ); } } catch (Exception e) { logError(e.toString()); fileMD5 = ""; } OutString = fileMD5; logInfo("Wrote \"" + filename + "\" successfully. MD5: " + fileMD5); } catch(NullPointerException npex){ OutString = ""; logError("The input binary is null " + npex.toString()); } catch (IOException ioex) { OutString = ""; logError("An I/O error has occured. Check if the target directory exists and has write previleges." + ioex.toString()); } catch(Exception ex){ OutString = ""; logError("Failed to write row" + ex.toString()); }finally { if (in != null) { try{in.close();}catch(Exception ex){} } if (out != null) { try{out.close();}catch(Exception ex){} } } } else { synchronized(lock) { countNullRows++; } }

4. On End of Data (Copy the following block of code to the On End of Data tab): BinaryReader:
synchronized(lock) { logInfo("The total number of null rows across partitions is : " + countNullRows); } logInfo("End of data");

BinaryWriter:
synchronized(lock) { logInfo("The total number of null rows across partitions is : " + countNullRows); } logInfo("End of data");

Step 8: Compile the transformation and save. Step 9: The imported transformation can be found under the Transformations folder group. Now you can use the transformation like any other transformation.