DShield IP List Import to Arcsights Malware IP

List Documentation

The flow
DShield html Program ip csv file Arcsight SmartConnector
Arcsight Malware IP List

Process
1. A batch program was used to download the DShield Html file which
contains the IP list. Refer figure 1 in appendix for the batch
program code.

2. A custom program was used to delete out the 0 in front of the IP

and # comments in the html file as Arcsight Express does not
recognize them. Program input DShield Html file and output IP csv
file. Refer figure 2 in appendix for the source code of the program.

3. Arcsight SmartConnector will read the csv file and send the contents
to Arcsight Express. The events will contain a special field to be
recognize by Express for a rule to parse the events into the Malware
IP List. The special field is specified as DShield. Refer figure 3 - 6
in appendix for the SmartConnector setup.

4. Created a rule in Arcsight Express to put the events into the

Malware IP List whenever the special field is DShield. Refer figure
7 in appendix for the rule used in Arcsight Express.

5. The program has been scheduled to run every week automatically

using cron job. The cron job will run the respective program at 8am
every Monday. Refer figure 8 in appendix for cron job setup.

Appendix
Figure 1 Batch Program
#!/bin/bash
touch /home/soc/IPList/ip.csv
#Create an empty list
wget dshield.org/ipsascii.html
#Download the DShield IP List
sleep 5
timeout 300 /home/soc/filereader/current/bin/arcsight agents
#Run Arcsight
SmartConnector
sleep 5
rm -rf ip*
#Clear all file used (DShield IP List and the csv file)
done

Figure 2 The Source code of the program

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
void main(){
char text[1024];
char ip[16]="\0";
char temp;
int j,i,x;
FILE *f=fopen("ipsascii.html","r+");
FILE *p=fopen("ip.csv","w+");

//Open the DShield html file

//Open the empty csv file

while(!feof(f)){
//Read till the end of the Html file
fgets(text, sizeof(text),f);
temp = text[0];
//Get the 1st letter in the sentence
if( temp != 35 ){
//Check whether is not hash
strncpy(ip, text, 15);
for (i=0; i<15; i++){
temp = ip[i];
if (temp == 48){ //Check whether is zero
if(x!=2){
for(j=i;j<15;j++)
ip[j] = ip[j+1]; //Move to replace the zero
i--;
x++;
}
else{
i++;
x=0;
}
}
else{ //Not zero
do{
i++;

temp=ip[i];
}while(temp!=46); //Find "."
}
}
fprintf(p,"%s\n",ip); //Print to file
}
strcpy(text, ""); //Clear text in text buffer
x=0;
}
fclose(f);
fclose(p);
}

Figure 3 SmartConnector Agent Setup

Figure 4 Set Delimiter which all the settings are in Default

Figure 5 Set the IP address mapping field to be shown in Arcsight Express

Figure 6 Set a special field for the rule to recognize

Figure 7 Arcsight Express rule configuration

Figure 8 Cron job setup