Secure file transfer in a heterogeneous environment

Using the AIX V6.1 security feature

Anto A. John April 14, 2009

File transfer is an essential and important activity in the day-to-day computing world. Security
lapses during file transfer can invite leak important data to the external world. As a result,
securing FTP is of primary importance. Hence, in AIX® V6.1, IBM® has introduced a secure
flavor of FTP (and ftpd) based on OpenSSL, using Transport Layer Security (TLS) to encrypt
both the command and the data channels of file transfer. This article shows the advantage of
using this AIX V6.1 feature and its usage between AIX and other heterogeneous systems that
already support this feature. This article focuses on AIX secure FTP with a Windows® server.

Introduction
Transport Layer Security (TLS) is a cryptographic protocol that provides secure communication
between clients and servers. When a server and client communicate, TLS ensures that no third
party may eavesdrop or tamper with any message. TLS is the successor to the Secure Sockets
Layer (SSL). This enables any user on the system to exchange files in a secure manner, as long
as their counterpart offers this extension as well.

You might wonder if you already have a secure file transfer protocol, or FTP, (sftp) which comes
with openSSH, why would you opt for a new way of securing the FTP? While it may seem that
using secure FTP only and no secure telnet might not be most desirable, this method is in fact a
sensible alternative for environments where you are not able to use OpenSSH. For example, if
your most trusted systems run on a dedicated and segregated network, it makes perfect sense to
use telnet for remote access within that very network zone (or working from the console). But even
in such scenarios, you might very well have the need to transfer data from or to this secure zone,
which can be accomplished now using secure FTP.

Another scenario could be that you are using OpenSSH already but you still have to exchange
data with outside systems that do not support any form of SSH (thus neither scp nor sftp). Most
often, such (legacy) systems offer "FTP via SSL" (often called ftps) instead.

Requirements for secure FTP

TLS relies on Secure Socket Layer, which needs OpenSSL to be installed on the AIX system
(ftps depends on the libssl.a and libcrypto.a libraries). The latest version of IBM-supported

© Copyright IBM Corporation 2009 Trademarks

Secure file transfer in a heterogeneous environment Page 1 of 15
developerWorks® ibm.com/developerWorks/

OpenSSL can be obtained from https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?

source=aixbp.

The secure communication between the FTP client and server makes use of the certificates, a
combination of a public-private key pair. Hence, the creation of certificates in-house or the import
of certificates from commercial certifying authorities is the second requirement for setting up FTP
over SSL.

Setting up FTP between AIX and Windows

There are four steps to set up FTP between AIX and Windows:

• Creation of certificates
• Setting up of FTP services at the Windows server
• Configuration of FTP at the AIX client
• Data transfer using secure FTP
The article looks into each of these activities separately.

Creation of certificates
As stated previously, certificates need to be created for negotiation between the FTP server and
the client if there is not already a commercial certificate.

First, install OpenSSL on the Windows machine if certificates are created at this machine. If not,
the commercial certificates just need to be brought into this machine.

Create a root-level private key and root-level certificate request (holding the public key).

Listing 1. Root-level certificate request

c:\>cd c:\openssl\bin\
openssl req -newkey rsa:2048 -sha1 -keyout root_key.pem -out root_req.pem
Generating a 2048 bit RSA private key
...................................+++
....+++
writing new private key to 'root_key.pem'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:TX
Locality Name (eg, city) []:Austin
Organization Name (eg, company) [Internet Widgits Pty Ltd]:IBM
Organizational Unit Name (eg, section) []:AIX
Common Name (eg, YOUR name) []:windows
Email Address []:windows@austin.ibm.com

Please enter the following 'extra' attributes

to be sent with your certificate request
A challenge password []:1234

Secure file transfer in a heterogeneous environment Page 2 of 15

ibm.com/developerWorks/ developerWorks®

An optional company name []:ibm

Generate the certificate for root (valid for approximately 10 years) by self-signing it:

Listing 2. Root certificate creation

openssl x509 -req -days 3650 -in root_req.pem -signkey root_key.pem -out root_cert.pem
Signature ok
subject=/C=US/ST=TX/L=Austin/O=IBM/OU=AIX/CN=windows/emailAddress=windows@austin.ibm.com
Getting Private key
Enter pass phrase for root_key.pem:

You can have a look at your root certificate just to make sure everything's right by using:

Listing 3. Root certificate

openssl x509 -in root_cert.pem -text -noout
Certificate:
Data:
Version: 1 (0x0)
Serial Number:
a3:fe:9c:32:97:5e:2b:6e
Signature Algorithm: sha1WithRSAEncryption
Issuer: C=US, ST=TX, L=Austin, O=IBM, OU=AIX,
CN=windows/emailAddress=windows@austin.ibm.com
Validity
Not Before: Nov 12 19:54:40 2008 GMT
Not After : Nov 10 19:54:40 2018 GMT
Subject: C=US, ST=TX, L=Austin, O=IBM, OU=AIX,
CN=windows/emailAddress=windows@austin.ibm.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public Key: (2048 bit)
Modulus (2048 bit):
00:da:6f:db:90:e2:ab:31:92:21:1e:a0:11:13:a9:
6e:00:1a:a5:69:cf:29:8f:60:7c:a0:08:ea:c9:0f:
b2:81:c2:84:6b:a0:4f:9e:12:bf:99:2d:30:60:c4:
e4:c5:f8:2b:db:cd:9b:fc:5e:fe:fb:d8:a0:1c:26:
83:fd:ad:c5:6f:cf:32:70:f8:77:45:cd:f7:7d:a6:
65:5b:5d:82:d3:06:74:b8:0e:4e:d9:e4:63:6f:0a:
d1:11:ea:b0:d0:cf:ae:68:2f:e3:5e:17:fc:a6:49:
c8:db:73:6c:e5:61:1b:8d:22:0d:46:54:9c:79:16:
b1:1b:2b:ef:c1:7b:3c:91:79:56:69:e8:fe:8e:33:
46:6a:cc:17:cb:33:6f:38:6d:5d:f9:5a:57:2f:39:
fb:ca:f6:d2:a0:3b:29:08:c0:ed:d0:bc:31:21:0d:
15:72:9b:c2:ac:fc:1d:f8:a0:d6:9c:16:19:43:11:
2a:fb:86:d8:c5:e8:66:11:2d:c9:96:9f:23:1f:ae:
92:5a:6b:97:98:48:a5:a6:61:67:2b:8d:6b:90:c2:
df:ac:78:57:c5:25:85:bc:ab:30:2e:65:f3:4e:bb:
7d:a0:33:82:d8:f8:0c:7c:b2:a8:80:e6:9c:85:ec:
77:3c:67:26:10:3a:9d:4b:9f:57:73:44:c2:d3:af:
d6:93
Exponent: 65537 (0x10001)
Signature Algorithm: sha1WithRSAEncryption
6b:01:12:d5:91:3f:79:8d:c2:50:e0:ff:bf:52:fd:f0:93:44:
dc:c4:ef:5c:f6:4c:71:76:bf:b8:18:32:2b:f3:dd:85:bf:ad:
14:6c:2b:df:a6:06:37:38:c8:5c:3f:bf:a4:39:09:7c:ae:a5:
04:dc:a4:73:4a:b7:db:10:47:c7:c9:c5:65:21:28:01:64:5d:
9e:65:70:05:0a:59:6f:4a:80:53:b0:60:4e:03:6a:ff:c3:69:
a6:91:fb:1d:95:38:c4:e3:85:21:0a:3b:17:7f:7d:3e:4a:40:
45:ea:61:ba:fa:a3:2f:13:20:88:e1:43:5e:0b:96:3a:77:df:
ca:06:40:f5:09:cc:dd:10:dc:c7:bb:c6:cf:40:8f:e6:5b:e9:
2d:2f:01:6c:49:11:18:ff:89:82:a7:06:15:c9:af:84:19:0f:
04:b7:e8:d9:a6:f2:88:33:6e:cd:ed:bd:ec:72:f2:52:89:33:
07:93:ab:2f:93:50:07:1e:6a:1a:10:b4:61:08:09:11:03:47:

Secure file transfer in a heterogeneous environment Page 3 of 15

developerWorks® ibm.com/developerWorks/

29:4d:d9:86:82:79:d5:c6:0e:f2:3e:4d:e6:7c:0a:15:bd:74:
2a:57:13:b4:aa:69:93:37:74:b0:f6:2b:77:08:cb:ee:77:d3:
33:35:dc:5d:c2:67:97:ec:9c:e7:88:14:e3:06:74:3e:84:42:
36:fe:3b:b8

Now you will create an RSA key for the Windows FTP server without a PEM passphrase; hence,
you will use a different command than the one used previously. It is important not to use any
passphrases on such server keys. Otherwise, it would be required to input that passphrase every
time the key gets used (which is impossible to accomplish when ftpd is using it).

Listing 4. Key for Windows FTP server

openssl genrsa 2048
Loading 'screen' into random state - done
Generating RSA private key, 2048 bit long modulus
................................................................................
.......+++
............+++
e is 65537 (0x10001)
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAveC0u4iRBje4YsosQ9/rS7g2ucrlPR7tDQ/NZCdSuFg3RX3c
j1ZxSmDDHbwkt1u6FSkzBvWhNkZ9SGxRZdChnsdOhXER9L0gAeBgIApuZvcLc9iN
KVJb7GpvvqWYgJnLtTBVRN6aaKKIKSRG1NIUihzrET/+9XK+u6rGZN5aqhrllrWg
uG4e02PAo4ZB5/iStOPFmAvQSifMTsKMX518U1tgBXCtckw7X+tj+1Bdksx3B+1R
OGRwUgM02UkuIOY70j6mxiNDgANzrS8NrM6RhuylbULa0BaOf01Il+716qgjSy7p
AQFLle6S6JdFJrAo0DA43bEKSDcXhdR7iSak6QIDAQABAoIBAQCNQMby4+uPRpBi
3agEbmXqlIYRvUDAMWuD1gejxCUjgmzLw4/7CgGJpWZBjp950EEkUNvnB4AAknvN
DteW+JfjOL3PQ4hqGu/etX6NF3V+H7ZjXevVm7FKtPugRiynLsJ+bhBhBYC06oSE
proSxJ4HCPBOit8u1UfS8H5at5eXBtbHBhsjfL5CUuGzVx2e9z1TqoujYdprtLyL
mmCXUrkI4GuWisZpLRj7gTRAQ3NBYL1U01oKOP3/53mg0RnzPSUNaJ104hrk7j4G
emKVlgTnk9iTrBChjMpqRSEsuZ3J37LgrdpUHxtUiFG/UamfNwLTBlsiWbPKnLDz
CW6Wr1uhAoGBAOl43WQQxIfF2BZmX7AnenEPHcAtq4Wa0mQbNNX+9kWRGxwQe+zs
kCCbNKcfamhxMZijcQlX2CS5YSs9QxT5aa2cy/QVprSrnwlmfTTaxVfWL5FTsp84
4eyHlSTj7F79Nq7lhJjX1TRmje2/Oh6h0L+URtXKE/WJXJgP04Iv0/flAoGBANAy
/Y8jLyGLM/Rj5czgRt60d1CGKpOZvG1rP++PlsF/txGbTRZdEyRPsT8iqQopbcKb
eAM1cDkbx+4dTnNyM036Ow4xgJTMuFRIJEynhaA920UlNNhqIqS5WiGSc/lHVHdN
2qWtI7vtWuuWgCswDBBpEsNMzS2XRSJthDSL4OC1AoGAP+R6udq43esvwpmLkDqN
kIkxfBlDelGvSAQGBpY6sTkPsTN1ppXqr3GqDB5V0XVAvGPmGEd4+DvfMqUWSZQj
7oYdbpPKRmE9KaZpl2BTZ0FPTTcZh+daNI+6KhlR44DqSRy4l9dhGnECYFDXx/BC
n1sls3YU1jocI+UY3eTW3l0CgYAwIcmqwLiRqaveMQYcYjEgJVp8HDhVnTIJncTP
LkPtAgihomSk6emV+HKCpRGtTryYQ/bvzvfdzAVhkBbl653exh3G/zy5qcZnfbIe
oo/q5sH9b0eTTJ4rtMoYftIe7BWj6zTEHo3nUU02r7l7woTXemMjEWtRwenytpzc
xQW9DQKBgQCy04VN+RfizwuGP0fe84xY7fzuWOH6jkNIt2kx91TFlJ0K8iEtndju
kIjqHbDlADjQ5voUrAGAs80/wDpphrgt4iUjWDZw+MJuxTtxfiiEPMemZ50ee8on
fu6Faz8OBaUz5sJK87oj0YoYlsKF4gzfO+bh74RWxulapE4FPbK5SA==
-----END RSA PRIVATE KEY-----

Copy the above key content into a file named server_key.pem.

Next, create a certificate request for the key you just created (including its public key):

Secure file transfer in a heterogeneous environment Page 4 of 15

ibm.com/developerWorks/ developerWorks®

Listing 5. Certificate request for the created key

openssl req -new -key server_key.pem -out server_req.pem
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:TX
Locality Name (eg, city) []:Austin
Organization Name (eg, company) [Internet Widgits Pty Ltd]:IBM
Organizational Unit Name (eg, section) []:AIX
Common Name (eg, YOUR name) []:server
Email Address []:server@austin.ibm.com

Please enter the following 'extra' attributes

to be sent with your certificate request
A challenge password []:1234
An optional company name []:ibm

Next, sign the server key request with the root CA's private and self-signed public key. This creates
the server certificate (again, valid approximately for 10 years):

Listing 6. Step 6: Server certificate creation

openssl x509 -req -days 3650 -in server_req.pem -CA root_cert.pem -CAkey root_key.pem
-CAcreateserial –out server_cert.pem
Signature ok
subject=/C=US/ST=TX/L=Austin/O=IBM/OU=AIX/CN=server/emailAddress=server@austin.ibm.com
Getting CA Private Key
Enter pass phrase for root_key.pem:

In order to make server configurations easier as well as the distribution of certified key files, it
is handy to have the server key, the server certificate, and the root certificate in one single file
(OpenSSL supports this). So combine all three files into one file. This file should be protected with
respective file permissions.

Listing 7. Single server file

Copy the contents of server_key.pem server_cert.pem root_cert.pem
into the file server.pem

Export the server certificate into the .pfx format.

Listing 8. Convert the server certificate to pfx format

openssl pkcs12 -export -out server.pfx -in server.pem -name "server cert pfx"
Enter Export Password:
Verifying - Enter Export Password:

Now you have the server certificate in the .pfx format, which Windows accepts. It is very important
that you add this certificate to the Microsoft® Management console's trusted certificate repository.
Failure to do so will result in making the certificate untrusted during the usage.

The Microsoft Management Console (MMC) can be accessed by Start -> Run -& type mmc .

Secure file transfer in a heterogeneous environment Page 5 of 15

developerWorks® ibm.com/developerWorks/

The MMC has an option to add the certificates as snap-ins.

Figure 1. Microsoft Management Console

Figure 2. Adding snap-ins

Once the Snap-in "certificate" is added to the console, you get the certificate categories as listed
below. Select the trusted root certificate or any other certificate category accordingly and import
the server certificate that you had created into it.

Secure file transfer in a heterogeneous environment Page 6 of 15

ibm.com/developerWorks/ developerWorks®

Figure 3. Importing the certificate

The import of the certificate will open up the Certificate Import Wizard, following which the import
of certificate becomes successful.

Figure 4. Certificate Import Wizard

You have now changed the OpenSSL certificate for the FTP purpose into a trusted certificate in the
Windows machine.

Secure file transfer in a heterogeneous environment Page 7 of 15

developerWorks® ibm.com/developerWorks/

Figure 5. Certificate added successfully to the snap-in

Now open up the Internet Information Services (IIS) manager and add this trusted certificate as a
part of the the ISS server certificate database and now it can be used for the different services of
IIS, which includes secure FTP.

Figure 6. Server Ceriticate repository

Secure file transfer in a heterogeneous environment Page 8 of 15

ibm.com/developerWorks/ developerWorks®

Figure 7. Import into Server Certificate repository

Importing the server certificate into this repository makes it usable for the secure FTP service.
Also, note that the certificate is OK and is trusted. After the certificate is successfully added, you
have to 'apply' the changes in the IIS Manager for the changes to take effect.

Figure 8. Imported certificate

Secure file transfer in a heterogeneous environment Page 9 of 15

developerWorks® ibm.com/developerWorks/

Setting up FTP services on a Windows server

To set up FTP on the Windows side, you have to open the console for Internet Information server
(IIS). IIS can be accessed through Start -> Administrative Tools -> Internet Information
Services(IIS) Manager.

Figure 9. Internet Information services (IIS) Manager

Create a new site for this purpose and the work area will create the FTP-specific services. In this
case, create a site called ssl_ftp_site.

Secure file transfer in a heterogeneous environment Page 10 of 15

ibm.com/developerWorks/ developerWorks®

Figure 10. Adding an FTP site

You can assign specific IP address and ports for which this FTP server needs to be associated. In
this step, there is an option that asks you to specify the certificate. If the certificate is created and
ready for usage, it can be specified now or could be left for later configuration.

Figure 11. Binding and SSL settings

Secure file transfer in a heterogeneous environment Page 11 of 15

developerWorks® ibm.com/developerWorks/

This step configures the FTP servers authentication mode and the users who are allowed, along
with their read/write permissions and completes the configuration of the FTP site in the IIS.

Figure 12. Authentication and authorization information

The panel at the right side under the "Manage FTP Site" section helps you to either stop or run the
service. It is always important to stop the server and restart it for any changes to take effect.

Secure file transfer in a heterogeneous environment Page 12 of 15

ibm.com/developerWorks/ developerWorks®

Figure 13. Site ready for use

Now you must set the FTP SSL settings for the site, which is available on the workspace of the
FTP site home.

Figure 14. FTP SSL settings

Again, do not forget to stop the server and restart it for the changes to take effect.

Secure file transfer in a heterogeneous environment Page 13 of 15

developerWorks® ibm.com/developerWorks/

Configuration of FTP at the AIX client

There is one further step of security that can be established with such self-signed certificates: The
rootCA's certificate needs to be copied to every client and referred to in their ~/.ftpcnf files of any
user wishing to use encrypted and authenticated FTP sessions.

Adjust the user's .ftpcnf file to point to the CA certificate by only changing this one line:
CA_PATH ./root_cert.pem

Data transfer using secure FTP

You can use /usr/bin/ftp to connect to the FTP daemon on any remote system, whether this target
host is TLS-enabled or not. Once you have completed the TLS setup on your server, in order to
transfer files to or from that server, you can issue the following command.
# ftp -s –M remote_host

This will establish an encrypted session without any modifications needed on the client side. If the
certificate, however, is self-signed, the client has to blindly trust the server, so the FTP command
will display the most important data of the certificate it received during the TLS handshake to leave
the decision up to the user whether he wants to connect or not.

The command ftp –s remote_host will work only for AIX-to-AIX machines. One has to use the ftp
–s –M remote_host option to get the secure FTP working between AIX and non-AIX machines.

Conclusion
This article covered the various aspects of the secure FTP setup, which includes the FTP service
configuration on the Windows server side and the FTP configuration on the client side. This feature
adds a very important security capability to the AIX operating system and helps it to harness the
similar functionality available with other operating systems, tthus helping the customer to have a
secure file transfer in a heterogenous environment.

Secure file transfer in a heterogeneous environment Page 14 of 15

ibm.com/developerWorks/ developerWorks®

Related topics
• "Information for System p" Visit this this site for additional information.
• The AIX 6 Advanced Security Features, Introduction and Configuration Guide —highlights and
explains the security features enhancements on AIX 6.1.
• Windows Server 2008 (Administrator's Pocket Consultant) by William R. Stanek
• Internet Information Services (IIS) 7.0 Administrator's Pocket Consultant by William R. Stanek
• Download AIX Web Download Pack like openssl, lsof.
• Download IBM product evaluation versions and get your hands on application development
tools and middleware products from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.

© Copyright IBM Corporation 2009

(www.ibm.com/legal/copytrade.shtml)
Trademarks
(www.ibm.com/developerworks/ibm/trademarks/)

Secure file transfer in a heterogeneous environment Page 15 of 15