Summary

Security vulnerabilities have been discovered in OpenSSL

Vulnerability Details

Abstract

IBM Systems Director is affected by vulnerabilities in OpenSSL (CVE-2014-0160 and CVE-2014-0076).

Content

Vulnerability Details:

CVE-ID: CVE-2014-0160 Description: OpenSSL could allow a remote attacker to obtain sensitive information, caused by an error in the TLS/DTLS heartbeat functionality. An attacker could exploit this vulnerability to expose 64k of private memory and retrieve secret keys. An attacker can repeatedly expose additional 64k chunks of memory. This vulnerability can be remotely exploited, authentication is not required and the exploit is not complex. An exploit can only partially affect the confidentially, but not integrity or availability.
CVSS Base Score: 5
CVSS Temporal Score: See <http://xforce.iss.net/xforce/xfdb/92322&gt;
CVSS Environmental Score*: Undefined
CVSS Vector: (AV:N/AC:L/Au:N/C:P/I:N/A:N)

Warning: We strongly encourage you to take action as soon as possible as potential implications to your environment may be more serious than indicated by the CVSS score.

CVE-ID: CVE-2014-0076 Description: OpenSSL could allow a local attacker to obtain sensitive information, caused by an implementation error in ECDSA (Elliptic Curve Digital Signature Algorithm). An attacker could exploit this vulnerability using the FLUSH+RELOAD cache side-channel attack to recover ECDSA nonces. This vulnerability can only be exploited locally, authentication is not required and the exploit is not complex. An exploit can only partially affect confidentially, but not integrity or availability.
CVSS Base Score: 2.1
CVSS Temporal Score: See <http://xforce.iss.net/xforce/xfdb/91990&gt;
CVSS Environmental Score*: Undefined
CVSS Vector: (AV:L/AC:L/Au:N/C:P/I:N/A:N)

Affected products and versions

• IBM Systems Director: 6.3.2.0, 6.3.2.1, 6.3.3.0. 6.3.3.1

Non-affected Products and Versions

• IBM Systems Director versions 5.2.x.x, 6.1.x.x, 6.2.x.x, 6.3.0.0, 6.3.1.0, 6.3.1.1 server and agents on all hardware platforms are NOT vulnerable to the OpenSSL Heartbleed vulnerability (CVE-2014-0160).

Remediation:

1. Click the following link: http://www-933.ibm.com/support/fixcentral/swg/downloadFixes?parent=ibm~Director&product=ibm/Director/SystemsDirector&release=All&platform=All&function=fixId&fixids=SysDir6_3_x_0_IT01062_IT01063_IT01199&includeRequisites=1&includeSupersedes=0&downloadMethod=ddp

2. Select the following fix pack: SysDir6_3_x_0_IT01062_IT01063_IT01199 Note: This fix package includes all releases, and all platforms.

3. Follow the Instructions beneath the table for your desired platform

Warning: Your environment may require additional fixes for other products, including non-IBM products. Please replace the SSL certificates and reset the user credentials after applying the necessary fixes to your environment.

Product

| VRMF | APAR | Associated Technote
—|—|—|—
IBM Systems Director and IBM Systems Director Platform Agent | Xlinux Platform Agent 6.3.2 to 6.3.3.1

| IT01199 | http://www.ibm.com/support/docview.wss?rs=0&uid=nas7faac5f301a762f0386257cc2006a66dd
IBM Systems Director and IBM Systems Director Platform Agent | Windows Platform Agent 6.3.2 to 6.3.3.1

| IT01063 | http://www.ibm.com/support/docview.wss?rs=0&uid=nas7f8e5d418493cb42c86257cc2006a4023
IBM Systems Director and IBM Systems Director Platform Agent | Power Linux Platform Agent 6.3.2 to 6.3.3.1

| IT01062 | http://www.ibm.com/support/docview.wss?rs=0&uid=nas7fc8e0d18414cb24886257cc2006a73c6
IBM Systems Director and IBM Systems Director Platform Agent | Zlinux Platform Agent 6.3.2 to 6.3.3.1

| IT01200 | http://www-947.ibm.com/support/entry/portal/support/ and search for Technote 704661448
IBM Systems Director and IBM Systems Director Platform Agent | AIX Platform Agent 6.3.2 to 6.3.3.1

| IT01060 | http://www.ibm.com/support/docview.wss?rs=0&uid=nas711965dd15d022ea386257cc2006a7afd

XLinux Platform Agent 6.3.2 to 6.3.3.1

Introduction:

These instructions describe how to manually install the Heartbleed patch on a xLinux systems with the Platform Agent, Common agent, or IBM Systems Director Server’s Common Agent. To update the Platform agent on the Flex System Manager, use fix ‘FSMApplianceFixPackage-1-3-1-1’

Notes and assumptions:

• These instructions assume that the affected endpoint is discovered and a Request Access has already been done.
• The Platform Agent is included as part of the Common Agent including the Common Agent on IBM Systems Director servers.
• It is important to remember that this patch is only applicable if Platform Agent’s OpenSSL binary is used for CIMOM communications. If the CIM RSAP is locked or not present, it is not used (ie: The Common Agent CIM RSAP is there but unused which greatly limits the impact).
• When the SSL keys are reset the keys will be 2048 bits in length and the certificate signing algorithm will be SHA256.

Confirm that you need the patch:

• This patch only applicable to Platform Agent versions 6.3.2, 6.3.2.1, 6.3.3, 6.3.3.1.
• The Platform Agent CIMOM runs OpenSSL on ports 5989 or alternatively 15989 (ie: if OS CIMOM is using 5989) and is exposed to the IBM Systems Director Server/Flex System Manager as the CIM RSAP of the agent security configuration. To test to see if your agent is affected open the command prompt and run the following command:

/opt/ibm/icc/bin/openssl version

Note: Only the 1.0.1 version of OpenSSL is affected (ie: not 0.9.8, or 1.0.0). If your version is 1.0.1 and dated earlier than April 7, 2014 then the endpoint is affected. There are alternative ways to test if your PA is affected (eg: perl scripts, modified OpenSSL clients, etc.) but it is not defined in this document.

To Install:

1. Obtain the appropriate RPM package for your xLinux distribution such as:

2. * RHEL5_x86 --> ibmcim-ssl-1.0.1-rhel5.i386.rpm
   

* RHEL6_x86 --> ibmcim-ssl-1.0.1-rhel6.i386.rpm
* SUSE10_x86 --> ibmcim-ssl-1.0.1-sles10.i386.rpm
* SUSE11_x86 --> ibmcim-ssl-1.0.1-sles11.i386.rpm  

Download here: http://www-933.ibm.com/support/fixcentral/systemx/selectFixes?parent=Flex+System+Manager+Node&product=ibm/systemx/8731&&platform=All&function=fixId&fixids=Flex1_3_1_Platform_Agents_IT00284

3. Copy the RPM file to the affected endpoint and install the RPM.

rpm -Uvh --force xxxxxxx.rpm

4. Verify that the package was delivered properly by running the version check again. The new OpenSSL version will be 1.0.1 dated April 7, 2014.

/opt/ibm/icc/bin/openssl version

5. The SSL encryption keys/certificates located in the Platform Agent keystore must be reset because we must assume they have been stolen before the patch was applied. To do so do the following on the target agent system:

1. At the endpoint command set variables by running these commands:  

KEYSTORE_PATH=/etc/opt/ibm/icc/keystore
ICC_PATH=/opt/ibm/icc
HOSTNAME=hostname
OPENSSL_CONF=$KEYSTORE_PATH/…/openssl.cnf**
Note**: “`” for the hostname line is the tic not the apostrophe “'”

2. Regenerate the certificates and keys:  

echo -e “US\nNORTH CAROLINA\nRTP\nIBM\nSTG\n$HOSTNAME\n.\n.\n” | $ICC_PATH/bin/openssl req -x509 -nodes -sha256 -days 3650 -newkey 2048 -config $OPENSSL_CONF -keyout $KEYSTORE_PATH/server.key -out $KEYSTORE_PATH/server.cert
Note: Running optional step 4 before this command is run will show the the ‘before’ certificate so you can compare it to the ‘after’ certificate if desired.

3. Stop and restart the CIMOM so it uses the new certificates and keys:  

service cimserverd restart

4. Optional Step: Run the command below dump the certificate information, including the new public key if desired.  

/opt/ibm/icc/bin/openssl x509 -in $KEYSTORE_PATH/server.cert -noout -text

6. Optional step: Confirm that the CIMOM is up and running with the new SSL certificate. These commands will list the CIMOM’s namespaces (eg: root/ibmsd) using the SSL connection (the port will normally be 15989 or 5989):

/opt/ibm/icc/bin/cimcli ns -l <IP_ADDRESS>:<PORT> -s -u <ADMINUSER> -p <PASSWORD>

7. IMPORTANT: At this point all user IDs that have been used to Request Access to any Platform Agent endpoint must be changed using common password procedures. This protects from an attacker who acquired the private SSL keys when a user in the past Requested Access.

8. Optional Step: Verify the CIM connection between the IBM Systems Director/Flex System Manager server and the Platform Agent endpoint is still good by right clicking on the endpoint within the console and choose Security–>Verify connection. If this is ok then the keystore SSL certificates ISD uses to communicate with the Platform Agent endpoint are working.
   Note: If the CIM RSAP is locked (or not there) such as when a Common Agent (CAS) is installed then it is unused this step is completely unnecessary.

Windows Platform Agent 6.3.2 to 6.3.3.1

Introduction:

These instructions describe how to manually install the Heartbleed patch on systems with the Platform Agent, Common agent, or IBM Systems Director Server’s Common Agent.

Notes and assumptions:

• These instructions assume that the affected endpoint is discovered and a Request Access has already been done.
• The Platform Agent is included as part of the Common Agent including the Common Agent on IBM Systems Director servers.
• It is important to remember that this patch is only applicable if Platform Agent’s OpenSSL binary is used for CIMOM communications. If the CIM RSAP is locked or not present, it is not used (ie: The
• ommon Agent CIM RSAP is there but unused which greatly limits the impact).
• When the SSL keys are reset the keys will be 2048 bits in length and the certificate signing algorithm will be SHA256.

Confirm that you need the patch:

This patch only applicable to Platform Agent versions 6.3.2, 6.3.2.1, 6.3.3, 6.3.3.1

The Platform Agent CIMOM runs OpenSSL on ports 5989 or alternatively 15989 (ie: if OS CIMOM is using 5989) and is exposed to the IBM Systems Director Server/Flex System Manager as the CIM RSAP of the agent security configuration. To test to see if your agent is affected open the command prompt and run the following command:

64 bit Windows

“c:\Program Files (x86)\Common Files\ibm\icc\cimom\bin\openssl.exe” version

32 bit Windows

“c:\Program Files\Common Files\ibm\icc\cimom\bin\openssl.exe” version

Note: Only the 1.0.1 version of OpenSSL is affected (ie: not 0.9.8, or 1.0.0). If your version is 1.0.1 and dated earlier than April 7, 2014 then the endpoint is affected. There are alternative ways to test if your PA is affected (eg: perl scripts, modified OpenSSL clients, etc.) but it is not defined in this document.

To Install:

1. Obtain the “OpenSSL_v101g.msi” msi for Windows. Here:
   http://www-933.ibm.com/support/fixcentral/systemx/selectFixes?parent=Flex+System+Manager+Node&product=ibm/systemx/8731&&platform=All&function=fixId&fixids=Flex1_3_1_Platform_Agents_IT00284

2. Copy the OpenSSL_v101g.msi to the affected endpoint and run the msi installer.
   Note : Installation of the msi patch will prompt to restart the machine. Please restart the machine after the installation of the openSSL_v101g patch.

3. Verify that the package was delivered properly by running the version check again. The new OpenSSL version will be 1.0.1 dated April 7, 2014.

64 bit Windows

“c:\Program Files (x86)\Common Files\ibm\icc\cimom\bin\openssl.exe” version

32 bit Windows

“c:\Program Files\Common Files\ibm\icc\cimom\bin\openssl.exe” version

es\ibm\icc\cimom\bin\openssl.exe" version

4. The SSL encryption keys/certificates located in the Platform Agent keystore must be reset because we must assume they have been stolen before the patch was applied. To do so do the following on the target agent system:
   Note: Keystore path will be located here:
   C:\Program Files (x86)\Common Files\ibm\icc\cimom\data\keystore\ for 64 bit agent
   C:\Program Files\Common Files\ibm\icc\cimom\data\keystore\ for 32 bit agent
   1. Open a command prompt and type net stop wmicimserver. This command is used to stop the Platform Agent CIMOM subagent process on the system.

2. CD into the correct Keystore directory based on your Windows version (ie: 32 vs 64 bits - see above)  


3. At this point, the wmicimserver.key and wmicimserver.cert files should be seen. The cert file is the current certificate (contains public key).  


4. Optional Step: Run the command below dump the certificate information, including the public key if desired.  

…\…\bin\openssl.exe x509 -text -noout -in wmicimserver.cert

5. Rename wmicimserver.cert and wmicimserver.key to wmicimserver.cert.old wmicimserver.key.old respectively.  


6. A new certificate and keys now need to be generated. To do this, type:  

…\…\bin\openssl.exe req -x509 -nodes -sha256 -days 3650 -newkey 2048 -config …\…\bin\openssl.cnf -keyout wmicimserver.key -out wmicimserver.cert

7. Country Name=US , State=NORTH CAROLINA , Locality=RTP , Organization=IBM , Organizational Unit=xSeries , Common Name=%machine_hostname% , Email=<Leave Blank> .  

Note: %machine_hostname% is literally the endpoint’s fully qualified hostname which should be placed here.

8. Now type net start wmicimserver  


9. At this point, the new certificate should be installed and working. Run Step 4 above to confirm that the new certificate is in place.  

5. Optional step: Confirm that the CIMOM is up and running with the new SSL certificate. These commands will list the CIMOM’s namespaces (eg: root/ibmsd) using the SSL connection (the port will normally be 15989 or 5989):

cimcli ns -l <IP_ADDRESS>:<PORT> -s -u <ADMINUSER> -p <PASSWORD>

6. IMPORTANT: At this point all user IDs that have been used to Request Access to any Platform Agent endpoint must be changed using common password procedures. This protects from an attacker who acquired the private SSL keys when a user in the past Requested Access.

7. Optional Step: Verify the CIM connection between the IBM Systems Director/Flex System Manager server and the Platform Agent endpoint is still good by right clicking on the endpoint within the console and choose Security–>Verify connection. If this is ok then the keystore SSL certificates ISD uses to communicate with the Platform Agent endpoint are working.
   Note: If the CIM RSAP is locked (or not there) such as when a Common Agent (CAS) is installed then it is unused this step is completely unnecessary.

Example script and automation:

The below script can be saved as a bat file and may help automating some of these commands. The script is untested but may provide some helpful ideas of how to automate the process. When writing scripts, be sure to account for if the Windows system is 32 or 64 bits. The path to the keystore (see above) is different for each.

================
REM Batch file helps for the certificate replacement once the openssl 101 g installed
REM This script automatically finds the Ipv4 address of the machine therby showing the certificate information before and after the certificate is replaced.
REM The Default path of the cerificate is C:\Program Files (x86)\Common Files\ibm\icc\cimom\data\keystore for 64bit Window and for 32 bit env is C:\Program Files\Common Files\ibm\icc\cimom\data\keystore.
for /f “tokens=1-2 delims=:” %%a in (‘ipconfig^|find “IPv4”’) do set ip=%%b
@echo off
net stop wmicimserver
@echo off
if defined ProgramFiles(x86) (
set OSBITS=C:\Program Files (x86^)
) else (
set OSBITS=C:\Program Files
)
pushd %OSBITS%\Common Files\ibm\icc\cimom\data\keystore
openssl x509 -text -noout -in wmicimserver.cert
dir
ren wmicimserver.cert wmicimserver.cert.old
ren wmicimserver.key wmicimserver.key.old
dir
echo “creating the new certificate…”
openssl req -x509 -nodes -sha256 -days 3650 -newkey 2048 -config …\…\bin\openssl.cnf -keyout wmicimserver.key -out wmicimserver.cert
echo “Starting the cimservr now …”
net start wmicimserver
echo “After the cert replace…”
openssl x509 -text -noout -in wmicimserver.cert
openssl s_client -connect %ip%:5989 -showcerts

You can pipe things to the openssl.exe command line to help with text entry (see step 7) such as:
- Create a file (incert.txt) that has the following contents:
US
NORTH CAROLINA
RTP
IBM
xSeries
AgentHostname.ibm.com
<CARRIAGE RETURN HERE FOR blank email>
<CARRIAGE RETURN HERE>

- Run something such as (see step 6).

type incert.txt | …\…\bin\openssl.exe req -x509 -nodes -sha256 -days 3650 -newkey 2048 -config …\…\bin\openssl.cnf -keyout wmicimserver.key -out wmicimserver.cert

Power Linux Platform Agent 6.3.2 to 6.3.3.1

Introduction:

These instructions describe how to manually install the Heartbleed patch on Power Linux with the 64bit PA. Other platforms will have a similar process.

Notes and assumptions:

• These instructions assume that the affected endpoint is discovered within IBM Systems director and Request Access has already been done.
• The PA included with the CAS agents and the ISD server can be patched by this PA patch instructions for their respective platforms.
• It is important to remember that this patch is only applicable if PA openssl binary is used for CIMOM communications. If the OS openssl binary is used then it must be patch by the OS vendor and the certificates must still be reset.
• When the SSL keys are reset the keys will be 2048 bits in length and the certificate signing algorithm will be SHA256.

Confirm that you need the patch:

The PA cimmom runs OpenSSL on ports 15989 or alternatively 15988 (ie: OS CIMOM is using 15989) and is exposed to the IBM Systems Director Server as the CIM RSAP of the agent security confiuration. To test to see if your agent is affected run the following command:
/opt/ibm/icc/bin/openssl version
**
Note**:
OpenSSL 1.0.1 through 1.0.1f (inclusive) are vulnerable
OpenSSL 1.0.1g is NOT vulnerable
OpenSSL 1.0.0 branch is NOT vulnerable
OpenSSL 0.9.8 branch is NOT vulnerable

If your version is 1.0.1 and dated earlier than April 7, 2014 then the endpoint is affected. There are alternative ways to test if your PA is affected (eg: perl scipts, modified openssl clients, etc.) but it is not defined in this document.

To Install:

1. Obtain the the RPM patch for your OS/Platform. For example consider “ibmcim-ssl-1.0.1.7-sles10_sles11.rpm” as the SLES 11 patch for Power Linux.

2. Copy the RPM to the affected endpoint and run the following commands to install (where xxxx is the RPM name):

rpm -Uvh --force xxxx.rpm

3. Verify that the package was delivered properly by running the version check again. The new OpenSSL version will be 1.0.1 dated April 7, 2014.

/opt/ibm/icc/bin/openssl version

4. Next the SSL encryption keys/certificates located in the PA keystore must be reset because we must assume they have been stolen before the patch was applied. To do so Do the following:
   1. Set variables by running the commands:
   KEY_SIZE=2048
   ALGORITHM_CSR=‘req -nodes -sha256’
   ALGORITHM_CERT=‘-req -sha256’
   KEYSTORE_PATH=/etc/opt/ibm/icc/keystore
   ICC_PATH=/opt/ibm/icc
   HOSTNAME=hostname (NOTE: “`” is the tic not the apostrophe “'”)
   OPENSSL_CONF=$KEYSTORE_PATH/…/openssl.cnf
   2. Regenerate the keys/certificates:
   echo “Creating a SSL private key…”
   $ICC_PATH/bin/openssl genrsa -out $KEYSTORE_PATH/server.key $KEY_SIZE
   echo “Generating a SSL certificate request…”
   echo -e “US\nNORTH CAROLINA\nRTP\nIBM\nSTG\n$HOSTNAME\n.\n.\n” | $ICC_PATH/bin/openssl ${ALGORITHM_CSR} -config $OPENSSL_CONF -new -key
   $KEYSTORE_PATH/server.key -out $KEYSTORE_PATH/server.csr >/dev/null 2>&1
   echo “Self-signing an SSL certificate based on system information…”
   $ICC_PATH/bin/openssl x509 -in $KEYSTORE_PATH/server.csr -out
   $KEYSTORE_PATH/server.cert ${ALGORITHM_CERT} -signkey $KEYSTORE_PATH/server.key -days 3650 -set_serial 00
   chmod 600 $KEYSTORE_PATH/server.key
   chmod 600 $KEYSTORE_PATH/server.csr
   chmod 600 $KEYSTORE_PATH/server.cert
   3. Stop and restart the CIMOM so it uses the new keys/certificate
   service cimserverd restart
   4. Optional step: Confirm that the keys/certificates were indeed replaced run the command below. The output should show that the new sha256 2048 bit key certificate was generated today.
   /opt/ibm/icc/bin/openssl x509 -in $KEYSTORE_PATH/server.cert -noout -text
   5. Optional step: Confirm that the CIMOM is up and running with the new SSL certificate. These commands will list the CIMOM’s namespaces (eg: root/ibmsd) using the SSL connection:
   /opt/ibm/icc/bin/cimcli ns -l <IP_ADDRESS>:<PORT> -s -u <ADMINUSER> -p <PASSWORD>
   6. At this point all userids that have been used to Request Access to any PA endpoint should be changed using common password procedures. This protects from an attacker who acquired the private keystore keys when a user in the past Requested Access.
   7. Optional Step: Verify the CIM connection between the ISD server and the PA endpoint is still good by right clicking on the endpoint within the ISD console and choose Security–>Verify connection. If this is ok then the keystore SSL certificates ISD uses to communicate with the PA endpoint are working.
   8. Optional Step: If you also want to reset the Agent trusstore certificates (not as important), right click on the endpoint within the ISD console and choose Security–>Revoke. Once revoked, request access again using a userid/password.

ZLinux Platform Agent 6.3.2 to 6.3.3.1

Introduction:

These instructions describe how to manually install the Heartbleed patch on a zLinux systems with the Platform Agent, Common agent, or IBM Systems Director Server’s Common Agent.

Notes and assumptions:

• These instructions assume that the affected endpoint is discovered and a Request Access has already been done.
• The Platform Agent is included as part of the Common Agent including the Common Agent on IBM Systems Director servers.
• It is important to remember that this patch is only applicable if Platform Agent’s OpenSSL binary is used for CIMOM communications. If the CIM RSAP is locked or not present, it is not used (ie: The Common Agent CIM RSAP is there but unused which greatly limits the impact).
• When the SSL keys are reset the keys will be 2048 bits in length and the certificate signing algorithm will be SHA256.

Confirm that you need the patch:

This patch only applicable to Platform Agent versions 6.3.2, 6.3.2.1, 6.3.3, 6.3.3.1.
The Platform Agent CIMOM runs OpenSSL on ports 5989 or alternatively 15989 (ie: if OS CIMOM is using 5989) and is exposed to the IBM Systems Director Server/Flex System Manager as the CIM RSAP of the agent security configuration. To test to see if your agent is affected open the command prompt and run the following command:

/opt/ibm/icc/bin/openssl version

Note: Only the 1.0.1 version of OpenSSL is affected (ie: not 0.9.8, or 1.0.0). If your version is 1.0.1 and dated earlier than April 7, 2014 then the endpoint is affected. There are alternative ways to test if your PA is affected (eg: perl scripts, modified OpenSSL clients, etc.) but it is not defined in this document.

To Install:

1. Obtain the appropriate RPM package for your zLinux distribution such as:
   RHEL5_RHEL6 –> ibmcim-ssl-1.0.1-rhel5_rhel6.s390x.rpm
   SLES10_SLES11 –> ibmcim-ssl-1.0.1-sles10_sles11.s390x.rpm
   Download here:

http://www-933.ibm.com/support/fixcentral/systemx/selectFixes?parent=Flex+System+Manager+Node&product=ibm/systemx/8731&&platform=All&function=fixId&fixids=Flex1_3_1_Platform_Agents_IT00284

2. Copy the RPM file to the affected endpoint and install the RPM.

rpm -Uvh --force xxxxxxx.rpm

3. Verify that the package was delivered properly by running the version check again. The new OpenSSL version will be 1.0.1 dated April 7, 2014.

/opt/ibm/icc/bin/openssl version

4. The SSL encryption keys/certificates located in the Platform Agent keystore must be reset because we must assume they have been stolen before the patch was applied. To do so do the following on the target agent system:
   1. 1. At the endpoint command set variables by running these commands:
   KEYSTORE_PATH=/etc/opt/ibm/icc/keystore
   ICC_PATH=/opt/ibm/icc
   HOSTNAME=hostname
   OPENSSL_CONF=$KEYSTORE_PATH/…/openssl.cnf
   Note: “`” for the hostname line is the tic not the apostrophe “'”
   2. Regenerate the certificates and keys:
   echo -e “US\nNORTH CAROLINA\nRTP\nIBM\nSTG\n$HOSTNAME\n.\n.\n” | $ICC_PATH/bin/openssl req -x509 -nodes -sha256 -days 3650 -newkey 2048 -config $OPENSSL_CONF -keyout $KEYSTORE_PATH/server.key -out $KEYSTORE_PATH/server.cert
   Note: Running optional step 4 before this command is run will show the the ‘before’ certificate so you can compare it to the ‘after’ certificate if desired.
   3. Stop and restart the CIMOM so it uses the new certificates and keys:
   service cimserverd restart
   4. Optional Step: Run the command below dump the certificate information, including the new public key if desired.
   /opt/ibm/icc/bin/openssl x509 -in $KEYSTORE_PATH/server.cert -noout -text
   5. Optional step: Confirm that the CIMOM is up and running with the new SSL certificate. These commands will list the CIMOM’s namespaces (eg: root/ibmsd) using the SSL connection (the port will normally be 15989 or 5989):
   /opt/ibm/icc/bin/cimcli ns -l <IP_ADDRESS>:<PORT> -s -u <ADMINUSER> -p <PASSWORD>
   6. IMPORTANT: At this point all user IDs that have been used to Request Access to any Platform Agent endpoint must be changed using common password procedures. This protects from an attacker who acquired the private SSL keys when a user in the past Requested Access.
   7. Optional Step: Verify the CIM connection between the IBM Systems Director/Flex System Manager server and the Platform Agent endpoint is still good by right clicking on the endpoint within the console and choose Security–>Verify connection. If this is ok then the keystore SSL certificates ISD uses to communicate with the Platform Agent endpoint are working.
   Note: If the CIM RSAP is locked (or not there) such as when a Common Agent (CAS) is installed then it is unused this step is completely unnecessary.

Important note: IBM strongly suggests that all System z customers subscribe to the System z Security Portal to receive the latest critical System z security and integrity service. If you are not subscribed, see the instructions on the System z Security web site. Security and integrity APARs and associated fixes will be posted to this portal. IBM suggests reviewing the CVSS scores and applying all security or integrity fixes as soon as possible to minimize any potential risk.

AIX Platform Agent 6.3.2 to 6.3.3.1

Introduction

The Platform Agent on AIX and VIOS uses OpenSSL, limiting the exposure to AIX and Virtual I/O Server (VIOS) using OpenSSL versions 1.0.1.500 and 1.0.1.501. These instructions describe how to manually install the Heartbleed fix on an AIX and VIOS systems with the Platform Agent, Common agent, or IBM Systems Director Server’s Common Agent.

Notes and assumptions

• These instructions assume that the affected endpoint is discovered and a Request Access has already been done.
• The Platform Agent is included as part of the Common Agent including the Common Agent on IBM Systems Director (ISD) Servers.
• It is important to remember that this fix is only applicable if Platform Agent’s OpenSSL binary is used for CIMOM communications. If the CIM RSAP is locked or not present, it is not used (for example: The Common Agent CIM RSAP is there but unused, which greatly limits the impact).
• When the SSL keys are reset, the keys will be 2048 bits in length and the certificate signing algorithm will be SHA256.

To install

Note: For VIOS endpoints, you should run the listed command(s) below under oem_setup_env.

1. Determine if the AIX OpenSSL fix is required.
   This fix is applicable to any versions of Director/Platform Agents running on AIX and VIOS versions which has OpenSSL 1.0.1.500 or 1.0.1.501.
   The Platform Agent CIMOM runs OpenSSL on ports 5989 (for example: if OS CIMOM is using 5989) and is used by IBM Systems Director Server/Flex System Manager as the CIM RSAP of the agent security configuration.

To test to see if your agent is affected, open the command prompt and run the following command for AIX and VIOS:
/user/bin/lslpp -l | grep -i openssl

Note: Only the 1.0.1.500 and 1.0.1.501 versions of OpenSSL are affected. If your version is 1.0.1.500 or 1.0.1.501, the endpoint is affected. There are alternative ways to test if your Platform Agent is affected (for example: perl scripts, modified OpenSSL clients, and so on); however, it is not defined in this document.

2. Download and Installation:
   1. Access the AIX/VIOS Heartbleed technical bulletin from here: http://www14.software.ibm.com/webapp/set2/subscriptions/onvdq?mode=18&ID=3549&myns=pwraix53&mync=E
   2. Review the bulletin.
   3. Under Remediation/Fixes, there is a link to obtain OpenSSL 1.0.1.502 image.

3. Verify that the package was installed properly by running the version check given in Step 1 again.

4. The SSL encryption keys/certificates located in the Platform Agent keystore must be reset because we must assume they have been stolen before the fix was applied.
   To do so, do the following on the target agent system:
   1. Steps to Revoke and Regenerate certificates:

   1. PEGASUS_HOME=/opt/freeware/cimom/pegasus/etc

   2. /usr/bin/openssl req -x509 -sha256 -days 3650 -newkey rsa:2048 -nodes -config $PEGASUS_HOME/orig/ssl.cnf -keyout $PEGASUS_HOME/file.pem.new -out $PEGASUS_HOME/cert.pem.new >/dev/null 2>/dev/null

   3. echo $?

   4. If command mentioned in Step iii (above) returns 0 then certificate revoke+generation was successful, in that case execute below commands

/usr/bin/mv -f $PEGASUS_HOME/file.pem.new $PEGASUS_HOME/file.pem >/dev/null 2>/dev/null
/usr/bin/mv -f $PEGASUS_HOME/cert.pem.new $PEGASUS_HOME/cert.pem >/dev/null 2>/dev/null

  5. If command mentioned in Step iii (above) returns nonzero, certificate revoke+generation was not successful. In that case, execute the following commands to clean files and repeat Step 5.  

/usr/bin/rm -f $PEGASUS_HOME/file.pem.new >/dev/null 2>/dev/null
/usr/bin/rm -f $PEGASUS_HOME/cert.pem.new >/dev/null 2>/dev/null

2. Restart CIMSERVER:  

For AIX endpoint:

/usr/bin/stopsrc -s platform_agent
/usr/bin/stopsrc -s cimsys
/usr/bin/startsrc -s cimsys
/usr/bin/startsrc -s platform_agent

For VIOS endpoint (see Note 1 above):

/opt/ibm/director/agent/bin/stopagent_vios
/usr/bin/cimserver –s
/opt/ibm/director/agent/bin/startagent_vios

3. Verify Certificates  

Login to any other endpoint in the same network. Execute the following command:
openssl s_client -connect <name of the system>:5989 -showcerts

5. Optional step: Confirm that the CIMOM is up and running with the new SSL certificate. These commands will list the CIMOM’s namespaces (for example: root/ibmsd) using the SSL connection (the port will normally be 5989):
   /usr/bin/cimcli ns -l <IP_ADDRESS>:<PORT> -s -u <ADMINUSER> -p <PASSWORD>

6. Important Note: At this point, all user IDs that have been used to Request Access to any Platform Agent endpoint must be changed using common password procedures. This protects from an attacker who acquired the private SSL keys when a user in the past Requested Access.

7. Optional Step: Verify the CIM connection between the IBM Systems Director/Flex System Manager server and the Platform Agent endpoint is still good by right clicking on the endpoint within the console and chooseSecurity–>Verify connection. If this is OK, the keystore SSL certificates ISD uses to communicate with the Platform Agent endpoint are working. Note: If the CIM RSAP is locked (or not there) such as when a Common Agent (CAS) is installed, it is unused and this step is completely unnecessary.

Workaround(s) & Mitigation(s):

None known

References:

• Complete CVSS Guide
• On-line Calculator V2
• OpenSSL Project vulnerability website
• Heartbleed

Related Information:
IBM Secure Engineering Web Portal
IBM Product Security Incident Response Blog

Acknowledgement
None

Change History
24 April 2014: Original Copy Published

• The CVSS Environment Score is customer environment specific and will ultimately impact the Overall CVSS Score. Customers can evaluate the impact of this vulnerability in their environments by accessing the links in the Reference section of this Flash.

Note: According to the Forum of Incident Response and Security Teams (FIRST), the Common Vulnerability Scoring System (CVSS) is an “industry open standard designed to convey vulnerability severity and help to determine urgency and priority of response.” IBM PROVIDES THE CVSS SCORES “AS IS” WITHOUT WARRANTY OF ANY KIND, INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. CUSTOMERS ARE RESPONSIBLE FOR ASSESSING THE IMPACT OF ANY ACTUAL OR POTENTIAL SECURITY VULNERABILITY.