Because this security system uses a layered architecture, you should begin with an MCP application (like Web Transaction Server or SocketSupport library) and work toward the system layer at the base to find the malfunctioning portion of the system. The modules to check are
-
Web Transaction Server
-
SocketSupport library
-
FTP
-
TCP/IP and SSL/TLS
-
MCAPISUPPORT library
-
MCP Cryptographic Services Manager of Security Center
-
Security Center/Socket Router
-
IPsec
Web Transaction Server
Use the command, NA ATLASSUPPORT STATUS, to check for the following status information:
-
Verify that SSL/TLS is active and linked. If the link to SSL/TLS or the SSL/TLS function is inactive for more than a few minutes, then check TCP/IP. See “TCP/IP and SSL/TLS” later in this section.
-
Verify that SSL/TLS-based ports are present and active. If the ports are inactive and SSL/TLS is active, check to make sure that the service name of the port matches exactly the service name entered through the MCP Cryptographic Services Manager snap-in of Security Center.
An example display of the status (showing only the relevant lines) follows:
NA ATLASSUPPORT STATUS SSL Configured, Linked To SSL, SSL Function Active Port Status: 80 Security = None Active Listens = 10 Total Connections = 10 443 Security = SSL Active Listens = 40 Total Connections = 40 CGI Status: WEBPCM Enabled Not Linked
Also, check the Web Transaction Server (WebTS) error log for details of any problems with SSL/TLS.
SOCKETSUPPORT Library
The SOCKETSUPPORT library is a shared library and is implicitly linked when called. To debug potential problems with SSL/TLS through sockets, do one of the following:
-
Set the socket option SO_DEBUG on the particular socket.
-
Enable socket tracing through the following system command:
NW TCPIP DEBUG TRACE + SOCKETS
This command places the diagnostic information into the system SUMLOG for analysis.
-
Ensure that the SSL/TLS run-time key has been installed. If there is a waiting entry indicating that the run-time key has not been installed, install the run-time key.
FTP
Refer to the TCP/IP Distributed Systems Services Operations Guide for troubleshooting suggestions for FTP.
TCP/IP and SSL/TLS
Use the command, NW TCPIP STATUS, to check for the following status:
-
Verify that TCP/IP is up and running. If TCP/IP is inactive, issue the NW TCPIP + command.
-
Verify that SSL/TLS is up and running. If SSL/TLS has been terminated, issue the NW TCPIP OPT + SSL command.
-
If TCP/IP Security is ENABLED and TCP/IP is SESSION ESTABLISHMENT PROHIBITED, check to see if the TCP/IP security rules file is present on the same pack as the TCPIPSECURITY library and the TCP/IP security rules file is the same version level as the TCPIPSECURITY code file. Also ensure that no errors have been logged while processing the rules file. The errors can be found in the system sumlog.
-
If SSL/TLS is “waiting for cryptography,” see “MCAPISUPPORT Library” later in this section.
Refer to the Troubleshooting section in the TCP/IP Implementation and Operations Guide for additional troubleshooting suggestions for TCP/IP.
An example display of the status follows:
NW TCPIP STATUS TCPIP IS CURRENTLY NETWORKING, RIP IS CURRENTLY ENABLED/RUNNING, TCPIP SECURITY IS CURRENTLY RUNNING SSL IS CURRENTLY RUNNING SSH IS CURRENTLY RUNNING IPSEC IS CURRENTLY ENABLED/RUNNING (IPV6 ONLY)
MCAPISUPPORT Library
Use the command, NA MCAPI CRYPTOPROXY, to check for the following status:
-
Verify that MCAPI is up and running. If MCAPI is inactive, issue the NA MCAPI + command. If MCAPI is still not initiated, check the distributed systems services entries found under “Installing MCP Cryptographic Services” earlier in this section.
-
Check for CryptoProxy services found and their locations. If none are found, then check WIN RPC. If none or not all environments are found, verify that MCP Cryptographic Services has been installed and is running in all Windows environments. In the Windows environment, from the Start menu, select Settings, then Control Panel, then Administrative Tools, and Services to verify that CryptoProxy services are installed and running.
-
Check on the status of the CryptoProxy services with the NA MCAPI STATUS command.
-
Verify that the Windows environment is visible from the MCP environment.
MCP Cryptographic Services Manager
Ensure that the service names for NXATLAS keys are the same as the service names defined through the Web Transaction Server Site Manager (Ports dialog).
Ensure that the container specified by the FTP SSL_SERVICENAME option is the same as a service name for NXATLAS keys.
Ensure that the service names for SOCKETS keys are the same as the service names set through the sockets application by means of setsockopt( ).
Security Center/Socket Router
For information on troubleshooting Security Center, see Introduction to Security Center.
IPsec
If IPsec is not working correctly, examine the response to the NW TCPIP STATUS command. The response shows the status of the IPsec module. For example,
TCPIP IS CURRENTLY NETWORKING (IPV6-ENABLED), RIP IS CURRENTLY ENABLED/RUNNING, TCPIP SECURITY IS CURRENTLY RUNNING, SSL IS CURRENTLY RUNNING, IPSEC IS CURRENTLY ENABLED/RUNNING (IPV6 ONLY)
If the state of IPsec is shown as WAITING FOR CRYPTOGRAPHY, then follow the troubleshooting information under “TCP/IP and SSL/TLS” for the cryptography environment.
If the state of IPsec is shown as WAITING FOR SECURITYCENTER, then check the waiting entries to see whether TCP/IP Networking could find the Security Center executable or ensure that the Security Center database has been initialized and is ready for use.
If IPsec is running, a detailed inquiry (NW TCPIP STATUS IPSEC) shows the status of the IPsec module as well as the current IPsec policy settings:
IPSEC IS CURRENTLY ENABLED/RUNNING (IPV6-ONLY) KEY EXCHANGE METHODS SUPPORTED = MANUAL, ENCRYPTION ALGORITHMS SUPPORTED = 3DES_CBC, AES_CBC INTEGRITY ALGORITHMS SUPPORTED = AUTH_HMAC_SHA1_96, SECURITY POLICIES = 28
An extended detail inquiry (NW TCPIP STATUS IPSEC ALL) also shows all IPsec security polices.
If all of the IPsec policies have been installed, then examine the SUMLOG for any possible discarded IPsec packets.
A TCPIP IPsec report is logged when a message is explicitly discarded by the IPsec module. When a message is discarded, an entry including detailed information such as the IP addresses, the message protocol, any IPsec information, and the port addresses (if applicable), or the message code and message type (ICMP) is stored in a buffer.
Before an entry is stored in the buffer, a check is made to see whether a similar entry already exists. If a similar entry exists, the count stored within the matched entry is incremented.
Every 3 minutes, the entries in the buffer are written to the log and the buffer is cleared. These actions minimize the effect of a Denial of Service attack on logging if large amounts of IPsec traffic, which is to be discarded, hits an MCP environment.
The following is a sample IPsec report:
TCPIP IPSEC REPORT: DISCARD REQUEST = LOCAL IP: FE80::2A0:D2FF:FEA5:E9F5, REMOTE IP: 2001:0DB8::1428:57AB, MESSAGE PROTOCOL: ICMPv6, MESSAGE TYPE: 5, MESSAGE CODE: 11 TRAFFIC: INBOUND COUNT: 5 MESSAGE DISCARDED DUE TO ABSENCE OF A SECURITY POLICY MATCH LOCAL IP: FE80::2A0:D2FF:FEA5:E9F5, REMOTE IP: 2001:0DB8::1428:57AB, MESSAGE PROTOCOL: UDP, LOCAL PORT: 4444, REMOTE PORT: 11111 TRAFFIC: INBOUND COUNT: 5 MESSAGE EXPLICITLY DISCARDED BY A SECURITY POLICY
The following are possible reasons for IPsec to discard a packet:
-
A message is explicitly discarded by a security policy.
-
A message is discarded because the security policies do not match.
-
No cryptography is available.
-
IPsec is disabled or not running.
-
Either the MCP host networking or network device (CNA, IEA-IOP, and so on) does not support IPsec.
-
An internal software error has occurred.
An IPsec Message Discard summary report is displayed on the ODT when a TCP/IP IPsec Message Discard report is logged to the SUMLOG.
SSL/TLS
If SSL/TLS is not working correctly, examine the response to the NW TCPIP STATUS SSL command. The command returns a detailed summary of the SSL/TLS module. For example, the following response is displayed if SSL/TLS is waiting for cryptography:
NW TCPIP STATUS SSL SSL IS CURRENTLY WAITING FOR CRYPTOGRAPHY
If the status of SSL/TLS shows as WAITING FOR CRYPTOGRAPHY, then check the MCAPISUPPORT/crypto environments to make sure that they are ready.
If the status of SSL/TLS shows as DISABLED/NOT RUNNING, then turn the option for SSL/TLS on (NW TCPIP OPT + SSL).
If the status of SSL/TLS shows as RUNNING, the response shows the versions of TLS and the ciphers that are supported.
NW TCPIP STATUS SSL SSL IS CURRENTLY RUNNING VERSIONS SUPPORTED = TLS 1.2 CIPHERS SUPPORTED = TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA256, RSA_WITH_AES_128_GCM_SHA256, RSA_WITH_AES_256_GCM_SHA384, DHE_RSA_WITH_AES_128_GCM_SHA256, DHE_RSA_WITH_AES_256_GCM_SHA384, ECDHE_ECDSA_AES_128_CBC_SHA256, ECDHE_ECDSA_AES_256_CBC_SHA384, ECDHE_RSA_WITH_AES_128_CBC_SHA256, ECDHE_RSA_WITH_AES_256_CBC_SHA384, ECDHE_ECDSA_AES_128_GCM_SHA256, ECDHE_ECDSA_AES_256_GCM_SHA384, ECDHE_RSA_WITH_AES_128_GCM_SHA256, ECDHE_RSA_WITH_AES_256_GCM_SHA384
Client Authentication
When a secure connection is not established due to SSL/TLS handshake failure and if the client authentication feature was enabled, then the user must view the SUMLOG/SECURITYLOG for more information.
The client system can send either a X.509 certificate or a NULL certificate. The server system can either accept or reject the certificate the client provides based on what mode is selected on the server system for client authentication.
For example, the following entries will be in the SUMLOG when the client sends a NULL certificate and the server is set to either the DESIRED or REQUIRED mode of client authentication:
13:56:33.8105 7782 TCPIP SSL The client at x.x.x.x sent a
NULL certificate, which is NOT accepted.When the client sends a certificate whose CA certificate is not installed on the server ROOT store, then you will see entries as mentioned below if the REQUIRED mode of client authentication is selected on the server side:
14:07:22.7309 7782 TCPIP SSL The client at x.x.x.x sent an
invalid/untrusted certificate, which is NOT
accepted.When the client sends a certificate whose CA certificate is not installed on the server ROOT store and the client authentication mode is set to DESIRED, you will see the following entries:
14:40:34.4815 7783 TCPIP SSL The peer at x.x.x.x sent a
untrusted certificate, which is valid.
14:40:34.5016 7783 TCPIP SSL Certificate accepted from
x.x.x.x, SUBJECT= C=US, S=Pennsylvania,
L=Malvern, O=Unisys Corporation,
.......When the client sends a certificate whose CA certificate is pre-installed on the server ROOT store and the client authentication mode is set to REQUIRED, you will see the following entries:
14:47:11.6290 7782 TCPIP SSL Certificate accepted from
x.x.x.x, SUBJECT= C=US, S=Pennsylvania,
L=Malvern, O=Unisys Corporation,
......For more information on which client authentication mode on the server system accepts different types of certificates (NULL, trusted, untrusted) from the client, refer to the TCP/IP Distributed Systems Services Operations Guide.
Secure Shell (SSH)
If SSH is not working correctly, examine the response to the NW TCPIP STATUS SSH command. The command returns a detailed summary of the SSH module. For example, the following response is displayed if SSH is disabled or not running.
NW TCPIP STATUS SSH SSH IS CURRENTLY DISABLED/NOT RUNNING
If the status of SSH shows as WAITING FOR CRYPTOGRAPHY, then check the MCAPISUPPORT/crypto environments to make sure that they are ready.
If the status of SSH shows as DISABLED/NOT RUNNING, then turn the option for SSH on (NW TCPIP OPT + SSH).
If the status of SSH shows as RUNNING, the response shows the versions of the supported algorithms and user authentication methods.
NW TCPIP STATUS SSH
SSH IS CURRENTLY RUNNING
KEY EXCHANGE ALGORITHMS SUPPORTED:
DIFFIE-HELLMAN-GROUP14,
DIFFIE-HELLMAN-GROUP1
ENCRYPTION ALGORITHMS SUPPORTED:
AES256-CBC,
AES128-CBC,
AES256-CTR,
AES128-CTR
MAC ALGORITHMS SUPPORTED:
HMAC-SHA2-256
HMAC-SHA1
HOST KEY ALGORITHMS SUPPORTED:
SSH-RSA
USER AUTHENTICATION METHOD SUPPORTED:
PUBLIC KEY,
PASSWORD 