SSL/TLS Secure Channel - SCHANNEL - Troubleshooting
When encrypted communication is enabled in YSoft SafeQ 6, the subsystems connect to each other using strong protocols or protocols defined in the underlying operating system. You may encounter some issues when your server runs on an out-of-date OS or when the OS is up to date but your MFD has old firmware. This guide should help you identify and fix problems related to security protocols.
Terminal/MFD Cannot Be installed, the Error "Could not create SSL/TLS secure channel" Appears in the terminalserver.log Log File
This is a general error message meaning that an error occurred during SSL/TLS cipher negotiation, the data encryption/decryption process or the data was corrupted.
The following sections describe several cases that may cause this message to occur.
The Device Only Supports Weak DH Keys
When the device and YSoft SafeQ 6 Server agree on the cipher suite containing a Diffie-Hellman key exchange (DH or DHE) algorithm and the DH keys provided by the device are shorter than Windows' default 1024 bits, Terminal Server, which inherits the security settings from the underlying operating system, it may refuse a connection to such a device.
Conditions/environment
YSoft SafeQ 6 Server (Terminal Server) is running on the Windows Server OS machine with one of the following updates installed:
June 2016 update rollup
KB3161608 (Windows 7, Windows Server 2008 R2)
KB3161606 (Windows RT 8.1, Windows 8.1, Windows Server 2012 R2)
July 2016 update rollup
KB3172605 (Windows 7, Windows Server 2008 R2)
KB3172614 (Windows RT 8.1, Windows 8.1, Windows Server 2012 R2)
June 2016 cumulative update
KB3163018 (Windows 10, Windows Server 2016 Technical Preview 4)
How to recognize this case
Try to connect to the MFD interface via the web browser on the same server as Terminal Server is running. Is the browser able to connect?
Internet Explorer – usually just the general failure appears: "This page can’t be displayed"
Mozilla Firefox – a connection is denied with the error code SSL_ERROR_WEAK_SERVER_EPHEMERAL_DH_KEY
Scan the device using the following nmap command:
nmap --script ssl-enum-ciphers -p DeviceSecurePort DeviceAddressA list of SSL/TLS protocol versions and cipher suites supported by the device similar to the following picture should display.
In parentheses after the DHE cipher suites is a DH key length. If this number is lower than 1024, you may have encountered this case.
How to fix this case
On Windows servers, this can only be avoided by setting the required DH key length lower using registers. You can add the registry key manually or use the following fix:
Create a dh.reg file with the following content:
Windows Registry Editor Version 5.00[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\KeyExchangeAlgorithms\Diffie-Hellman]"ClientMinKeyBitLength"=dword:00000200Open this file in the machine with Terminal Server running. Windows should automatically create the given registry key.
Try to install the device again.
Do not set a lower length of DH keys unless really necessary. Supporting DH keys of lower than 1024 bits makes your machine vulnerable to some attacks.
The Incorrect Implementation of the DH Algorithm in the Device
When using a cipher suite containing the Diffie-Hellman key exchange (DH or DHE) algorithm, occasionally, a device detects a fatal error resulting in an SSL/TLS session termination.
Conditions/environment
YSoft SafeQ 6 (Terminal Server) is running on the Windows Server OS machine with one of the following updates installed:
June 2016 update rollup
KB3161608 (Windows 7, Windows Server 2008 R2)
KB3161606 (Windows RT 8.1, Windows 8.1, Windows Server 2012 R2)
July 2016 update rollup
KB3172605 (Windows 7, Windows Server 2008 R2)
KB3172614 (Windows RT 8.1, Windows 8.1, Windows Server 2012 R2)
June 2016 cumulative update
KB3163018 (Windows 10, Windows Server 2016 Technical Preview 4)
How to recognize this case
The installation of a Konica Minolta device randomly crashes on different installation steps when trying reinstallation several times.
How to fix this case
On Windows servers, this can only be avoided by disabling the DH key exchange algorithm using registers. You can add the registry key manually or use the following fix:
Create a dh.reg file with the following content:
Windows Registry Editor Version 5.00[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\KeyExchangeAlgorithms\Diffie-Hellman]"Enabled"=dword:00000000Open this file in the machine with Terminal Server running. Windows should automatically create the given registry key.
Try to install the device again.
This registry modification disables DH/DHE ciphers for the whole machine, thus, no communication links dependent on the operating system settings will be able to use them.
The MFD Only Supports Older SSL/TLS Protocol Versions, but also Accepts Higher Ones
YSoft SafeQ 6 Server needs to connect to the device during terminal installation. YSoft SafeQ Server and the device negotiate the SSL/TLS protocol version they will be communicating with. During this negotiation, the parties always try to connect using the highest protocol version both of them support. The problem here is, for example, when YSoft SafeQ Server supports TLSv1.2 and the device does not but pretends it does, they agree on this version and the connection fails.
How to recognize this case
Try to connect to the MFD interface via a web browser (with TLSv1.2 enabled) on the same server as Terminal Server is running on. Is the browser able to connect?
Mozilla Firefox – the connection is denied with the error code SSL_ERROR_BAD_MAC_ALERT
Google Chrome – the connection is denied with the error code ERR_SSL_BAD_RECORD_MAC_ALERT
How to fix this case
In the YSoft SafeQ 6 management interface, go to System settings (Expert options) and search for the property securityProtocolTypesForOutboundCommunication. Set this system property to only the SSL/TLS protocol versions supported by your MFD.
Be careful when modifying this property. Some other devices may stop communicating when the SSL/TLS protocol versions are not set properly.
Communication with YSoft SafeQ Payment System Fails with the Error "Could not create SSL/TLS secure channel" in the terminalserver.log Log File
Older Windows servers do not support cooperation of TLSv1.2 and certificates using the SHA-512 hash function. This causes failures in the connection to YSoft SafeQ Payment System.
Conditions/environment
YSoft SafeQ Server (Terminal Server) is running on the Windows Server OS machine without one of the following updates installed:
August 2014 update rollup
KB2973337 (Windows 7, Windows Server 2008 R2)
KB2975331 (Windows RT 8, Windows 8, Windows Server 2012)
KB2975719 (Windows RT 8.1, Windows 8.1, Windows Server 2012 R2)
How to recognize this issue
Try to connect to Payment System from the Terminal Server machine via Internet Explorer with TLSv1.2 enabled. Is it able to connect?
Internet Explorer – the error message is displayed: "Make sure TLS and SSL protocols are enabled."
Be aware that this is only for troubleshooting purposes. YSoft SafeQ no longer supports Internet Explorer.
How to fix this issue
If you encounter this issue with TLSv1.2 enabled on the machine with Terminal Server running (in the registry or in the securityProtocolTypesForOutboundCommunication expert system property), apply the following update to this machine https://support.microsoft.com/en-us/kb/2973337 .
The Device Supporting Only Higher Versions of the SSL/TLS Protocol Cannot Connect to YSoft SafeQ 6 Server
Older Windows servers do not have higher versions of SSL/TLS protocol enabled. Terminal Server inherits these settings from the system, and, thus, devices only supporting higher versions cannot connect.
Conditions/environment
YSoft SafeQ 6 Server (Terminal Server) is running on the Windows Server OS machine with one of the following systems:
Windows Server 2008 R2 with KB3080079 installed
Windows Server 2012
Windows Server 2012 R2
How to recognize this issue
Scan the machine with Terminal Server running using the following nmap command:
nmap --script ssl-enum-ciphers -p 5012 MachineAddressA list of SSL/TLS protocol versions and cipher suites supported by Terminal Server should display.
Scan the device using the nmap command:
nmap --script ssl-enum-ciphers -p DevicePort DeviceAddressA list of SSL/TLS protocol versions and cipher suites supported by the device should display.
If the protocols supported by the device and the protocols supported by Terminal Server are exclusive lists, you have encountered this issue.
How to fix this issue
On Windows servers, this can only be avoided by setting the required SSL/TLS protocol versions using registers. You can add the registry keys manually or use the following fix:
Create a tls.reg file with the following content:
Windows Registry Editor Version 5.00[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 3.0\Server]"Enabled"=dword:00000000"DisabledByDefault"=dword:00000001[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0\Server]"Enabled"=dword:00000000"DisabledByDefault"=dword:00000001[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.1\Server]"Enabled"=dword:00000001"DisabledByDefault"=dword:00000000[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server]"Enabled"=dword:00000001"DisabledByDefault"=dword:00000000These values will disable support for SSLv3 and TLSv1.0 and enable TLSv1.1 and TLSv1.2. Of course, you can modify them before proceeding to the next step according to your requirements.
Open this file is in the machine with Terminal Server running, Windows should automatically create the given registry keys.
Restart the machine with Terminal Server.
The Device Is Installed Correctly, but Cannot Connect to YSoft SafeQ 6 Server
This usually happens when the device is not able to verify the YSoft SafeQ 6 Server (Terminal Server) certificate or the server certificate uses a cryptographic algorithm not supported by the device.
The Certificate Is Not Trusted
By default, the communication between the terminal and Terminal Server is encrypted using the same default certificate for all YSoft SafeQ 6 deployments. This is not trusted in the MFDs, so a new key/certificate needs to be created and Terminal Server configured to use it (follow the chapter Configuring secured connection between terminals and Terminal Server). Also, the certificate must be imported to the device as trusted.
The Device Does Not Support the Cryptographic Algorithm Used for the Certificate or the Key Length
Since YSoft SafeQ 6 MU17 (and newer builds), the default certificate in the Terminal Server uses the algorithm SHA-256 (instead of the previous SHA-1) and has an RSA key of 2048 bits long.
Some older devices do not support either the SHA-256 algorithm or RSA keys of a maximum length of 1024 bits. When it really is a must to communicate in an encrypted way with such a device, a new key/certificate compatible with it needs to be created. For more information about the Terminal Server certificate setting see Configuring secured connection between terminals and Terminal Server .