UAG and FIPS Compliance – How to implement 3DES in your SSLF environment


During the hardening of our DMZ domain we had applied the recommended Windows Server 2008 R2 SSLF Specialized Security Limited Functionality templates. What most people will know about the joys of implenting hardening policies is that you are bound to break every single application and the UAG is no exception.

If you apply the local security policy setting (as you should) “System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing” you will break the UAG.  The problem is as documented here. What you are doing by enabling this security policy is informing applications that they should only use cryptographic algorithms that are FIPS 140 compliant and in compliance with FIPS approved modes of operation.

In my case I could hit my login page fine but as soon as I got authenticated and passed through to the portal in my trunk I saw an error message:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.
[HttpException (0x80004005): Unable to validate data.]
   System.Web.Configuration.MachineKeySection.EncryptOrDecryptData(Boolean fEncrypt, Byte[] buf, Byte[] modifier, Int32 start, Int32 length, IVType ivType, Boolean useValidationSymAlgo, Boolean signData)  +4308283

Yep, it was a complete mystery to me too.

The reason it happens is because ASP.NET 2.0 uses the RijndaelManaged implementation of the AES algorithm when it processes view state data. The RijndaelManaged implementation has not been certified by the National Institute of Standards and Technology (NIST) as compliant with the Federal Information Processing Standard (FIPS). Therefore, the AES algorithm is not part of the Windows Platform FIPS validated cryptographic algorithms and web pages are not served correctly.

The workaround is to configure ASP.NET to use 3DES instead of RijndaelManaged AES and is documented here:

  1. In a text editor such as Notepad, open the application-level Web.config file. In my case this was D:\Program Files\Microsoft Unified Access Gateway\von\PortalHomePage\web.config
  2. In the Web.config file, locate the <system.web> section.
  3. Add the following <machineKey> section to in the <system.web> section: <machineKey validationKey=”AutoGenerate,IsolateApps” decryptionKey=”AutoGenerate,IsolateApps” validation=”3DES” decryption=”3DES”/>
  4. Save the Web.config file.
  5. Restart the Microsoft Internet Information Services (IIS) service. To do this, run the following command at a command prompt: iisreset
  6. If you have additional UAG servers in an array, run iisreset on the other UAG servers.
  7. Test connection to your trunk. If you can login succesfully and hit your portal page, continue to next step.
  8. Enable the “System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing” GPO setting.
  9. Run gpupdate /force on all servers in your array.
  10. Re-Test connection to your trunk. If you can login succesfully and hit your portal page, continue to next step.

Microsoft never make it easy, eh!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s