You may be unable to connect to a VPN server running on OS X Server. This may be related to how Windows handles IPSec NAT traversal by default.  This article will explain how to change this behavior to allow VPN connections to OS X Server VPN.

The steps in this article involve making changes to the Windows Registry using the Registry Editor (Regedit). Even if you are very comfortable editing the registry, you should make a backup of the registry prior to editing it. Making mistakes in Regedit can cause Windows issues or can prevent Windows from starting. The changes may cause the software that installed the entries to not work correctly until you restore the entries. Follow the appropriate article below for steps on how to back up your Windows Registry.

Change Windows IPSec NAT traversal behavior

1. Open Registry Editor:

   XP: From the Start menu choose Run. In the resulting dialog, type "regedit" (without quotes) and click OK.
   Vista: From the Start menu, in the Start Search dialog, type "regedit" (without quotes) and press Enter.
   Windows 7: From the Start menu, in the "Search programs and files" dialog, type "regedit" (without quotes) and press Enter.

2. If Windows needs your permission to continue, click Continue.
3. Click the plus sign (XP) or arrow (Vista and Windows 7) next to HKEY_LOCAL_MACHINE to expand its contents.
4. Expand the contents of SYSTEM.
5. Expand the contents of CurrentControlSet.
6. Expand the contents of services.
7. Click to select (you just need to highlight this folder) the folder called PolicyAgent.
8. From the File menu, choose Export.
9. In the dialog box that appears, make sure the "Selected branch" option is enabled. Then, save the file somewhere that you can find it later, such as the desktop; this is a backup of this Windows Registry key. You should keep this in case you need to reimport your original settings later.
10. Make sure PolicyAgent is still selected, from the Edit menu, choose New and select DWORD (32-bit) Value.

11. Edit the name of the value to be: "AssumeUDPEncapsulationContextOnSendRule" (without quotes) and press Return.

12. Double-click "AssumeUDPEncapsulationContextOnSendRule" and set the Value data to 2.

13. Click OK.
14. Close Registry Editor.

Change the Local Security Policy

1. Open Local Security Policy:
   XP: From the Start menu choose Run. In the resulting dialog, type "secpol.msc" (without quotes) and click OK.
   Vista: From the Start menu, in the Start Search dialog, type "secpol.msc" (without quotes) and press Enter.
   Windows 7: From the Start menu, in the "Search programs and files" dialog, type "secpol.msc" (without quotes) and press Enter.
2. Click the plus sign (XP) or arrow (Vista and Windows 7) next to Local Policies to expand its contents.
3. Click to select (you just need to highlight this folder) the folder called Security Options.
4. On the right hand side of the Local Security Policy, locate and double-click on Network security: LAN Manager authentication level.
5. In the drop-down list, select "Send LM & NTLM - use NTMLv2 session security if negotiated".

6. Click OK.
7. On the right hand side of the Local Security Policy, locate and double-click on: "Network security: Minimum session security for NTML SSP based (including secure RPC) clients".
8. Uncheck "Require 128-bit encryption".

9. Click OK.
10. Close Local Security Policy.
11. Restart the computer.

You should now be able to create a VPN connection on Windows and be able to connect to OS X Server VPN.

Important: Mention of third-party websites and products is for informational purposes only and constitutes neither an endorsement nor a recommendation. Apple assumes no responsibility with regard to the selection, performance or use of information or products found at third-party websites. Apple provides this only as a convenience to our users. Apple has not tested the information found on these sites and makes no representations regarding its accuracy or reliability. There are risks inherent in the use of any information or products found on the Internet, and Apple assumes no responsibility in this regard. Please understand that a third-party site is independent from Apple and that Apple has no control over the content on that website. Please contact the vendor for additional information.