Configuring Kerberos Authentication on Edge SWG with BCAAA

1. Introduction

In enterprise environments, secure web access requires user identity verification, not just IP-based identification. Symantec/Broadcom Edge SWG (Secure Web Gateway, formerly ProxySG/ASG) supports multiple authentication methods. One of the most commonly deployed is Integrated Windows Authentication (IWA).

IWA can use two different protocols: – NTLM: legacy, weaker security and less efficient – Kerberos: modern, ticket-based, secure, high performance, and supports Single Sign-On (SSO) To enable Kerberos on Edge SWG, you need BCAAA (Blue Coat Authentication and Authorization Agent) integrated with Active Directory (AD).

2. Lab Topology

Environment used for testing:

• AD Domain Controller: ad01.example.local, running DNS and KDC
• BCAAA Server: a Windows Server (domain member), running service account example\bcaaa_svc\
• Edge SWG: proxy hostname edge01.example.local:8080
• Client: Windows 10/11 domain member, browser configured to use Edge SWG as proxy

Simplified topology:

[Client] ---> [Edge SWG] ---> [BCAAA Server] ---> [AD Domain Controller]

3. Deployment Steps

3.1 Prepare Service Account in AD

1. Create a domain account in Active Directory Users and Computers (dsa.msc)
   • Username: bcaaa_svc
   • Set password never to expire
   • Uncheck “User must change password at next logon”
2. For Windows SSO/DCQ on Server 2019+, add the account to Server Operators or configure NetSessionEnum ACL rights.

⚠️ Do not grant Domain Admin rights to the service account.

3.2 Install and Configure BCAAA on a Domain Member Server

1. Prepare a Windows Server, join it to the domain example.local.
2. Download the BCAAA installer:
   • Option A: In the Edge SWG Management Console → Authentication → Realms → Advanced Settings, click the download link for BCAAA MSI.
   • Option B: Direct link download (current release):http://appliance.bluecoat.com/sgos/bcaaa/v6/bcaaa_windows32.zip
3. Unzip the installer package (e.g., bcaaa_windows32.zip) and run the .exe setup file.
4. Installation wizard:
   • Click Next to start.
   • Destination Folder → accept the default (C:\\Program Files\\Blue Coat Systems\\BCAAA) or choose another path.
   • Port Number → default is 16101. If you change it, you must configure the same port on Edge SWG and ensure it’s open in firewalls.
5. SSL Settings between BCAAA and Edge SWG:
   • Options: Permitted, Required, or Forbidden.
   • If using SSL, enter the Certificate Subject (hostname of the BCAAA server, not IP). The installer can generate a self-signed cert if none exists.
6. Windows SSO Support:
   • Select Yes if using BCAAA with Windows SSO or IWA/Kerberos.
   • Provide the domain service account (example\\bcaaa_svc) and its password.
   • If not using Windows SSO, you may choose to run under LocalSystem or a domain account. For Kerberos, always use the domain account.
7. Finish installation: the wizard will configure and start the BCAAA service automatically.

3.3 Configure SPNs (Service Principal Names)

Kerberos authentication relies on SPNs. Clients requesting Kerberos tickets must match the SPN registered in AD.

Before registering SPNs, make sure that DNS records exist in AD for the proxy and virtual URL hostnames (e.g., auth.example.local and edge01.example.local). These must resolve to the Edge SWG appliance.

Register SPNs for the BCAAA account:

setspn -S HTTP/edge01.example.local example\bcaaa_svc
setspn -S HTTP/edge01 example\bcaaa_svc

Verify SPNs:

setspn -L example\bcaaa_svc

Check duplicates:

setspn -X

3.4 Configure Realm on Edge SWG

1. In Management Center, launch the SG Admin Console.
2. Go to Configuration → Authentication → Realms and Domains.
3. Create a new Realm (IWA-BCAAA).
4. Configure:
   • BCAAA Server IP
   • Port: 16101 (default)
   • Virtual URL: auth.example.local
5. Test configuration→ should return “Test passed”.

3.5 Join Edge SWG to the Active Directory Domain

1. In Management Center, launch the SG Admin Console.
2. Go to Configuration → Authentication → Realms and Domains.
3. Click Add Domain.
4. In Properties → Domain Name Alias, enter a short alias for the domain (e.g., EXAMPLE).
5. Click Join, then provide:
   • DNS Domain Name: example.local
   • Username: a domain account with permission to join (e.g., administrator)
   • Password: the account password
6. Click Join. When the confirmation message appears, click Close.

3.6 Configure Authentication Policy in VPM

1. Open Visual Policy Manager (VPM).
2. Add an Authentication Layer:
   • Action: Force Authenticate → Auto
   

4. Client-Side Requirements

1. Domain membership: Client must be joined to example.local.
2. DNS:
   • Client DNS must point to AD Domain Controller.
   • Must resolve edge01.example.local and auth.example.local.
3. Time synchronization: Clock skew < 5 minutes between client and DC.
4. Browser settings:
   • IE/Edge/Chrome:
     • add SWG hostname/Virtual URL toLocal Intranet Zone
     • enable “Automatic logon only in Intranet zone”.
   • Firefox: about:config
     • network.negotiate-auth.trusted-uris = .example.local.
5. Proxy configuration:
   • Must use FQDN (e.g., edge01.example.local:8080)
   • IP address will cause Kerberos to fail → fallback to NTLM.

5. Experiment: IP Proxy vs Domain Proxy

5.1 Proxy set as IP (172.16.x.x:8080)

• Browser sends NTLM authentication (Proxy-Authorization: NTLM ...).
• klist shows no HTTP tickets.
• Kerberos not triggered.

5.2 Proxy set as FQDN (edge01.example.local:8080)

• Browser sends Kerberos authentication (Authorization: Negotiate ...).

• klist shows HTTP ticket:Server: HTTP/edge01.example.local @ EXAMPLE.LOCAL
• Kerberos works successfully.

Conclusion: Kerberos requires FQDN and valid SPN; using IP will always fall back to NTLM.

6. Troubleshooting

• No HTTP tickets in klist: check proxy setting (must use FQDN), browser auto-login, SPN config.
• Message: user does not belong to any group of interest: authentication succeeded but user not in policy-relevant AD group.
• Always NTLM fallback: DNS wrong, SPN not matching, duplicate SPN, or time sync issues.

7. Best Practices

• Separate Proxy Hostname and Virtual URL (e.g., edge01.example.local vs auth.example.local).
• Register all necessary SPNs on the BCAAA service account.
• Ensure client proxy configuration always uses FQDN, not IP.
• Install BCAAA on a domain member server, not on the DC.

8. Conclusion

The key to enabling Kerberos authentication on Edge SWG is:

1. Correct BCAAA installation with a dedicated service account.
2. Proper SPN registration aligned with Proxy hostname/Virtual URL.
3. Client environment preparation (domain join, DNS, time, browser).
4. Validation using klist and Policy Trace.

With these steps, Edge SWG can provide secure, seamless Single Sign-On authentication using Kerberos instead of NTLM.