Call this the perfect storm upgrade. If you have to perform a vSphere 5, vShield 5 and Trend DS 8 upgrade (whether or not you happen to have a vBlock 300HX), read the following for what TO do and what NOT to do!
The main caveats to remember when performing this upgrade are:
- vShield Endpoint v3.x and vShield Endpoint v5.x are NOT compatible.
- You cannot upgrade to the latest VMware Tools if you have the old endpoint thin agent installed on your Windows VMs. It has to be removed first.
Your final approach will depend on whether you are upgrading your hosts with VUM or rebuilding them withvia ISO. I took the ISO route as I thought it would be cleaner.
Before we get started, there is some documentation you should read:
- vSphere 5 Upgrade Guide including vCenter, ESXi
- vShield 5 Quick Start guide
- Trend Manager 8 Getting Started Guide
Step-by-Step Deployment Guide:
I’ll tell you what you should do to avoid the pain and suffering I went through. If you prefer testing the upgrade on a single host to ensure the process works, update accordingly. It will still work.
- Upgrade Trend Manager to v8
- Power of all your VMs except Trend appliances.
- De-activate your Trend Appliances from Trend Manager
- You should see the Trend service account in Virtual Center updating the configuration (.vmx) files of all your VMs.
- Confirm all VFILE line entries have been removed from the VMs .vmx files before continuing
- Power off and delete your Trend appliances from Virtual Center
- Put all hosts into Maintenance mode.
- Remove Virtual Center from Trend Manager.
- Login and un-register vShield Manager 4.1 from Virtual Center
- Power off vShield Manager 4.1
- Disconnect and remove all hosts from cluster
- Upgrade Virtual Center to v5
- If any your hosts are disconnected during the upgrade, just reconnect them.
- Upgrade VMware Update Manager to v5
- Deploy vShield Manager v5
- Register vShield Manager v5 with Virtual Center
- Rebuild hosts manually with vanilla ISO
- Setup management IP address on each host
- Add hosts back into the cluster
- Patch hosts with VUM and apply any host profiles
- Add hosts back to the 1000V if present
- Setup all vDS virtual adapters
- Add virtual center back into the Trend Manager
- Deploy vShield Endpoint v5 driver to all hosts
- Ensure vShield Manager is reporting Endpoint is installed before continuing
- Deploy Trend 8 dvfilter-dsa to all hosts via Trend Manager
- Ensure Trend Manager is reporting hosts are prepared before continuing
- Deploy and activate all Trend 8 virtual appliances
- Ensure all virtual appliances are reporting as ‘vShield Endpoint: Registered’
- Power on your VMs
- Remove vShield Endpoint Thin Agent from all your Windows VMs and reboot
- Upgrade VMware Tools on all your VMs, ensuring vShield option is selected. Reboot required.
- Confirm all VMs are protected by the local virtual appliance. Anti-malware should report ‘real time’.
- Update all your DRS groups as all the hosts and appliances will have been removed.
If you want to upgrade, rather than rebuild, do the following between steps 3 and 4:
- Uninstall Trend filter (dvfilter-dsa) from all hosts
- Uninstall Endpoint v3 filter (epsec_vfile) from all hosts
and upgrade vShield Manager instead of deploying new version. Refer to Page 29 of the vShield Quick Start Guide.
Things to Watch Out For:
Steps 2 and 3 are crucial.
Step 2 – vShield Endpoint v3 includes a loadable kernel module (LKM) called VFILE, which loads into the kernel on a vSphere 4.1 host at boot up. Whenever a VM is powered on, on a host running the VFILE LKM, the virtual machine’s .vmx file is updated with the following two line entries:
VFILE.globaloptions = “svmip=169.254.50.39 svmport=8888?
scsi0:0.filters = “VFILE”
vShield endpoint v5 does not do this! No VFILE LKM is loaded, no VFILE line entries are added to the .vmx files of the VMs. Therefore if you do not correctly decommission vShield Endpoint v3, your VMs will not power on, on your vSphere 5 hosts.
This is implied in the vShield 5 Quick Start guide on Page 31 under ‘Upgrading vShield Endpoint’:
2. Deactivate all Trend DSVAs. This is required to remove vShield related VFILE filter entries from the virtual machines.
What they don’t tell you above though is that all your VMs must be powered off. If you de-activate your Trend appliances while your VMs are on, well mine just had their .vmx files updated again immediately afterwards!
If you missed that step the first time around, you’ll have to manully update the .vmx file of every virtual machine to remove the vfile line entries as per KB1030463.
Step 3 – If you don’t remove and re-add Virtual Center from Trend Manager after you have installed vShield Manager 5, your DS virtual appliances will not register with vShield Endpoint.
Step 7 – First time I deployed vShield Manager 5 I didn’t have any issues, although I did have to re-deploy it a 2nd time as it stopped synchronising with vCenter. Unfortunately then it no longer recognised vShield Endpoint was installed and I had to rebuild all my hosts.
Besides these issues, things went relatively smoothly. Its just a matter of time.