Trend Micro Deep Security 8 \ vShield Endpoint EPSEC and UNC\SMB Scanning


‘Another day another dollar’…. no no that’s not quite right…

‘Another day another Deep Security issue.’

Next up is Deep Security and UNC\SMB Scanning. This isn’t exactly Deep Security’s fault as this is another limitation of the vShield EPSEC driver.

Executables that are accessed via a SMB share will loop and need to be manually killed. This is a known limitation of the EPSEC driver as disccused in Trend Micro KB1059280.

In our case one of our applications was launching an executable from a UNC path which was crashing. We couldn’t figure out why but unmanaging the virtual machine fixed the problem.

It is relatively easy to fix this problem, but it does leave you exposed.

The EPSEC driver does not support exclusions of a particular server name, i.e. \\servername, nor can you exclude a directory on the server nor can you exclude a specific file even if you know the name. The only way to fix this problem is to exclude all UNC paths\SMB scanning, by updating your security policy and adding the exclusion ‘\\’ to your Directory exclusion list.

Even unticking ‘Scan Network Drives’ from within Trend has no effect. This has been raised as an incident and I am yet to hear back from Trend Micro.

I have been assured by VMware this will be resolved in vShield 5.1 released in Q4 2012.

Advertisements

New vShield Endpoint Driver available to improve Deep Security 8 performance


Thanks to http://www.joulupukki.nl/wordpress/?p=523 for alerting me to this issue.

VMware made a pre release of the new vShield Endpoint Driver (5.0.0.2 build-813867) available last week to customers who are experiencing issues with their current vShield Driver. This will be released in Q4 but if you are using an anti malware product in your virtual environment that relies on vShield Endpoint Driver I would contact VMware to get the patch.

This hotfix needs to be applied on top of vShield Endpoint Driver build 652273 which is available with the VMware tools included with ESXi 5 Express Patch 3 (build 702118).

In the words of VMware this fixes two main issues: performance issues with network files and sharing violation issues.

1. Sharing violations – It was discovered that, while you had the thin agent installed and real-time AV scanning running, if you opened a file on a network share a few times in quick succession, the 3rd or 4th attempt could result in the file being locked. This was due to the lack of caching for network files, which is the recommend AV practice, but caused this locking

2. Performance issues – This was to due to the general overhead when our thin agent called some MS filter methods.

I also found that this version fixes a BSOD issue with vsepflt.sys. More about that in my next post.

Trend Deep Security Warning Message ‘Machine was unprotected during move from one esx host to another’


I wanted to post some more information on this Trend DS error message – ‘Machine was unprotected during move from one esx host to another’ as it seems to come up regularly.

The description of the error message is, ‘a virtual machine was moved to an ESX that does not have an activated Deep Security Virtual Appliance.’

In essence this warning message is saying that the ESXi host you vMotioned your VM too is not currently protecting the virtual machine.

This can be because there is no virtual appliance on the target ESXi host, the Trend Virtual Appliance is not offering Anti Malware protection, is not Activated or is Offline.

This error message will not show for unactivated virtual machines — A virtual machine has to be activated to generate this error message.

There is a known bug with this error message too – even though your VM is being protected by the appliance, the error message is always reported as an Agent error. Apparently Trend are working on this.

Back to the error message: When you receive this error message, what is the next step?

Trend is a complicated beast – An appliance can have issues for a number of reasons – whether there is a fault with the appliance or one of its dependencies is what you need to figure out. It could be something as basic as the appliance dropping off the network, losing connectivity back to the DSM or to the vShield Endpoint VMkernel port, or possibly its no  longer activated (not registered as a security appliance in vShield Manager.)

If you get this warning  message, open the virtual appliance that the VM is currently residing on and first ‘Clear Warnings/Errors’  so you remove any old status\error messages and then run ‘Check Status’ to see if there are any new issues. If there are errors reported on the appliance try and resolve them by following the patented ‘Trend DS Virtual Appliance Health Check’ below.

My main bugbear with Trend is that it is too complicated and it does not report its current state accurately and concisely. When I run a Check Status I want to know exactly what is going on. It would be most useful to have a health check screen on the appliance where the health check tests I mention below in the article are run sequentially in full view for the benefit of the administrator. Issue could be highlighted immediately and it would give us confidence that the appliance and its dependencies are all configured correctly, rather than having to check all the different components individually.

For example if you check the status of your appliance and it reports back that it is Managed and Online you would expect it to be managed, online and offering anti malware protection. In my testing after I changed the vShield VMkernel IP address on my ESXi host from 169.254.1.1 to 169.254.1.2, so the appliance could not offer anti malware protection, I ran a Check Status and the virtual appliance would still report that it was managed, online and offering anti malware protection.

On the plus side when I migrated a VM to the ESXi host with the misconfigured VMkernel port, the warning message was still generated that the VM is unprotected. What this shows is this error message is symptomatic of an underlying issue with your virtual appliance or ESXi host. While the issue may not be immediately noticable because the DSM reports that all is well, you should dig deeper following the ‘Trend DS Virtual Appliance Health Check’ below.

Bottom line — You cannot fully trust the DSM when you notice this error message. The only way to verify for sure that the appliance is actually working or not would be to drop the EICAR virus on the VM to confirm whether anti malware protection is working.

‘Trend DS Virtual Appliance Health Check’:

  1. Synchronise your Virtual Center(s) in Trend DSM
  2. Confirm your credentials for VVC and vShield are uptodate
  3. Confirm filter driver is installed on ESXi host via Trend DSM
  4. Confirm vShield driver is installed on ESXi host via vShield Manager
  5. Confirm Trend Appliance is registered as Security VM with vShield Manager
  6. Confirm the appliance is in the correct VLAN
  7. Confirm the appliance network configuration is correct
  8. Confirm you can ping the Appliance from the DSM.
  9. Confirm the VMkernel IP address for vShield Endpoint is correct on ESXi host – 169.254.1.1

and if nothing works follow my last resort:

10. Deactivate and reactivate the appliance

And if that fails…. Follow the blocksandbytes ‘Triple D’ process:

11. Deactivate, Delete and Deploy the appliance.

When I’m being lazy and I know the config hasn’t changed I will Deactivate and reactivate the appliance immediately. What I find with Trend is that as long as your environment is static, Trend will continue to stay Green, but if your environment is fairly dynamic and hosts are being rebooted, VMs are being built and vMotioned, you are performing SRM fail overs and fail backs, etc. it struggles to keep up with environment changes.

Every week I have to try and figure out why virtual machines are unhappy and do not have anti-malware protection. Hopefully this will help others stay on top of Trend DS 8.