Archives 2018

Rescanning ESXi storage in a parallel way

Lately, I’ve been doing a lot of work provisioning and decommissioning LUNs on some Fibre Channel arrays. As you all know, this process can be somewhat tedious when provisioning a new LUN on vSphere or removing said LUN.

In the past, I would always use this PowerCLI command

This would first get all the ESXi hosts in the cluster “Cluster1” and then launch the refresh cmdlet on each host. The problem with this is that it can take a really long time for the command to complete because the refresh is executed host by host. Before starting the next refresh, the command waits for the previous one to finish. While this is fine for smaller environments, the time really does add up when you have a decent sized cluster.

Using PowerShell jobs

In order to reduce the time this process takes, I decided this would be a great use case for PowerShell jobs. A job is essentially another PowerShell instance that runs in the background. This means that your main script or PowerShell window doesn’t wait for it to finish before going to the next step. Using jobs will allow us to start the rescan on each host as a job and then start the next job. Meaning the rescans will happen in parallel.

Code

The script is provided as is and can be found on GitHub


ERROR: The host returns esxupdate error code:15

Today I was preparing some updates with VUM on Lenovo servers running vSphere 6.0. Things did not go as expected. While staging the patches I was greeted with this error.

ERROR: The host returns esxupdate error code:15. The package manager transactions is not successful. Check the update Manager log files and esxupdate.log files for more details.

Looking into the esxupdate.log file I could find the following entries:

This seems to be a known issue for Lenovo servers. It turns out they cram a whole lot of drivers in there, more than other vendors. The only known workaround is to remove VIBs from the host that are not needed. This is also described in https://kb.vmware.com/s/article/2144200.

To find out what VIBs are installed, log onto the affected host using SSH and run this command:

Once you have identified a driver that you want to remove, put the host in maintenance mode and this command:

In my case I wasn’t using the qlogic nx2 driver so I removed it. Now you should be able to update the host using VUM,


Event ID 1000 rundll32.exe_aepdu.dll

While doing my morning check of our monitoring system I came across a strange issue…Two of our servers had been unavailable during the night, this had also happened the night before. Time to dig a bit deeper!

Both servers are domain controllers running Server 2012 R2, no updates were installed in the last few weeks and nothing had been changed either. Pulling up the event log of both servers, I found the exact same event logged on both machines at the time they were unavailable to our monitoring system.

After I quick search, I found out that aeinv.dll is part of the Microsoft Customer Experience Improvement Program (CEIP). If you’re signed up to the CEIP, which you are by default when you install Server 2012 R2, three scheduled tasks will run each night to upload your anonymized date to Microsoft.

  • AitAgent
  • Microsoft compatibility Appraiser
  • ProgramDataUpdater

We had the unavailability after the first and third task ran.

Luckily you can easily opt out of the CEIP by opening Server Manager, going to Local Server and clicking on the link behind Customer Experience Improvement Program.

There you will see that the radio button, Yes, I want to participate in the CEIP is selected. Click on No, I don’t want to participate and click OK.

You’re now opted out and the scheduled tasks will no longer run.

UPDATE 11/07: Turns out that just opting out of the CEIP does not actually prevent the scheduled tasks from running. In order to fully disable them, you need to disable them through the task scheduler or, even better, disable them through PowerShell