Rescanning ESXi storage in a parallel way

Lately, I’ve been doing a lot of work provisioning and decommissioning LUNs on some Fibre Channel arrays. As you all know, this process can be somewhat tedious when provisioning a new LUN on vSphere or removing said LUN.

In the past, I would always use this PowerCLI command

Get-VMhost -Cluster "Cluster1" | Get-VMHostStorage -Refresh

This would first get all the ESXi hosts in the cluster “Cluster1” and then launch the refresh cmdlet on each host. The problem with this is that it can take a really long time for the command to complete because the refresh is executed host by host. Before starting the next refresh, the command waits for the previous one to finish. While this is fine for smaller environments, the time really does add up when you have a decent sized cluster.

Using PowerShell jobs

In order to reduce the time this process takes, I decided this would be a great use case for PowerShell jobs. A job is essentially another PowerShell instance that runs in the background. This means that your main script or PowerShell window doesn’t wait for it to finish before going to the next step. Using jobs will allow us to start the rescan on each host as a job and then start the next job. Meaning the rescans will happen in parallel.

Code

The script is provided as is and can be found on GitHub


Get exclusions for all Veeam jobs

This will be another short one, but I figured someone else will have run into this.

While I was doing a rework for a Veeam implementation, I noticed on several jobs that there were exclusions set inside the jobs. I wanted a list of all jobs with their respective exclusions, time for Powershell!

The script starts by getting a list of all Veeam jobs. Next, it will go through all jobs and look for objects that have the type “Exclude” set. What follows is a bit of code to match the job name to the different exclusions and dump everything into a CSV. I struggled a bit with getting the contents of the array listed properly in the CSV, I kept getting the array listed as “System Object[]“. Turns out I just needed to put the $VMExclusions variable between quotes.

The CSV will look something like this, the job name is listed on the left and the excluded objects on the right.

excludedvms-csv

One last note, this script needs to be run from the Veeam server itself.

As always, you can find the most recent version of the script on Github. The initial version can be found below.

Add-PSSnapin VeeamPSSnapin

$JobList = Get-VBRJob
$ExclusionList = @()
foreach($Job in $JobList)
{
    #Get exclusion per job
    $exclusions = $job.GetObjectsInJob() | Where-Object {$_.Type -eq "Exclude"}
    $VMExclusions = @()
    foreach ($exclusion in $exclusions)
    {
        $VMExclusions += ($exclusion.name)
    }
    
    $ExclusionList += New-Object -typeName psobject -Property @{
        Name = $job.Name
        ExcludedVMs = "$VMExclusions"
    }
}

$ExclusionList | Export-Csv ExcludedVMs.csv -delimiter ";" -NoTypeInformation

 

 


Remove Logs

A couple of our web servers were running into some issues with disk space. Turns out the logs weren’t being cleaned up properly.
In order to remediate this, I wrote a function that can be reused anywhere.

The function accepts 3 parameters:

  • FilePath: The directory where you want to remove the logs from
  • CutOff; Specifies the age (in days) that a file must have before being deleted.
  • LogPath; This parameter specifies the directory where you want the CSV log file. If this is left open, no CSV will be saved.

Example

As an example, I’m going to remove all files, older than 30 days, from the folder “C:\temp\W3SVC2030036971\W3SVC2030036971\”. I want a CSV log to be written in the “C:\temp” folder.

remove-logs1

A view from the Powershell window

As you can see, you always get feedback in your window, even if you don’t specify a log path.

The CSV looks something like this:

remove-logs2

Code

You can find the script code below. This is provided as-is. You can find the most recent version of the code on GitHub.

#Requires -Version 3.0

Function Remove-Logs{ 

<#
.SYNOPSIS
  Delete old log files
.DESCRIPTION
  This function will delete all .log and .txt files older than a certain number of days.
  You have the ability to specify log path, extension (log or txt) and path.
.NOTES
  Author:  Maarten Van Driessen
.PARAMETER FilePath
  Specify the path where you want to delete the logs from.
.PARAMETER CutOff
  Specify how many days you want to go back.
.PARAMETER LogPath
  Where do you want to store the log file. Path must end with a \
.PARAMETER FileExtension
  Specify whether you want to delete *.log, *.etl or *.txt files. If left blank, the function will delete both.
  Must be written as *.log, *.etl or *.txt
.EXAMPLE
  Remove-Logs -FilePath C:\inetpub\ -Cutoff 30 -LogPath c:\temp\
  
  Delete all txt and log files older than 30 days from the c:\inetpub folder and write the log to c:\reports
.EXAMPLE
  Remove-Logs -FilePath C:\inetpub\ -Cutoff 20 -LogPath c:\temp\ -Filter *.txt
  
  Delete all txt files older than 20 days from the c:\inetpub folder and write the log to c:\reports
#>

[Cmdletbinding()]
#Parameters
Param
(
    #Check to see if there are any invalid characters in the path
    [Parameter(Mandatory=$true)][ValidateScript({
            If ((Split-Path $_ -Leaf).IndexOfAny([io.path]::GetInvalidFileNameChars()) -ge 0) {
                Throw "$(Split-Path $_ -Leaf) contains invalid characters!"
            } Else {$True}
        })][string]$FilePath,
    #You can only delete logs older than 1 day
    [Parameter(Mandatory=$true)][ValidateRange(1,365)][int]$Cutoff,
    #Check to see if there are any invalid characters in the path
    [ValidateScript({
            If ((Split-Path $_ -Leaf).IndexOfAny([io.path]::GetInvalidFileNameChars()) -ge 0) {
                Throw "$(Split-Path $_ -Leaf) contains invalid characters!"
            } Elseif(-Not ($_.EndsWith('\'))){
                Throw "Logpath must end with \ !"
            } Else {$True}
        })]$LogPath="",  
    
    #you can only delete log and txt files
    [ValidateSet("*.log","*.txt","*.etl")][string]$FileExtension ="*.*"
    
)

    #Check if the file & log path exist
    if(-not (Test-Path $FilePath))
    {
        throw "Invalid file path! Please make sure you enter a valid path."
    }
    #If statement to handle optional parameter
    elseif($LogPath -eq "") {}
    elseif(-not (Test-Path $LogPath))
    {
        throw "Log path does not exist! Please make sure you enter a valid path."
    }

    $CutOffDate = (Get-Date).AddDays(-$Cutoff)

    #Get all items inside the path
    $LogFiles = Get-ChildItem -Path $FilePath -Filter $FileExtension | Where-Object{$_.LastWriteTime -lt $CutOffDate}
    $DeletedItems = @()
    if($LogFiles)
    {
        foreach($LogFile in $LogFiles)
        {
            #Remove all items
            $DeletedItems += $LogFile
            $LogFile | Remove-Item
            Write-Host "Removing file $LogFile" -ForegroundColor Green
        }
        if($LogPath)
        {
            $LogName = Get-Date -Format "yyyy-MM-dd"          
            $DeletedItems | select Name,creationtime,lastwritetime | Export-Csv -path "$LogPath\DeletedLogs - $LogName.csv" -Delimiter ";" -NoTypeInformation -Append
        }
    }
    else 
    {
        Write-Host "No files were found." -ForegroundColor Red
    }
}