Compress and Archive IIS logs using Powershell Script – Introduction

We have observed that in our daily routine task, we have to perform some cleanup tasks once in a while or when disk space is full.
The low disks space may lead to cause service outage to the customer if that unnoticed.
As we know most of the disk space gets occupied by log files and that could be any log files on windows or third party application running on windows.

Exchange servers can consume lot of IIS log files over time.As a best practice administrators configure IIS to store logs on a different disk to avoid problems, rest will wait for free disk space alerts and manually remove old logs from time to time.

Objective

There are number of PowerShell scripts availble in online for automating the cleanup of IIS log files. Here is the objectives for this script,

• No other dependencies
• Must compress log files into monthly archive zip files, not just delete them
• Must have the ability to store the zip files in a central archive location
• once the script completed ( either through manual or auto scheduled method ), the output log will get attached and will send mail notification to the individual or team which mentioned in the recipient list portion of this script

In this article , I will be showing very simple Power Shell cleanup script that will clean IIS log files. we can use this sample and modify it to fit to other log files cleanup tasks as well like IMAP, transport logs or logs files generated by any third party applications

How this IIS Logs Compress and Archive script will works

This PowerShell script to compress and archive IIS log files.

This script will check the folder that we specify, and any files older than the first day of the previous month will be compressed into a zip file and the zip file will be moved to that location which specified

The recommended use for this script is a once-monthly scheduled task run on the first day of each month. This will compress all files older than the first day of the previous month, resulting in only 1 to 2 month of log files being stored on the server.

If the script detects any issues with the archive process that may indicate that a file was missed it will not delete the log files from the folder.
The script also writes a log file each time it is run so you can check the results or troubleshoot any issues.

Once the script completed , the end part of the script will trigger the notification to the team with the resulted output log file to verify .
The recipient will verify the log and check if they found any error.

Note : Please test the script in the test environment first or at least make sure you have backed up your IIS log files before trying this script for the first time.

You can download the script from TechNet Script Gallery also but this article contains the modified version of the script which also included with mail notification

How to run this script

The script takes two parameters,

Log path – this is a mandatory parameter to specify the path to the IIS logs you want to clean up, such as “C:\inetpub\logs\LogFiles\W3SVC3″
Archive Path – this is an optional parameter to specify the path to the central archive location, such as “\nas01\archives\iislogs”

Example Usage

This example will compress the log files in “D:\inetpub\logs\LogFiles\W3SVC3” and leavethe zip files in that location.

.\IISLogsCleanup.ps1 -Logpath “D:\inetpub\logs\LogFiles\W3SVC3”

This example will compress the log files in “D:\inetpub\logs\LogFiles\W3SVC3” and move the zip files to the archive path.

.\IISLogsCleanup.ps1 -Logpath “D:\IIS Logs\W3SVC1” -ArchivePath “Z:\IISLogsBackup\archive\W3SVC3”

Scheduling the script

To run the script as a scheduled task use the following task settings

• Run whether user is logged on or not
• I would recommend to trigger the script on first day of each month
• Action: Start a program
• Program: powershell.exe
• Batch file Arguments: C:\WINDOWS\system32\WindowsPowerShell\v1.0\PowerShell.exe -PSConsoleFile “E:\Exchsrvr\Bin\exshell.psc1” -command “. ‘Z:\Monitoring\MailStats\IISLogsCleanup.ps1 -Logpath “D:\IIS Logs\W3SVC1” -ArchivePath “Z:\IISLogsBackup\archive\W3SVC3′”

Download the script

You can copy paste the below script content in to the notepad and save it as .Ps1 file and use it as per your need.

The script starts from the below ,

#-------------------------------------------------
#  Variables
#-------------------------------------------------

$sleepinterval = 5

$computername = $env:computername

$now = Get-Date
$currentmonth = ($now).Month
$currentyear = ($now).Year
$previousmonth = ((Get-Date).AddMonths(-1)).Month
$firstdayofpreviousmonth = (Get-Date -Year $currentyear -Month $currentmonth -Day 1).AddMonths(-1)

$myDir = Split-Path -Parent $MyInvocation.MyCommand.Path
$output = "$myDir\IISLogsCleanup.log"
$logpathfoldername = $logpath.Split("\")[-1]

#...................................
# Logfile Strings
#...................................

$logstring0 = "====================================="
$logstring1 = " IIS Log File Cleanup Script"


#-------------------------------------------------
#  Functions
#-------------------------------------------------

#This function is used to write the log file for the script
Function Write-Logfile()
{
	param( $logentry )
	$timestamp = Get-Date -DisplayHint Time
	"$timestamp $logentry" | Out-File $output -Append
}

# This function is to test the completion of the async CopyHere method
# Function provided by Alain Arnould
function IsFileLocked( [string]$path)
{
    If ([string]::IsNullOrEmpty($path) -eq $true) {
        Throw “The path must be specified.”
    }

    [bool] $fileExists = Test-Path $path

    If ($fileExists -eq $false) {
        Throw “File does not exist (” + $path + “)”
    }

    [bool] $isFileLocked = $true

    $file = $null

    Try
    {
        $file = [IO.File]::Open($path,
                        [IO.FileMode]::Open,
                        [IO.FileAccess]::Read,
                        [IO.FileShare]::None)

        $isFileLocked = $false
    }
    Catch [IO.IOException]
    {
        If ($_.Exception.Message.EndsWith(“it is being used by another process.”) -eq $false)
        {
            # Throw $_.Exception
            [bool] $isFileLocked = $true
        }
    }
    Finally
    {
        If ($file -ne $null)
        {
            $file.Close()
        }
    }

    return $isFileLocked
}


#-------------------------------------------------
#  Script
#-------------------------------------------------

#Log file is overwritten each time the script is run to avoid
#very large log files from growing over time

$timestamp = Get-Date -DisplayHint Time
"$timestamp $logstring0" | Out-File $output
Write-Logfile $logstring1
Write-Logfile "  $no"
Write-Logfile $logstring0w


#Check whether IIS Logs path exists, exit if it does not
if ((Test-Path $Logpath) -ne $true)
{
    $tmpstring = "Log path $logpath not found"
    Write-Warning $tmpstring
    Write-Logfile $tmpstring
    EXIT
}


$tmpstring = "Current Month: $currentmonth"
Write-Host $tmpstring
Write-Logfile $tmpstring

$tmpstring = "Previous Month: $previousmonth"
Write-Host $tmpstring
Write-Logfile $tmpstring

$tmpstring = "First Day of Previous Month: $firstdayofpreviousmonth"
Write-Host $tmpstring
Write-Logfile $tmpstring

#Fetch list of log files older than 1st day of previous month
$logstoremove = Get-ChildItem -Path "$($Logpath)\*.*" -Include *.log | Where {$_.CreationTime -lt $firstdayofpreviousmonth -and $_.PSIsContainer -eq $false}

if ($($logstoremove.Count) -eq $null)
{
    $logcount = 0
}
else
{
    $logcount = $($logstoremove.Count)
}

$tmpstring = "Found $logcount logs earlier than $firstdayofpreviousmonth"
Write-Host $tmpstring
Write-Logfile $tmpstring

#Init a hashtable to store list of log files
$hashtable = @{}

#Add each logfile to hashtable
foreach ($logfile in $logstoremove)
{
    $zipdate = $logfile.LastWriteTime.ToString("yyyy-MM")
    $hashtable.Add($($logfile.FullName),"$zipdate")
}

#Calculate unique yyyy-MM dates from logfiles in hashtable
$hashtable = $hashtable.GetEnumerator() | Sort Value
$dates = @($hashtable | Group -Property:Value | Select Name)

#For each yyyy-MM date add those logfiles to a zip file
foreach ($date in $dates)
{
    $zipfilename = "$Logpath\$computername-$logpathfoldername-$($date.Name).zip"

    if(-not (test-path($zipfilename)))
    {
        set-content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
        (dir $zipfilename).IsReadOnly = $false 
    }

    $shellApplication = new-object -com shell.application
    $zipPackage = $shellApplication.NameSpace($zipfilename)

    $zipfiles = $hashtable | Where {$_.Value -eq "$($date.Name)"}

    $tmpstring = "Zip file name is $zipfilename and will contain $($zipfiles.Count) files"
    Write-Host $tmpstring
    Write-Logfile $tmpstring

    foreach($file in $zipfiles) 
    { 
        $fn = $file.key.ToString()
        
        $tmpstring = "Adding $fn to $zipfilename"
        Write-Host $tmpstring
        Write-Logfile $tmpstring

        $zipPackage.CopyHere($fn,16)

        #This sleep interval helps avoids file lock/conflict issues. May need to increase if larger
        #log files are taking longer to add to the zip file.
        do
        {
            Start-sleep -s $sleepinterval
        }
        while (IsFileLocked($zipfilename))
    }

    #Compare count of log files on disk to count of log files in zip file
    $zippedcount = ($zipPackage.Items()).Count
    
    $tmpstring = "Zipped count: $zippedcount"
    Write-Host $tmpstring
    Write-Logfile $tmpstring
    
    $tmpstring = "Files: $($zipfiles.Count)"
    Write-Host $tmpstring
    Write-Logfile $tmpstring

    #If counts match it is safe to delete the log files from disk
    if ($zippedcount -eq $($zipfiles.Count))
    {
        $tmpstring = "Zipped file count matches log file count, safe to delete log files"
        Write-Host $tmpstring
        Write-Logfile $tmpstring
        foreach($file in $zipfiles) 
        { 
            $fn = $file.key.ToString()
            Remove-Item $fn
        }

        #If archive path was specified move zip file to archive path
        if ($ArchivePath)
        {
            #Check whether archive path is accessible
            if ((Test-Path $ArchivePath) -ne $true)
            {
                $tmpstring = "Log path $archivepath not found or inaccessible"
                Write-Warning $tmpstring
                Write-Logfile $tmpstring
            }
            else
            {
                #Check if subfolder of archive path exists
                if ((Test-Path $ArchivePath\$computername) -ne $true)
                {
                    try 
                    {
                        #Create subfolder based on server name
                        New-Item -Path $ArchivePath\$computername -ItemType Directory -ErrorAction STOP
                    }
                    catch
                    {
                        #Subfolder creation failed
                        $tmpstring = "Unable to create $computername subfolder in $archivepath"
                        Write-Host $tmpstring
                        Write-Logfile $tmpstring

                        $tmpstring = $_.Exception.Message
                        Write-Warning $tmpstring
                        Write-Logfile $tmpstring
                    }
                }

                if ((Test-Path $ArchivePath\$computername\$logpathfoldername) -ne $true)
                {
                    try
                    {
                        #create subfolder based on log path folder name
                        New-Item -Path $ArchivePath\$computername\$logpathfoldername -ItemType Directory -ErrorAction STOP
                    }
                    catch
                    {
                        #Subfolder creation failed
                        $tmpstring = "Unable to create $logpathfoldername subfolder in $archivepath\$computername"
                        Write-Host $tmpstring
                        Write-Logfile $tmpstring

                        $tmpstring = $_.Exception.Message
                        Write-Warning $tmpstring
                        Write-Logfile $tmpstring
                    }
                }

                #Now move the zip file to the archive path
                try 
                {
                    #Move the zip file
                    Move-Item $zipfilename -Destination $ArchivePath\$computername\$logpathfoldername -ErrorAction STOP
                    $tmpstring = "$zipfilename was moved to $archivepath\$computername\$logpathfoldername"
                    Write-Host $tmpstring
                    Write-Logfile $tmpstring
                }
                catch
                {
                    #Move failed, log the error
                    $tmpstring = "Unable to move $zipfilename to $ArchivePath\$computername\$logpathfoldername"
                    Write-Host $tmpstring
                    Write-Logfile $tmpstring
                    Write-Warning $_.Exception.Message
                    Write-Logfile $_.Exception.Message
                }   
            }
        }

    }
    else
    {
        $tmpstring = "Zipped file count does not match log file count, not safe to delete log files"
        Write-Host $tmpstring
        Write-Logfile $tmpstring
    }

}


#Finished
$tmpstring = "Finished"
Write-Host $tmpstring
Write-Logfile $tmpstring
Start-Sleep -s 10
$From = "Reports.messaging@computercarriage.com"
$To = "User1@computercarriage.com”
$Cc = "User1@computercarriage.com"
$Subject = "IIS Logs Cleanup status"
$Body = "
Hi Team, find the attached log file and check the same for errors if found !


"
$Attachment = "Z:\IISLogsBackup\archive\W3SVC3\IISLogsCleanup.log"
$SMTPServer = "SMTP Server name space (or) IP address"
Send-MailMessage -From $From -to $To -Cc $Cc -Subject $Subject -Body $Body -BodyAsHtml -SmtpServer $SMTPServer -Attachments $Attachment
#...................................
# Finished
#...................................

Home