Monitoring certificate expiration in Windows on NetXMS
Recently, we faced the task of monitoring the validity of certificates on Windows servers. Well, as she got up, after the certificates were turned into a pumpkin several times, at the very time when the bearded colleague responsible for their extension was on vacation. After that, we were suspicious of something decided to think about it. Since we are slowly introducing the NetXMS monitoring system, it has become the main and, in principle, the only candidate for this task.
The result was finally obtained in this form:
')
And the process itself further.
Go. There is no built-in counter for expiring certificates in NetXMS, so you need to create your own and use scripts to provide it with data. Of course, on Powershell, this is also Windows. The script should read all the certificates in the operating system, take out the expiration date in days and transfer this number to NetXMS. Through his agent. Here we start.
Option one , the easiest. Just get the number of days until the expiration date of the certificate with the next date.
In order for the NetXMS server to know about the existence of our custom parameter, it must receive it from the agent. Otherwise, this parameter cannot be added due to its absence. Therefore, in the configuration file of the agent nxagentd.conf we add an external parameter line with the name HTTPS.CertificateExpireDateSimple , in which we write the script to run:
Considering that the script is launched over the network, you need to not forget about the Execution Policy , as well as not to forget the others "-NoLogo -NoProfile -NonInteractive", which I dropped for better readability of the code.
As a result, the agent config looks like this:
After that, you need to save the config and restart the agent. You can do this from the NetXMS console: open the config (Edit agent's confuguration file), edit, execute Save & Apply, as a result of which, in fact, the same happens. Then re-read the configuration (Poll> Configuration), if there is absolutely no power to wait. After these actions, you should be able to add our custom parameter.
In the NetXMS console, we go to the Data Collection Configuration of the experimental server, on which we are going to monitor the certificates and create a new parameter there (later, after setup, it makes sense to transfer it to the templates). We select HTTPS.CertificateExpireDateSimple from the list, enter a Description with a clear name, type set Integer and set the polling interval. At the time of debugging it makes sense to make it shorter, 30 seconds, for example. Everything is ready, yet enough.
You can check ... no, it's too early. Now, naturally, we will not get anything. Just because the script has not yet written. We correct this omission. The script will give out just a number, the number of days remaining until the expiration of the certificate. The most minimal of all available. Sample script:
It turns out like this:
723 days, until the certificate expires for almost two more years. It is logical, because I rewritten the certificates on the Exchange test bench quite recently.
It was a simple option. Probably, someone and this suit, but we wanted more. We set ourselves the task to get a list of all certificates on the server, by name, and to see the number of days remaining until the certificate expires.
The second option is somewhat more complicated.
Again we edit the agent config and there instead of a line with the ExternalParameter we write two others:
In ExternalList we get just a list of strings. In our case, a list of strings with the names of the certificates. A list of these lines we get a script. The list name is HTTPS.CertificateNames .
Script NetXMS_CertNames.ps1:
And already in the ExternalParameter we input the rows from the ExternalList list, and at the output we get the same number of days for each. The identifier is the Thumbprint certificate. Please note that HTTPS.CertificateExpireDate in this variant contains an asterisk (*). This is necessary in order for it to accept external variables, just our CertificateId.
Script NetXMS_CertExpireDate.ps1:
In the server's Data Collection Configuration we create a new parameter. In the Parameter, select our HTTPS.CertificateExpireDate (*) from the list, and (attention!) Change the asterisk to {instance} . This important point will allow you to create a separate counter for each instance (certificate). The rest is filled as in the previous version:
In order for the counters to be created from, on the Instance Discovery tab, select the Agent List from the list and enter the name of our ExternalList from the script in the List Name field - HTTPS.CertificateNames.
Almost ready, wait a bit or force Poll> Configuration and Poll> Instance Discovery to be done, if it’s impossible to wait. As a result, we get all our certificates with expiration dates:
What you need? Well, yes, only the perfectionism worm looks at this unnecessary Thumbprint in the name of the counter with sad eyes and does not allow to finish the article. In order to feed him, we open the counter properties again and on the Instance Discovery tab, in the “Instance discovery filter script” field, add the script written in NXSL ( NetXMS internal language):
which will filter Thumbprint:
To display it filtered, on the General tab, in the Description field, change CertificateExpireDate: {instance} to CertificateExpireDate: {instance-name} :
Everything, finally the finish from KDPV:
Beauty is the same?
It remains to set up alerts so that they arrive in the mail when the certificate expires at its logical end.
1. First you need to create an Event Template to activate it when the counter value decreases to some threshold we set. In the Event Configuration, create two new templates with the names, say CertificateExpireDate_Threshold_Activate with the Warning status:
and similar CertificateExpireDate_Threshold_Deactivate with the status Normal.
2. Next, go to the properties of the counter and on the Tresholds tab, set the threshold:
where we select our created certificateExpireDate_Threshold_Activate and CertificateExpireDate_Threshold_Deactivate events, set the number of measurements (Samples) 1 (it makes no sense to put more time for this counter), a value of 30 (days), for example, and, importantly, set the event repeating time. For certificates in production, I put it once a day (86400 seconds), otherwise you can drown in alerts (which, by the way, happened once, and so that the mailbox overflowed over the weekend). At the time of debugging it makes sense to put a smaller, 60 seconds, for example.
3. In the Action Configuration, create a letter alert template, such as:
All these% m,% S, etc. - macros in which values ​​from our parameter will be substituted. They are described in more detail in the NetXMS manual .
4. And finally, combining the previous points, in the Event Processing Policy we create a rule according to which the Alarm will be created and the letter sent:
Save the policy, everything can be tested. We set the threshold higher for verification. My closest certificate expires in 723 days, for verification I set 724. As a result, we get the following alarm:
and such notification by mail:
Now everything is exactly. It would be possible, of course, to set up dashboards, to build graphics, but for certificates it would be somewhat meaningless and boring straight lines, in contrast to schedules of CPU or memory usage, for example. But, about this some other time.
The result was finally obtained in this form:
')
And the process itself further.
Go. There is no built-in counter for expiring certificates in NetXMS, so you need to create your own and use scripts to provide it with data. Of course, on Powershell, this is also Windows. The script should read all the certificates in the operating system, take out the expiration date in days and transfer this number to NetXMS. Through his agent. Here we start.
Option one , the easiest. Just get the number of days until the expiration date of the certificate with the next date.
In order for the NetXMS server to know about the existence of our custom parameter, it must receive it from the agent. Otherwise, this parameter cannot be added due to its absence. Therefore, in the configuration file of the agent nxagentd.conf we add an external parameter line with the name HTTPS.CertificateExpireDateSimple , in which we write the script to run:
ExternalParameter = HTTPS.CertificateExpireDateSimple: powershell.exe -File "\\server\share\NetXMS_CertExpireDateSimple.ps1" Considering that the script is launched over the network, you need to not forget about the Execution Policy , as well as not to forget the others "-NoLogo -NoProfile -NonInteractive", which I dropped for better readability of the code.
As a result, the agent config looks like this:
# # NetXMS agent configuration file # Created by agent installer at Thu Jun 13 11:24:43 2019 # MasterServers = netxms.corp.testcompany.ru ConfigIncludeDir = C:\NetXMS\etc\nxagentd.conf.d LogFile = {syslog} FileStore = C:\NetXMS\var SubAgent = ecs.nsm SubAgent = filemgr.nsm SubAgent = ping.nsm SubAgent = logwatch.nsm SubAgent = portcheck.nsm SubAgent = winperf.nsm SubAgent = wmi.nsm ExternalParameter = HTTPS.CertificateExpireDateSimple: powershell.exe -File "\\server\share\NetXMS_CertExpireDateSimple.ps1" After that, you need to save the config and restart the agent. You can do this from the NetXMS console: open the config (Edit agent's confuguration file), edit, execute Save & Apply, as a result of which, in fact, the same happens. Then re-read the configuration (Poll> Configuration), if there is absolutely no power to wait. After these actions, you should be able to add our custom parameter.
In the NetXMS console, we go to the Data Collection Configuration of the experimental server, on which we are going to monitor the certificates and create a new parameter there (later, after setup, it makes sense to transfer it to the templates). We select HTTPS.CertificateExpireDateSimple from the list, enter a Description with a clear name, type set Integer and set the polling interval. At the time of debugging it makes sense to make it shorter, 30 seconds, for example. Everything is ready, yet enough.
You can check ... no, it's too early. Now, naturally, we will not get anything. Just because the script has not yet written. We correct this omission. The script will give out just a number, the number of days remaining until the expiration of the certificate. The most minimal of all available. Sample script:
try { # $lmCertificates = @( Get-ChildItem -Recurse -path 'Cert:\LocalMachine\My' -ErrorAction Stop ) # , "10 " if ($lmCertificates.Count -eq 0) { return 3650 } # Expiration Date $expirationDates = @( $lmCertificates | ForEach-Object { return $_.NotAfter } ) # Expiration Date $minExpirationDate = ($expirationDates | Measure-Object -Minimum -ErrorAction Stop ).Minimum # Expiration Date $daysLeft = [Math]::Floor( ($minExpirationDate - [DateTime]::Now).TotalDays ) # return $daysLeft } catch { return -1 } It turns out like this:
723 days, until the certificate expires for almost two more years. It is logical, because I rewritten the certificates on the Exchange test bench quite recently.
It was a simple option. Probably, someone and this suit, but we wanted more. We set ourselves the task to get a list of all certificates on the server, by name, and to see the number of days remaining until the certificate expires.
The second option is somewhat more complicated.
Again we edit the agent config and there instead of a line with the ExternalParameter we write two others:
ExternalList = HTTPS.CertificateNames: powershell.exe -File "\\server\share\netxms_CertExternalNames.ps1" ExternalParameter = HTTPS.CertificateExpireDate(*): powershell.exe -File "\\server\share\netxms_CertExternalParameter.ps1" -CertificateId "$1" In ExternalList we get just a list of strings. In our case, a list of strings with the names of the certificates. A list of these lines we get a script. The list name is HTTPS.CertificateNames .
Script NetXMS_CertNames.ps1:
# $nameTypeList = @( [System.Security.Cryptography.X509Certificates.X509NameType]::SimpleName, [System.Security.Cryptography.X509Certificates.X509NameType]::DnsName, [System.Security.Cryptography.X509Certificates.X509NameType]::DnsFromAlternativeName, [System.Security.Cryptography.X509Certificates.X509NameType]::UrlName, [System.Security.Cryptography.X509Certificates.X509NameType]::EmailName, [System.Security.Cryptography.X509Certificates.X509NameType]::UpnName ) # , $certList = @( Get-ChildItem -Path 'Cert:\LocalMachine\My' | Where-Object { $_.HasPrivateKey -eq $true } ) # , " - - Thumbprint" foreach ($cert in $certList) { $name = '(unknown name)' try { $thumbprint = $cert.Thumbprint $dateExpire = $cert.NotAfter foreach ($nameType in $nameTypeList) { $name_temp = $cert.GetNameInfo( $nameType, $false) if ($name_temp -ne $null -and $name_temp -ne '') { $name = $name_temp; break; } } Write-Output "$($name) - $($dateExpire.ToString('dd.MM.yyyy')) - [T:$($thumbprint)]" } catch { Write-Error -Message "Error processing certificate list: $($_.Exception.Message)" } } And already in the ExternalParameter we input the rows from the ExternalList list, and at the output we get the same number of days for each. The identifier is the Thumbprint certificate. Please note that HTTPS.CertificateExpireDate in this variant contains an asterisk (*). This is necessary in order for it to accept external variables, just our CertificateId.
Script NetXMS_CertExpireDate.ps1:
# $CertificateId param ( [Parameter(Mandatory=$false)] [String]$CertificateId ) # if ($CertificateId -eq $null) { Write-Error -Message "CertificateID parameter is required!" return } # Thumbprint $CertificateId Expiration Date $certId = $CertificateId; try { if ($certId -match '^.*\[T:(?<Thumbprint>[A-Z0-9]+)\]$') { $thumbprint = $Matches['Thumbprint'] $certificatePath = "Cert:\LocalMachine\My\$($thumbprint)" if (Test-Path -PathType Leaf -Path $certificatePath ) { $certificate = Get-Item -Path $certificatePath; $certificateExpirationDate = $certificate.NotAfter $certificateDayToLive = [Math]::Floor( ($certificateExpirationDate - [DateTime]::Now).TotalDays ) Write-Output "$($certificateDayToLive)"; } else { Write-Error -Message "No certificate matching this thumbprint found on this server $($certId)" } } else { Write-Error -Message "CertificateID provided in wrong format. Must be FriendlyName [T:<thumbprint>]" } } catch { Write-Error -Message "Error while executing script: $($_.Exception.Message)" } In the server's Data Collection Configuration we create a new parameter. In the Parameter, select our HTTPS.CertificateExpireDate (*) from the list, and (attention!) Change the asterisk to {instance} . This important point will allow you to create a separate counter for each instance (certificate). The rest is filled as in the previous version:
In order for the counters to be created from, on the Instance Discovery tab, select the Agent List from the list and enter the name of our ExternalList from the script in the List Name field - HTTPS.CertificateNames.
Almost ready, wait a bit or force Poll> Configuration and Poll> Instance Discovery to be done, if it’s impossible to wait. As a result, we get all our certificates with expiration dates:
What you need? Well, yes, only the perfectionism worm looks at this unnecessary Thumbprint in the name of the counter with sad eyes and does not allow to finish the article. In order to feed him, we open the counter properties again and on the Instance Discovery tab, in the “Instance discovery filter script” field, add the script written in NXSL ( NetXMS internal language):
instance = $1; if (instance ~= "^(.*)\s\-\s\[T\:[a-zA-Z0-9]+\]$") { return %(true, instance, $1); } return true; which will filter Thumbprint:
To display it filtered, on the General tab, in the Description field, change CertificateExpireDate: {instance} to CertificateExpireDate: {instance-name} :
Everything, finally the finish from KDPV:
Beauty is the same?
It remains to set up alerts so that they arrive in the mail when the certificate expires at its logical end.
1. First you need to create an Event Template to activate it when the counter value decreases to some threshold we set. In the Event Configuration, create two new templates with the names, say CertificateExpireDate_Threshold_Activate with the Warning status:
and similar CertificateExpireDate_Threshold_Deactivate with the status Normal.
2. Next, go to the properties of the counter and on the Tresholds tab, set the threshold:
where we select our created certificateExpireDate_Threshold_Activate and CertificateExpireDate_Threshold_Deactivate events, set the number of measurements (Samples) 1 (it makes no sense to put more time for this counter), a value of 30 (days), for example, and, importantly, set the event repeating time. For certificates in production, I put it once a day (86400 seconds), otherwise you can drown in alerts (which, by the way, happened once, and so that the mailbox overflowed over the weekend). At the time of debugging it makes sense to put a smaller, 60 seconds, for example.
3. In the Action Configuration, create a letter alert template, such as:
All these% m,% S, etc. - macros in which values ​​from our parameter will be substituted. They are described in more detail in the NetXMS manual .
4. And finally, combining the previous points, in the Event Processing Policy we create a rule according to which the Alarm will be created and the letter sent:
Save the policy, everything can be tested. We set the threshold higher for verification. My closest certificate expires in 723 days, for verification I set 724. As a result, we get the following alarm:
and such notification by mail:
Now everything is exactly. It would be possible, of course, to set up dashboards, to build graphics, but for certificates it would be somewhat meaningless and boring straight lines, in contrast to schedules of CPU or memory usage, for example. But, about this some other time.
Source: https://habr.com/ru/post/460193/
All Articles