One of the most simple but effective ways of managing your software assets against malware, cyber attacks, and performance issues is to apply patches or updates. However, if done ineffectively, patching can also cause system crashes or may even introduce new vulnerabilities. In this guide, we’ll detail the benefits, common mistakes, and best practices of patch management to help businesses know why, when, and how to apply software and firmware patches.
Why Patch Management Is Important
Before we talk about why patch management is important, we need to underline the importance of patching first. Why patch at all? Software and firmware patches (a.k.a. updates) are primarily designed to accomplish two objectives:
- To fix bugs and/or
- To fix security vulnerabilities.
Meaning, patching can improve a system’s performance, eliminate exploitable vulnerabilities, and address other known issues.
To appreciate how crucial patching really is, let’s recall some of the most recent malware outbreaks that could have been prevented by conscientious patching.
WannaCry outbreak
Last year (2017), ransomware known as WannaCry swept through 150 countries, crippling no less than 200,000 computer systems – including thousands of computers in NHS trusts in England – and amounting to billions of dollars in damages. That massive outbreak could have been prevented by a Windows patch that was already released 2 months before the first wave of infections.
Petya/NotPetya outbreak
About a month after WannaCry struck, another malware known as Petya/NotPetya wreaked havoc in Ukraine, France, Germany, Italy, Poland, the UK, and the US. Although disguised as ransomware, it actually wiped out systems and data. What makes this highly destructive malware interesting for us is that it exploited the same vulnerability targeted by WannaCry.
Because the Petya outbreak happened a month after WannaCry, it meant that businesses put off patching despite the many warnings they likely received from news outlets and social media regarding WannaCry, and consequently paid a heavy price for ignoring them.
The exploitation of known vulnerabilities is integral to most cyberattacks, with threat actors often targeting vulnerabilities in operating systems, web browsers, browser plugins, network devices, security solutions themselves, and other pieces of software. This is the reason why patch management is a key component in defence-in-depth. Some threats can pass through firewalls and evade antivirus software but many of these threats can be stopped in their tracks if there are no vulnerabilities to exploit.
To patch or not to patch now? Why that is also a valid question
When you consider the risk of downtime (and the resulting revenue loss, opportunity loss, etc.) that may be caused by a malware outbreak and how it can be mitigated by patching, the immediate reaction would be to patch as soon and as often as possible. It’s actually not that simple.
A patch can also lead to undesirable results. Let me give you an example. Very recently, security researchers discovered a couple of vulnerabilities in certain Intel processor chips. These vulnerabilities have been dubbed Meltdown and Spectre. Because Intel processors are so ubiquitous, the potential impact (should an actual exploit be developed) is expected to be substantial.
The good news is that patches for these vulnerabilities have already been released for Windows and Linux OSes. But there’s a problem. Those patches are now known to potentially cause a system performance reduction by up to 30%.
That amount of performance hit, especially in a business-critical system, is not something to be sniffed at. That’s just one of the many possible side effects of a patch. Some patches can even spawn new vulnerabilities or cause systems to crash – in turn also causing downtime.
These eventualities are the reasons why patches need to go through intensive quality assurance testing. They need to be verified, reviewed and validated to ensure that they were installed successfully or whether they have any negative impact on the system.
But because QA processes can take time, there exists a conflict between patching and quality assurance. On one hand, companies need to secure their software assets using technical tools and employee training. And on the other, they also need to give ample time for QA to run tests and determine the possible impact on operations and the presence of new vulnerabilities.
These conflicts can be resolved through proper patch management.
Looking for Software Asset Management?
Compare the best SAM solutions in the UK. For FREE.
Common Patch Management Mistakes
Patch management is a crucial and sophisticated process that should never be taken lightly. Here are four patch management mistakes many companies are guilty of doing.
Relying Solely on Technology
Because of the misguided principle of patching as soon and as often as possible, some system administrators are tempted to rely on auto-updates whenever the software supports it. As we now know, some updates can actually break systems. So if you don’t incorporate testing, risk assessments, backout plans, and other critical processes into your patch management activities, you run the risk of incurring lengthy, disruptive downtimes as a result of a defective patch.
Relying Purely on Manual Methods
While it’s bad to rely solely on auto-updates for patching, it’s also equally unacceptable to rely purely on manual methods of patching, even if it involves scripting. Patching endeavours that are 100% manual can be prone to human errors. Besides, it’s tedious, time-consuming, and doesn’t scale well. Hence, it’s particularly unsuitable for large enterprises with a multitude of IT assets.
Absence or Lack of Reports
Just like in any mission-critical process, patch management requires the presence of reliable reports. Whoever is in charge of patching needs to know key information such as which operating systems, applications, hardware, and other systems lack updates or underwent failed updates in order to make informed decisions.
Delegating Patching to End Users
This isn’t just a mistake. It’s outright laziness, and it can lead to cyber or productivity consequences. A large majority of end-users will never bother to carry out updates, as they don’t understand the importance of the activity and the gravity of failing to do it. An organization that leaves patching to their end-users will end up with a network riddled with critical vulnerabilities.
Effective Methods for Patch Management
If those were some of the mistakes organisations make when doing patch management, what then are the best practices for it? Let’s tackle those now.
Collecting Baseline Data
The first thing that needs to be done is to identify the systems in your environment. That way, you’ll know what might need patching. Build an inventory of all your hardware including servers, routers, printers, desktops, laptops, mobile devices, and other network devices. Similarly, build an inventory of all your software including office applications, operating systems, databases, etc. Wherever possible, use a database for storing inventory data of all your IT assets and automated tools for collecting them. That way, your data will always be current.
Using a Test Environment
As mentioned earlier in the article, it is important to test systems prior to patching. This would typically come in the form of a virtual environment, with your servers, operating systems, applications, etc., replicated on virtual machines. Here, you can test how a patch might affect a system before deploying it into production. Results of the tests should be properly documented for future reference.
Devising a Backout Plan
A backout plan is basically a disaster recovery plan. This ensures everything you need for reverting back to a working state is ready in the event a patch goes awry. This backout plan must likewise be tested periodically to ensure that you are able to recover seamlessly and within your RPO (recovery point objective) and RTO (recovery time objective).
Evaluating Patches
Not all patches are created equal. There are patches that are mission-critical and need to be carried out as quickly as possible. There are also those that are simply deemed useful and those that are really not necessary. Apply a risk assessment and devise a system for evaluating patches.
Applying Configuration Management
Once the patches have already been tested and ready to be applied in a production environment, make sure everyone who will and might be needed are on standby. This will include your Help Desk, system owners, personnel assigned to conduct testing after patching, etc.
Applying the Patching Process Conscientiously
The patching process itself should be carried out with care. Some patches can be carried out through automated means. Automation will enable you to patch during off-peak hours to minimise disruptions. Some patches, e.g. patches to servers, need to be closely monitored to enable quick recovery if something goes wrong. Once the patch is complete, monitor the system for any issues and create reports.
Establishing Patch Management Procedures and Policies
Patches should be guided by well-documented policies and procedures. This will ensure that best practices can be replicated even if the people currently in charge of patching will leave your organisation. Because of their possible impact, patching processes are critical to business operations. As such, it is important to carry out these processes in the best possible way. We hope this article will guide you on that path.