The essentials of security incident response for SMBs

Many small companies, especially startups and SMBs, won’t have full-time security staff when hit with their first security incident. The process looks simple on paper, but several “gotchas” can trip up a team. As a result, understanding how to manage a security incident from start to finish correctly is critical for minimizing damage and preventing future breaches.

This guide provides a lightweight, minimal framework for resolving security incidents. The goal is to enable anyone to confidently and cleanly close out security incidents while minimizing business risk.

Preparation phase

This step is pre-work, designed to make your process smooth and repeatable so you’re not trying to develop a process under pressure.

1. Designate the key players that form your incident response team. Usually, your first indications of a potential incident will be from people (internal employees, external reporters, other companies) who cannot assess the risk or harm themselves and seek someone to take charge.
2. Document the bones of a plan and who has the authority to make decisions about business risk.
   • Who is allowed to declare “all hands on deck”?
   • Who is authorized to shut down business systems or accounts?
   • Who is allowed to communicate with customers on behalf of the company?
   • Who is responsible for making the legal calls regarding breach notification or communicating with law enforcement?
   • At what point is it necessary to bring in professional help? (e.g. an incident response firm)
3. Practice your plan, at least with a tabletop walkthrough, so the team knows where to start rather than devolving to personalities and politics. The outcome of an incident is often determined by how it’s managed in the first 30 minutes.

Identification phase

This step focuses on verifying the incident’s facts to prove it warrants a total response effort. Many investigations never turn into incidents; focus on finding the evidence that proves you have a probable loss of confidentiality, integrity, or availability due to malicious activity.

1. Describe and document the alleged harm. Try to verify it across multiple sources.
   • What systems seem to be affected?
   • What logs or data show indications of harm?
   • What patterns or behaviors stand out as unusual?
2. Activate your incident response team. Notify leadership that you may need additional resources depending on where the investigation takes you. Designate at least an Incident Commander (responsible for driving the incident to close and making business risk decisions) and a Technical Lead (responsible for collecting data, preserving evidence in a forensically sound way, and performing technical analysis).
3. Create a coordination channel. Sometimes teams prefer a Slack channel, a video call, or a phone conference bridge to keep the team aligned. Set the expectation that anyone pulled into the incident must be available and responsive until the harm is contained.

Containment phase

This stage is about scoping and stopping the harm; it’s the most crucial step to prevent further harm. If poorly executed, you may have to revisit the containment stage multiple times, discovering additional affected resources and expanding your exposure window. Focus on being comprehensive rather than quick.

1. Determine and document the full incident scope.
   • What machines were affected?
   • What user accounts were affected?
   • What credentials were affected?
   • What data or records were modified or tampered with?
   • What evidence can you point to that proves a given resource was or was not affected? How confident can you be with the records on hand?
2. Isolate or contain any affected resources.
   • What machines need to be disconnected from the network?
   • What accounts need to be locked?
   • What passwords need to be changed or login sessions suspended?
   • What files or data need to be quarantined, locked, or made read-only?
   • What firewall blocks need to be put in place?
3. Collect relevant evidence. Think about it through the lens of the lawyers, the insurance company, or the customers. What data do you need to prove what happened and what impact the alleged harms will have? This step is critical for future determinations of risk or liability. Evidence includes:
   • Relevant system logs
   • Full disk images (if possible)
   • Full memory images (if possible)
   • Network packet captures
   • Screenshots (e.g. text messages, threats, communication from bad actors)

Eradication phase

This step is about being confident that all sources or artifacts of the intrusion are removed, ensuring bad actors don’t have another way in. Sophisticated attackers usually leave backdoors or alternate “persistence methods” to regain access if you remove their initial point of intrusion.

1. Remove all artifacts of the intrusion, restoring the affected systems to a known-good state.
   • What malware or file system artifacts need to be removed?
   • What patches or compensating controls need to be applied?
   • What backups can you restore from?
   • What machines can be wiped and rebuilt?
2. Validate the new configuration is safe by verifying patch levels, scanning filesystems, or rescanning the network to ensure any holes are closed. Don’t forget adjacent systems.

Recovery phase

This step is about restoring normal operations and monitoring for re-infection or anomalous behavior. Some time must pass after the last signs of intrusion before you can safely declare the incident resolved.

1. Put operational monitoring in place to catch reinfection or resumption of the malicious activity. This could include scripted searches for attacker artifacts, configuration changes, or network traffic indicating the attacker has returned.
2. Restore normal operations by safely bringing up affected systems and re-enabling any affected accounts or resources that were isolated.
3. Document any system or configuration changes necessary to close the incident so they become part of your new baseline.

Lessons Learned phase

This step is about capitalizing on the experience and ensuring this type of incident or root cause cannot happen again. Any repeated root cause findings across incidents should lead to a thorough analysis of why your environment is susceptible to this type of incident and what can be done to prevent it in the future.

1. Document your current understanding of the incident as soon as you can.
   • What indications did you see?
   • How did you prove your scope?
   • What artifacts did you collect?
   • What was the activity timeline from the first signs of an intrusion to now?
2. Hold a retrospective with the other incident responders.
   • What did the team decide were the root causes?
   • How could this incident be prevented, detected, remediated, or recovered faster the next time a similar incident occurs?
     • Consider tools, training, policy, process, and documentation
   • What recommendations or insights do other team members have?
   • What in your retrospective document is incomplete or inaccurate?
   • What perspectives did you miss?
3. Integrate all feedback and team perspectives into the retrospective document.
4. Brief leadership on the incident.
   • Get management buy-in on the required changes
   • Provide resource recommendations for any new tools or capabilities
   • Highlight any organizational or structural issues that slowed the team down
   • Provide an estimate of the time and resources spent
5. Publish the retrospective document internally for other employees to learn from.
6. Deactivate your incident response team. Release your ad-hoc incident response team members back to their day jobs.

Three common pitfalls

Despite our best efforts, incidents can go awry. Below are some common pitfalls that prevent security issues from being closed quickly and cleanly. While every environment is different, these issues will likely impede small businesses without full-time security teams.

Not empowering your incident response team

Whether due to competing business priorities, fear of the consequences of taking systems down, or general “analysis paralysis”, incident responders need to feel like this immediate incident is their one-and-only concern. Everything else can wait.

That level of priority must be recognized by managers and leadership as well—there should only be one Incident Commander at a given time. Outside parties interfering with the flow of the incident can result in incomplete knowledge, sloppy incident handling, a much longer time-to-close, and significantly more damage to the company.

Finally, executives tend to focus on the external perception of the incident (public relations, stock impact, customer disclosure) or fast recovery before all the facts are known. Once the evidence is trampled, it’s nearly impossible to get it back. Security news is full of stories from companies who had repeated re-infections or “rolling remediations” for months because they could not be sure they had fully evicted their intruder. Avoid the temptation to let outside parties drive the incident.

Failure to learn the lesson

Your initial understanding of the incident is rarely your final understanding. Every person on an incident response team has key knowledge and perspective that has to be brought together to provide the proper corrective actions to prevent the incident from happening again.

Create an environment where everyone can speak up, entertain any reasonable theory, and prove/disprove it with the facts. Staring an inconvenient truth in the face and feeling that pain creates the motivation and consensus to close vulnerabilities for good (rather than cutting corners). Likewise, rushing the retrospective or glossing over the paperwork will likely result in repeat root causes and preventable future incidents. Take the time to do it right, once.

Letting the lawyers run the incident

Lawyers have a crucial role in some incidents, especially if mandatory notifications are required (e.g. GDPR Breach Notifications) or regulatory assessments are performed. Pull them in as soon as the incident is fully scoped (Containment) and ensure they have all the evidence and understanding they need to do their part.

They may label the incident “Privileged and Confidential” and want to direct specific response actions (especially during employee investigations) to protect the company from potential lawsuits. Recognize that they should not be making technical decisions on isolating, containing, recovering, or preventing future incidents.

Your Incident Commander is empowered to make the business calls and close the incident clearly and efficiently. Your Technical Lead should provide expertise on the “how”. Finally, suppose lawyers fully “lock up” the incident. In that case, it’s unlikely your team will be able to do a thorough retrospective or inform your customers of the additional steps they might need to take to protect themselves. You owe your customers the chance to defend themselves if their data is exposed.

Effective management of security incidents is not just about immediate containment. Instead, it involves thorough investigation and long-term prevention strategies. Failing to address any phase—identification, containment, eradication, recovery, or lessons learned—exacerbates the situation and puts your organization at greater risk. As cyber threats evolve, staying ahead requires a well-planned, dynamic approach to incident response.

Grow Your Business on the Cloud Provider for Builders

Choose DigitalOcean for a simple cloud solution that drives business growth. Experience reliable cloud computing services, robust documentation, scalability, and predictable pricing.

Sign-up for DigitalOcean