Incident Response & Digital Forensics: NIST 800-61, Runbooks & Evidence Handling - Cybersecurity Hub

← Back to Home Incident Response & Digital Forensics: NIST 800-61, Runbooks & Evidence Handling Table of Contents Introduction: When Detection Becomes Response NIST 800-61 Incident Response Lifecycle Phase 1: Preparation Phase 2: Detection & Analysis Phase 3: Containment, Eradication, Recovery Phase 4: Post-Incident Activity Digital Forensics Fundamentals Evidence Handling & Chain of Custody Legal Considerations Conclusion Introduction: When Detection Becomes Response Incident response is the operational phase of cybersecurity—the moment when an alert becomes an investigation, when detection becomes containment, and when chaos becomes structured recovery. NIST Special Publication 800-61 provides the canonical lifecycle that most mature programs adapt. Digital forensics is the evidentiary arm of incident response—the disciplined collection, preservation, and analysis of digital evidence to answer "what happened, when, and how" with courtroom-grade rigor. NIST 800-61 Incident Response Lifecycle NIST SP 800-61r2 defines four phases, with recent drafts adding "Incident Analysis Coordination" as an implicit phase. Four-Phase Lifecycle Phase Activities Output 1. Preparation Build team, runbooks, tools, comms, training, exercises IR plan, on-call rotation, war room 2. Detection & Analysis Triage alert, scope, Identify, validate Validated incident, scope, severity 3. Containment, Eradication, Recovery Limit damage, eliminate threat, restore service Contained, clean systems restored 4. Post-Incident Activity Lessons learned, metrics, updates Postmortem report, improvements Phase 1: Preparation 80% of successful incident response happens before the incident—preparation compresses time-to-contain and prevents panic. Preparation Checklist # 1. IR Plan (Document) # - Roles and responsibilities (IR lead, comms, legal, IT) # - Severity definitions (SEV1-SEV4) # - Escalation paths (24/7 on-call) # - External partners (law enforcement, IR retainer) # - Contact list (updated quarterly) # - Approval: CISO, General Counsel # 2. Runbooks (per scenario) # - Phishing (email, attachment, link) # - Ransomware # - Malware (endpoint) # - Data exfiltration # - Account compromise # - DDOS # - Insider threat # - Cloud misconfiguration # - Supply chain compromise # - Lost/stolen device # 3. Tooling # - EDR with remote response capabilities # - SIEM with hunting queries # - Forensic imaging tooling (FTK, EnCase, dd) # - Memory capture (Volatility, Rekall) # - Network capture (tcpdump, Zeek) # - IR chat bridge (Slack/Teams bridge, Mattermost) # - Out-of-band comms (cell, satellite, if network down) # 4. Jump Kit # - Pre-staged laptop with IR tools installed # - External hard drives for imaging # - Write blockers (hardware) # - Network capture appliance (lightweight) # - Bootable USB with forensic OS (SIFT, Kali) # - Chain-of-custody forms (printed) # - Tamper-evident bags # - Portable labels, Sharpies, gloves # 5. Authority & Decision Tree # - Who can authorize: system shutdown, network isolation, public statement, law enforcement involvement # - "Breach notification decision tree" (Legal) # 6. Training & Exercises # - Tabletop exercises (quarterly) # - Purple team exercises (annually) # - External IR retainer integration drill # - Lessons learned from previous incidents Cloud-Age Preparation # New prep items for cloud: # - Cloud provider escalation path (AWS TAM, Azure FastTrack) # - Ability to isolate cloud workload programmatically # - Cloud-native forensic snapshot capability # - Cloud account-level logging (CloudTrail, Activity Log, Audit Logs) # - IR retainer experienced with cloud # - Cloud security budget for surge capacity (additional log ingestion, isolation hosts) # Script: Isolate AWS EC2 instance aws ec2 modify-instance-attribute --instance-id i-123456789 \ --groups sg-isolation-quarantine # Script: Revoke Azure AD user sessions # Revoke all refresh tokens POST /users/{id}/revokeSignInSessions # Script: Snapshot GCP disk for forensics gcloud compute disks snapshot my-disk --snapshot-names forensic-snap-$(date +%s) Phase 2: Detection & Analysis Alert Triage # Alert intake -> Triage -> Validate # # Speed goals: # SEV1: 5 min from alert to investigation # SEV2: 15 min # SEV3: 60 min # SEV4: 24 hours # # Triage Questions: # 1. Real? (vs false positive) # 2. Scope? (single asset vs widespread) # 3. Severity? (impact potential) # 4. Confidence? (verified vs needs more info) # 5. Initial containment? (or let run for observation) # # Documentation: # - Incident ticket created at triage # - Chain of events: entries timestamped, source cited # - Analyst notes preserved for forensics Scoping # Scope = Blast radius. Determined by: # - Initial vector: how did adversary get in? # - Lateral movement: where did they go? # - Privilege escalation: what level did they achieve? # - Persistence: what mechanisms did they install? # - Data access: what could they have accessed? # - Exfiltration: what did they take out? # Techniques for scoping: # 1. Pivot off initial IOCs # IP -> search all logs for that IP -> find hosts # Host -> search EDR for that host -> find processes # Process -> search SIEM for hashes -> find other hosts # # 2. ATT&CK behaviors (not just IOCs) # Find all hosts with: PowerShell + encoded + parent=word # Find all hosts with: LSASS access by non-system process # # 3. Behavior baselines # Find unusual logon hours per user # Find new user accounts created in last 30 days # Find unusual service installs # Find new scheduled tasks # # 4. Network pivots # Smooth: ASN, subnet, geo # Traffic: who else talked to suspicious IPs? # Splunk example: Pivot from one host to find related search index=edr host=compromised-host | stats values(process_name) as procs, values(parent_process) as parents, values(user) as users, values(ProcessHash) as hashes | foreach procs hashes [ search index=edr ProcessHash=< > | stats values(host) as other_hosts ] # Sentinel KQL: Find hosts with same malicious file hash DeviceFileEvents | where SHA256 == "abc123..." | summarize hosts = make_set(DeviceName) by SHA256 Confirming the Incident # Validation is critical before declaring incident # Apply "swiss cheese" model: line up evidence from multiple sources # Sources of confirmation: # 1. Multiple EDR detections on same host # 2. Network egress to known C2 # 3. Scheduled task / service registered post-exploit # 4. Multiple failed logins followed by success from unusual geo # 5. Modification of security sensitive registry keys # 6. Document access out of normal pattern # 7. EDR flag on malicious file (sandbox detonation) # 8. AV signature hit on file # 9. DLP violation tied to user # 10. Threat intel match on observed artifact # Confidence ladder: # Suspected - 1 signal # Indicated - 2 independent signals # Confirmed - 3+ signals across different data sources Phase 3: Contment, Eradication, Recovery Containment Strategies Strategy When to Use Risk Network isolation Host is beaconing, ransomware staging Service outage, user disruption Account disable Account compromised Business process interruption Block C2 C2 known, several victims Adversary detects, switches channels Monitor only Need more data, low risk Adversary escalates while observed Shoot the host Ransomware imminent, egress large Data integrity risk, evidence destroyed Decoy operations Need attribution Extended dwell time, risk Containment Runbook: Ransomware # Ransomware Containment Script (immediate, automated via SOAR) # Trigger: ransomware detected # 1. ISOLATE INFECTED HOSTS (within seconds) # - AWS: isolate security group aws ec2 modify-instance-attribute --instance-id i-xxx --groups sg-quarantine # - Azure: change NIC NSG az network nic update --name nic-name --resource-group rg --network-security-group quarantine-nsg # - GCP: change network tag / firewall rule gcloud compute instances add-tags instance --tags=quarantine --zone=us-central1-a # - On-prem / EDR-administered: # SentinelOne / CrowdStrike / Defender: network isolation command # RPC to EDR API: POST /api/v1/agents/{id}/isolate # 2. KILL PROCESSES # - Kill known ransomware process names # - Kill suspicious PowerShell, cmd.exe spawned by office apps # 3. PRESERVE EVIDENCE # - Snapshot cloud disk (for forensic analysis later) # - Memory dump (if accessible without powering on) # - Network capture if attacker still on system # 4. BLOCK KNOWN C2 # - Firewall block C2 IPs/domains # - Email security block sender domains # - Web proxy block C2 URLs # 5. DISABLE CREDENTIALS # - Revoke Kerberos tickets (krbtgt rotation) # - Reset all passwords in affected area # - Disable affected service accounts # - Revoke SSO/OAuth tokens # 6. NOTIFICATIONS # - IR Lead notified # - IT/Network Engineering notified # - Business owner notified # - Legal/Compliance notified (potential breach) # - Law enforcement contacted (FBI IC3) Eradication # Goal: remove ALL persistence and malicious artifacts # # 1. Identify ALL persistence mechanisms # - Registry persistence keys # - Scheduled tasks / cron # - Services # - WMI subscriptions # - Startup folders # - DLL hijacking # - Boot batch scripts # - Browser extensions # - Office macros/templates # - LSASS-loaded DLLs # # 2. Identify ALL malicious binaries # - Hash -> scan whole fleet via EDR # - Yara rules on EDR file scan # - Length / parent / first-seen heuristics # # 3. Identify ALL credentials compromised # - Stored credentials (browsers, RDP, CredSSP) # - Service accounts # - Certificates (could be used for auth later) # - Cookie theft (web sessions) # # 4. Verify with hunt # - Run YARA scan against fleet # - Review host-level EDR timeline for each infected host # - Check SIEM for similar TTPs anywhere in fleet # # 5. Rebuild from known-good (most reliable) # - Don't try to "clean" - rebuild from golden image # - Apply patches before reconnecting # - Rotate all credentials that touched the asset Recovery # Recovery Strategy Decision Tree: # # 1. Can we restore from backup? # Yes -> Backup is offline/immutable >> restore, validate, reconnect # No or backup encrypted -> go to step 2 # # 2. Can we rebuild from known-good image? # Yes -> Provision new, install software, restore app data # No -> go to step 3 # # 3. Do we rebuild from compromised system? # Yes (only option) -> forensic image, rebuild OS, reinstall apps, restore app data only # # 4. Validation: # - Run AV/EDR full scan # - Compare hashes against known-good (Tripwire, OSSEC baseline) # - Monitor for 24-48h before returning to production traffic # - Higher alerting sensitivity for recovered hosts # Recovery timeline example: Ransomware in data center # Day 1: Isolate, snapshot, begin eradication, restore critical backups # Day 2-3: Rebuild critical servers from golden image # Day 3-5: Validate systems, restore data # Day 5-7: Phase back to production, monitor # Day 7: Lessons learned meeting scheduled Phase 4: Post-Incident Activity Lessons Learned Meeting # Hold within 2 weeks (while memory fresh) # No-fault: focus on systems, processes - not blame # Attendees: IR team, IT, business stakeholder, optional: external # # Agenda: # 1. Timeline reconstruction (10 min) # 2. What went well? (15 min) # 3. What went poorly? (30 min) # 4. What unusual happened? (15 min) # 5. What could have been detected earlier? (15 min) # 6. What improvements to make? (30 min) # 7. Action items & owners (15 min) # Action items categories: # - Detection rule updates (rule_owner) # - Logging gaps (logging_owner) # - Process changes (process_owner) # - Training updates (training_owner) # - Tooling gaps (tooling_owner) # - Vendor escalation (procurement) Postmortem Report # Postmortem Template (blameless) # Title: [Incident name] - [severity] - [start_date] # Authors: [name] # Status: Final # Date: [completion_date] # 1. Summary (TL;DR - 5 sentences) # What happened, impact, duration, root cause # 2. Impact # - Users affected # - Systems affected # - Data lost / accessed / exfiltrated # - Revenue impact (if known) # - Customer trust impact # - Regulatory impact # 3. Timeline (events with timestamps) # Time | Event | Source # 2026-01-01 12:00:00 | Initial phishing email arrived | Email gateway log # 2026-01-01 12:30:00 | User clicked link | Proxy log # ... # 2026-01-03 14:30:00 | Incident declared closed | IR ticket # 4. Root Cause Analysis # - Initial access vector # - Why detection delayed # - Why containment took X hours # - Why recovery took Y days # - Systematic issues (not individual mistakes) # 5. What Worked Well # - Detection rule fired # - IR runbook followed smoothly # - Comms worked # - Containment decision quick # 6. What Didn't Work Well # - Lack of log coverage on system Z # - IR runbook out of date # - Approval bottleneck for network isolation # - Forensic laptop not configured # 7. Action Items # ID | Action | Owner | Due | Status # 1 | Deploy logging on system Z | IT | 2026-02-01 | Open # 2 | Update ransomware runbook | IR | 2026-02-15 | Open # 3 | Pre-authorize network isolation | CISO | 2026-02-01 | Done # 4 | Refresh jump kit | SOC | 2026-02-15 | In progress # 8. Metrics # MTTD: X hours (target: < 24h) # MTTR: Y hours (target: < 4h for SEV1) # MTTC: Z hours (target: < 24h for SEV1) # Detection rule performance: triggered correctly # Alerts closed without escalation: A (low) # Alerts successfully escalated to incident: B (correct) # 9. Appendices (forensic details, IOCs, ATT&CK mapping, etc.) Digital Forensics Fundamentals Forensic Soundness Principles Do no harm: Minimize modifications to evidence Document everything: Every action, command, and tool Acquire early: Volatile data first (RAM, network state) Work on copies: Never analyze the original Validate integrity: Hash every acquisition Maintain chain of custody: Continuous, signed, dated Order of Volatility # Collect most volatile first (RFC 3227) # # 1. CPU cache, registers (lost on reboot) # 2. Routing tables, ARP cache, process table, kernel stats # 3. Memory (RAM) # 4. Temporary filesystems (/tmp, /var/tmp) # 5. Network state (connections, listening ports) # 6. Running processes # 7. Network connections # 8. Active network connections (tcpdump) # 9. Mounted filesystems (recently accessed) # 10. Remote logs (syslog servers) # 11. Physical disk - non-volatile but mutable # 12. Remote archival storage (offline backups) Live Response Collection # Minimal-impact scripts to run on live compromised host # Use thumb drive with trusted binaries (hash-verified) # Memory acquisition (Windows) # winpmem C:\\capture.mem # or: Magnet RAM Capture # or: Dumplt / RamCapture # Memory acquisition (Linux) # LiME kernel module # insmod lime.ko "path=/tmp/memory.lime format=lime" # Memory acquisition (cloud) # AWS: snapshot EC2 i