Penetration Testing and Vulnerability Scanning

Author: Marcus Hale, OSCP, CEH — Offensive Security Specialist. Penetration tester with 11 years of hands-on experience across network, web application, and social engineering assessments. 

Consultant: Marcus Dreiling, OSCP, CEH — certified penetration tester and vulnerability assessment specialist with twelve years of hands-on experience in offensive security, red team operations, and security program development for mid-market and enterprise organizations.

The techniques described apply only to systems you own or have explicit written authorization to test. Unauthorized testing of any system is illegal and can result in criminal prosecution. When in doubt, consult a licensed cybersecurity professional before taking any action.

The confusion between these two methods costs organizations more than almost any other procurement mistake in security. A company treating a clean scan report as a passed test may be spending confidently on a false sense of safety. One that skips straight to penetration testing without any scanning baseline may be paying expert rates for work a cheap automated tool could have handled first.

Both methods probe systems for weaknesses — that surface-level similarity is enough to blur the line, especially when vendor marketing sometimes uses the terms interchangeably. They answer different questions, require different resources, and produce different kinds of evidence. Getting this wrong means either wasting budget or leaving real attack paths unexamined.

How Automated Scanning Finds and Ranks Weaknesses

Vulnerability scanning is automated. A tool examines a network, application, or system and compares what it finds against a database of known weaknesses — outdated software, open ports that should be closed, misconfigured services. Results are ranked using the Common Vulnerability Scoring System (CVSS), a severity framework maintained by NIST that assigns each flaw a score from 0 to 10 based on exploitability and potential impact.

Scans can run daily or weekly because they require no human analyst to execute. Most organizations run two types: authenticated scans, where the tool logs in with valid credentials and sees the environment from the inside, and unauthenticated scans, which simulate an outside attacker with no prior access. Together they give different views of the same exposure.

Tools on the market vary significantly in how they deploy and what they cover:

Tool	Deployment	Agent required	Best suited for
Tenable Nessus	On-premise or cloud	Optional	Mid-size to enterprise networks
Qualys VMDR	SaaS	Optional	Large, distributed environments
OpenVAS	On-premise, open-source	No	Budget-conscious or air-gapped setups
Rapid7 InsightVM	SaaS + on-premise hybrid	Yes	Environments needing live endpoint visibility

What scanning cannot do is confirm whether a found weakness can actually be exploited in your specific environment, or whether it sits on a path that leads anywhere sensitive. That is its limit — and where penetration testing begins.

Penetration Testing — From First Reconnaissance to Final Report

Penetration testing — pen testing — is a structured, manual effort by a certified professional to compromise a target system within agreed boundaries. A tester holding credentials such as OSCP (Offensive Security Certified Professional) or CEH (Certified Ethical Hacker) attempts to exploit vulnerabilities the way a real attacker would, then documents the full path from entry to impact.

A typical engagement begins with reconnaissance — gathering publicly available information about the target, mapping DNS records, identifying exposed infrastructure. From there, the tester looks for entry points, attempts exploitation, and moves through the environment the way a threat actor would after gaining an initial foothold.

The 2017 Equifax breach illustrates why this matters. Attackers exploited a known Apache Struts vulnerability (CVE-2017-5638) that had gone unpatched for months, then moved laterally through internal systems to reach personal data belonging to approximately 147 million people. A vulnerability scanner would have flagged the unpatched software. A penetration test would have demonstrated the full attack path — from entry point through privilege escalation to the data itself — showing real-world impact rather than a severity score on a list.

Marcus Dreiling, OSCP, CEH, who consults on penetration testing and red team operations for mid-market and enterprise organizations, describes a pattern he sees regularly:

“The most common mistake is treating a clean scan report as evidence that defenses work. Scanning tells you what’s exposed. A pen test tells you what an attacker could actually do with that exposure — and those are very different answers.”

How Testers Classify Engagements by Scope and Access

Tests are described along two axes. The first is how much information the tester starts with. In a black-box test, the tester has no prior knowledge and works like an external attacker. In a white-box test, full documentation and credentials are provided upfront, allowing for deeper and more efficient coverage. Grey-box — partial information shared — is the most common setup for internal security audits.

The second axis is where the tester starts from. An external penetration test simulates an attacker with no network access — someone trying to get in from the internet. An internal test assumes the attacker is already inside the perimeter, which reflects the reality of phishing-based intrusions and compromised credentials. These are separate scopes that surface different categories of risk; running only one can leave significant blind spots.

According to the Verizon Data Breach Investigations Report 2024, attackers frequently chain multiple lower-severity vulnerabilities to achieve high-impact breaches — the exact scenario a pen test is designed to surface and model.

Choosing Between the Two Methods

For most small businesses with limited security budgets, regular vulnerability scanning is the right starting point. It provides continuous visibility into known weaknesses, helps prioritize patching, and integrates with many endpoint protection platforms. Scanning is a mandatory requirement under several compliance frameworks, including PCI DSS and ISO/IEC 27001.

Penetration testing becomes necessary when the risk stakes are higher. Financial institutions, healthcare providers, and organizations handling large volumes of personal data should conduct pen tests at minimum once a year and after any major infrastructure change. CISA’s Binding Operational Directive 23-01 (2022) encourages critical infrastructure operators to treat penetration testing as a recurring practice. Organizations undergoing a merger, launching a new public-facing application, or recovering from a security incident should treat it as a standard step.

For organizations with public-facing applications and an internal security team capable of triaging incoming reports, bug bounty programs offer a third layer — paying independent researchers for verified vulnerabilities they discover. They work best alongside scanning and pen testing, not as a replacement for either.

	Vulnerability scanning	Penetration testing
Execution	Automated	Manual, certified professional
Frequency	Weekly, monthly, or continuous	Annual minimum; after major changes
Output	Prioritized list of known weaknesses	Full attack path with business impact
Skill required	Low — tool-based	High — OSCP, CEH, or equivalent
Typical cost	$50–$500/month (SaaS tools)	$5,000–$50,000+ per engagement
PCI DSS requirement	Quarterly scans for in-scope systems	Annual pen test for in-scope systems

Hiring a Pen Test Vendor

Credentials matter, but they are the starting point, not the finish line. An OSCP or CEH tells you someone passed a certification exam — it does not tell you whether their methodology suits your environment, whether they will handle your data responsibly, or whether their report will be worth acting on.

Questions to Ask Before You Sign

Before engaging a vendor, get clear answers to these questions:

• Which methodology do they follow — PTES (Penetration Testing Execution Standard), the OWASP Testing Guide, or another documented framework? Vendors without a named methodology are improvising.
• Will they sign a non-disclosure agreement before receiving any information about your environment? This should be standard; hesitation is a warning sign.
• What does their Statement of Work cover — scope boundaries, out-of-scope systems, rules of engagement, emergency contacts if something breaks during testing?
• What does a sample report look like? Ask for a redacted example. A report that lists CVE numbers without showing exploitation paths or business impact has limited value.
• Do they offer a retest after remediation, and is it included in the price or billed separately?

A vendor who cannot answer these questions clearly is worth walking away from regardless of price.

From Findings to Fixes

Getting results back is where most organizations quietly fail. The report arrives, the findings look alarming, and then it gets filed as a PDF while teams return to regular work. Six months later, the same vulnerabilities are present.

A penetration test report is only useful if findings are tracked as actionable items in a ticketing system, assigned to owners, and given remediation deadlines. Findings left in a document do not get fixed on their own. More importantly, remediation without a follow-up retest is incomplete — patching a vulnerability and assuming it is closed is not the same as verifying that the patch held under the same conditions the tester used.

Treat the retest as a mandatory part of the engagement, not an optional add-on. Schedule it before the initial test concludes, while the tester still has context about what they found and how they got there. This single step separates organizations that improve their security posture from those that pay for a report and stay equally exposed.

Scanning and pen testing work best when they are not isolated events but part of a continuous process — scans running on a fixed schedule, results feeding into annual pen test planning, and both feeding into a remediation workflow with real accountability. The ENISA Threat Landscape 2023 notes that most successful attacks exploit both technical vulnerabilities and human factors. No combination of testing methods addresses that entirely on its own, but without baseline visibility into what you are defending, everything else is guesswork.

Frequently Asked Questions