A system that runs without errors on the first try feels like success. It compiles, it responds, it passes the demo. But known unknowns in software development are the reason that feeling is sometimes premature: a program can work as expected today and still fail catastrophically the day its assumptions stop holding.
The distance between a system that works because it was designed to and a system that works because nothing has broken it yet is where most expensive production incidents come from. This article covers what separates those two states, why engineers at different experience levels read the same green checkmark differently, and how to build a risk assessment practice that catches structural problems before they become outages.
Table of Contents
Why "It Works" Is Not the Same as "It Is Correct"
Software is built from layers of code written by people who cannot hold the entire system in their head at once. It runs alongside other software, so a change in one place can ripple into systems nobody on the current team remembers connecting. Many applications reach millions of users, so a defect that would be trivial in a five-user internal tool becomes a headline in a consumer product.
Getting a program to run is the first milestone, not the last one. The moment a system appears to work is exactly when a certain kind of engineer starts asking harder questions, because a system that works by coincidence and a system that works by design produce identical output right up until the moment they do not.
What Are Known Unknowns in Software Development?
The clearest framing for this problem is not a software engineering term at all. It comes from a 2002 Department of Defense press briefing, later adapted widely in risk management: there are known knowns, things a team knows it knows; known unknowns, things a team knows it does not know; and unknown unknowns, things a team does not know it does not know.
Applied to a codebase, known knowns are the parts of the system the team has tested, documented, and can explain. Known unknowns are the parts flagged as risky, the integration nobody has load-tested, the edge case in the backlog. Known unknowns in software development are the assumptions the system depends on that nobody has stated out loud, let alone tested. Every production incident postmortem eventually traces back to one of those three categories, and the third one is always the expensive surprise.
The NIST Guide for Conducting Risk Assessments formalizes this same distinction for information systems: risk assessment is not a single event but a three-step process of preparing, conducting, and maintaining an assessment precisely because what counts as a known unknown changes as a system evolves.
How Junior, Mid-Level, and Senior Engineers Read the Same Green Checkmark
Experience changes what a working system means to the person looking at it. The same passing test suite produces three different reactions depending on who is reading the result, and that difference is not about talent. It is about which category of unknown each person has learned to look for first.
A junior engineer who sees a system working feels relief and moves on, focused on known knowns: the system runs, they understand the immediate mechanism, and further questions feel secondary until something visibly breaks. A mid-level engineer feels relief tempered by habit, and shifts attention to known unknowns: the specific bugs, compatibility gaps, and edge cases the team has already flagged as risk. A senior engineer feels something closer to concern, because their first question is why the system works, not whether it does. They are hunting for unknown unknowns before shipping, not after.
| Experience Level | Primary Focus | First Question After a System Passes |
| Junior engineer | Known knowns: confirming the system runs as shown | "Does it work?" |
| Mid-level engineer | Known unknowns: flagged bugs, compatibility issues, edge cases | "What could still break this?" |
| Senior engineer | Unknown unknowns: unstated assumptions, untested interactions | "Why does it work, and what happens when that stops being true?" |
None of these reactions is wrong at the stage it happens. The problem shows up when an organization only ever asks the junior question, because a team that stops at "does it work" has no mechanism for surfacing the unknown unknowns that eventually cause the expensive incidents.
When a Working System Hides a Structural Flaw
A pattern that shows up repeatedly in infrastructure engineering: a certification or validation tool passes every check for months, and everyone downstream trusts the green result because it has always been green. Then someone eventually inspects the tool itself and finds it was silently loading the wrong test suite, producing false positives on exactly the property it was supposed to verify. Nothing about the visible system changed. The system that was supposed to catch the risk was itself the unknown unknown.
This is not a story about a careless team. It is a story about what happens when "the tests pass" becomes the entire definition of correct. A working system and a well-understood system produce the same dashboard for a long time. The gap between them only becomes visible at the worst possible moment, which is usually under load, at scale, or in front of a customer.
Google's Site Reliability Engineering practice treats this gap directly rather than hoping it never surfaces. The Embracing Risk chapter of the SRE book argues that 100 percent reliability is the wrong target for almost every service, because the pursuit of an impossible standard hides more risk than a deliberately chosen, monitored error budget does. Assuming a system is fully understood is its own form of risk.
How Do You Build Risk Assessment Into a Team That Is Already Shipping?
Most engineering organizations cannot pause delivery to conduct a formal risk audit before every release, and they should not need to. The practical version of this discipline is a small set of habits that surface unknown unknowns without slowing the team to a crawl.
Assess risk before writing the fix, not after the incident. The NIST Risk Management Framework structures this as a continuous cycle rather than a one-time gate: categorize what the system touches, select the controls that matter for that category, and monitor continuously rather than certifying once and moving on. Applied to a product team, this means every significant change gets a short, honest answer to "what would have to be true for this to fail," not just a passing test.
Build a real understanding of the codebase, not just familiarity with it. Understanding the architecture, the dependencies, and the reasons behind past decisions is what lets a team predict failure modes instead of discovering them. Teams that carry unmanaged technical debt lose this understanding gradually, because every undocumented shortcut is one more unknown unknown waiting for the wrong conditions to align. Carnegie Mellon's Software Engineering Institute has studied this pattern directly and found that most organizations have no systematic practice for tracking technical debt at all, which means the risk it represents stays invisible until it is expensive.
Write blameless postmortems that ask why, not who. Google's SRE team documents this as core practice in Postmortem Culture: Learning from Failure: a postmortem that focuses on the people involved fixes nothing, because the goal is to understand the system well enough that the same category of unknown cannot cause the same category of failure twice.
DORA's 2024 State of DevOps Report found that teams with stable priorities and a genuine focus on the end-user experience consistently outperform teams optimizing for speed alone, reinforcing that understanding what a system is actually for is inseparable from understanding whether it truly works. A shared vocabulary for what the team does and does not understand about its own codebase, defined by IEEE's Software Engineering Body of Knowledge, gives teams a consistent way to talk about this instead of relying on individual instinct.
We cover a related measurement problem, how teams mistake activity for understanding, in our analysis of software development productivity.
What This Means for Engineering Leaders
Mid-market software companies managing lean teams
For independent software companies with lean internal teams carrying broad responsibilities, the risk is not usually a lack of skill. It is a lack of time to ask the senior-level question on every change when the roadmap only leaves room for the junior-level one. Teams under constant delivery pressure default to "does it work" because "why does it work" takes longer, and that tradeoff compounds every sprint it goes unexamined.
A nearshore engineering partner integrated as a dedicated team adds senior-level capacity specifically for this kind of systemic review, without requiring the core team to choose between shipping and understanding what they shipped.
PE-backed software portfolios preparing for exit
For PE-backed software portfolios the known unknowns question becomes a diligence question. Technical due diligence exists precisely to find the unknown unknowns in an acquired codebase before they become the acquirer's problem, and portfolio companies that cannot demonstrate a systematic risk assessment practice going into a sale process typically see that gap priced into the valuation.
If your team is preparing for a hold-period review or an exit and wants a second set of senior eyes on where your known unknowns actually are, our team would be glad to talk through it.
Frequently Asked Questions
What do known unknowns actually mean in a software context?
These are the risks a team is already aware of but has not resolved: flagged edge cases, untested integrations, or dependencies that have not been fully validated. They differ from unknown unknowns, which are the assumptions and interactions nobody on the team has identified as risky at all, and which are the more common source of production incidents because nobody is watching for them.
Why does a passing test suite not guarantee a system is correct?
A passing test suite confirms a system behaves as expected under the specific conditions the tests check, not that the team understands every condition under which the system will actually run in production. Tests validate known knowns and known unknowns that someone thought to write a test for. They cannot validate an assumption nobody realized needed testing, which is exactly the category of failure that causes the most damaging outages.
How can engineering teams find unknown unknowns before shipping?
The most reliable methods are structural rather than instinctive: formal risk assessment frameworks like NIST's Guide for Conducting Risk Assessments, blameless postmortems that trace failures back to root cause rather than individual blame, and deliberately assigning senior engineers to ask "why does this work" rather than only "does this work" on changes to critical systems. None of these guarantee catching every unknown unknown, but each one systematically narrows the space where they can hide.
Why do senior developers react differently to working software than junior developers?
Senior developers have typically been surprised by a system that worked right up until it did not, and that experience shifts their attention from confirming a system runs to understanding why it runs. Junior and mid-level developers are not wrong to feel relief when something works. They simply have not yet built the pattern-matching that tells a senior engineer which kinds of "it works" are trustworthy and which kinds are coincidences waiting to expire.
What is the business cost of ignoring known unknowns in software?
The direct cost is production incidents that a systematic risk review would have caught earlier and cheaper. The indirect cost is larger: technical due diligence findings that reduce a company's valuation during an acquisition, engineering time consumed by firefighting instead of building, and customer trust that erodes every time a "working" system fails in a way nobody had flagged as possible.
The Bottom Line
A system that works and a system that is understood are not the same claim, and treating them as identical is where most expensive production failures start. Known unknowns in software development are manageable the moment a team agrees to look for them systematically instead of waiting for a senior engineer's gut feeling to catch what a passing test suite missed.
The teams that avoid the worst surprises are not the ones with the fewest unknowns. They are the ones with a working practice for finding them before a customer does. If your organization is trying to build that practice into a team that cannot afford to slow down, we would be glad to talk through what that looks like in practice.
References and Further Reading
- NIST, Guide for Conducting Risk Assessments (SP 800-30). Federal guidance defining the three-step risk assessment process of preparing, conducting, and maintaining an assessment, directly applicable to identifying known and unknown risk in software systems. https://www.nist.gov/publications/guide-conducting-risk-assessments
- NIST, Risk Management Framework. NIST's overview of the Risk Management Framework, a structured and continuous process for managing risk across the system development lifecycle. https://csrc.nist.gov/projects/risk-management
- Google, Site Reliability Engineering: Embracing Risk. The chapter of Google's SRE book arguing that 100 percent reliability is the wrong target, and that deliberately managed risk produces more reliable systems than assumed certainty. https://sre.google/sre-book/embracing-risk/
- Google, Site Reliability Engineering: Postmortem Culture. Google's documented practice for blameless postmortems, focused on understanding root cause rather than assigning individual fault. https://sre.google/sre-book/postmortem-culture/
- Carnegie Mellon Software Engineering Institute, A Field Study of Technical Debt. SEI research finding that most development organizations have no systematic practice for tracking or managing technical debt, leaving its associated risk largely invisible. https://www.sei.cmu.edu/blog/a-field-study-of-technical-debt/
- Carnegie Mellon Software Engineering Institute, 5 Recommendations to Help Your Organization Manage Technical Debt. Practical guidance from SEI on making technical debt visible and associating it explicitly with risk rather than treating it as an abstract concern. https://www.sei.cmu.edu/blog/5-recommendations-to-help-your-organization-manage-technical-debt/
- DORA, Accelerate State of DevOps Report 2024. Annual research on the practices that predict software delivery performance, including the finding that stable priorities and user-centricity outperform speed-only optimization. https://dora.dev/research/2024/dora-report/
- IEEE Computer Society, Software Engineering Body of Knowledge (SWEBOK). The consensus-driven reference defining the knowledge areas of professional software engineering, used to build shared vocabulary across engineering teams. https://www.computer.org/education/bodies-of-knowledge/software-engineering