Design as Functions

The Error Model

The Fit Check Error Model

A solution can pass every test we give it and still fail to produce the change we wanted.

The error may lie in what we observed, the outcome we chose, our assumptions about the world, the requirements we derived, the alternatives we generated, the choice we made, what we built, or how we interpreted the result.

This model makes those dependencies inspectable. It provides a way to justify a design and locate errors when the result disappoints—or succeeds for reasons we do not understand.

The order shows what must justify what. Discovery can happen in any order, and every part remains open to revision.

You can also check out an interactive walkthrough of the Fit Check Error Model.

The abstract model

x → f() → y

This is the design claim. It says that a proposed solution will move a situation toward an intended outcome. It does not establish that the change will happen.

Final check: even if the solution satisfies the requirements governing acceptance, does deploying it actually move reality toward y?

Intent is the underlying human or organizational want from which y is formed. Several parties may want different things. Turning those wants into one outcome involves a priority decision that often goes unaudited. Record whose outcomes count, how conflicts are handled, and who can authorize a change. The model can expose that decision; it cannot settle it by itself.

Imposed boundaries are limits set outside the design choice: a binding deadline, policy, contractual obligation, fixed platform, or product category. Their implications belong in R as constraints, with their source recorded. “Must be confectionery” can be a legitimate constraint without being a customer need.

Appetite is the time and resources we choose to spend on this bet. Track it separately from R. A candidate can meet the requirements and still cost more than we are willing to invest. Appetite guides search, scope, and selection; it should not be silently expanded to fit the answer we already like.

A binding external deadline is a constraint. A chosen six-week investment is appetite. If they coexist, record both.

The two tests

Conformance

Eᶜ(f, R, M) asks whether a solution satisfies the applicable requirements under our model of the operating world, M.

It applies at two levels:

The evaluator may use inspections, tests, prototypes, benchmarks, simulations, or research. Its verdict is fallible. Evaluator acceptance and actual conformity are different things. A weak test can accept a failing solution, and a misleading test can reject a conforming one.

Where R includes preferences or unresolved items, keep them visible without treating them as mandatory. In this article, “conforms to R” means meeting the requirements that govern acceptance for the current decision. Nice-to-have benefits cannot compensate for a Must-have failure.

Effect

Eᵉ(evidence, x, y, M) asks what the evidence supports about whether deploying the solution moved reality toward y.

This involves two questions:

1. Did the relevant outcome change?
2. How much of that change can credibly be attributed to the intervention?

The baseline in x helps describe where we started. It does not, by itself, establish what would have happened without the intervention. Attribution needs a credible comparison and assumptions appropriate to the situation.

Conformance and effect differ by the question they ask, not simply by whether a check happens before or after launch. Conformance continues during operation. Evidence about effect can come from a pilot before a wider deployment. When a requirement expresses part of the desired outcome, the same evidence may inform both judgments; that does not make their scope identical.

How the conclusions can differ

Conformance is accepted, but effect fails. The requirements may omit something essential. The operating model may be wrong. The evaluator may have accepted a design or realization that did not actually conform. Or the effect judgment may be mistaken. Execution remains a possible cause, but it has no privileged place on the list.

A requirement is unmet, but the desired outcome occurs. Suppose a workflow reduces repetitive editing through a reusable rule even though it lacks a required multi-select control. That is evidence that multi-select was not necessary for this route to the outcome. Reopen the requirement and its justification. One success does not show that the criterion is unnecessary in every context, and achieving y does not waive a binding constraint.

Effect succeeds, but the explanation remains uncertain. A solution may work through a mechanism we did not identify, or through several mechanisms whose contributions we have not separated. Success does not confirm every cell in a fit matrix. The proposed explanation may be incomplete rather than false. An unexplained success is a finding, not a verdict on the whole theory.

Conformance and effect both succeed against the wrong outcome. We may have measured y accurately while serving the wrong party, choosing the wrong priority, or pursuing a want that has changed. Recheck Intent → y. Better execution and better measurement cannot settle that choice.

Keep four responsibilities visible in field evaluation:

These checks can share evidence. They should not disappear into one success metric.

The full model

The full model makes the operating assumptions, candidate search, selection, realization, and evidence explicit. The Fit Check method provides the companion workflow for comparing candidates and inspecting their parts.

Name the relations rather than making one arrow carry every meaning:

R and M do not derive a solution. They constrain what an answer must account for and provide a basis for judging it. Generation adds invention. An evaluator cannot reject a better candidate that nobody generated.

Forming x, y, and M

Before deriving requirements, reconstruct enough of the situation to distinguish the current path, what made its limitations salient, the forces acting on change, and the result people want.

Useful questions:

1. What is happening, for whom, where, and when? Preserve concrete observations, including quantities and variation.
2. What is the incumbent path? It may be another product, manual work, a workaround, delegation, tolerance, or doing nothing.
3. What changed or became salient? Why does this situation matter now?
4. What helps or resists change? Consider habits, anxieties, costs, dependencies, incentives, and enabling conditions.
5. What should become different, and for whom? Separate the relevant parties’ outcomes before making priority decisions.
6. What happens without an intervention? What persists, worsens, improves anyway, or remains acceptable?

These questions inform one another. A trigger, anxiety, or workaround is evidence about the frame; it is not automatically a requirement.

Different justifications

x and y need different justifications. Evidence supports our description of the situation. Intent and explicit priority decisions justify the outcome we choose.

That does not require independent discovery. Learning about the situation can change what we want. Choosing an outcome can focus further observation. But y → x contamination begins when that focus turns into a filter that makes the world look like the deficit our preferred answer repairs.

Keep observations that do not support the preferred frame. Preserve evidence collected before the solution occurred to us. Seek disconfirming cases or a reading from someone who does not know which answer we favor.

Likewise, the current diagnosis does not determine the full ambition. What is possible to observe now and what is worth achieving are related questions with different grounds.

What belongs in M

M combines what we know about the operating world with assumptions about how relevant conditions and relationships will behave. It can fail by misdescribing the present or by projecting the future incorrectly.

The distinction between x and M is their role in the reasoning. An observed outage may be part of x; an expectation about the frequency of future outages may belong in M. Keep one traceable evidence record and distinguish observation, interpretation, and projection. The same evidence can inform both without becoming two independent confirmations.

General causal knowledge can belong in M. The conclusion that a particular candidate will succeed cannot be assumed there. “Recognition depends partly on attention and memory” is an operating hypothesis. “This creative execution will transfer the hunger association to Snickers” is a claim in that candidate’s theory of action.

The solution neighborhood may influence which conditions we investigate. That is legitimate scoping as long as it does not install the candidate’s success as a premise.

Deriving R

R records the conditions an answer is judged against and why they matter. It draws from five overlapping sources:

A requirement can have several sources. Record them without pretending each requirement has one exclusive origin. Labels such as FROM_X, FROM_Y, FROM_GAP, FROM_M, and FROM_BOUNDARY make that provenance easy to inspect.

Status and authority

The requirements record should distinguish:

Status differs from authority. A proposed Must-have can still be Working rather than Accepted. Agree which criteria govern acceptance before using them to select a candidate. Keep Out items as a record of the boundary, without treating them as active obligations.

Each active requirement needs a stable identifier, its rationale, its status and authority, and enough acceptance detail to support a decision. Do not hide two independently failing needs inside one row merely because they occur in the same sentence.

Test the derivation

Ask:

1. Can the requirement be traced to the frame, operating model, or an imposed boundary?
2. Is it specific enough to judge, with relevant conditions and thresholds made explicit?
3. Does it prescribe a mechanism? If so, what justifies that restriction?
4. Could materially different solutions satisfy it?
5. Can it coexist with the other governing requirements?

The fourth question is a diagnostic. A requirement that only permits the preferred answer deserves scrutiny, but it is not automatically wrong: an external compatibility constraint may genuinely restrict the mechanism.

Materially different means changing how a requirement is satisfied, a consequential dependency, or a substantial cost, feasibility, or performance tradeoff. Cosmetic variants do not supply a meaningful comparison.

Preserve the reason

Suppose the observed situation is:

Repeating the same edit across 200 items takes 45 minutes.

Compressing this to “Need bulk editing” preserves an answer while losing the reason. Replacing “45 minutes” with “too long” also loses evidence and inserts a judgment about the desired outcome into the observation.

Keep the layers visible:

If the brief requires compatibility with an existing import format, record that as a constraint and cite the brief. Do not present it as something the observed editing time implied.

A solution idea may arrive first. Work backward to the requirement that would justify it, then check that requirement against independent evidence and intent. A mechanism can reveal a requirement; it cannot prove one.

Generating, selecting, and realizing candidates

Build a meaningful comparison set

S contains the alternatives actually considered. In a practical comparison, label the existing path CURRENT and new directions A, B, and C. Reserve labels such as B1 for parts inside a direction.

Include CURRENT when changing an existing system. It exposes the relevant baseline and gives derivation an additional check: if CURRENT appears to satisfy every governing requirement while the desired outcome remains absent, something needs investigation. Requirements may be incomplete, thresholds weak, conformance overstated, or the operating model mistaken.

In open design problems, the complete solution space is usually unavailable. In bounded decisions it may be possible to enumerate the options. Either way, inspect the set actually considered: does it contain genuinely different mechanisms and consequential tradeoffs?

Do not manufacture weak alternatives to meet a quota. Search far enough to support the decision at stake. A missing better candidate creates no red test because it never entered the comparison.

Evaluate before selecting

At the reasoning level, a conformance judgment can be accepted, rejected, or unresolved. Unknown is not a factual pass, and it is not automatically a reason to abandon the candidate.

The companion Fit Check table uses binary marks: an unknown remains ❌ until resolved or explicitly accepted. Record the uncertainty and the decision behind any acceptance. Accepting uncertainty authorizes reliance at a stated risk; it does not establish that the claim is true.

Choose the next move according to the uncertainty. It may call for a focused spike, a prototype, a structural breadboard, a clearer requirement, an explicit acceptance decision, an improved candidate, or rejection. A spike is useful when its result could change selection or establish whether the candidate fits appetite.

Conformance does not select the winner. Selection also considers appetite, preferences, uncertainty, maintenance, and other declared tradeoffs. A Must-have failure cannot be averaged away by attractive scores elsewhere.

Allow no acceptable candidate

The outcome may be that no candidate is acceptable, or that every acceptable candidate exceeds appetite. Improve the search, investigate a decisive unknown, correct a bad requirement, explicitly change the bet, or stop.

Do not quietly loosen a constraint or expand the budget to force a selection.

Including CURRENT does not mean that “do nothing” must pass the new requirements. Retaining an inadequate current path may still be preferable to funding an unacceptable intervention. The decision to stop investment remains available independently of candidate conformance.

Realize the selected design

Implementation contains its own design judgments. Teams choose data structures, interfaces, operations, and details not settled in the higher-level shape. Representation, modeling, derivation, generation, and judgment errors can recur within those decisions.

Send the frame with R: the relevant x, y, and M, requirement origins, constraints, accepted uncertainties, and reasons for selection. A requirement list alone can lose the information needed to make a good local choice.

Then check the realization. The intended design may conform while the built version does not. Conditions may also have changed during implementation. When that happens, revise the affected frame or model explicitly and repeat the relevant checks.

Where repeated optimization against visible checks is a concern, reserve some independent evaluation cases. Keep the requirements visible. Withholding evaluation cases can test generalization; withholding requirements deprives the builder of the intended task.

Worked example: Snickers and Milky Way

Ryan Singer’s account of Snickers and Milky Way illustrates how similar products can be understood through different situations and desired outcomes. The following frames and fit judgments are constructed examples. They do not reconstruct either company’s research or establish how either brand actually works.

These are two frames, each with its own requirements. They are not two candidates competing under one R.

Form the frames

The consumer and brand outcomes need separate evidence. Pleasure for the consumer does not establish brand recognition; remembering a brand does not establish the consumer benefit. In a real decision, the relationship and priority between these aims would need to be agreed.

Record imposed boundaries

Suppose both briefs require confectionery, existing retail and distribution channels, a specified mass-market price range, and compatibility with available manufacturing capabilities.

These are constraints in the full R, with FROM_BOUNDARY provenance. The consumer frame did not imply “must be candy”: a missed meal could be answered by a sandwich.

Existing distribution channels restrict the available routes to market. They do not ensure that the product is stocked where and when someone needs it. Availability still requires a mechanism or a verified dependency.

Form M

For the exercise, suppose the operating model contains hypotheses about convenience in routine confectionery choice, the role of enjoyment in that category, sensory properties influencing experience, and attention and memory influencing brand recognition.

These assumptions need evidence appropriate to their importance. Their plausibility does not establish that a particular texture will relieve hunger or that a particular advertisement will attach the intended meaning to a brand. Those are candidate claims.

Derive the requirements

The two sets below separate availability from consumption and make the consumer and company reasons visible.

Snickers frame

Milky Way frame

For this exercise, treat R1 as the Core goal and R2–R5 as Must-haves in each frame, alongside the imposed constraints. A real project would need to accept those priorities and define the population, occasion, thresholds, and evidence before making a conformance decision. Words such as “sufficient” do not supply a test by themselves.

Generate candidate systems

Official US product descriptions identify peanuts, nougat, caramel, and milk chocolate in Snickers, and nougat, caramel, and milk chocolate in Milky Way.^snickers-product Those descriptions establish composition, not the proposed psychological or physiological effects.

The hypothetical Snickers system combines a portion and formulation, flavor, chew and crunch, a wrapped format, placement and replenishment, and communication linking hunger, changed behavior, and the named brand. The documented campaign line is “You’re not you when you’re hungry.”

The hypothetical Milky Way system combines flavor, softness and melt, a wrapped portion, placement and replenishment, and communication linking pleasurable absorption to the named brand. Mars documents the campaign “Sorry I Was Eating a Milky Way.” Its distraction premise illustrates absorption; it does not establish the private-comfort frame used here.

A campaign line is one part of each proposed system. A real selection exercise would compare materially different systems under the same frame. Here, one system per frame lets us inspect the proposed links between mechanisms and requirements.

Inspect the proposed fit

This is the rotated view: mechanisms down the side, requirements across the top.

● records a decision not to seek more evidence now. It does not mean verified. For significant accepted claims, record the basis for acceptance and what would reopen it.

The following marks illustrate hypothetical decisions. They report no research results. Accepting one contribution also does not establish that the whole requirement is satisfied.

Snickers: proposed contributions

Milky Way: proposed contributions

The positive contribution matrices do not cover the entire R. Check the imposed constraints separately and carry their results into the overall conformance judgment:

Read the matrix in both directions

Forward fit asks whether every active requirement has an account of how it will be satisfied. An empty column exposes a coverage gap. A wrapped bar addresses consumption convenience; it does not explain availability. Placement and replenishment supply that missing part of the account.

Reverse fit asks what justifies each meaningful part of the answer. An empty row exposes a missing justification, not proof that the part is useless. A collectible wrapper might contribute to pleasure or brand recognition, but that connection needs an argument. If it serves a genuinely missing requirement, derive and accept that requirement from the frame, then rerun the fit check. Do not invent a requirement merely to rescue the feature.

Implementation convenience alone does not authorize new product scope. Remove an unjustified addition, put it outside this bet, or justify it through an accepted requirement.

Decompose enough to expose the reasoning

A single row called “product and brand experience” could place a dot in every column while explaining almost nothing.

At the other extreme, ingredients and their resulting sensory properties are different causal levels. Counting the ingredient, texture, and experience as independent support can count one path several times. Separate the properties whose contributions need inspection, and show dependencies in the accompanying explanation.

One property may have two proposed effects. Chew might contribute to pleasure and perceived relief. That is a claim to investigate, not evidence of two independent benefits. The dots are not scores to add.

Coverage is not sufficiency

A filled column says that something claims to contribute. It does not establish enough contribution, the right duration, reliable operation, or an acceptable result across the intended population.

Inspect the whole configuration:

• Does each contribution reach the required level?
• Do several parts need to work together?
• Does a contribution depend on an unverified condition outside the design?
• Does a part undermine another requirement?
• Is a constraint satisfied by the deliberate absence of a feature or behavior?

Annotate adverse relationships explicitly. For example, more chew might increase effort and undermine consumption convenience. A positive dot elsewhere does not cancel that conflict.

Some constraints require evidence of absence rather than a supporting mechanism. A restriction against an added subscription, for example, can be met by excluding one. Record that check separately; do not invent a positive mechanism to fill the matrix.

Check design and realization

Design conformance asks whether there is sufficient reason to accept the proposed system. Can the distribution plan provide the required availability? Is the intended sensory benefit supported? Does the communication connect the occasion to the named brand? What evidence resolves the consequential uncertainty?

Realized conformance asks whether the actual recipe, portion, wrapper, distribution, price, and communication meet the accepted requirements. A plausible placement plan can fail through stockouts. An intended brand cue can be lost in execution. Check those conditions during operation as well as before launch.

Check effect and investigate mechanisms

For Snickers, distinguish the consumer’s experienced relief and ability to continue from the company’s recognition and consideration outcome. For Milky Way, distinguish the consumer’s pleasurable absorption from the company’s recognition and consideration outcome. A change in one does not establish a change in the other.

Then ask what supports attribution to the intervention. Product outcomes, communication outcomes, and the effect of the whole system may need different comparisons.

Give consequential uncertain links their own evidence plan:

These are investigation directions, not complete research protocols. Testing the total effect alone will not identify each link. Some uncertainties can be resolved before deployment; others require evidence from use.

Now suppose people remember the Snickers advertisement but fail to associate the hunger occasion with Snickers. That hypothetical finding should reopen the ● in the story-to-R5 cell: the team had accepted that contribution without more evidence.

Possible diagnoses include an ineffective candidate mechanism, weak acceptance evidence, a realization that lost necessary brand cues, an inadequate threshold in R5, or misleading outcome measurement. Revisit M only if evidence challenges a general operating assumption. Do not hide this candidate’s failed transfer claim inside M after the fact.

The example shows how different frames justify different requirements and mechanisms. It does not prove either constructed frame or the actual public meaning of either brand.

Where the model can fail

Five reasoning failure kinds recur across these locations: representation, operating-model formation and projection, derivation, generation, and judgment. They are useful diagnostic distinctions, not mutually exclusive root causes. Implementation can contain several of them.

A practical summary asks whether we pursued the right change, modeled the world adequately, specified appropriate conditions, searched for strong alternatives, chose well, realized the design, and judged its consequences correctly. Leaving generation out would make a faithfully built mediocre answer look fully explained by the other checks.

Diagnose backward

When effect disappoints—or succeeds without a sound explanation—walk backward through the dependencies:

1. Was the effect conclusion warranted? Examine the comparison, attribution, and uncertainty. Separately ask whether y still represents the relevant intent.
2. Was the evidence adequate? Inspect measurement, timing, population, missing signals, and the baseline representation.
3. Did the operating solution actually conform? Include continuing constraints and external dependencies.
4. Did implementation realize the selected design? Inspect consequential decisions made during the build.
5. Was selection justified? Revisit appetite, tradeoffs, accepted uncertainty, and the option to stop.
6. Was design conformance justified? Reopen weak tests and accepted contribution claims where evidence warrants it.
7. Was the candidate search adequate? Ask which mechanisms or tradeoffs never entered S.
8. Were the requirements right? Check omissions, unnecessary restrictions, bundled needs, priorities, and thresholds.
9. Was the frame sound? Revisit x, y, M, intent, and the origins and continuing force of imposed boundaries.

This is a route for investigation, not a rule that every failure has one upstream cause. Shared assumptions can make several stages wrong together.

M deserves particular attention because it informs derivation, search, conformance, and attribution. Those judgments can agree with one another while depending on the same mistaken account of the world.

The loop

Deployment changes the situation represented by x. The intervention becomes part of the next incumbent path, and learning can require changes anywhere in the model.

Revisit the frame, intent, operating model, boundaries, requirements, candidate generation, selection, implementation, observation, and both evaluators according to the evidence. Change appetite only through an explicit decision.

Preserve the versions used for each judgment. Record what was accepted, on what evidence, and under which assumptions. When a material input changes, identify the affected comparisons and rerun them. Do not rewrite the old frame until the result looks inevitable.

This loop can run before a full deployment. Observation can correct x; research can challenge y or M; a spike can test a mechanism; a prototype can expose an implementation problem; a comparison with CURRENT can expose weak requirements. Early contact with reality uses the same correction logic at a smaller cost.

The goal is not to declare every gap permanently closed. It is to keep narrowing the gaps that matter as evidence and conditions change.

Principles

Preserve the reasons. Requirements need their originating frame. Implementation needs the frame and the reasons for selection. Later diagnosis needs the evidence and assumptions behind earlier acceptance.

Search as well as test. Conformance can examine the candidates in front of us. It cannot reveal the answer we never invented. Deliberate comparison and exploration address a different failure from testing.

Prefer mechanisms that do more justified work. One causal property can sometimes serve several requirements. Simplicity is useful when it preserves required behavior, adequate robustness, and understandable dependencies. Fewer parts can also create tighter coupling or a single point of failure. Count neither parts nor dots as a substitute for judgment.

Correct upstream inputs openly. New evidence can justify changing a frame or requirement. A preferred answer cannot supply that justification by itself. Record the change and reevaluate the decisions that depended on the old version.

Better execution does not repair a bad frame. It can make pursuit of that frame more effective. Implementation quality, framing quality, operating assumptions, and candidate search each need their own attention.

Capability can turn omissions into affordances when the builder optimizes against the encoded checks. Missing requirements and incomplete evaluators can become routes to apparent success. More capable implementation does not supply the intent left out of the task; it can supply more ways to meet the recorded conditions. Carry the frame into the build, examine consequential omissions, and preserve independent evaluation.

Gaps differ in detectability and in how far their consequences reach. A failing test supplies an observable contradiction. A missing requirement, missing candidate, or poor effect inference may produce no such signal. Those omissions need deliberate investigation. An upstream correction can invalidate requirements, search, selection, and evaluations that previously agreed; a local implementation correction may be more contained.

Build forward. Verify locally. Diagnose backward. Correct against reality.

Reality gets the final say, but we still have to hear it through evidence.