Every MBSE program hits the same meeting eventually. Two or three years into the rollout, somebody in finance asks what the modeling investment returned, and the systems engineering lead opens a deck full of architecture diagrams and the word "alignment." The tool licenses are a real line item. The training was a real line item. The answer is a feeling.

This is not a presentation problem. The honest version is that the industry has been claiming MBSE benefits for two decades and measuring them almost not at all, and the people who went looking for the evidence found a lot less of it than the marketing suggests.

The evidence gap behind the business case

Henderson and Salado's 2021 review in Systems Engineering went through the major SE journals and conference proceedings and sorted every claimed MBSE benefit by the kind of evidence behind it: measured, observed without formal measurement, perceived (claimed with no evidence at all), or simply citing another paper that made the claim. Roughly two thirds of the claimed benefits fell into the perceived bucket. Two papers reported measured evidence. Two.

Campo and colleagues followed in 2023 with a broader look at perceived value and metrics in the literature and landed in a similar place. The top benefits people name are remarkably consistent across studies: better communication and information sharing, increased traceability, improved consistency, better handling of complexity, reduced cost, reduced time, fewer errors. Seven of the top ten overlapped between the studies. Consistency of expectation is not the same as consistency of results, though, and that distinction is the whole problem.

None of this means MBSE doesn't pay off. It means the profession has been running on expectation, and expectation is fragile when budgets tighten.

The framework that already exists

The most useful response to this came out of a government and industry working group rather than a vendor. The Systems Engineering Research Center, working with Practical Software and Systems Measurement, INCOSE, NDIA, the Aerospace Industries Association, and OUSD(R&E), published a Digital Engineering Measurement Framework in 2022 (version 1.0 in May, version 1.1 that July, with version 2.0 work since underway to fold in implementation lessons).

Its central argument, captured in the companion paper title "Measuring the RoI of Digital Engineering: It's a Journey, Not a Number," is that the aggregate ROI figure leadership wants is a lagging indicator sitting on top of a stack of base measures almost nobody collects. You cannot skip to the number. You have to build the measurement chain underneath it first.

The framework starts from eight direct benefits of DE and MBSE, each with a definition and an associated measurable effect: support for automation, early verification and validation, strengthened testing, better accessibility of information, increased traceability, multiple viewpoints of the model, reusability, and support for integration. Each one maps to something countable. Early V&V, for example, shows up as more discrepancies found in reviews, defects resolved earlier, and less rework. Traceability shows up as design correctness and completeness, and as reduced requirements volatility.

Tool adoptionModel qualityCycle timeCost outcome
The measurement chain the DE framework builds toward a cost answer.

Adoption measures come first, and they are unglamorous

The framework is blunt about sequencing. Most DE benefits evaporate if artifacts get produced outside the digital environment, so the foundation is measuring whether people actually work in the models. The adoption side looks like this:

That last point is where tooling choice becomes a measurement question rather than a feature question. If your repository can't tell you how many people opened a model last month, how many elements changed between two baselines, or how many requirements have a verification relationship, you will be reconstructing your own adoption data by hand. Dassault's Cameo Systems Modeler and CATIA Magic products, Eclipse Capella, and Dalus all expose model contents programmatically, through APIs, repository queries, or text-based model sources that can be diffed in version control. Whichever you use, the question to ask during evaluation is whether the measures you intend to report can be pulled without a human counting things in a GUI.

Four measures worth starting with

If you want somewhere concrete to begin, the DE work points at four:

  1. Defects and errors by phase. Count them, and count when they were found versus when they were introduced. The MBSE argument is that errors stop persisting across phase boundaries. That is directly testable.
  2. Functional completeness and correctness. Quantify how much of the capability and function decomposition is actually modeled, peer reviewed, simulated, and tested. This is harder than it sounds and most organizations have no baseline for it.
  3. Review efficiency. Track discrepancies and action items raised per review, and decision signoffs achieved at the review. If reviews are happening in the model, these should move.
  4. Cost. It comes last on purpose. Cost is the lagging indicator that depends on all the others, and it is best evaluated across programs rather than within one.

As of the 2022 publication, the authors noted only a small handful of DoD programs had begun collecting this kind of data. The elusive ROI figure stays elusive until enough programs measure and publish.

What this won't solve

Measurement is not free, and a measurement program that nobody uses is just more overhead on a team that already feels overloaded. Some honest trade-offs:

Attribution is genuinely hard. If defect counts drop after an MBSE rollout, you also changed team composition, review cadence, and probably the tool chain. Isolating the model's contribution in a real program is close to impossible, and anyone promising a clean causal number is selling something.

You probably have no baseline. Most organizations never measured the document-based process they're comparing against. Reconstructing it after the fact is guesswork, which is why the most credible MBSE ROI cases come from bounded comparisons like interface requirements management or change impact analysis, where the before and after are narrow enough to actually compare.

Metrics get gamed. "Number of users in the tool" becomes a target and suddenly everyone has a license they open once a quarter. Adoption measures are leading indicators, not goals, and treating them as goals produces exactly the theater you were trying to avoid.

The timeline is long. Cost effects show up in later phases, and on a program with a seven-year development cycle, the people who funded the transformation may have moved on before the data arrives. That is an organizational problem more than a technical one, and no framework fixes it.

Bottom line

The demand for an MBSE ROI number is reasonable. The expectation that it exists today, as a clean percentage anyone can quote, is not. The DE Measurement Framework is free, it has serious institutional backing, and it gives you a defensible structure for what to collect. Start with adoption measures because they're cheap and they gate everything else. Add defect timing and review efficiency next, because those are where the mechanism should be visible if it's working at all. Treat cost as the destination rather than the starting point.

And if you're building the business case right now, the most useful thing you can do for the next person is publish what you measured. The reason the evidence base is thin is that almost nobody does.

Sources: Value and benefits of MBSE: Evidence from the literature (Henderson & Salado, 2021), MBSE: Evaluating perceived value, metrics, and evidence through literature (Campo et al., 2023), Measuring the RoI of Digital Engineering: It's a Journey, Not a Number (McDermott et al., 2022), PSM Digital Engineering Measurement Framework, SERC on the DE Measurement Framework

← Back to all articles