Prove the Value of Enterprise Architecture With SMART KPIs
This note outlines some of the fundamental key performance indicators (KPIs) that you should measure to show the success of the enterprise architecture (EA) team. It also discusses how you measure them and visualize the result.
There are four types of metrics covered here:
EA Value Index covers the metrics that are directly attributed to the architecture team alone. These are great metrics to track not only to show the value of enterprise architecture, but to use over time as a great motivator for the architecture team.
Business Alignment Index covers how well enterprise architecture is satisfying the need of the business. When these metrics increase, it is usually accompanied by architecture job satisfaction and increased effectiveness of relationships between the business and architecture.
“Everyone needs to know that their job matters to someone – anyone. Without seeing a connection between their work and the satisfaction of other people, an employee will not find lasting fulfillment.” – Patrick Lencioni
Standards are an effective way to work smarter and reduce required effort on business change. Measuring how much the organization is standards based is a good way to motivate architecture and development to work as efficiently as possible.
Technical debt is generally incurred when tactical decisions are made for a good reason but do not align to the strategic objective. It has always been difficult to reduce technical debt over time because fixing something that is perceived to be working is inefficient. The metrics we cover in this section will show how much technical debt is being incurred against that which is being resolved over time.
Recommendations
This section covers some key metrics split into the four types that can accelerate and gain efficiencies across enterprise architecture.
EA Value
Metric Title: Plateau alignment
SMART Measure: How many plateaus have development iterations been aligned to?
How: In your enterprise continuum, there should be several artifacts that describe your enterprise architecture. Best practice dictates that you create an implementation and migration viewpoint that defines the architecture plateaus (or interim architectures). These plateaus represent base camps on the way to the summit of your development program. If you trace these plateaus to the work packages that represent your sprints or work iterations, you will be able to track how many development iterations relied on enterprise architecture as part of the planning process. This can be achieved in any good enterprise architecture tool.
Metric Title: Business capability satisfaction
SMART Measure: Satisfaction of business capabilities that have been realized by EA oversight
How: During show and tells or with a survey, you can easily establish which capabilities were useful to the business. If the architectures are traced to the business capability model, it is possible to establish a direct relationship between enterprise architecture effectiveness and business satisfaction. This can all be modeled in an architecture tool.
Metric Title: Regulation compliance
SMART Measure: Number of compliant regulation components based on reusable architecture
How: When you break down compliance and regulation into its constituent parts, each clause or section of the regulation may have implications or requirements for the architecture. It is important as architecture is created to trace it back to those constituent parts to establish that enterprise architecture is helping risk and compliance. This is an important metric because regulation is not normally optional but mandatory and the consequences of non-compliance can be severe. The traceability can be modeled in an architecture tool.
Metric Title: Reduction in development/integration effort
SMART Measure: Reusable ABB, SBB, architecture patterns by effort
How: When you create reusable enterprise architecture artifacts in the enterprise continuum, each one should have an effort measure associated with them. This can be used to determine how much effort is saved when those artifacts are reused in a project. These could be architecture building blocks (ABB), solution building blocks (SBB), reference architectures, or standard patterns. Using an architecture tool here will greatly reduce the effort in tracking reusability. When you can tell the story of cost savings on reusability, if done well this can almost pay for the enterprise architecture practice with a single metric.
Business Alignment
Metric Title: Number of level-three business capabilities implemented
SMART Measure: Count of business capabilities associated to plateau architectures
How: As with the business satisfaction of capabilities, this metric relies on the traceability from business capabilities to architectures. If you measure the capabilities that have been implemented during the sprint with supporting architecture, you can equate architecture’s contribution to the business goals. This modeling is possible in most enterprise architecture tools.
Metric Title: Number of high-value level-three capabilities implemented
SMART Measure: Count (or sum if capabilities are weighted) of high-value capabilities implemented
How: Similar to the previous metric, however, this focuses on the critical capabilities to the business. Usually the business architecture includes a capability model that contains a heatmap view of the priority business capabilities against goals. This can be used to filter the critical capabilities implemented by architecture to achieve a more powerful metric.
Metric Title: Number of business issues resolved
SMART Measure: Count of business issues resolved between sprints
How: Show and tells for the business should not be passive meetings; the business should be encouraged to evaluate the product from the sprint and critique which parts of the product are effective and satisfy requirements and which need work. Each issue identified that needs work should be addressed by the next show and tell. This will give the business confidence that development is progressing for the exclusive benefit of the business. If these issues or tasks are recorded and attributed to the ceremony in a tracking tool, reporting on this metric can be achieved easily.
Metric Title: Number of business issues identified
SMART Measure: Number of business issues identified per show and tell
How: As per the previous metric, it is good practice to track how many issues are identified by the business daily. The stronger the relationship between architecture and the business becomes, and the more aligned they are to the goals, the fewer issues will be raised during the show and tells.
Standards
Metric Title: Number of deviations from technology standard approved
SMART Measure: Approved deviations from mainstream or emerging standards
How: Your standards landscape, including architecture building blocks, solution building blocks, patterns, and technology standards, should be centrally documented, ideally in a wiki or intranet. These standards should be classified as:
- Emerging (something new that should become a mainstream standard in future)
- Mainstream (the preferred standard that most initiatives should use)
- Contained (standards that should not be used for new initiatives but may remain in the current state architecture from a legacy perspective)
- Decommissioned (standards that were once mainstream in the organization but have since been retired deliberately due to expense, out of support, or obsolete in some way; standards in this category should never be used)
When initiatives come through architecture governance to request a waiver for misalignment to standards, they should be recorded and tracked so that deviations from standards are reduced over time.
Metric Title: Resolvable standard deviations
SMART Measure: Deviations from standards that have a resolution path
How: A subset of the waivers granted in governance will have a future initiative or program that will enable the waiver to be released. This subset of waivers should also be tracked to show the volume of misaligned initiatives that will be resolved in time.
Metric Title: Payloads and data stores modeled on standards
SMART Measure: Entity coverage that is traced to a standard data model
How: Best-practice data architecture states that data models are traced to standards such as EDI, HL7, or UBL. This decreases the ambiguity of the models, speeds up the modeling process, and reduces the transformation needed between payloads in services to the model of the data at rest. A good data model has traceability from all entities to the standards on which they are based. This can be used to generate the metric. This facility is available in most good enterprise architecture modeling tools.
Technical Debt
Metric Title: Story points
SMART Measure: Always have a pointed story that represents technical debt with story points
How: When you incur technical debt, if it is documented in a story or requirement in a catalog with an associated story point or effort estimation, you can measure the increase or decrease of technical debt over time. Project management tools, especially those used for Agile projects have this facility built in.
Metric Title: Resolvable
SMART Measure: Technical debt stories that are resolvable
How: Some technical debt stories that you create when the debt is incurred can immediately be associated with an epic story in the backlog for a future initiative that will resolve the technical debt when complete. This is worth documenting so that you can project how much technical debt you will have at a point in the future.
Metric Title: Average time to resolve
SMART Measure: Time between current date and expected completion date
How: Going back to the last metric, it’s possible to measure the time it takes to resolve each technical debt item if you have an epic story planned for the future that resolves the debt item. You can report on this in project management tools to ensure the time to resolve reduces over time.
Bottom Line
Many of the metrics mentioned here required an enterprise architecture tool. This is one great reason for modeling architecture rather than creating images of architecture. However, it is still possible to capture some of these KPIs without the tooling, but it may be a high effort and very manual process.
Once these metrics have been established and you can track progression over time, the result should be better business engagement, reduction in effort of development cycles, and a business case for more architecture resources!
Want to Know More?
Traditional Enterprise Architecture Is Dead. We Need to Move With the Times.