Implementation is not improvement: how to measure the impact of healthcare technology

How many technologies has your organization implemented without being able to demonstrate, a year later, what actually changed? A healthcare institution's adoption of a new technology does not lead to immediate improvement. After selecting the most suitable solution, as we explained in the previous article, it needs to be properly integrated into the organization's operations, with a clear way to verify whether it is actually meeting the established goals.

Implementation does not necessarily mean improvement. And that is precisely why healthcare technology should be evaluated based on the results it delivers, not just the features it offers.

To do this, the first step is to define what "improvement" means for the organization and establish how it will be measured.

Defining what "improvement" means

When a new technology is introduced into an established process, some type of improvement is expected. However, improvement can relate to different aspects and does not necessarily mean the same thing for every institution.

Depending on the goal of the implementation, an organization may seek improvements in areas such as:

• Efficiency of the care process: reducing time spent or administrative tasks.
• Quality of care: enabling more consistent processes or improving continuity of care.
• Clinical or care outcomes: achieving better results on specific indicators.
• Patient and healthcare professional experience: facilitating access, reducing friction, or improving the experience throughout the process.
• Care capacity: enabling healthcare professionals to care for or monitor more patients.
• Financial impact: reducing operating costs, optimizing resources, or achieving a return on investment (ROI).

What to measure before implementation

The first step in evaluating a new healthcare technology should take place before implementation. It is necessary to understand how the process targeted for improvement currently works so that results can be compared later.

This initial state forms a baseline. Without it, it can be difficult to determine whether changes observed after introducing the technology are actually due to its use or to other factors. For example, a facility could measure how much time the team currently spends on triage and gathering information before an appointment, then use that value as a reference to evaluate whether the tool reduces that workload. In the case of Nexi Health, estimates suggest a reduction of up to 60% in the time spent on preliminary clinical assessment, an improvement that can only be properly evaluated by comparing it with the previous situation.

The defined objectives can be used to establish KPIs (Key Performance Indicators), which measure specific aspects of how the process works.

Indicators may vary depending on the objective, but some common dimensions include:

Clinical and care KPIs	Adoption KPIs	Financial KPIs
Time	Usage	ROI
Healthcare professional workload	Healthcare professional acceptance	Freed-up resources
Capacity	Abandonment	Additional capacity
Follow-up	Ease of use	Cost per patient
Detection	Integration	Implementation costC

The goal is not to measure everything. KPIs should be directly related to the problem the technology aims to solve.

From pilot to the decision to scale

Once the technology has been implemented in an initial phase, data for the defined indicators needs to be collected and compared with the baseline.

This evaluation reveals what changes have occurred, where value has been created, and where challenges remain. It also helps identify differences between expectations and what actually happens when the technology is used.

The results may show progress in some areas and more limited outcomes in others. This does not necessarily mean the technology does not work. It may indicate a need to adjust the process, improve the tool's integration, or make it easier for healthcare professionals to adopt it.

Feedback from those involved in the process, especially patients and healthcare professionals, is also important for interpreting the data and identifying aspects that quantitative indicators alone do not reveal.

That is why a pilot should not serve only to determine whether a technology should be scaled. It should also provide an opportunity to learn how to integrate it into the organization's day-to-day operations.

If the results are positive and the process is ready to grow, the next phase is to assess its scalability: whether the results can be sustained in a broader context, whether healthcare professionals can sustainably incorporate it into their work, and whether the organization has the resources needed to expand its use.

Measuring the change, not just the technology

Healthcare technology should not be evaluated solely on what it does, but on the change it brings to the care process.

Understanding that change requires establishing a baseline, defining what needs to improve, setting appropriate indicators, and using the results obtained during the pilot to make decisions.

A tool can deliver all its features perfectly and still fail to have a meaningful impact on the organization. That is why the question guiding any adoption process should not just be "What can this technology do?" but also "What problem do we want to solve, and how will we know whether we have solved it?"

Technology adoption should be understood as a process of continuous evaluation and learning: measure, analyze, adjust, and, when there is sufficient evidence, scale.

Because the true value of healthcare technology lies not in bringing it into an organization, but in demonstrating that doing so improves what it was intended to improve.