How can a performance problem be investigated with evidence rather than assumptions?

How is a measurement question framed?

“The system is slow” is an observation, not a question you can measure. A measurement question covers the operation or scenario, the affected user group, the time window, the expected and observed duration, and the business impact. An investigation that starts without these five items tends to focus on whatever is easiest to measure, not on what matters most.

Measurement question template:

Why is a baseline needed?

A baseline is a measurement of the current state under defined conditions. Without one, the effect of a change cannot be shown. It is set from a distribution rather than a single measurement: alongside the average, it records percentiles showing the time within which most operations finish, and the longest durations.

An average can hide operations that are rare but take a very long time. The problem users feel is often in that long tail. So the baseline states plainly which statistic is used, and later measurements use the same one.

Why record runtime and data volume context?

The same operation runs for different lengths of time under different conditions. If the concurrent load, running batch jobs, volume of data processed, data distribution, selection criteria and cache state are not recorded at the time of measurement, two measurements cannot be compared.

This context matters as much as the measurement itself, because it shows under which conditions the result holds. Data volume matters especially. An operation that runs quickly on a small data set can slow down disproportionately as the data grows. So the number of records processed and the selection criteria are written down with the measurement; otherwise you cannot tell whether the slowness comes from the code or from the data.

Which variables distort a measurement?

Some variables change the measured duration even though they have nothing to do with the problem. If they are not controlled, or at least recorded, a difference between two measurements can be pinned on the wrong change. The variables below are reviewed for each measurement; any that cannot be controlled are noted and taken into account when interpreting the result.

• a cache that is empty on the first run and warm on later runs
• batch jobs, backups or heavy user traffic running at the same time
• changes in data volume or distribution between measurement periods
• users with different authorisations seeing different amounts of data
• fluctuating network latency and the current state of other systems
• the overhead of the measurement or tracing tool itself
• periodic peaks such as month-end

If a variable is suspected, an extra measurement is made that changes only that variable. That shows separately how much of the observed difference comes from it.

Which sources does the evidence come from?

No single source shows the whole of a performance problem. System-level monitoring data shows resource usage, traces show how much time an operation spends in each step, and database access statistics show which queries run, how often and with how much data.

Application logs and batch job records show timing, and the duration a user measures shows how the work actually feels.

These sources are used together so they can confirm one another. If a trace shows that a query takes a long time, database statistics can confirm how much data the same query read in that period. If the sources contradict each other, the contradiction is itself a finding, and the measurement conditions are checked again. Which tool provides these sources varies with the product and release; the evidence pack records which source each measurement came from.

Actual product screen

Scroll horizontally to read the screen, or select the image to open it at full size.

Where is the time being spent?

Until the total time is broken down along the call chain, you cannot know which change will help. Time is usually spread across these items: database access, application logic, calls to external systems, the network, lock and resource waits, and the user interface. The investigation measures the share falling to each.

Patterns commonly seen in ABAP code are also confirmed by this measurement, not by guesswork. Database access repeated inside a loop, reading more data than needed, or processing the same data several times can all be sources of trouble. But finding a pattern in the code does not show that it is the main bottleneck; measurement does.

How is the bottleneck told apart from the symptom?

A symptom is the observed result: a long duration, high processor usage, a full queue. The bottleneck is the constraint that, once reduced, actually reduces the total time. High processor usage may be the symptom of an inefficient loop, while the real bottleneck is elsewhere, for example in waiting for an external call.

Reducing a symptom does not mean removing the bottleneck. The distinction is made with a hypothesis: “the time grows because of item X; if X is reduced, total time should fall by this much.” The hypothesis is tested with a measurement that isolates X. If it is not confirmed, the bottleneck is elsewhere and the investigation moves on to the next item.

How is measurement made repeatable?

A repeatable measurement gives the same result under the same conditions. That means using the same data and selection criteria, and measuring either in the same time window or under a controlled load. The measurement is repeated several times and the spread of results is recorded; a single run can make a random deviation look like a result.

The measurement protocol is written down: how it was started, under which conditions, how many times and which statistic was used to summarise it. The same protocol is applied to the measurement after the change. If the protocol changes, the two measurements cannot be compared, and the report says so.

How can another team reproduce the evidence?

Evidence that only the person who collected it can repeat is not strong enough for a decision. Reproducible evidence is evidence that another team can reach by following the same steps. That means the measurement steps, the data used, the version and configuration of the environment, and the raw results are kept separately from their interpretation.

Small differences are normal when evidence is reproduced. So how far apart two results can be and still count as “the same” is defined before measuring. If the difference falls outside that limit, the difference in conditions between the two measurements is investigated, and the result is not used for a decision until it is explained. Keeping raw results also allows the interpretation to be revisited later.

How are the before and after states verified?

Make one change at a time and measure its effect with the same protocol. When several changes are made at once, you cannot tell which one helped; one may have improved things while another made them worse. The comparison uses the same distribution statistics as the baseline, not just the average.

The result is reported in the same form as the baseline. Side effects are checked too: other operations using the same data, batch jobs sharing the same resources, or integrations may have been affected. A change that speeds up one operation and slows down another may not count as an improvement for the business.

Why is an unmeasured gain not claimed?

An estimated improvement is not a measured one. “This change will shorten the duration” is a hypothesis and is written as one. Only before and after values measured under the same conditions go into the results report. If measurement was not possible, that is stated plainly, along with which measurement could be made instead.

This rule protects management decisions. An unmeasured gain can be used as if it were real data in the next budget or priority decision, and when it fails to materialise, the basis for that decision disappears.

What are the limits of comparing environments?

Test and production environments can differ in data volume, data distribution, configuration, concurrent load and hardware. A measurement in a test environment points in a direction; it is not evidence of behaviour in production. The report states which environment the measurement was taken in and how that environment differs from production.

If measurement in production is needed, the overhead of the measurement tools and the measurement window are planned in advance. The measurement is carried out so that it does not affect the business process, and with the knowledge of the relevant owners.

What goes into an evidence pack?

Common wrong assumptions

• “The operation using the most resources is the source of the problem.” Heavy resource use does not make it the bottleneck; measurement has to show its effect on total time.
• “If it is fast in test, it will be fast in production.” Differences in data volume and load can change the result completely.
• “If the average improved, users will feel the improvement.” If the operations in the long tail did not change, users may notice no difference.
• “If the code contains a known bad pattern, that is the problem.” The presence of a pattern does not show its effect; its effect is measured.
• “Making several changes together saves time.” You cannot tell which one helped, and one may mask the effect of another.

Checklist

• The measurement question is written with its five pieces of information
• The baseline is set with distribution statistics
• Environment, load, data volume and selection criteria are recorded
• Variables that could distort the measurement are controlled or recorded
• Time is broken down into items along the call chain
• A bottleneck hypothesis is written and tested with an isolating measurement
• A measurement protocol is written and applied over several repetitions
• Measurement steps, data, environment details and raw results are kept so another team can reproduce them
• One change is made at a time
• The after measurement uses the same protocol; side effects are checked
• Environment differences and what could not be measured are written into the report
• No unmeasured gain is reported as an outcome

Limits of this guide

This guide is a general engineering explanation and does not rest on any product documentation. SAP tool names and product-specific settings are deliberately left out; they vary by product and release and should be confirmed in your landscape. We do not claim any particular speed-up, resource usage or duration outcome.

Frequently asked questions

Related pages and guides

• Our working approach: why we do not report outcomes that were not measured
• Error handling and safe reprocessing in integrations: monitoring waiting messages and timings for integration delays
• Custom code inventory and technical debt: where performance findings sit in the inventory
• All technical guides

Sources

This guide does not rest on any product documentation. The content is general engineering knowledge and Castintech’s investigation framework, and is not attributed to SAP. The guide contains no externally sourced claims.

Last verified: 23 September 2026

Independence note

• Castintech does not claim any partnership, authorization, endorsement or sponsorship relationship with SAP SE.
• SAP and the SAP product names mentioned on this page are trademarks of SAP SE or its affiliates.
• The work Castintech offers is independent technical consulting and support.