Measuring Performance – Base Level Accuracy Metrics

Relative Accuracy
• Relative accuracy compares the total predicted quantity for a
period against the actual yield, illustrating the relationship
between the two numbers
– Most meaningful at the lowest forecasted level
– To calculate relative accuracy, capture the forecast and the actual orders
for a period and divide the smaller number by the larger

Units
Forecast 1,000 800
Orders 800 1,000
Relative Accuracy 80.0% 80.0%
– The result is an absolute percentage that can be interpreted against a scale
with a range of zero to 100%

dp lanning
Global Supply Chain
&
eployment
Business Forecasting The Demand Planning Process
 Measuring Performance – Base Level Accuracy Metrics

Relative Accuracy
• Relative accuracy can have value at an aggregated level
– Similar products bundled into a single portfolio
• Common applications
• Shared components and physical characteristics
• Uniform cost and pricing structures
• Provide value to the sales, marketing, and financial teams
• The disadvantage in using relative accuracy at an aggregated level is that
its does not account for product mix performance
Portfolio A
Relative
Product Forecast Orders
Accuracy
ATP1001 10,000 9,905 99.1%
ATP1002 100 20 20.0%
ATP1003 200 100 50.0%
ATP1004 50 200 25.0%
ATP1005 10 30 33.3%
Total 10,360 10,255 99.0%

dp lanning
Global Supply Chain
&
eployment
Business Forecasting The Demand Planning Process
 Measuring Performance – Base Level Accuracy Metrics

Weighted Mean Absolute Percentage Error (WMAPE)

• WMAPE is the measurement used for reporting aggregated accuracy results

• WMAPE is the global reported accuracy measurement
• WMAPE solves the problem of using Relative Accuracy for product groups by
accounting for product mix characteristics
– To calculate WMAPE determine the absolute error for each product within the portfolio,
convert each error value to a percentage of the total forecast, and sum the percentages to
determine the WMAPE for the group. To convert the error into accuracy, subtract it from one
– The effect of considering the error contribution of each product on the accuracy results is
apparent when we compare the relative accuracy results to the WMAPE accuracy

Portfolio A
Relative Absolute Percent of
Product Forecast Orders
Accuracy Error Total Forecast
ATP1001 10,000 9,905 99.1% 95 0.9%
ATP1002 100 20 20.0% 80 0.8%
ATP1003 200 100 50.0% 100 1.0%
ATP1004 50 200 25.0% 150 1.4%
ATP1005 10 30 33.3% 20 0.2%
Total 10,360 10,255 99.0% WMAPE 4.3%
Accuracy 95.7%

dp lanning
Global Supply Chain
&
eployment
Business Forecasting The Demand Planning Process
 Measuring Performance – Base Level Accuracy Metrics

Weighted Mean Absolute Percentage Error (WMAPE)

• WMAPE es la medida usada para divulgar los resultados agregados de la exactitud.

• WMAPE es la medida global divulgada de la exactitud.
• WMAPE soluciona el problema de usar la exactitud relativa para los grupos de
producto por caracteristicas de la mezcla del producto.
– Para calcular WMAPE determinar el error absoluto para cada producto dentro del portafolio,
convertir cada valor del error a un porcentage del pronóstico y sumar los porcentages para
determinar el WMAPE para cada grupo. Para convertir el error en exactitud, restarla a partir de la
primera.
– El efecto de considerar la contribución del error de cada producto en los resultados de la exactitud
es evidente cuando comparamos los resultados de la exacitud relativa con la precisión de WMAPE.
Portfolio A
Relative Absolute Percent of
Product Forecast Orders
Accuracy Error Total Forecast
ATP1001 1 10,000 2 9,905 99.1% A 95 =1-2 0.9% =A-B
ATP1002 100 20 20.0% 80 0.8%
ATP1003 200 100 50.0% 100 1.0%
ATP1004 50 200 25.0% 150 1.4%
ATP1005 10 30 33.3% 20 0.2%
Total B 10,360 10,255 99.0% WMAPE 4.3%
Accuracy 95.7%
dp lanning
Global Supply Chain
&
eployment
Business Forecasting The Demand Planning Process
 Measuring Performance – Base Level Accuracy Metrics

Forecast Tolerance
• Forecast tolerance is another approach to measuring
aggregated forecast performance
• Forecast tolerance quantifies the number of products
within a group that meet or exceed a predetermined
forecast accuracy target or range
– Extremely stringent
– Often misused and misunderstood
– Limited in value for improving forecast performance
– Should be used to focus efforts on a specific segment

dp lanning
Global Supply Chain
&
eployment
Business Forecasting The Demand Planning Process
 Measuring Performance – Base Level Accuracy Metrics

Forecast Tolerance
• To determine the tolerance result for a segment of products, first calculate
the relative accuracy for each product, then determine whether the
relative results meets or exceeds the established threshold, total the sum of
products meeting their targets, and divide the result by the total number
of products within the portfolio
Portfolio A
Forecast Relative Meets
Product Forecast Orders
Target Accuracy Target?
ATP1001 95% 10,000 9,905 99.1% Y
ATP1002 85% 100 20 20.0%
ATP1003 85% 200 100 50.0%
ATP1004 85% 50 200 25.0%
ATP1005 85% 10 30 33.3%
ATP1006 90% 400 375 93.8% Y
ATP1007 90% 250 300 83.3%
ATP1008 95% 1,000 980 98.0% Y
ATP1009 95% 500 480 96.0% Y
ATP1010 90% 700 850 82.4%
Total 4
Total Products 10
Accuracy 40.0%

dp lanning
Global Supply Chain
&
eployment
Business Forecasting The Demand Planning Process
Measuring Performance – Base Level Accuracy Metrics

Forecast Accuracy
Portfolio A

120%
96.3% 94.6% 97.0% 96.9% 97.4% 96.5%
100% 88.6%
92.7% 92.0% 91.4% 93.8%
86.6%

80% 89.4% 89.9% 88.8% 89.9% 89.0%

84.2% 86.8% 83.4% 83.2%
86.6% 85.2%
81.2%

60% 52.0% 51.0% 49.0% 52.0%

47.0% 48.0% 47.0%
44.0% 44.0% 43.0%
41.0% 39.0%
40%

20%

0%
Jan` Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Relative Accuracy WMAPE Tolerance

dp lanning
Global Supply Chain
&
eployment
Business Forecasting The Demand Planning Process
 Measuring Performance – Base Level Accuracy Metrics

Forecast Bias
• Forecast bias is a continued prejudice in the forecast, the tendency to
always over-forecast or under-forecast
– Over-forecasting (positive forecast bias) contributes to an increased inventory
investment, reducing operational cash flow
– Under-forecasting (negative forecast bias) reduces inventory investment, increases
the likelihood of poor customer service, creates lost profit risk and increases
expenses
– To calculate forecast bias capture the forecast and the actual orders for a period and
subtract the orders from the forecast, then divide the result by the larger of the
forecast or orders. This will generate a positive or negative percentage that can be
plotted on a scale relative to zero

Relative
Product Forecast Orders Bias
Accuracy
ATP1003 200 100 50.0% 50.0%
ATP1004 50 200 25.0% -75.0%

dp lanning
Global Supply Chain
&
eployment
Business Forecasting The Demand Planning Process
Measuring Performance – Base Level Accuracy Metrics

Forecast Bias
Portfolio A

100.00%
80.00%
60.00%
40.00%
11.4% 7.3% 8.0% 8.6%
20.00% 3.7% 5.4% 3.0% 3.1% 2.6% 3.5%

0.00%
-20.00% -6.2%
-13.4%
-40.00%
-60.00%
-80.00%
-100.00%
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Bias

dp lanning
Global Supply Chain
&
eployment
Business Forecasting The Demand Planning Process