Académique Documents
Professionnel Documents
Culture Documents
Project NCD-1-CD-FLOW
The information contained in this document is the property of the EUROCONTROL Agency and no part should be
reproduced in any form without the Agency’s permission.
The views expressed herein do not necessarily reflect the official views or policy of the Agency.
This document has been collated by mechanical means.
Should there be missing pages, please report to:
Distribution Statement :
(a) Controlled by : Head of NCD (Network Capacity and Demand management)
(b) Special Limitations (if any) : None
(c) Copy to NTIS : No
Descriptors (keywords): Uncertainty, Smoothing, ATFCM, ATFM, Indicators,
CFMU, Performance, FMP, Flow, Capacity
Abstract :
Objective: This note presents the interim conclusions of a study launched in April 2006.The
study aimed at proposing a list of ATFM performance indicators that reflect the variety of the
ATFM users' objectives.
The study was part of the yearly NCD "ATFCM Studies" work programme, in support of and
funded by CFMU.
Results: The notion of ATFM performance should be associated to a wide range of users'
objectives which lies well over the traditional resolution of the hourly demand excesses
problem. Other objectives are: Reduction of the traffic bunching problem, prevention from the
forming of complex traffic clusters within the sectors, improvement of flows predictability,
optimisation of the declared capacity.
Different indicators are defined to assess whether the system is able or not to meet these
objectives. In some cases, it is possible to observe that a given regulation can be very
efficient for the achievement of one particular objective while providing modest results for the
achievement of the other objectives. That is why, it is proposed to assess the performance by
applying the whole set of indicators.
Intentionally blank
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
TABLE OF CONTENTS
1 INTRODUCTION 1
1.1 DOCUMENT RATIONALE ................................................................................... 1
1.2 CONTEXT ....................................................................................................... 1
1.3 STRUCTURE OF THE DOCUMENT ...................................................................... 2
2 ATFM REGULATION EFFICIENCY: PERFORMANCE MEASURE
ACCORDING TO USERS’ OBJECTIVE 3
2.1 OBJECTIVE N°1: IMPROVEMENT OF THE INCOMING TRAFFIC FLOWS SMOOTHING . 3
2.2 OBJECTIVE N°2: IMPROVEMENT OF FLOWS PREDICTABILITY ............................... 4
2.3 OBJECTIVE N°3: OPTIMISATION OF CAPACITY USE ............................................. 4
2.4 CONCLUSIONS................................................................................................ 5
3 PERFORMANCE INDICATORS 6
3.1 SMOOTHING INDICATORS ................................................................................ 6
3.2 UNCERTAINTY REDUCTION INDICATORS .......................................................... 14
4 ILLUSTRATIVE APPLICATION CASE 17
4.1 SMOOTHING PERFORMANCE APPLICATION CASE: ANALYSIS OF ONE REGULATION
PERIOD (EUHL26A REGULATION - LFEUHL4 TV) .......................................... 17
4.2 UNCERTAINTY REDUCTION APPLICATION CASE: LFEUF4 TV (REIMS ACC)
ANALYSIS, OVER ONE AIRAC CYCLE .............................................................. 20
5 APPROPRIATENESS OF THE USE OF ALL-FT FILES VS. THE USE OF
SIMULATION RESULTS (TACOT) 23
5.1 RELEVANCE OF A STEP-BY-STEP SIMULATION OF THE SLOT ALLOCATION PLAN
LIFE-CYCLE .................................................................................................. 23
5.2 SIMULATIONS RESULTS ................................................................................. 24
6 CONCLUSION & NEXT STEPS 29
6.1 CONCLUSION: A SEGMENTED ATFM GROUND REGULATION PERFORMANCE
ASSESSMENT ............................................................................................... 29
6.2 NEXT STEPS ................................................................................................. 30
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
List of Figures
Figure 2: 2D view (average amplitude and duration of the traffic load fluctuations).......... 8
Figure 3: 2D view (average amplitude and duration of the traffic load fluctuations).......... 9
Figure 4: Flow evolution curves (measures of max. and average excesses) ........................ 11
Figure 5: Flow evolution curves (measure of the average deviation from the flow rate).. 12
Figure 6: Distributions of TV entries over the reg. period (measure of the time during
which TV entries are over a certain level) ......................................................... 13
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
GLOSSARY
WP Work Package
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
REFERENCES
EUROCONTROL
Tasking Support for ATFM – Impact of uncertainty and smoothing on
declared capacity – Interim results (November 2005)
EUROCONTROL
Tasking Support for ATFM – Impact of uncertainty and smoothing on
declared capacity – General framework (November 2005)
EUROCONTROL
Tasking Support for ATFM – Intermediate note on ATFM efficiency indicators
- ATFCM Performance and Declared Capacity study (July 2006)
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
1 INTRODUCTION
1.2 Context
Therefore, the ATFM performances should reflect the variety of the ATFM users’
various objectives and especially the ones of the FMPs and FMDs. The identification
of the users’ objectives associated to ATFM performance has constituted the
baseline for the instigation of the work on ATFM performances indicators. Once the
objectives have been identified, a set of indicators has been proposed to assess the
performances of the ground regulation process for each of the identified objectives.
Those indicators have been further “tested” on sundry application cases.
1 The FTFM is the “initial” profile as it reflects the status of the demand before activation of the regulation
plan. It is computed with the latest flight plan version, sent by each AO to the CFMU/IFPS.
2 The RTFM is the “regulated” profile as it reflects the status of the demand after activation of the regulation
plan. It is computed with the latest ATFM slot (CTOT) issued to the AO, by the ground regulation system.
1
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
The document is broken down into six sections and one annex:
9 Section 1 – Introduction – presents the purpose and structure of the present
document;
9 Section 2 – ATFM regulation efficiency: Performance measure according
to users’ objective – recalls the users’ objectives identified in the previous
2005 study and presents how the performance indicators are related to them;
9 Section 3 – Performance Indicators – presents the indicators proposed to
give account of the system’s capability to fulfil the users’ objectives and traffic
counting functions on which these indicators are based;
9 Section 4 – Illustrative application cases – presents the results, which
could be obtained from the application of the indicators on a particular case;
9 Section 5 – Appropriateness of the use of ALL-FT files vs. the use of
simulation results (TACOT) – assesses the suitability in using “after the
fact” data extracted from ALL-FT files, as compared to the use of results
provided by an ATFM fast-time simulator (TACOT), for the measurement of
ATFM performances;
9 Section 6 - Conclusion and Next steps – presents the interim conclusions
on the proposed performance assessment method, the next work steps and
the associated requirements.
9 Annex 1 – ATFM smoothing performance assessment (methodology) –
presents the methodology applied for the traffic data analysis and the building
of the different observables of the ATFM slot allocation smoothing efficiency.
2
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
In interviewing FMPs and FMDs, it appeared that the smoothing objective could be
parted in three different sub-objectives:
9 Firstly, the traditional resolution of the hourly demand / capacity
imbalance. This refers to the original mandate attributed to the ATFM
regulation system and relies on a commitment vis-à-vis a number of
movements per hour.
The corresponding performance indicators will therefore derive from the
analysis of hourly flow counts.
9 Last but not least, the prevention from the forming of complex traffic
clusters within the sectors. This last objective may be considered out of
ATFM scope, and is normally addressed directly by the ACCs: FMPs,
together with ATCOs collaboratively coordinate with upstream ACCs for
obtaining a regular spacing of aircraft and preventing from the forming of
complex traffic clusters within the sectors. However, we were reported that
FMPs may also wish to improve in advance the regularity of the flows
3
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
throughput and use the “limit” smoothing capability of the ATFM system for
this “à la marge” traffic smoothing objective.
The corresponding performance indicator derives from the analysis of
instantaneous – 1 minute flow counts.
The last class of objective is an indirect, longer term objective that is out of the
performance measurement scope presented in this document. It focuses on the
direct results of ATFM ground regulation application (smoothing and predictability
improvement) on the declared capacity over the years.
As a matter of fact, the calculation of the declared capacity could be sometimes an
empirical process. A way of assessing a suitable value for the declared capacity of a
given TV consists in issuing some ATFM ground regulations with gradual increased
values. The process allowing the suitable declared capacity value could be explained
as follows:
At first the FMP issues a low value for the declared capacity and if there is no ATCO
report on over-delivery for some times, the FMP will gain confidence in the system
and will increase gradually the declared capacity value to a point at which some
ATCOs will complain on over-deliveries.
Therefore, the objective – that draws the link between the ATFM service performance
and the capacity offer – could be assessed using a specific performance indicator
accounting for the positive evolution of capacity figures, if not over the entire life
cycle of the ATFM ground delay service, at least over months or years.
3 The regulations restrict departure times’ variations down to a 15 minutes tolerance whilst the tolerance
4
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
2.4 Conclusions
The performance of a given ATM ground regulation should reflect the objectives
sought by the users when issuing the ground regulation. That is why, we propose to
assess the overall performance of a ground regulation, by applying the whole set of
indicators on this regulation. We will probably observe that a given regulation can be
very efficient for the resolution of the bunching effect when providing modest results
for the prevention of hourly flows over-deliveries. In other cases, we will maybe find
that a given ground regulation has been issued just to reduce the uncertainty
conveyed by the stochastic nature of air transport.
In any case, the performance of a given regulation will be assessed in applying the
whole set of indicators reflecting all the users’ objectives.
5
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
3 PERFORMANCE INDICATORS
Performance indicators are derived from a model, which describes the different input
databases, counting functions and observation parameters used for the computation
of each indicator. The model is presented in Annex 1.
Performance indicators are presented in the following chapters.
4
A time index is represented on the (X axis). The origin (Index “0”) corresponds to the time of activation.
Index “178” corresponds to the start of regulation period. The regulation process is therefore activated at
anticipation “178” minutes, i.e. 2 h 58 min.
6
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
7
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
The primary purpose of ATFM service users (FMPs) in each ACC is to avoid
situations of unbearable ATCO workload. The objective is the elimination of the
uncontrolled loads of traffic (traffic peaks during significant periods of time).
Another objective is to make the most efficient use of the available capacity (deduced
from the flow rates requests) by minimising capacity wastes (traffic shortfalls during
significant periods of time), and to limit the burden (delays) imposed on airspace
users induced by the implementation of regulations.
Therefore, ATFM smoothing performance will be firstly evaluated with regards to the
reduction, in terms of duration and amplitude, of the traffic load fluctuations (“peaks”
and “shortfalls”) around the flow rate.
In order to discriminate between the different fluctuations, we propose to segment the
regulation period into series of shorter intervals, within which the flow evolution curve
is continuously either strictly above (peak interval) or below (shortfall interval) the
requested flow rate.
The result is a discrete characterisation of the flow evolution curve in terms of
average amplitude of a fluctuation (positive or negative) and duration of the
fluctuation.
Flow Rate: 47
Fluctuation of average
amplitude -10 (-21%)
and duration 106 min.
5 The average amplitude is measured in terms of number of aircraft above or below the flow rate and in
8
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
In the above figure, it could be shown that the blue curve (before activation of the
reg. process) is represented by one dot showing one over-delivery of average
amplitude 13% and one under-delivery of average amplitude 21%.
It could be also noticed that the pink curve (after activation of the reg. process) is
very fluctuating, and close to the flow rate as being represented by several dots close
to the “flow rate (i.e. 47)” axis.
Pink dots are inside a zone (in green) delimited by -10% and +10% below and above
the flow rate. Then, it could be inferred that the regulation is “successful”, as the
over-deliveries are fully eliminated according to a traditional7 tolerance margin
accepted by the FMP (in case of hourly counts measurements).
Figure 3: 2D view (average amplitude and duration of the traffic load fluctuations)
The aggregated results figure6 demonstrates clearly that regulations – most of the
time – prevent or at least decrease substantially over-deliveries (most of the pink
dots inside the -10% ; +10% green zone). However, some exceptions are recorded:
3 pink dots inside the {> 10% ; > 20 minutes}7 orange zone.
9
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
The observation of the blue zone {< - 10% ; > 20 minutes} depicts a more
preocupaying situation, in terms of capacity efficiency use: still significant 10 under-
deliveries records, revealing capacity wastes.
Thus, the use of the 2D views, as illustrated by the last presented figures are a good
way to discriminate between the successes patterns (a “success” zone that would
include the points for which the smoothing is successful according to a tolerance
margin, in this case: between -10% and +10%) and exceptional or failure situations
(the outside zones).
The reason for experimenting peaks of significant amplitude and duration, in zones
outside the tolerance margin within which regulations are considered “successful” by
the ATFM service user, are to be investigated using additional information recorded
at the CFMU in heavy databases (Oplog files). In these files, all the relevant ATFM
events, including regulation requests tracks are archived.
In many cases, sources of inefficiency situations are related to the system’s inertia
(regulations have not been issued before the time required by CFMU to operate
properly a regulation).
10
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
Given one regulation or given several regulations, the 2D view (amplitude; duration –
Indicator I1) is a very good way to look into the success or the failure of the TV(s)
protection and use of the available capacity. This analysis can be done for several
regulations, since each fluctuation is characterised individually.
Now, when trying to figure out if one particular regulation is in line with what the FMP
was expecting; one has to figure out if the implementation of this specific regulation
could allow the ATCO on his position to deal with the actual traffic.
To give account of this, the shape of the flow evolution curve can be characterised by
the maximum peak and average traffic excesses records, over the regulation period.
The two are complementary since an excess may not climax at the maximum, for a
long time, which in turn mitigates the average excess record.
Then, another pair of indicators derives from the measures of the maximum of
the flow evolution curve (I2) and of the average of the Flow evolution curve
excesses (I3) over the regulation period.
Note: In the above figure, the shape of the flow evolution curve is evaluated in terms
of numbers of aircraft and in proportion (%) to the flow rate (e.g. 53 aircraft
corresponds to (53-47)/47, i.e. 13%)
11
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
The other component of the ATFM smoothing performance relates to an efficient use
of the achievable capacity offered by the ATCO (deduced from the flow rate
requests).
Therefore, when looking at the performance of one single regulation, one has to look
on the ability of the system to maintain the load as close as possible to the flow rate,
over the regulation period. The system is all the more efficient as the average
deviation from the flow rate is low.
Then, a fourth indicator derives from the measure of the average deviation (in
absolute value) from the flow rate, over the regulation period.
Note: In the above figure, average deviations are evaluated in number of aircraft and
in proportion to the flow rate (e.g. an average deviation of 8 aircraft corresponds to
8/47, i.e. 17%).
12
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
Dispersion around other references than the flow rate, capacity management
Indicator I5 (definition of a “tolerance” margin for over-deliveries)
With the last presented measures, the smoothing is characterised with respect to a
“standard” reference, the requested flow rate.
In addition to the compliance with the requested flow rate, ATFM users may have
different criteria against which their own evaluation of the system’s performance is
made. Depending on the user, the “tolerance” margin within which a traffic excess
remains acceptable could be for example of 0%, 10% or 20% above the flow rate. In
the same way, the efficiency in the capacity use could be considered as achieved
when the load is above a limit of -20%; -10% or 0% below the flow rate.
Therefore, the smoothing quality could be also evaluated when that natural reference
(the flow rate) changes.
Then, a fifth indicator derives from the measure of the amount of time
(expressed in terms of percentage of the total regulation time) during which the
traffic flow has been over a certain level. That “certain level” is the new reference
and is defined over a range of values centred on the optimum (the requested flow
rate). Then it is another way of measuring the ability to keep the regulated traffic as
close as possible to the flow rate.
New Reference:
Flow Rate: 47 In the illustration, the initial (blue
Flow Rate + 20%: 56
curve) traffic excesses account for
47% of the total regulation time
whereas the system output a
Before regulation:
distribution with excesses
% of reg. period with
accounting for only 8% of the
TV Entries > Flow
total regulation time.
Rate: 47%
Before regulation:
When the reference changes (new
% of reg. period with
reference: Flow Rate + 20%, i.e.
TV Entries >
After regulation: 56), initial traffic excesses account
Flow Rate + 20%: 5%
% of reg. period with for 5% of the total reg. time,
TV Entries > Flow while the system output a
Rate: 8% distribution with all the period
below that reference (no excesses
> 20%).
Figure 6: Distributions of TV entries over the reg. period (measure of the time during which TV
entries are over a certain level)
Note: The ideal distribution would be a regulated curve confounded with the “Y axis”.
13
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
Call-signs are then put to the test individually to determine whether the measurement
is statistically meaningful. The “succeeding” call-signs are those for which
significance is ensured with a confidence level of at least 95%, a standard value in
statistics.
Significant call-signs are then correlated to figure out (if possible) a relationship
between the average flight time deviation without regulation and the average flight
time deviation with regulation. The type10 and coefficients of the relationship are
the indicators of the impact of ATFM measures on uncertainty reduction (I6).
A graphical illustration, recapping the main steps of the methodology, is presented
below.
8
The ETO is the expected time of penetration within the TV, when the flight is not regulated. It is obtained
from the latest flight plan sent by the AO to the CFMU.
9
The CTO is the expected time of penetration within the TV, when the flight is regulated. It is deduced from
the latest slot (CTOT) issued to the AO.
10 Premininary results seem to derive a linear relationship (see § 4.2)
14
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
NO REG
Movements are sorted by:
2/ Reshuffle 1/ call-sign
REG 2/ status (regulated or not)
Call-sign 1 Call-sign 2 Call-sign -
∆ vs. occurrences
Call-signs are processed one-by-one
Yes No
5/ Integration
Non-significant call-signs
Mean REG
Significant call-signs
Correlation coefficient
Mean NO REG
15
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
1/ Data Collection:
The studied TV is put under the microscope over a significant amount of time,
typically several AIRAC cycles. Every movement on record is taken into account
within that scope.
2/ Data Reshuffle:
All movements sharing the same call-sign are clustered. Each cluster is partitioned:
non-regulated movements vs. regulated movements (whatever the origin of the most
penalising regulation).
3/ Data Processing:
For each cluster, the two subsets are the baseline samples from which the
distributions of operational deviations are drawn:
9 Deviations from ETO11 for non-regulated movements;
9 Deviations from CTO12 for regulated movements.
4/ Data Analysis:
The two distributions are weighed one against the other to determine whether the
cluster is statistically significant. Is it a meaningful indicator of how the call-sign
behaves routinely, whether or not it is submitted to restrictions? Does it denote a
deep-rooted operational change? A confidence level of 95% is retained to make a
decision. Presumably: the broader the initial scope, the greater the number of
significant call-signs.
5/ Data Integration:
Significant call-signs are “benchmarked” to try and set out a trend. In other words, if
the correlation between them is strong enough, then a relationship can be drawn to
model the impact of ATFM measures on predictability.
11 The ETO is the expected time of penetration within the TV, when the flight is not regulated. It is obtained
from the latest flight plan sent by the AO to the CFMU.
12 The CTO is the expected time of penetration within the TV, when the flight is regulated. It is deduced from
16
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
The case of the French North East ACC is interesting since located in an area with
high concentration of traffic and complex flows crossing. A particular point on this
case is the mix of cruising flows, of low complexity from an Air Traffic Controller
viewpoint, with the climbing or descending flows from or to the airports of Zurich,
Geneva, etc. which are of higher complexity.
Hence, in order to avoid the uncontrolled accumulation of complex aircraft, FMPs
frequently request regulations applied to the complex flows, even when the planned
demand hourly excesses are low. Here, the objective is more to tackle the bunching
threat of the climbing or descending aircraft on the working positions, than to
maintain the hourly load below a given barrier.
Because the resolution of the bunching problem is one of the intended objectives, the
emphasis here is put on the assessment of that performance of the regulation system
as well as on the (mandatory) performance of resolution of the hourly demand /
capacity imbalance. As the illustration is focused on one single regulation case, the
analysis is performed using the indicators I2, I3, I4 and I5.
Analysis of the load evolution for initial (before activation of the reg. process) and
“after regulation” (just after activation of the reg. process) traffic (60 min. counting
step).
Flow Rate
17
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
It could be noticed that the hourly situation is perfectly corrected (I2 and I3 after reg.
– pink curve), albeit not critical initially (I2 and I3 before reg. – blue curve) with an
average excess below 10%.
Analysis of the load evolution for initial (before activation of the reg. process) and
“after regulation” (just after activation of the reg. process) traffic (20 min. counting
step) 20 min counts depict a more preoccupying situation, unveiling greater demand
excesses (I2 and I3 before reg.) than expected with 1 h counts.
Flow Rate
% of reg. period
with TV Entries >
Flow Rate + 10%:
1% (after reg.)
By the way, 20 min excesses and 1 h excess are not in phase. The regulation
process proves to be efficient for the bunching objective; traffic is smoothened below
10%.
18
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
% 60 min. 20 min.
Before reg. After reg. Before reg. After reg.
I2 22 0 78 11
I3 9 0 39 4
I4 9 16 37 17
I5 (0%) 56 0 30 8
I5 (10%) 19 0 27 1
I5 (20%) 4 0 21 0
19
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
20
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
18
16
14
Regulated movement
12
10
0
0 2 4 6 8 10 12 14 16 18
Non-regulated m ovem ent
The data points lie on the same side of the pink solid line, pointing to a reduction of
operational drifts when regulations are applied. Furthermore, they tend to flock to the
green dashed line, profiling a linear relationship with a twofold reduction in deviation
when the flight is submitted to restrictions.
Such preliminary results echo the initial assumptions15; they seem to derive a trend,
to be confirmed by the consolidation of the study, which will address more TVs, over
longer time periods.
Uncertainty has different components. One of them is the average deviation of flight
time estimates over TV entry, which seems to be half-reduced when regulations are
applied. The other component of uncertainty is the dispersion of the values around
the average deviation, which under a statistical approach, is referred as the standard
deviation.
The standard deviation characterises uncertainty in terms of predictability and can be
very small even with a significant average deviation.
Therefore; in order to assess the predictability improvement for regulated flights of
the studied application case, a measure of the standard deviation for the call-signs
has also been performed. Results are illustrated below.
15Namely that uncertainty reduction seems to be half-reduced as the shortening of the ground tolerance
window, which is twofold.
21
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
18
16
14
12 Non-regulated movement
Standard deviation
10
8
Regulated movement
6
0
0 2 4 6 8 10 12 14 16 18
Mean
The same call-signs are spotted twice hereunder, in the non-regulated (diamonds)
and regulated (triangles) modes. The standard deviation is exhibited in addition to the
mean.
A slip towards the origin is noticeable for the whole cluster. The magnitude of the
shift is a rough marker of uncertainty reduction. The attracting power of the origin
points out the overall impact of ATFM measures in that sense. Ideally, a call-sign
lying on the focal point would become more predictable, with no operational
deviation.
In the case study presented, the comparison between the average flight time
deviation with regulation and the average flight time deviation without
regulation derives a trend: a linear relationship of slope 0.5.
The present result needs to be confronted with an analysis on other TVs and longer
periods so as to assess if this is a regular pattern and/or if it depends from the TVs
characteristics such as e.g. the distances to the departure aerodromes.
The correlation coefficient will indicate the strength of the relationship.
22
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
The previously outlined indicators are herein run on data records to measure
smoothing performance from an analytical perspective. However different options are
available to shape the data into a purposeful observation model. They range from the
static batch of “after-the-fact” flight models (ALL-FT-based)16 to the dynamic
simulation of the evolution of the slot allocation plan.
The former is a fast-track (permanent availability of ALL-FT files) but partial way to
measure performance, as the observation model does not strictly reflect the traffic
load situation that FMP / FMD would have observed on the field.
The latter is a time-consuming solution, but it enriches considerably the performance
assessment, and guarantees quality of the results since it provides the observables
at different time events of the regulation life-cycle, when the FMP / FMD decides to
activate, monitor, modify or cancel a regulation.
The following chapters present the results of a first set of simulations launched in
order to “test” the quality of the observations obtained when using the ALL-FT model.
latest ATFM slot (CTOT) issued to the AO, by the ground regulation system.
19 “Actual” integrates the actual entry time of the flights in the regulated TV. It is computed with the Radar
23
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
A first set of simulations was launched in order to find out about the demand status
before the implementation of a regulation, as well as to derive initial conclusions on
the slot allocation process performance.
The simulation results were also confronted to results obtained with the processing of
“ALL-FT” data so as to “test” the quality of the observations obtained when using that
ALL-FT model.
For one day of simulation and one TV under analysis, the flow evolution is depicted
below, for the two observation models:
9 The one output by the simulator, at the following prediction time: just before
the implementation of the regulation, herein, 113 minutes19 prior to the period
start (Pred- curve): this is the demand, as figured out by the user when he or
she decides to take action;
9 The one that is derived from the ALL-FT file (FTFM profile): this is the
demand that is known “after the fact”, i.e. that integrates subsequent flight
plan cancellations or later flight plan submissions.
24
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
It could be observed from the figure that the prediction simulated curve (Pred-) can
present significant deviations from the ALL-FT (FTFM) one. It could be derived the
following two trends:
9 An overestimation of the demand (FTFM < Pred-), for anticipation periods20
of up to 9 hours (540 minutes). For that this trend, our assumption (to be
confirmed by further analyses) is that the impact of flight plans subsequent
cancellations or modifications is predominant in the deviations observed
between the prediction and the “after-the-fact” data;
9 An underestimation of the demand (FTFM > Pred-), for anticipation
periods19 of more than 9 hours. For this trend, our assumption (to be
confirmed by further analyses) is that a significant proportion of the flight
plans has not been submitted yet, leading to the deviations observed between
the prediction and the “after-the-fact” data.
As the time of events is getting closer, the two curves naturally reconcile one
another.
Therefore, it could be shown from this example that the ALL-FT curve (FTFM) may
deviate from the curve that the FMP would have observed on field, when deciding to
create a regulation process.
In some cases, deviations are of significant amplitude, which lead to the following
question: Could we estimate the appropriateness or inappropriateness of
implementing ground regulations, only by the observation of the FTFM profile? In
particular, a decision-making criterion which will recommend that ground regulations
20 A time index is represented on the (X axis). The origin (Index “0”) corresponds to the time where the user is
taking the action of creation. Index “113” corresponds to the start of regulation period. The regulation process
is therefore activated at anticipation “113” minutes, i.e. 1 h 53 min.
25
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
should not be activated for demand excesses lower than 10%, could be challenged if
the observation is only based on the use of the ALL-FT files.
The response to this question leads to push further the investigations about the
deviations between FTFM and prediction curves, and to launch an analysis for other
regulation cases.
As a preliminary result from the analysis, we found in particular that for 33% of the
studied sample of analysis (44 regulation cases); the FTFM curve was deviating
more than 10% in average with respect to the prediction curve (Pred-).
Such result tends to confirm the issues raised by the using of “FTFM” curves only:
frequent deviations of significant amplitude with what the FMP has observed on field,
when deciding the regulation settings.
Paris CDG
(4 TVs)
Aggregated results (average amplitude
and duration of the traffic load
fluctuations)
Madrid
(1 TV)
Amsterd.
(2 TVs)
Lon. Gat.
(1 TV)
Reims
(2 TVs)
26
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
Results analysis
In the analysis of the results, two different scenarios have been considered:
1. Scenario Pred(+): evaluation of the full sample of analysis (6 days of
simulation, 10 TVs), obtained with the data output by the simulator, at T+
prediction time, i.e. just after the activation of the user’s regulation request –
distribution output by the system;
2. Scenario ALL_FT (RTFM): evaluation of the equivalent sample (6 same days,
10 TVs) obtained with the processing of ALL-FT data (RTFM – regulated
profile), so as to measure the deviations in the results of this ALL-FT model
with the results obtained by simulations.
Traffic load fluctuations average amplitude is evaluated in proportion to the flow rate
(e.g. 0.05 means 5% above the flow rate, -0.05 means 5% below the flow rate).
An illustration is presented below. The curve represents the number of records (in
proportion to the total number) comprised in a] x-1%; x%] interval (step of 1%).
]-0.01; 0] interval:
- Scenario Pred (+): 43%
]-0.01; 0] interval:
- Scenario RTFM: 24%
The analysis clearly confirms that ALL-FT RTFM profiles under-estimate the
performance of the system. In the ALL FT scenario, only 66% of the traffic load
fluctuations average amplitude contained within an interval of ]-5%; 5%], whereas the
simulation shows that 87% of the distribution output by the system is contained within
that interval.
27
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
When we strictly focus on the reference, i.e. the flow rate, the difference is equally
important: 43% of the traffic load fluctuations average amplitude is contained within
an interval of ]-1%; 0%] in the case of the Pred(+) scenario, 24% of the traffic load
fluctuations average amplitude is contained within an interval of ]-1%; 0%] in the case
of the ALL FT RTFM scenario.
It could be shown that the use of the RTFM files does not guarantee the quality of the
results, in the analysis of the performance of the slot allocation process. As a matter
of fact, RTFM data are altered by the real-time events occurring after T+ (as it takes
into account the latest slot issued to the AO, which can be improved in function of
subsequent real-time events, such as airborne deviations from the flight plan, etc.).
28
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
As it has been demonstrated by the previous study and by the interviews conducted
with the users of the ATFM ground regulation system, FMPs / FMDs have developed
methods of using the ATFM regulation process to fulfil a number of various
operational objectives that goes beyond the traditional objective of resolving the
hourly demand / capacity imbalances.
ATFM service users will still look for the elimination of the hourly over-deliveries, but
some will also look for increased traffic predictability, and others for the elimination of
the bunching peaks. Assessing whether the system is able or not to meet these
objectives therefore required to specify an appropriate ATFM performance
assessment matrix of indicators.
That matrix of indicators21 has the following axes of measurements:
In the four above categories of indicators, the three first categories are referring to
the smoothing performance of the ATFM ground regulation solution, and the fourth to
the uncertainty reduction performance.
It is also relevant to notice that in the class of the smoothing indicators, the indicators
are associated to different TV entry counting parameters depending on the
smoothing objective that is measured: 60 minutes (hourly over-deliveries resolution),
20 minutes (traffic bunching reduction) or 1 minute (prevention from the forming of
complex traffic clusters within the sectors).
In various cases, we could observe that a given regulation could be very efficient for
one given objective, when providing modest results for another objective. Therefore,
the regulation performance is to be assessed globally by the application of the whole
set of indicators reflecting the various users’ objectives.
21 A full description of the indicators developed (incl. formulae) is available on a referenced report
(Intermediate note on ATFM efficiency indicators).
29
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
It shall be borne in mind that this “extrapolation” will certainly require the use of
simulators (TACOT, available in the EEC/NCD), in order to rebuild the slot allocation
plans “life-cycle” and to make an assessment of the slot allocation plan evolution with
regards to the events inherent to the functioning of the system (actions of creation,
modification, cancellation of a reg. process and also the continuous adjustments of
the plan triggered by the real-time events, e.g. airborne deviations to the flight plans,
etc.).
30
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
22 “Initial” reflects the status of the demand before activation of the regulation plan. It is computed with the
latest flight plan version, sent by each AO to the CFMU/IFPS.
23 “Regulated” reflects the status of the demand after activation of the regulation plan. It is computed with the
latest ATFM slot (CTOT) issued to the AO, by the ground regulation system
24 “Actual” integrates the actual entry time of the flights in the regulated TV. It is computed with the Radar
31
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
• Time events:
The objective is to build the views that FMP / FMD would observe on the field, at
different times events of the regulation life-cycle, when deciding to create, modify or
cancel a regulation.
For each event – activation, modification or cancellation – two time events are
considered:
9 Event T-: latest flight list version available in the system before the creation,
modification or cancellation of the regulation process;
9 Event T+: flight list version output by the system, just after the creation,
modification or cancellation of the regulation process.
The “T- view” reflects the status of the demand, at a time when a decision for
regulating and for setting the regulation parameters is made by the FMP / FMD.
The “T+ view” reflects the status of the demand, output by the system and according
to the regulation settings. At this time, ideally, the traffic load should be aligned with
the flow rate(s) requested.
In the EEC, TACOT is available for fast time ATFM simulations. TACOT integrates
an implementation of the CASA slot allocation algorithm. We are able to generate,
with this tool, slot lists at the different time events here above identified (T-; T+) and
reconstitute the slot allocation plan, that would have been observed in operations by
FMPs / FMDs.
The following data, as required for the analyses, is extracted from the simulations:
9 The “Flight List” status of each TV under study, and at the different times
here above mentioned (T-, T+ for the processes of creation, modifications
and cancellation).
The “Flight List” reports time estimates. For each flight of the TV, it gives the
information related to the latest flight status, i.e. ETO25 if the flight is not
regulated, eventually CTO26 if the flight is regulated and eventually ATO27 if
the flight has already taken off.
9 The “Flight List” status of each TV under study, at the end of simulation
day. Such “Flight List” is equivalent to the “CTFM” view (real flown profile)
extracted from the processing of “ALL-FT” files.
25 The ETO is the expected time of penetration within the TV, when the flight is not regulated. It is obtained
from flight plan sent by the AO to the CFMU.
26 The CTO is the expected time of penetration within the TV, when the flight is regulated. It is deduced from
Position Reports)
32
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
For each regulation case, these data aim to build different traffic pictures before and
after an event of creation, modification or cancellation of the regulation process. The
data obtained at the end of simulation day is used for the building of the real traffic
pictures, as perceived by the controller on his working position.
2. Analytical model
The analytical model is granted on the processing of each “Flight List” (TACOT
outputs), at the different time events here above identified: T-, T+. Each of these
events triggers ATFM information updates (creation, modifications or cancellations).
Different traffic pictures are drawn before and after the time event, and compared to
the regulation settings.
Reg. parameters:
Flow Rate, Period
Actual (ACT)
33
EUROCONTROL Experimental Centre
ATFCM Performance and Declared Capacity Study
Pred - (T-), Pred + (T+) and ACT (Actual) reflects predictions at different statuses and
anticipation levels:
9 Pred + (T+): Flight List status after activation of a regulation process (creation
/ modification / cancellation). It reflects the status of the demand after action
of the system, according to the regulation settings;
9 ACT (ACT): Flight List status at the end of the day. It integrates the actual
entry time of flights in the regulated TV.
The Real-time events are reintegrated into the system and are a source of alteration
of the distribution initially output by the system (Pred + (T+)). Real-time events can be
grouped into the following categories:
9 Flight plans: The uncertainty vis-à-vis the initial demand profile in the
regulated TV.
Part of the uncertainty on the traffic loads also results from the late filing and
sending of flight plans to the CFMU. Also, a significant proportion of the flight
plans is subjected to subsequent modifications and cancellations, leading to
modifications of the initial demand profile.
9 Real-time traffic: The real-time deviations from the flight plan (also evaluated
through the analysis of flight time estimates on TV entry deviations, as
detailed in the “uncertainty reduction” sections of this document).
The category refers to the operational deviations from the flight plan, inherent
to the real-time execution of flights (pilot / ATC instructions).
34