Silvia Giacinti

Principal Economist-Statistician BIRD test

DG-S

for discussion
BIRD methodology – Reference / non-reference
Rubric

What is expected involvement of the software vendors:

• The SV are invited to build a prototype to apply the transformation
rules to a given input in order to reach the output specified by the
BIRD.
• The final scope is to let the market to develop a software solution
based on BIRD and make the BIRD executable.
• The prototype can be based on VTL or use the BIRD
documentation as a starting point.
– Pure VTL engine
– Different software guided by the BIRD docuemntation

Mappings 2 www.ecb.europa.eu ©
BIRD methodology – Reference / non-reference
Rubric

Different test scale :

1) Input real data by a specific bank following the BIRD input layer,
the resulting output using the VTL engine to be compared with the
actual reported data in AnaCredit
Implications
• Input data expected from banks
• Actual reported data as it is reported in AnaCredit secondary
reporting (ECB regulation)
Potential issues
- Confidentiality
- Indirect test of the quality of reporting, this is not in the scope

Mappings 3 www.ecb.europa.eu ©
BIRD methodology – Reference / non-reference
Rubric

Different test scale :

2) Input sample data by a specific bank following the BIRD input

layer, the sample is based on specific functions (e.g carrying amount,
SME calculation) the resulting output using the VTL engine

Implications
• Input data expected from banks
• Actual reported data as it is reported in AnaCredit secondary
reporting (ECB regulation)

Mappings 4 www.ecb.europa.eu ©
BIRD methodology – Reference / non-reference
Rubric

Different test scale :

3) The test is based on the data created by tester on the bases of
specific test case scenarios, the testers will provide also the
expected output:
Implications
• Input data expected from tester
• The expected output layer is provided by tester

Mappings 5 www.ecb.europa.eu ©
BIRD methodology – Reference / non-reference
Rubric

Different test scale :

4) Full set of input data generated randomly and ensuring

consistency across different input cubes
Implications
• Full input data available
• The test is focused on
– the capability of the BIRD to generate the various output cubes, no testing of
specific hypothesis
– Check the treshold for AnaCredit
• Any reporting requirements can be checked

Mappings 6 www.ecb.europa.eu ©
BIRD methodology – Reference / non-reference
Rubric

Possible steps
1) Test the random data and the general capability of the
BIRD to generate an output

2) specific test cases, defining the list the test cases with data
provided by the test team or a SV or a bank. Only the needed
variable are provided in the test cases

3) Combine the random data with specific values of variables

to test specific test cases

3) Test real full dataset, up to the bank to verify the results

depending on the capability to implement the VTL engine

Mappings 7 www.ecb.europa.eu ©
BIRD methodology – Reference / non-reference
Rubric

Constraints:
• The data should follow the BIRD reference codification

• The VTL used to develop the engine should be taken as

provided in the BIRD DB without any manual manipulation in
case of VTL engine

• The test scope is not the software performance but the quality of
the resulting output

• The enrichment layer should be implemented in the software

• Respect the deadline of March 2020

• The software should be able to show the data lineage and the
specific phase that is generating the error
Mappings 8 www.ecb.europa.eu ©
BIRD methodology – Reference / non-reference
Rubric

Plan for test case development:

Different Securities
loans issued/
input to be tested off balance instruments derivatives owned liabilities collateral
AnaCredit X X
Finrep X X X X X X
Asset encumbrance X
Resolution plan X X X X
SHS Group X

100 test cases expected on:

- Loans ( factoring, leasing, deposits, credit cards, revers repo)
- Securities ( equity, investment funds, debt securities, sec issued
- Derivatives ( credit derivatives, options, forwards and swaps)
- Deposits and repos

Mappings 9 www.ecb.europa.eu ©
BIRD methodology – Reference / non-reference
Rubric

Open discussion
Please provide your input

Mappings 10 www.ecb.europa.eu ©