CIA.Models

Question 1

define ‘Model’

Answer 1

a practical representation of relationships among entities or events using FEMS (Financial, Economical, Mathematical, Statistical) concepts

Question 2

what are the elements of a model?

Answer 2

SIR
- Specification
- Implementation
- Runs

Question 3

define ‘model specification’

Answer 3

a description of the parts of a model and their interactions (includes data, assumptions, methods, entities, events)

Question 4

define ‘model implementation’

Answer 4

the systems that perform the calculations (computer programs, spreadsheets…)

Question 5

define ‘model run’

Answer 5

the inputs and outputs of the implementation

Question 6

define ‘model risk’

Answer 6

risk that user will draw inappropriate conclusions due to shortcomings of model use

Question 7

what is the main distinction between a calculation and a model?

Answer 7

a model requires more documentation on how it was chosen and how it is used

Question 8

why is there always risk in using a model?

Answer 8

because a model is a simplification of reality

Question 9

3 strategies to mitigate model risk

Answer 9

• choose a model for a task
• oversee usage
• communicate model results

Question 10

how can model risk be measured?

Answer 10

• severity of model failure
• likelihood of model failure

Question 11

describe 4 considerations in assessing the severity of model failure

Answer 11

severity of rapping (need to ID rapper NF because he swears FU FS)
ID NF FU FS
- Impact of Decision being made using model
- Non-Financial impact (reputational)
- Frequency of Use
- Financial Significant of results model produces

Question 12

describe 4 considerations in assessing the likelihood of model failure

Answer 12

how i met your mother (TED.C)
- Testing: sufficiency of testing
- Expertise: required level of expertise of users
- Documentation: adequacy of documentation
- Complexity: higher complexity means higher likelihood of misuse of model

Question 13

does the actuary have more control over the severity or likelihood of model failure? Justify.

Answer 13

More control over likelihood through
- choose a more reliable model
- test the model more thoroughly
- tighter controls over model run

Question 14

identify the 4 steps an actuary should take before using a new model

Answer 14

SLID
- review Specification
- deal with Limitations
- validate Implementation
- keep Documentation

Question 15

describe what actuary does when reviewing a model’s specifications

Answer 15

verify DAM
- your Data fits the model requirements
- Assumptions are appropriate
- Methods are sound

Question 16

describe what an actuary does when validating a model’s implementation

Answer 16

• compare with other tested models
• maintain a set of test cases
• back testing ( testing with historical data where you already know the answer)
• run an entire live file through successive versions of the model (for models with a higher risk rating)
• peer review of testing procedure

Question 17

describe what an actuary does when dealing with model’s limitations

Answer 17

understand the range of uses of which the model was designed and tested

Question 18

describe 3 items an actuary should include when documenting a model

Answer 18

• how the model was chosen
• how it was tested
• what are its limitations

Question 19

what is an important tool for validating models?

Answer 19

a model’s risk rating
riskier models need more thorough validation

Question 20

how should an actuary evaluate an existing model that’s being used in a new way?

Answer 20

• check that the initial model was properly validated
• review limitations in the new application that may not have been relevant in the initial application

Question 21

how should an actuary evaluate a model approved for use by others

Answer 21

actuary should review and approve the initial validation report

Question 22

how should an actuary evaluate a model outside actuary’s expertise

Answer 22

make a reasonable attempt at understanding the model’s
- specifications
- validation
- risk rating
- complexity
- control framework

Question 23

give an example of a model outside actuary’s expertise

Answer 23

a credit scoring model

Question 24

what is the purpose of sensitivity testing regarding models

Answer 24

validate comfort relationship
- validate model
- develop a sense of comfort within the model
- understand the relationship between inputs and outputs

Question 25

compare sensitivity testing and scenario testing

Answer 25

sensitivity:
- incremental changes to risk factors
- shock is more immediate and shorter time horizon
- simpler
scenario:
- significant changes to risk factors
- observe future state including ripple effects and actions over longer time horizon
- more complex

Question 26

how can model assumptions be tested in the context of sensitivity testing?

Answer 26

• outside expected range
• singly & in combination
• with a non linear relationship between inputs and outputs

Question 27

what types of validation should be done when using a model

Answer 27

DAR
- Data should be RelSuff (reliable and sufficient)
- validate non global Assumptions that vary by model run
- Results should be reasonable relative to input

Question 28

what does it mean for data to be reliable?

Answer 28

• data reconciles to audited sources (ex: balance sheet)
• data is reasonable with respect to prior period data

Question 29

what does it mean for data to be sufficient?

Answer 29

• data fits model specification
• data is available in a consistent format

Question 30

what checks can be done to validate the results of a model?

Answer 30

• inputs and outputs should be consistent
• results should be reasonable in both magnitude and direction

Question 31

compare uni dimensional model risk rating with two dimensional rating

Answer 31

uni:
- rating from 1- 20 (20 is high)
- based on financial significant, complexity, expertise of users and documentation
two:
- assessed separately for severity and likelihood of failure
- final rating is a balance of these