Week 2: Empirical Evaluation of Models for EA Flashcards
(44 cards)
Nature of comparison
- Single grammar comparison:
+ Inter-grammar comparison
+ Intra-grammar comparison - Multi grammar comparison.
Inter-grammar comparison
- BPMN vs Activity Diagram
- Entity Relationship vs Class Diagram
Intra-grammar comparison
Example: Within an ER Diagram.
- Cardinality
- Attributes
- Relationships
- Inheritance
Multi-grammar comparison
UML vs DODAF
Empirical Evaluation of models
Setting:
- Assume that we want to compare model A and model B.
- Choose at least one case, that are informationally equivalent.
- Randomly assign study participants to the two groups.
- Group 1 is given model A and Group 2 is given model B.
- Make everything similar or they would say the result was because of the difference.
- Evaluate all subjects using similar test.
- If experts are grading the outcome, use at least two experts and have Inter-Rater-Reliability (IRR) >0.8
- Compare the average of two groups (e.g. ANOVA).
+ Check if there is a significant difference between the averages.
Breakdown
Errors that are identified from the verbal protocols of sessions.
Error
A breakdown that lasts and not corrected by the end of the process.
Confidence
In correctness is an indicator of comfort level generated throughout the process.
Diagrama comprehension
A set of questions about elements (constructs) in the model. -> number of items remembered, recall (redraw the model) fill-in-the-blank questions.
Problem solving
Number of suggested solutions, then, check which diagram is better; BPMN or an activity diagram.
Ontology
Addresses what exists and happens in the world and how they can be grouped.
Zachman Framework
Approach: Linguistic
Complete representation occurs if .. : Models answers all questions (What, Where, Who, …) from different perspectives.
Not empirically supported.
Bunge, Wand, and Weber (BWW)
Approach: Ontological
Complete representation occurs if .. : Models have a corresponding construct for each construct in the ontology.
Empirically supported.
Ontological deficiencies:
- Ontological incompleteness (construct deficit)
- Ontological completeness but vagueness construct overload.
- Ontological completeness but vagueness construct redundancy.
- Ontological completeness but vagueness construct excess.
Ontological incompleteness (construct deficit)
An ontological construct exists that has no mapping from any grammatical construct (a 1:0 mapping).
Ontological completeness but vagueness construct overload.
A single grammatical construct maps to two or more ontological constructs (a m:1 mapping).
Ontological completeness but vagueness construct redundancy.
Two or more grammatical constructs map to a single ontological construct (a 1:m mapping).
Ontological completeness but vagueness construct excess.
A grammatical modelling construct does not map onto any ontological construct (a 0:1 mapping).
Empirical support for ontological expressiveness.
- Ontological expressiveness. \+ Ontological completeness on perceived useful of modelling -> construct deficit. - Ontological clarity on perceived ease of use of modelling: \+ Construct redundancy \+ Construct overload \+ Construct excess - Path significance \+ >0.1 \+ Variability explained
THING (BWW)
A thing is the elementary unit in the BWW model. The real world is made up of things. Two or more things (composite or simple) can be associated into a composite thing.
PROPERTY (BWW)
Things possess properties. A property is modeled via a function that maps the thing into some value.
CLASS (BWW)
A class is a set of things that can be defined via their possessing a single property.
KIND (BWW)
A kind is a set of things that can be defined only via their possessing two or more common properties.
STATE (BWW)
The vector of values for all property functions of a thing is the state of a thing.
