4. Automated Test Generation

Question 1

Explain what the following kinds of testing are well suited for:

• random testing (fuzzing)
• systematic testing (e.g., Korat)
• feedback-directed random testing (e.g., Randoop)

Answer 1

Random testing (fuzzing) is useful for:

• security
• mobile apps
• concurrency

Systematic testing (e.g., Korat) is useful for:
- linked data structures

Feedback-directed random testing (e.g., Randoop) is useful for unit testing:

• classes
• libraries

Question 2

What is a concise test?

Answer 2

A concise test avoids illegal and redundant tests.

Question 3

What is a diverse test?

Answer 3

A diverse test gives good coverage.

Question 4

What technique does Korat use to avoid generating inputs that don’t satisfy the pre-condition?

Answer 4

Instrument the pre-condition.
- add code to observe its actions
record fields accessed by the pre-condition

Observation: If the pre-condition doesn’t access a field of the test input then the pre-condition doesn’t depend on the field.

Question 5

Give an example of a pre-condition for binary trees.

Answer 5

• Root may be null

If root is not null:

• no cycles
• each node (except root) has one parent)
• root has no parent

Question 6

Strengths and weaknesses of Korat?

Answer 6

• Strong when we can enumerate all possibilities (e.g., four nodes, two edges per node)

Good for:

• linked data structures
• small, easily specified procedures
• unit testing

Weaker when enumeration is weak:
- integers, floating-point numbers, strings

Question 7

Explain the recipe Randoop uses for generating tests?

Answer 7

• Build new sequences incrementally, extending past sequences
• As soon as a sequence is created, execute it
• Use execution results to guide test generation towards sequences that create new object states