PARAMETERIZED UNIT TEST (PUT)

Parameterized unit tests (PUTs) is a new methodology where parameters can be passed to the open unit tests by which the behavior of the test can be tested under certain scenario as well as by instantiating the parameterized test methods with given argument sets. Instantiations should be chosen so that they exercise different code paths of the methods-under-test.

UNIT TESTING AT A GLANCE[i]:

Three sections of Unit Tests:

• Data
• Method Sequence
• Assertions

void Add()

{

int item = 3;

var list = new List();

list.Add(item);

var Count = list.Count();

Assert.AreEqual(1, Count);

}

INCOMPLETE CODE COVERAGE WITH TEST DATA:

list.Add(3);

• Bad data may result in redundant, or worse, incomplete test suites.
• Fixed data values become invalid when product changes
• Why choose a value which may not make sense?

WHAT IS PARAMETERIZED UNIT TEST?

Parameterized Unit Test is a Unit Test with Parameters.

Separation of concerns:

• Data is generated by a tool
• Developer can focus on functional specification

Traditional Test method:

[TestMethod]

void TestAdd()

{

ArrayList a = new ArrayList(0);

object o = new object();

a.Add(o);

Assert.IsTrue(a[0] == o);

}

Parameterized Test Methods:

[TestAxiom]

void TestAdd(ArrayList a, object o)

{

Assume.IsTrue(a!=null);

int i = a.Count;

a.Add(o);

Assert.IsTrue(a[i] == o);

}

[TestMethod]

void TestAddNoOverflow()

{

TestAdd(new ArrayList(1), new object());

}

[TestMethod]

void TestAddWithOverflow() {

TestAdd(new ArrayList(0), new object());

}

Which Splits axioms and test cases and thus TestAdd(. . .) succeeds for all array lists and all objects

PARAMETERIZED UNIT TESTING (PUT):

• Some popular unit testing tools, like JUnit, NUnit doesn’t support the automation of creating unit tests
• In some cases it even takes more lines of code to create the unit tests than more implementation being tested
• Some Automatic test generation tools may not even track the divide-by-zero type errors rather than specific error message
• PUT or Parameterized Unit Test accepts behavioral parameters/argument sets for a specific test method

PUT CONTRIBUTIONS:

• They allow unit tests to play a greater role as specifications of program behavior. In fact, PUTs are axiomatic specifications.
• They enable automatic case analysis, which avoids writing implementation-specific unit tests.
• Their generated test cases often result in complete path coverage of the implementation, which amounts to a formal proof of the PUT’s assertions.

PUT FRAMEWORK:

• Symbolic State
• Constraints
• Symbolic Evaluation
• Axioms
• Test Case generation

SYMBOLIC STATE:

It is basically the state of program execution. It can contain expressions with symbolic variables.

Symbolic expressions:

E =

o object ids, infinite set of potential object identifiers

v variables, set of symbolic variable identifiers

t types, set of type identifiers

f(E bar) function application, set of function symbols

For All (v bar).E universal quantification,

where x to denote lists of items x1, . . . , xn.

Function symbols: equals(x, y) denotes whether x and y represent the same value for value Types, type(x) denotes the runtime type of object x, and len(x) the length of array x

Heaps: No of Times a method being called in a nested method call

Symbolic state: A symbolic state is a 5-tuple S = (O, A, He,Hi, X), where the current set of objects O is a subset of ObjectId, the program stack A is a stack of activation records, He and Hi are expressions denoting the extensional heap and the intentional heap respectively, and finally, X, an object expression, denotes the current exception. Say: O(S) is being written for projection on S.

SYMBOLIC EVALUATION:

Symbolic Evaluation describes the effect of the current instruction from a given constrained state (S,C).

AXIOMS:

Axiom is basically of summary of external behavior

[TestAxiom]

public void TestQuickSort(int[] a)

{

Assume.IsTrue(a != null);

QuickSort.Sort(a, 0, a.Length - 1);

for (int i = 0; i < a.Length - 1; i++)

Assert.IsTrue(a[i] <= a[i + 1]);

}

TEST CASE GENERATION:

Each transition sequence (S0,C0) ! (S1,C1) ! · · · represents a unique execution path of the program.

[TestMethod]

public void TestQuickSort597()

{

int[] ints = new int[2];

ints[1] = 1;

this.TestQuickSort(ints);

}

[TestMethod]

public void TestQuickSort512()

{

int[] ints = new int[3];

ints[0] = -2147475453;

ints[1] = 1073750016;

ints[2] = 8194;

this.TestQuickSort(ints);

}

CONSTRAINTS:

A constraint on a symbolic state is a pair C = (BG, PC), where BG is the static background, which only depends on the program declarations, and PC is the dynamic path condition, which is built up during symbolic evaluation.

---

 

[i] Parameterized Unit Testing: Principles, Techniques, and Applications in Practice, Nikolai Tillmann, Peli de Halleux, Wolfram Schulte (Microsoft Research) Tao Xie (North Carolina State University)http://research.microsoft.com/en-us/projects/pex/pexpublictutorialslides.pptx