Comp 611

System Testing and Evaluation

Fall 2019 | MW 1:00 to 2:15 pm in Benbow Hall 109

Home Course Info Course Project Readings

Instructor

Schedule

Navigate to a topic: Foundations | Test Automation | Test-Driven Development | Coverage Fundamentals | Input Space Partitioning | Graph-Based Coverage | Mutation Testing | Research Exploration

Lecture Date Description Course Materials
Lecture 1 8/21 Introduction
Concepts: What is a testing?
Lecture 2 8/26 Foundations
Concepts: Faults, Errors, and Failures
  • Definitions
  • Deriving tests to propagate errors
  • In-Class exercise: [Exploring faults]
Lecture 3 8/28 Foundations
Concepts: Model Driven Test Design
  • How should we design tests?
  • High-level view of testing process
  • In-Class exercise: [Release the Monster]
 [HW1 Out] Due: 9/4
- 9/2 No Class: Labor Day
Lecture 4 9/4 Test Automation
Concepts: Introduction to JUnit 		Quiz 1 In Class
[HW2 Out] Due: 9/11
Lecture 5 9/9 Test Automation
Concepts: JUnit and Beyond 		Project Checkpoint 1
Lecture 6 9/11 Test-Driven Development
Concepts: Putting Tests First
  • Explore test creation strategies
  • Example: Calculator Class
  • When to mock
  • In-Class exercise: [Triangle Class TDD]
 Quiz 2 In Class
[HW3 Out] Due: 9/18
Lecture 7 9/16 Test-Driven Development
Concepts: Code Retreat
  • Hands on practices with TDD
  • Conway's Game of Life
Lecture 8 9/18 Coverage Fundamentals
Concepts: What is Coverage?
  • Coverage Criteria and Test Requirements
  • Infeasible Coverage Criteria
  • Coverage Criteria Subsumption
  • In-Class exercise: [Coverage for Writing Utensils]
 Quiz 3 In Class
[HW4 Out] Due: 9/25
Lecture 9 9/23 Input Space Partitioning
Concepts: Basics of ISP
  • Partition Characteristics
  • Interface-Based vs. Functionality-Based
  • In-Class exercise: [ISP for a Stack]
Lecture 10 9/25 Input Space Partitioning
Concepts: Choosing Values 		Quiz 4 In Class
[HW5 Out] Due: 10/2
Lecture 11 9/30 Graph Coverage
Concepts: Graph Syntax Coverage
- 10/2 Concepts: Project Work Day
Lecture 12 10/7 Graph Coverage
Concepts: Graph Syntax Coverage 		Quiz 5 In Class
Lecture 13 10/9 Graph Coverage
Concepts: Graph Syntax Coverage
  • Def, Use, and DU Pairs
  • DU-Paths
  • All Def/Use/DU-Paths Coverage
  • In-Class exercise: [Apply DU Coverage]
 [HW6 Out] Due: 10/16
- 10/14 Fall Break
Lecture 14 10/16 Graph Coverage
Concepts: Graphs from Source Code 		Quiz 6 In Class
[HW7 Out] Due: 10/23
- 10/21 Concepts: Project work day.
Lecture 15 10/23 Mutation Testing
Concepts: Grammar-Based Mutation Testing
  • Review: Regular Expression
  • Backus Naur Form (BNF) Grammars
  • Grammar Coverage Criteria
  • In-Class exercise: [BNF Exercise]
 Quiz 7 In Class
[HW8 Out] Due: 10/30
Lecture 16 10/28 Mutation Testing
Concepts: Program-Based Mutation Testing
  • Categories of Mutants
  • Mutation Coverage
  • Designing Mutation Operators
  • In-Class exercise: [Mutating Code]
Lecture 17 10/30 Mutation Testing
Concepts: Code Retreat - Attack and Defend 		Quiz 8 In Class
- 11/04 Concepts: Project work day.
Lecture 18 11/06 Switching Gears to Research
Concepts: Current Software Testing Research
  • How to Read Academic Papers
  • Hot Topics in Software Testing Research
  • Common Challenges in Software Testing Research
Lecture 19 11/11 Research Presentations and Discussions
Concepts: A Critical look at software testing research 		Project Checkpoint 2
Lecture 20 11/13 Research Presentations and Discussions
Concepts: Debugging
  • Presentation 0: [Reading 2]
  • Presentation 1: [Reading 3]
  • Intro to Test Generation Research
Lecture 21 11/18 Research Presentations and Discussions
Concepts: Test Generation
  • Presentation 0: [Reading 4]
  • Presentation 1: [Reading 5]
  • Intro to Test Selection Research
Lecture 22 11/20 Research Presentations and Discussions
Concepts: Test Selection
  • Presentation 0: [Reading 6]
  • Presentation 1: [Reading 7]
  • Intro to Fault Localization Research
Lecture 23 11/25 Project Check-In
Concepts: Project Check-In
  • One-on-one meetings with groups
  • Discuss checkpoint 2 feedback
  • Convenience me you are on track!
- 11/27 Thanksgiving Break!
Lecture 24 12/2 Research Presentations and Discussions
Concepts: Fault localization
  • Presentation 0: [Reading 8]
  • Presentation 1: [Reading 9]
  • Intro to Testing Machine Learning Research
Lecture 25 12/4 Research Presentations and Discussions
Concepts: Testing Machine Learning Systems
- 12/9 Final Project Due.