Chapter 7: Testing and Debugging
Introduction
Learning Outcomes
Distinguish unit, module, sub-system, system and acceptance testing, and say what each one checks.
Write automated tests with @Test, @BeforeEach and assertions such as assertEquals and assertThrows; read a failure message.
Choose boundary values, write parameterized tests, keep tests deterministic and repeatable, and explain why regression tests matter.
Set breakpoints, step through a program (Step Into / Over / Out) and watch variables to find the cause of a bug.
Know which question a profiler, coverage tool, static analyzer (SpotBugs), property-based testing, Selenium, Memory Analyzer and VisualVM each answer.
Decide whether a problem calls for a test, a debugger or a profiler.
Introduction
Testing and Debugging: Why?
“Testing is not about finding bugs; it is about making sure the system works as expected.”
Michael Bolton (software testing expert)
“The best way to predict the future is to create it.”
Often attributed to Abraham Lincoln; also to Peter Drucker and Alan Kay
“A bug is just an undocumented feature.”
Anonymous programmer joke
“Quality is not an act, it is a habit.”
Attributed to Aristotle; actually Will Durant's paraphrase of him
“Not testing is like playing Russian roulette with your software.”
Anonymous
“Testing shows the presence, not the absence of bugs.”
Edsger W. Dijkstra
Introduction
Error, Defect, Failure: Testing vs. Debugging
| Testing | Debugging | |
|---|---|---|
| Goal | Check whether the program works as expected | Find the cause of a known failure and fix it |
| Question | “Does it work?” | “Why doesn't it work?” |
| Result | Pass / fail (✔ / ✘) | The faulty line and a fix |
| Tools | JUnit, Selenium, code coverage | Debugger, stack trace, println |
| When | Continuously, after every change | When a test fails or a user reports a bug |
Test Levels
Levels of Testing in Software Development
Unit Testing
- Focuses on individual, independent components (a class, a method).
- Makes sure each component works correctly on its own.
- Example: does
BankAccount.withdraw()throw an exception when funds are insufficient?
Unit tests: the smallest parts, the most tests, the fastest to run.
Module Testing
- Tests a module made of related components.
- Checks the integration and behavior of the components inside it.
- Example: do the
Account,TransactionandReceiptclasses transfer money correctly together?
Module tests: related components together.
Sub-System Testing
- Tests for interface mismatches between sub-systems.
- Checks communication and data flow between components.
- Example: do the account and database sub-systems use the same date format?
Sub-system tests: the interfaces where sub-systems meet.
System Testing
- Tests the interactions between sub-systems as a whole.
- Checks that functional (what it does) and non-functional (speed, security, usability) requirements are met.
- Example: with 1000 concurrent users, does a transfer finish in under 2 seconds?
System tests: the whole system, functional and non-functional requirements.
Acceptance Testing
- Tests the system with real data instead of simulated data.
- Confirms its real-world usability and performance.
- Done together with users: alpha testing at the developer's site, beta testing in the user's own environment.
Acceptance tests: can the customer say “I accept this system”?
Test Levels
Test Levels at a Glance
| Level | What is tested? | Who tests? | Data |
|---|---|---|---|
| Unit | One class / method | Developer | Small, hand-picked |
| Module | A few classes working together | Developer | Scenario data |
| Sub-system | Interfaces between sub-systems | Developer, test team | Scenario data |
| System | Whole system; speed, security | Test team | Realistic |
| Acceptance (alpha / beta) | Customer requirements | With the users | Real data |
Tests only input → expected output, without looking inside the code. Acceptance and system tests are mostly like this.
Uses knowledge of the code to exercise every branch and loop. Unit tests and code coverage take this view.
A bug caught at a low level is cheap and quick to find; the same bug found in acceptance testing or by a user costs far more to fix.
Unit Testing and JUnit
First: Testing by Hand
Class under test
public class Calculator { public int add(int a, int b) { return a + b; } public int multiply(int a, int b) { return a * b; } }
Manual test program
public class ManualTest { public static void main(String[] args) { Calculator calc = new Calculator(); System.out.println("2 + 3 = " + calc.add(2, 3)); System.out.println("2 * -3 = " + calc.multiply(2, -3)); // Correct? We look at the screen and decide. } }
2 + 3 = 5 2 * -3 = -6
Problems with this approach
- A human eye judges whether the result is right; it gets tired and makes mistakes.
- When the code changes, everything must be retried by hand.
- The test code sits in
mainand is later deleted: the effort is lost. - Who will read 100 output lines for 100 methods?
Unit Testing and JUnit
JUnit
- JUnit is the most widely used unit testing framework for Java. Tests are ordinary Java methods; JUnit finds them, runs them and reports the results.
- In Java a unit usually means a class; its tests check the class's methods one by one.
- JUnit was designed for unit testing but is quite flexible; it is also used for:
- Module testing: scenarios that involve several classes.
- System testing: system-level interactions.
- IntelliJ IDEA, Eclipse, NetBeans, VS Code and Maven / Gradle run JUnit tests directly.
| Version | How is a test written? |
|---|---|
| JUnit 3 | Extend TestCase; method names start with test |
| JUnit 4 | @Test annotation, org.junit |
| JUnit 5 | @Test, org.junit.jupiter.api; lambdas, parameterized tests |
The examples use JUnit 5 (Jupiter). Adding it with Gradle:
testImplementation("org.junit.jupiter:junit-jupiter:5.11.4")Test classes go in src/test/java; the classes under test go in src/main/java.
Unit Testing and JUnit
A First JUnit 5 Test
import org.junit.jupiter.api.Test; import static org.junit.jupiter.api.Assertions.assertEquals; class CalculatorTest { @Test void addsTwoNumbers() { Calculator calc = new Calculator(); assertEquals(5, calc.add(2, 3)); } @Test void multipliesByNegative() { Calculator calc = new Calculator(); assertEquals(-6, calc.multiply(2, -3)); } }
- ✔
addsTwoNumbers() - ✔
multipliesByNegative()
@TestTells JUnit the method is a test. It is void with no parameters; no main is needed, and it need not be public.
assertEquals(expected, actual)Expected value first, then the actual result. If they differ, the test fails (✘). import static saves writing the class name.
A name like addsTwoNumbers tells you what broke when the test fails.
Unit Testing and JUnit
Assertion Methods
| Method | Passes when | Example (all pass) |
|---|---|---|
assertEquals(e, a) | e and a are equal (equals) | assertEquals(5, 2 + 3) |
assertEquals(e, a, delta) | Floating-point difference ≤ delta | assertEquals(0.3, 0.1 + 0.2, 1e-9) |
assertNotEquals(u, a) | Not equal | assertNotEquals(4, 2 + 3) |
assertTrue / assertFalse | Condition is true / false | assertFalse("abc".isEmpty()) |
assertNull / assertNotNull | Value is / is not null | assertNotNull("x") |
assertSame(a, b) | Same object (==) | assertSame(s, s) |
assertArrayEquals | Arrays equal element by element | assertArrayEquals(new int[] {1, 2}, new int[] {1, 2}) |
assertThrows(T.class, code) | The code throws an exception of that type | assertThrows(ArithmeticException.class, () -> …) |
assertAll(…) | All assertions hold (all are tried) | assertAll(() -> …, () -> …) |
fail("message") | Never; fails the test on purpose | fail("not written yet") |
Without a delta, assertEquals(0.3, 0.1 + 0.2) fails: expected: <0.3> but was: <0.30000000000000004>. The last parameter is an optional message: assertEquals(4, 2 + 3, "wrong sum") → wrong sum ==> expected: <4> but was: <5>
Unit Testing and JUnit
The Class Under Test: BankAccount
public class BankAccount { private double balance; public BankAccount(double balance) { this.balance = balance; } public void deposit(double amount) { if (amount <= 0) { throw new IllegalArgumentException("Invalid amount"); } balance += amount; } public void withdraw(double amount) { if (amount > balance) { throw new IllegalStateException("Insufficient funds"); } balance -= amount; } public double getBalance() { return balance; } }
Which behaviors should we test?
| Call (balance 100) | Expected |
|---|---|
deposit(50) | balance 150 |
withdraw(30) | balance 70 |
withdraw(500) | IllegalStateException; balance stays 100 |
deposit(-10) | IllegalArgumentException |
deposit(0) (boundary) | IllegalArgumentException |
withdraw(100) (boundary) | balance 0, no exception |
Each row becomes a separate test method. Cover normal cases, invalid input and boundaries.
Unit Testing and JUnit
Anatomy of a Test: Arrange, Act, Assert
class AaaTest { @Test void withdrawReducesBalance() { // Arrange BankAccount acct = new BankAccount(100); // Act acct.withdraw(30); // Assert assertEquals(70.0, acct.getBalance(), 0.001); } }
- ✔
withdrawReducesBalance()
Set up the object and the data.
Call the method under test once.
Compare the result with the expected one.
Each test checks one behavior. The balance is a double, so we pass delta = 0.001.
Unit Testing and JUnit
Shared Setup: @BeforeEach
BankAccountTest
class BankAccountTest { private BankAccount acct; @BeforeEach void setUp() { acct = new BankAccount(100); } @Test void depositIncreasesBalance() { acct.deposit(50); assertEquals(150.0, acct.getBalance(), 0.001); } @Test void overdraftThrows() { assertThrows(IllegalStateException.class, () -> acct.withdraw(500)); assertEquals(100.0, acct.getBalance(), 0.001); } }
- The
@BeforeEachmethod runs before every test: each test starts with a fresh account (balance 100). - JUnit creates a new instance of the test class for every test method; one test's changes to
acctdo not affect another. assertThrowsruns the code in its second argument and expects an exception of that type.() -> acct.withdraw(500)is a lambda expression: “a piece of code to run”.- We also check that the balance is unchanged after the exception.
- ✔
overdraftThrows() - ✔
depositIncreasesBalance()
Unit Testing and JUnit · Step by step
How Does JUnit Run Tests?
JUnit finds the @Test methods and creates a new test object for the first test. The field acct is still null.
@BeforeEach runs before the test: a new account, balance 100.
JUnit picks the order: overdraftThrows runs first. assertThrows calls the lambda: withdraw(500) throws.
The exception has the expected type: assertThrows passes. The balance is still 100, so the second assertion passes too.
A brand-new test object (#2) for the second test. The old object and its account are no longer used.
@BeforeEach runs again: this test also starts at balance 100.
Act: deposit(50); the balance becomes 150.
Assert: 150.0 = 150.0 (difference ≤ 0.001). Both tests passed, and they ran independently.
class BankAccountTest { private BankAccount acct; @BeforeEach void setUp() { acct = new BankAccount(100); } @Test void depositIncreasesBalance() { acct.deposit(50); assertEquals(150.0, acct.getBalance(), 0.001); } @Test void overdraftThrows() { assertThrows(IllegalStateException.class, () -> acct.withdraw(500)); assertEquals(100.0, acct.getBalance(), 0.001); } }
Unit Testing and JUnit
Testing Exceptions: assertThrows
class BankExceptionTest { @Test void overdraftMessage() { BankAccount acct = new BankAccount(100); IllegalStateException e = assertThrows( IllegalStateException.class, () -> acct.withdraw(500)); assertEquals("Insufficient funds", e.getMessage()); } @Test void negativeDepositRejected() { BankAccount acct = new BankAccount(100); assertThrows(IllegalArgumentException.class, () -> acct.deposit(-10)); } @Test void wrongExpectation() { BankAccount acct = new BankAccount(100); assertThrows(IllegalStateException.class, () -> acct.withdraw(50)); } }
assertThrowsreturns the thrown exception, so its message can be checked too.- Testing that bad input is rejected matters as much as testing good input.
- The last test is wrong on purpose: withdrawing 50 TL is valid, so nothing is thrown.
- ✔
overdraftMessage() - ✔
negativeDepositRejected() - ✘
wrongExpectation()Expected java.lang.IllegalStateException to be thrown, but nothing was thrown.
In JUnit 4: @Test(expected = …class).
Unit Testing and JUnit
Test Lifecycle Annotations
class LifecycleTest { @BeforeAll static void start() { System.out.println("BeforeAll"); } @BeforeEach void setUp() { System.out.println(" BeforeEach"); } @Test void testA() { System.out.println(" testA"); } @Test void testB() { System.out.println(" testB"); } @AfterEach void cleanUp() { System.out.println(" AfterEach"); } @AfterAll static void finish() { System.out.println("AfterAll"); } }
BeforeAll
BeforeEach
testA
AfterEach
BeforeEach
testB
AfterEach
AfterAll| Annotation | When does it run? |
|---|---|
@BeforeAll | Once before all tests (static) |
@BeforeEach | Before each test |
@AfterEach | After each test (even if it fails) |
@AfterAll | Once after all tests (static) |
The order of the tests is deterministic but need not match the source order: never write tests that depend on each other.
Unit Testing and JUnit
A Failing Test: From Red to Green
ArrayUtils (first version)
public class ArrayUtils { public static int max(int[] arr) { int best = 0; for (int i = 0; i < arr.length; i++) { if (arr[i] > best) { best = arr[i]; } } return best; } }
ArrayUtilsTest
class ArrayUtilsTest { @Test void positiveArray() { int[] arr = {3, 9, 4}; assertEquals(9, ArrayUtils.max(arr)); } @Test void allNegativeArray() { int[] arr = {-5, -2, -8}; assertEquals(-2, ArrayUtils.max(arr)); } }
The code compiles and looks right at first glance. Let's write two tests: one positive array, one all-negative array.
ArrayUtils (first version)
public class ArrayUtils { public static int max(int[] arr) { int best = 0; for (int i = 0; i < arr.length; i++) { if (arr[i] > best) { best = arr[i]; } } return best; } }
- ✔
positiveArray() - ✘
allNegativeArray()org.opentest4j.AssertionFailedError: expected: <-2> but was: <0>
expected = what we wanted (-2), but was = what came back (0). 0 is not even in the array!
Red: the initial value best = 0 is never exceeded when all elements are negative.
ArrayUtils (fixed)
public class ArrayUtils { public static int max(int[] arr) { int best = arr[0]; // first element for (int i = 1; i < arr.length; i++) { if (arr[i] > best) { best = arr[i]; } } return best; } }
- ✔
positiveArray() - ✔
allNegativeArray()
The maximum now starts from the first element. The test stays for good: if the bug returns, it is caught at once.
Green: after the fix the same tests were run again, and both passed.
Unit Testing and JUnit
Which Values to Test? Boundary Values
public class Grade { public static String letterGrade(int score) { if (score < 0 || score > 100) { throw new IllegalArgumentException("Invalid score: " + score); } if (score >= 90) return "AA"; if (score >= 80) return "BA"; if (score >= 70) return "BB"; if (score >= 60) return "CB"; if (score >= 50) return "CC"; return "FF"; } }
- Bugs cluster at boundaries:
>instead of>=,< 100instead of<= 100… - Boundary value analysis: pick both ends of each range (49 / 50, 89 / 90) and the first invalid values (−1, 101).
- Equivalence classes: 61, 65 and 68 lie in the same range and give the same result; one of them is enough.
Unit Testing and JUnit
Parameterized Tests JUnit 5
class GradeTest { @ParameterizedTest @CsvSource({"0, FF", "49, FF", "50, CC", "59, CC", "60, CB", "89, BA", "90, AA", "100, AA"}) void boundaries(int score, String expected) { assertEquals(expected, Grade.letterGrade(score)); } @Test void outOfRangeRejected() { assertThrows(IllegalArgumentException.class, () -> Grade.letterGrade(-1)); assertThrows(IllegalArgumentException.class, () -> Grade.letterGrade(101)); } }
@ParameterizedTest: the same test method runs again and again with different data.@CsvSource: each string is one row; its comma-separated values are converted to the parameters (score,expected) in order.- Dependency:
junit-jupiter-params(included injunit-jupiter).
- ✔
[1] 0, FF - ✔
[2] 49, FF - ✔
[3] 50, CC - ✔
[4] 59, CC - ✔
[5] 60, CB - ✔
[6] 89, BA - ✔
[7] 90, AA - ✔
[8] 100, AA - ✔
outOfRangeRejected()
Unit Testing and JUnit · Step-by-step trace
A Parameterized Test, Step by Step
class GradeTest { @ParameterizedTest @CsvSource({"0, FF", "49, FF", "50, CC", "59, CC", "60, CB", "89, BA", "90, AA", "100, AA"}) void boundaries(int score, String expected) { assertEquals(expected, Grade.letterGrade(score)); } }
JUnit reads the 8 strings in @CsvSource; each becomes a separate test call.
Row 1: "0, FF" is split at the comma; "0" is converted to an int for score, and "FF" goes to expected.
49: the upper end of FF. Had we written > 48 instead of >= 50, it would fail here.
50: the lower end of CC; with 49 it tests the boundary from both sides.
59: the upper end of CC.
60: the lower end of CB.
89: the upper end of BA.
90: the lower end of AA.
100: the last valid value; if the score > 100 check is right, it returns AA.
Unit Testing and JUnit
What Makes a Good Unit Test: FIRST
Finishes in milliseconds, so thousands can run after every change. No files, network or database.
Does not depend on another test, on test order, or on state left by an earlier test.
Gives the same result on every machine, every time: randomness and the clock are under control.
The result is ✔ or ✘; no human needs to read the output.
Written with the code (or even before it), not months later.
A descriptive name, the AAA pattern, one behavior per test. A test documents how the code is meant to be used.
Unit Testing and JUnit
Unit Test or Integration Test?
| Unit test | Integration test | |
|---|---|---|
| Scope | One class / method, isolated | Several components working together |
| Speed | Milliseconds | Seconds (database, files, network) |
| External dependency | Replaced by a simple fake (a test double) | Real database, file, service |
| Bugs it finds | Local logic errors; location obvious | Mismatched interfaces, bad configuration |
Test names that describe behavior
@Test void test1() { … } @Test void discountForSixItems() { … } @Test void noDiscountForFive() { … }
- Test observable behavior, not implementation details.
- Unit tests avoid
Thread.sleep, real networks and files: they make tests slow and flaky. - Tests share no mutable state (e.g. a
staticlist).
Regression and Automation
Regression Testing
- Checks that everything that worked before still works after a change. A regression is a step backward: something that worked breaks again.
- Method: rerun all existing tests after every change.
Under the same conditions a test must give the same result every time. A test that sometimes passes and sometimes fails (“flaky”) destroys trust.
Design tests so they can be repeated at different times, on different machines and in different states.
Regression and Automation
Testing Random Code Deterministically
import java.util.Random; public class Die { private final Random random; public Die(Random random) { // injected this.random = random; } public int roll() { return random.nextInt(6) + 1; } }
Five rolls with the same seed (42)
3 4 1 3 1
class DieTest { @Test void sameSeedSameRolls() { Die d1 = new Die(new Random(42)); Die d2 = new Die(new Random(42)); for (int i = 0; i < 100; i++) { assertEquals(d1.roll(), d2.roll()); } } }
The Random object is passed in (dependency injection). With a fixed seed the test gets the same numbers on every run; had the class called new Random() itself, the results would change each time.
Regression and Automation
Automated Tests
- Write automated tests instead of testing code by hand.
- The effort is about the same, but you end up with an automated test.
- That test can be rerun whenever you like (every build, every change).
- It is wrong to think that software that worked once needs no more testing: code changes, libraries change, environments change.
- Continuous integration (CI, e.g. GitHub Actions, Jenkins) runs all tests on a server after every
git push; a red test blocks the merge.
| Manual | Automated | |
|---|---|---|
| Initial cost | Low | Somewhat higher |
| Cost to repeat | Same every time | ≈ 0 |
| Speed | Minutes | Seconds |
| Reliability | Human error | Same every time |
| Regression | Rarely done | On every change |
| As documentation | None | Usage examples |
Regression and Automation
Test-Driven Development (TDD)
- Test-Driven Development: write the test before the code.
- Red: write a test for behavior that does not exist yet; it fails (or does not compile).
- Green: write the simplest code that makes it pass.
- Refactor: clean up the code; as long as the tests stay green, the behavior is preserved.
- Example: first write
assertEquals("CC", Grade.letterGrade(50)), then theletterGrademethod. Each new grade range is one more cycle.
Debugging
The Debugger
A debugger lets you inspect the state of a running program. With a debugger you can:
- Run the program one statement (one line) at a time
- See the source code with the current line marked
- View variable values and change them if needed
- Set breakpoints in the program
- Run the program up to a given breakpoint
- Inspect a crashed or stopped program through a dump file (a core file in C; in Java a
.hprofheap dump and a thread dump)
IntelliJ IDEA, Eclipse, NetBeans, VS Code: click next to a line number to set a breakpoint, then start with Debug.
Quick, but it clutters the code, needs recompiling and is easy to forget to remove. A debugger leaves the code alone.
Debugging
Breakpoints and Stepping Commands
| Command | What it does | IntelliJ | Eclipse |
|---|---|---|---|
| Breakpoint | When execution reaches this line, the program stops before running it | Ctrl+F8 | Ctrl+Shift+B |
| Step Over | Runs the line; any method call completes without stepping into it | F8 | F6 |
| Step Into | Steps into the method called on the line | F7 | F5 |
| Step Out | Finishes the current method and returns to the caller | Shift+F8 | F7 |
| Resume | Runs until the next breakpoint | F9 | F8 |
| Evaluate / Watch | Evaluates an expression (e.g. arr[i] > best) now, or keeps watching it | Alt+F8 | Ctrl+Shift+I |
Stops only when a condition holds, e.g. i == 999: catches the exact moment in a loop of thousands of iterations.
Stops the moment a given exception (e.g. NullPointerException) is thrown.
Shortcuts are for the default keymap; the same commands are in the Run / Debug menu.
Debugging · Step by step
Finding the Defect with a Debugger
A breakpoint is set on line 5 (if). The program was started with Debug.
Step Into: we step into the call to max.
A new frame: arr refers to the same array. best is not assigned yet.
At the breakpoint: best = 0, arr[0] = -5. The watched expression is false.
Resume: i = 1, -2 > 0 is false again; best is unchanged.
i = 2, -8 > 0 is false. Suspicious: best never changes!
The loop ends and 0 is returned. Defect found: the initial value 0 is wrong when all elements are negative. It should be best = arr[0].
The failure on screen: Max: 0. After the fix, this case is added as a unit test.
public class FindMax { public static int max(int[] arr) { int best = 0; for (int i = 0; i < arr.length; i++) { if (arr[i] > best) { best = arr[i]; } } return best; } public static void main(String[] args) { int[] temps = {-5, -2, -8}; int result = max(temps); System.out.println("Max: " + result); } }
Debugging · Step by step
Step Into, Step Over, Step Out
Stopped at a breakpoint Debug
The breakpoint is on line 13: the line has not run yet.
Step Into F7
Step Into: we are inside sumSquares; a new frame is pushed on the stack.
Step Over F8
Step Over: square(3) ran, but we did not step into it; sq1 = 9.
Step Into F7
Step Into: this time we are inside square(4); only x is visible.
Step Out Shift+F8
Step Out: square returned 16; we are back on the calling line, about to finish the assignment.
Step Over F8
Step Over: sq2 = 16.
Step Out Shift+F8
Step Out: sumSquares returned 25; we are back in main.
Step Over F8
Step Over: s = 25.
Step Over F8
Step Over: the output is printed. Resume ends the program.
public class StepDemo { static int square(int x) { return x * x; } static int sumSquares(int a, int b) { int sq1 = square(a); int sq2 = square(b); return sq1 + sq2; } public static void main(String[] args) { int s = sumSquares(3, 4); System.out.println("s = " + s); } }
Debugging
Debugging on the Command Line: jdb
1public class Jdb { 2 static int square(int x) { 3 int result = x * x; 4 return result; 5 } 6 7 public static void main(String[] args) { 8 int sum = 0; 9 for (int i = 1; i <= 3; i++) { 10 sum += square(i); 11 } 12 System.out.println("Sum: " + sum); 13 } 14}
Compile and start under the debugger
javac -g Jdb.java jdb Jdb
-g: also writes local variable names into the class file; the locals command needs them.
jdb (Java Debugger) is the command-line debugger that ships with the JDK; it does with commands what an IDE does with clicks.
| Command | Meaning |
|---|---|
stop at Class:line | sets a breakpoint |
clear Class:line | removes a breakpoint |
run / cont | starts / continues |
step / next / step up | Step Into / Over / Out |
where | call stack |
locals, print x | local variables, one value |
To attach to a running application, start it with java -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 App; then an IDE or jdb -attach 5005 connects.
Debugging
A jdb Session
> stop at Jdb:4 Deferring breakpoint Jdb:4. It will be set after the class is loaded. > run VM Started: Set deferred breakpoint Jdb:4 Breakpoint hit: "thread=main", Jdb.square(), line=4 bci=4 main[1] where [1] Jdb.square (Jdb.java:4) [2] Jdb.main (Jdb.java:10) main[1] locals Method arguments: x = 1 Local variables: result = 1
main[1] cont Breakpoint hit: "thread=main", Jdb.square(), line=4 bci=4 main[1] print result result = 4 main[1] clear Jdb:4 Removed: breakpoint Jdb:4 main[1] cont > Sum: 14
- The breakpoint is on line 4 (
return result;): the program stops before running it. where:main(line 10) →square.locals:x = 1,result = 1.contstops again on the loop's second pass:result = 4.- With the breakpoint cleared, the program finishes: 1 + 4 + 9 = 14.
Debugging
Systematic Debugging
{-5, -2, -8}).A bug you cannot reproduce, you cannot debug.
The test will also show that the fix works.
Look for the line where a value first goes wrong. assert statements in the code (enabled with -ea) catch the problem where it starts.
best starts at 0, so it never updates”, then confirm it in the debugger.Changing code on a hunch breeds new bugs.
The test stays in the suite for good.
Analysis Tools
Profilers
- A profiler shows where a program spends its time: which method was called how often, how long it took, how much memory it allocated.
- It can show the call graph (who called whom) in several forms: a flat list, a tree, a flame graph.
- The technical details vary with the compiler and JVM version.
- Rule: measure first, then optimize. Most of the time goes into a small “hot spot”; optimizing on a hunch usually hits the wrong place.
java -XprofThe built-in profiler of older JVMs; removed in Java 10. On Java 21:
OpenJDK 64-Bit Server VM warning: Ignoring option -Xprof; support was removed in 10.0
JDK Flight Recorder (JFR) + JDK Mission Control, VisualVM, IDE profilers. Below, the program runs with a JFR recording, then we summarize it.
java -XX:StartFlightRecording=filename=rec.jfr Profile jfr summary rec.jfr
Analysis Tools
Measuring the Hot Spot
public class Profile { static String concatPlus(int n) { String s = ""; for (int i = 0; i < n; i++) { s += i; // new String each pass } return s; } static String concatBuilder(int n) { StringBuilder sb = new StringBuilder(); for (int i = 0; i < n; i++) { sb.append(i); } return sb.toString(); } // main: see the next box }
s += ibuilds a new String each pass and copies the old one: total work grows as n².StringBuilderappends to one buffer: work grows as n. Hundreds of times faster.
public static void main(String[] args) { long t0 = System.nanoTime(); String a = concatPlus(50_000); long t1 = System.nanoTime(); String b = concatBuilder(50_000); long t2 = System.nanoTime(); long ms1 = (t1 - t0) / 1_000_000; long ms2 = (t2 - t1) / 1_000_000; System.out.println("+= : " + ms1 + " ms"); System.out.println("SB : " + ms2 + " ms"); System.out.println(a.equals(b)); }
+= : 873 ms SB : 2 ms true
Analysis Tools
Code Coverage Tools
- Shows which parts of the code ran during a test run.
- Reveals code that never runs or is never used.
- Helps find untested regions of code.
- Coverage is a key tool for making software more reliable.
- Tools: JaCoCo (Maven / Gradle), IntelliJ “Run with Coverage”, Eclipse EclEmma. Results are shown by coloring the source lines.
| Metric | What is counted? |
|---|---|
| Line | Lines run at least once |
| Branch | Both the true and false outcome of every if / loop condition |
| Method | Methods called at least once |
assert raises coverage too.Analysis Tools
Code Coverage: An Example
Sign: coverage after two tests
public class Sign { public static String sign(int x) { if (x > 0) { return "positive"; } else if (x < 0) { return "negative"; } return "zero"; }}
ran never ran
| Metric | Coverage |
|---|---|
| Line (5 executable lines) | 4 / 5 = 80% |
| Branch (2 conditions × true / false) | 3 / 4 = 75% |
SignTest
class SignTest { @Test void positive() { assertEquals("positive", Sign.sign(5)); } @Test void negative() { assertEquals("negative", Sign.sign(-3)); } }
Both tests are green (✔), but the false branch of x < 0 and return "zero" were never tried. The coverage report points to the missing test: sign(0).
Analysis Tools · Step-by-step trace
Code Coverage, Step by Step
public class Sign { public static String sign(int x) { if (x > 0) { return "positive"; } else if (x < 0) { return "negative"; } return "zero"; } }
There are 5 executable lines (3, 4, 5, 6, 8) and 4 branches from 2 conditions. No test has run yet: all red.
sign(5): line 3 runs and the condition takes the true branch.
Returns "positive"; the test passes. Coverage: 2/5 lines, 1/4 branches.
sign(-3): x > 0 is false, x < 0 is true. Two new branches.
Returns "negative". Both tests are green.
Report: 80% lines, 75% branches. Line 8 and the false branch of x < 0 were never tried: the missing test is sign(0).
New test sign(0): both conditions are false. Branch coverage 4/4.
Returns "zero": 100% line and branch coverage. Still, coverage only says the lines ran; assertEquals checks correctness.
Analysis Tools
Static Analysis: FindBugs and SpotBugs
- FindBugs is a tool for static analysis of Java code: it examines the compiled code without running the program.
- It finds bugs quickly:
nulldereferences, wrong comparisons, unclosed resources… - FindBugs is no longer developed; its successor is SpotBugs (Maven, Gradle, IDE plugins). Similar tools: PMD, Checkstyle, SonarQube.
The input is yes, yet the output is Rejected. The compiler says nothing; SpotBugs flags both lines.
import java.util.Scanner; public class SpotDemo { public static void main(String[] args) { Scanner sc = new Scanner(System.in); String answer = sc.nextLine(); // result ignored: answer.trim(); // String compared with ==: if (answer == "yes") { System.out.println("Approved"); } else { System.out.println("Rejected"); } } }
| SpotBugs pattern | Warning |
|---|---|
ES_COMPARING_STRINGS_WITH_EQ | Strings compared with ==; use equals |
RV_RETURN_VALUE_IGNORED | trim() returns a new String; the result is discarded |
Analysis Tools
The First Static Analyzer: the Compiler (-Xlint)
1public class Lint { 2 public static void main(String[] args) { 3 int day = 2; 4 switch (day) { 5 case 1: 6 System.out.println("Monday"); 7 case 2: 8 System.out.println("Tuesday"); 9 case 3: 10 System.out.println("Wednesday"); 11 break; 12 } 13 int x = 10 / 0; 14 } 15}
Tuesday
Wednesday
Exception in thread "main" java.lang.ArithmeticException: / by zero
at Lint.main(Lint.java:13)Lint.java:13: warning: [divzero] division by zero
int x = 10 / 0;
^
Lint.java:7: warning: [fallthrough] possible fall-through into case
case 2:
^
Lint.java:9: warning: [fallthrough] possible fall-through into case
case 3:
^
3 warnings- Plain
javac Lint.javacompiles without a single warning. -Xlint:allreports suspicious constructs:cases with a missingbreakand division by a constant zero.- Reading warnings is the cheapest way to find a bug before running the code.
Analysis Tools
QuickCheck: Property-Based Testing
- QuickCheck generates tests automatically: the tool, not the programmer, picks random inputs.
- Instead of single examples, the programmer writes a property: “this must hold for every input”.
- If an input breaks the property, it is reported as a counterexample (shrunk when possible).
- The original is for Haskell; there are versions for many languages; in Java: jqwik and junit-quickcheck.
The test on the right applies the idea in plain JUnit: 1000 random strings.
- ✔
reverseTwice…()
class PropertyTest { static String randomText(Random r) { int len = r.nextInt(20); // 0..19 StringBuilder sb = new StringBuilder(); for (int i = 0; i < len; i++) { sb.append((char) ('a' + r.nextInt(26))); } return sb.toString(); } @Test void reverseTwiceGivesSame() { Random r = new Random(42); // fixed seed for (int k = 0; k < 1000; k++) { String s = randomText(r); String t = TextUtils.reverse(s); assertEquals(s, TextUtils.reverse(t)); } } }
Analysis Tools
A Property Test Finds the Bug in the First Version
class PropertyTest2 { @Test void maxIsLargestElement() { Random r = new Random(7); for (int k = 0; k < 1000; k++) { int[] arr = new int[1 + r.nextInt(5)]; for (int i = 0; i < arr.length; i++) { arr[i] = r.nextInt(201) - 100; // -100..100 } int best = ArrayUtils.max(arr); boolean found = false; for (int x : arr) { assertTrue(best >= x); // property 1 if (x == best) found = true; } assertTrue(found, // property 2 "Counterexample: " + Arrays.toString(arr)); } } }
Run against the first (buggy) version of max:
- ✘
maxIsLargestElement()Counterexample: [-60, -69, -39, -65, -13] ==> expected: <true> but was: <false>
- Property 1 (the result is not less than any element) passes even for the buggy version: 0 is greater than every negative.
- Property 2 (the result is an element of the array) catches the bug.
- With the fixed version all 1000 inputs pass. Lesson: write complete properties; a weak property approves buggy code too.
Analysis Tools
Selenium
- Selenium is a framework for testing web applications.
- It makes browser-based tests easy: it drives a real browser from code, clicks links, fills in forms, presses buttons and checks that the expected text appears.
- It works with Java, Python, C#, JavaScript and more; in Java it is usually called from a JUnit test.
Open the login page → type user name and password → “Log in” → does the title say “Welcome”?
Tests the whole as the user sees it: system and acceptance tests.
Much slower and more brittle than unit tests; keep them few (the top of the pyramid).
Analysis Tools
A Memory Leak Example
import java.util.ArrayList; import java.util.List; public class Leak { static List<byte[]> cache = new ArrayList<>(); static void handle(int id) { byte[] data = new byte[1024 * 1024]; // 1 MB cache.add(data); // never removed! } public static void main(String[] args) { for (int i = 1; ; i++) { handle(i); if (i % 20 == 0) { System.out.println(i + " requests handled"); } } } }
20 requests handled
…
120 requests handled
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at Leak.handle(Leak.java:8)
at Leak.main(Leak.java:14)The garbage collector (GC) frees only unreachable objects; a leak means unneeded objects still have a reference. -Xmx256m: heap at most 256 MB.
Analysis Tools
Eclipse Memory Analyzer (MAT)
- Finds memory leaks by analyzing the Java heap.
- Used for memory optimization: which objects take how much space, and who holds them?
- Input: a heap dump (
.hprof): a snapshot of every object at one moment. - Key reports: Leak Suspects, Dominator Tree (who holds the most memory), Histogram (object count per class).
Taking a heap dump
// on OutOfMemoryError java -XX:+HeapDumpOnOutOfMemoryError Leak // from a running program jps jcmd <pid> GC.heap_dump heap.hprof
The Leak Suspects report shows that the Leak.cache list and its byte[] arrays hold almost the whole heap. Fix: remove data once it is done, or use a size-bounded cache.
Analysis Tools
VisualVM
- Visualizes memory use, thread activity, CPU load and more, live.
- Used to monitor and analyze the performance of Java applications: it attaches to running JVMs, profiles them and takes heap and thread dumps.
- Free and open source (visualvm.github.io); it used to ship with the JDK as
jvisualvm. - Command-line counterparts:
jps(list JVMs),jstat,jcmd,jconsole.
The heap graph gives a leak away: if the lows after each garbage collection keep rising, objects are not being freed. Next, take a heap dump and examine it in MAT.
Summary
Summary: Which Tool for Which Question?
| Question | Technique / tool |
|---|---|
| Does this method give the right result? | Unit testing: JUnit, @Test, assertEquals, assertThrows |
| Do the parts work together, as a whole and for the user? | Module, sub-system, system and acceptance (alpha / beta) testing; Selenium for the web |
| Did the change break something old? | Regression testing: all automated tests run on every change (CI) |
| Which inputs should I pick? | Boundary values, equivalence classes, @ParameterizedTest |
| Is it right for inputs I did not pick? | Property-based testing (QuickCheck, jqwik) |
| How much of the code do my tests run? | Code coverage: JaCoCo, EclEmma |
| Are there bugs visible without running the code? | Static analysis: javac -Xlint, SpotBugs (FindBugs) |
| Why does it misbehave? | Debugger: breakpoints, Step Into / Over / Out, watches; jdb |
| Where does the time go? | Profiler: JFR, VisualVM |
| Why does memory fill up? | Monitor with VisualVM; examine a heap dump in Eclipse Memory Analyzer |
Summary
Common Mistakes
assertEquals(calc.add(2, 3), 5) also works, but on failure the message reads backwards: expected shows the actual result.
assertEquals(0.3, 0.1 + 0.2) fails: expected: <0.3> but was: <0.30000000000000004>.
import staticassertEquals(5, x) → compile error: cannot find symbol: method assertEquals(int,int).
org.junit.Test and org.junit.jupiter.api.Test differ; the wrong import can stop tests from running at all.
@BeforeAll method not staticJUnit 5 will not run the tests: @BeforeAll method '…' must be static unless the test class is annotated with @TestInstance(Lifecycle.PER_CLASS).
Relying on an object left by another test: when the order changes, tests fail at random. Each test sets up its own data.
Calling a method without any assert: the test is always ✔ and proves nothing.
Boundaries (0, empty array, negatives), invalid input and exceptions get forgotten; that is exactly where bugs live.
Summary
Sample Problems (1)
Problem 1: Which tests pass, which fail?
class QuizTest { @Test void a() { assertEquals(7, 3 + 4); } @Test void b() { assertEquals(0.3, 0.1 + 0.2); } @Test void c() { assertTrue("Java".length() == 4); } @Test void d() { assertThrows(ArithmeticException.class, () -> System.out.println(10 / 2)); } }
Problem 2
If letterGrade had score > 50 by mistake instead of score >= 50, which row of the parameterized test would fail? With what message?
Problem 3
What is the minimum number of tests for 100% branch coverage of the method below? Choose the inputs.
public class Abs { static int abs(int x) { if (x < 0) { return -x; } return x; } }
Show answers
- a ✔; b ✘
expected: <0.3> but was: <0.30000000000000004>(no delta); c ✔; d ✘Expected java.lang.ArithmeticException to be thrown, but nothing was thrown.(it prints 10 / 2 = 5). [3] 50, CCfails:expected: <CC> but was: <FF>. 49 and 59 still pass; only the exact boundary catches the bug.- 2 tests: one negative (e.g. −3 → 3), one zero or positive (e.g. 5 → 5). Adding 0 as a boundary is a good idea.
Summary
Sample Problems (2)
Problem 4: What is wrong with this test class?
class CounterTest { static int count = 0; @Test void increment() { count++; assertEquals(1, count); } @Test void incrementTwice() { count += 2; assertEquals(2, count); } }
Problem 5
In StepDemo, the debugger is stopped at int s = sumSquares(3, 4);. After two Step Overs in a row, which line are we on, what is on the screen, and what is s?
Show answers
- The
staticfield is shared between the tests, so the result depends on test order. Whichever runs second fails (ifincrementruns first:expected: <2> but was: <3>). Fix: make the field non-static(JUnit creates a new instance per test) or reset it in@BeforeEach. - The first Step Over finishes the line (
s = 25) and moves to theprintlnline; the second printss = 25and moves to the closing line ofmain.s= 25.
Summary
Sample Problems (3): Which Tool?
Problem 6: Which technique or tool fits each situation?
- The program crashes with
OutOfMemoryErrorafter 10 minutes. - Generating a report takes 30 seconds; which method is slow?
- Did the new discount code break the old invoice calculation?
- Does the site's sign-up form work in every browser?
- Are there forgotten
breaks in aswitch? - A loop produces a wrong value on iteration 10,000; what are the variables at that moment?
Show answers
- Watch the heap graph in VisualVM; examine a heap dump (
-XX:+HeapDumpOnOutOfMemoryError) in Eclipse Memory Analyzer. - A profiler: JDK Flight Recorder or VisualVM.
- Regression testing: rerun all automated tests (ideally in CI on every change).
- Browser-based tests with Selenium, using different browser drivers.
- Static analysis:
javac -Xlint:all([fallthrough]), SpotBugs. - A conditional breakpoint in the debugger:
i == 9999.
Summary
Exercises (1)
- Write a JUnit 5 test class for a
Rectangleclass (width,height,area(),perimeter()). A negative side length must throwIllegalArgumentException. - Use
@ParameterizedTestto teststatic boolean isPrime(int n)at the boundary values (0, 1, 2, 3, 4, 97, 100). - Add
transfer(BankAccount target, double amount)toBankAccount; write its tests first (TDD). Which tests do you need?
Show answers
- Tests:
areaIsCorrect(3 × 4 → 12),perimeterIsCorrect(→ 14), a zero side,assertThrows(IllegalArgumentException.class, () -> new Rectangle(-1, 2)). Floating-point sides need a delta. @CsvSource({"0, false", "1, false", "2, true", "3, true", "4, false", "97, true", "100, false"})andassertEquals(expected, isPrime(n)). 0, 1 and 2 are the boundaries most often gotten wrong.- A successful transfer (both balances change), insufficient funds (exception, neither balance changes), a negative amount, a transfer to itself. Each test is red first, then green.
Summary
Exercises (2)
- A method that averages an array gives wrong results for some arrays (e.g. 1.0 for
{1, 2}). Write down the steps to find the defect with a debugger. - Write a test that checks the property “in a sorted array no element is greater than the next one” on 1000 random arrays. Is this property enough on its own?
- Which test must be added to bring branch coverage of
Signto 100%?
Show answers
- Write a failing test (
{1, 2}→ 1.5) → set a breakpoint on thereturnline → watchsumand the division → see thatsum / arr.lengthis integer division (3 / 2 = 1) → use(double) sum / arr.length→ run all tests. - No: a buggy “sort” that ignores its input and always returns an empty array or all zeros also satisfies it. Also check that the result is a permutation of the input (same elements, same counts).
assertEquals("zero", Sign.sign(0)): it runs both the false branch ofx < 0and the finalreturn.
Summary
References
- JUnit Team, JUnit 5 User Guide, junit.org/junit5/docs/current/user-guide (Writing Tests, Assertions, Parameterized Tests).
- Oracle, The Java Tutorials and JDK 21 Tool Specifications:
javac(-Xlint),jdb,jcmd,jfr. - Oracle, Programming With Assertions (Java SE documentation).
- J. Gosling et al., The Java Language Specification, Java SE 21 Edition (14.10 The assert Statement).
- P. Deitel and H. Deitel, Java: How to Program, 11th ed., Pearson.
- C. S. Horstmann, Core Java, Volume I: Fundamentals, 12th ed., Pearson (Chapter 7: Exceptions, Assertions, and Logging; debugging tips).
- K. Beck, Test-Driven Development: By Example, Addison-Wesley.
- Tool documentation: SpotBugs (spotbugs.github.io), JaCoCo (jacoco.org), Selenium (selenium.dev), Eclipse Memory Analyzer (eclipse.dev/mat), VisualVM (visualvm.github.io), jqwik (jqwik.net).