Software testing glossary
19 terms every QA team uses, each defined in a sentence or two.
- Acceptance criteria
- Conditions a feature must meet to be accepted by its stakeholders. Test cases that verify a requirement usually check its acceptance criteria one by one.
- Blocked test
- A test that could not be executed because of an external problem, such as a broken environment or an unfinished dependency. Blocked results count against release readiness until resolved.
Learn more - Burndown chart
- A chart of how many test cases in a run still have no final result on each day, compared with an ideal straight line from the run's start to its due date.
Learn more - Defect
- A difference between expected and actual behaviour found during testing, also called a bug or issue. A good defect report links to the failed test, the steps and the environment.
Learn more - Flaky test
- A test that passes and fails on the same code without a relevant change. Flakiness is measured by how often a test's outcome flips between pass and fail across recent results.
Learn more - JUnit XML
- A widely supported XML format for automated test results, written by test runners for most languages. Test management tools import it to record CI results next to manual ones.
Learn more - Pass rate
- The share of executed tests that passed. TestPlum's 30-day pass rate divides passed results by passed, failed and blocked results, so untested and skipped cases do not inflate it.
- Regression testing
- Re-running existing tests after a change to confirm that features which worked before still work. Regression suites are often run for every release candidate.
- Release readiness
- Whether a build meets the agreed quality bar to ship. It is best judged with explicit checks such as completion, pass rate, critical failures, open critical defects and requirement coverage.
Learn more - Requirement coverage
- The percentage of requirements that have at least one test case linked to them. It shows how much of the specified behaviour is tested at all, not whether it passes.
Learn more - Requirements traceability matrix (RTM)
- A table mapping each requirement to the tests that verify it, their latest results and related defects. It reveals untested requirements and tests that verify nothing.
Learn more - Risk-based testing
- Prioritising tests by the likelihood and impact of failure, so the most important and most fragile areas are tested first when time is limited.
Learn more - Smoke test
- A small, fast set of tests that checks the most critical functions of a build before deeper testing starts. A failed smoke test usually stops the rest of the run.
- Test case
- A documented set of preconditions, steps and expected results used to check one behaviour of a system. Structured test cases record an expected result for each step.
Learn more - Test management
- The practice and tooling for planning, writing, executing and reporting on software tests, including traceability to requirements and defects.
Learn more - Test plan
- A reusable definition of which test cases to run for a purpose such as a release or a smoke check. In TestPlum a plan selects all cases, cases matching a filter or a fixed list.
- Test run
- One execution of a set of test cases against a specific build, release or environment, with assignments and recorded results.
Learn more - Test suite
- A group of related test cases, usually organised by product area or feature. Suites can contain nested sections.
- Traceability
- The ability to follow links from a requirement to its tests, their executions and the defects they found, in both directions.
Learn more
See your release readiness in minutes
Create a workspace, add a project and record your first run. Everything on this site is in the product today.