Software Testing - UMD

2y ago
377.50 KB
20 Pages
Last View : 1d ago
Last Download : 1d ago
Upload by : Maxine Vice

Software TestingTesting: Our ExperiencesTest CaseSoftwareto be testedOutput1

Test CaseGenerationWhen to Stop?Test CaseSoftwareto be testedVerificationOutputNoEnough?Test CoverageYesA Real Testing ExampleTest Cases{1,3,2}{1,2,3}{3,2,3}{}{-1, -2}Just a list.A sorted list.Repeated entry.Empty list.SortingNegative numbers.ProgramSPECS:Takes a listof numbers;returns asorted list.{2,3,-1}{1,2,3}{-2,{}Output2

Test CaseGenerationAutomated TestingTest CaseSoftwareto be testedVerificationOutputNoEnough?Test CoverageYesAutomated TestingTest CaseGeneratorTest CaseSoftwareto be testedTestSpecsOutputVerifierORTest OracleCoverageEvaluator3

Testing the New ftwareNewTestCasesRegression ftwareNewTestCases4

What is Testing? Process of determining whether atask has been correctly carried out[Schach ’96] Goals of testing– Reveal Faults ceConflicting Goals?Types of Testing Execution-based Testing Non-execution based Testing Discussion5

Execution-based Testing Generating and Executing TestCases on the Software Types of Execution-based Testing– Testing to Specifications Black-box Testing– Testing to Code Glass-box (White-box) TestingBlack-box Testing Discussion: MAC/ATM MachineExample– Specs Cannot withdraw more than 300 Cannot withdraw more than your accountbalanceBalancexSoftware6

White-box Testing ExampleGenerate test casesx: 1.1000;1to cover each statementINPUT-FROM-USER(x);If (x 300) {2INPUT-FROM-FILE(BALANCE);If (x BALANCE)3GiveMoney x;4else Print “You don’t have x in your account!!”}else56Print “You cannot withdraw more than 300”;Eject Card;Discussion Which is superior? Each technique has its strengths –Use both7

Determining Adequacy Statement coverageBranch coveragePath coverageAll-def-use-path coverageSurprise Quiz Determine test cases so that eachprint statement is executed at leastoncebegininput(x);if (x 100)print “Line 1”;else {ifx 100x 1001x 100x 50if (x 50) print “Line 2”else print “Line 3”;2ifx 100x 503}end8

Non-execution Based Walkthroughs– Manual simulation by team leader Inspections– Developer narrates the reading Key Idea– Review by a team of experts: Syntaxchecker? Code Readings Formal Verification of Correctness– Very Expensive– Justified in Critical Applications Semi-formal: Some AssertionsSimulation Integration with system hardware iscentral to the design Model the external hardware Model the interface Examples Discussion9

Boundary-value Analysis Partition the program domain intoinput classes Choose test data that lies bothinside each input class and at theboundary of each class Select input that causes output ateach class boundary and within eachclass Also known as stress testingTesting Approaches Top-down Bottom-up Big Bang Unit testingIntegration testingStubsSystem testing10

Mutation Testing Errors are introduced in theprogram to produce “mutants” Run test suite on all mutants andthe original programTest Case Generation Test Input to the Software Some researchers/authors alsodefine the test case to contain theexpected output for the test input11

Category-partition Method Key idea– Method for creating functional testsuites– Role of test engineer Analyze the system specification Write a series of formal test specifications– Automatic generator Produces test descriptionsSteps Decompose the functional specificationinto functional units– Characteristics of functional units They can be tested independently Examples– A top-level user command– Or a function Decomposition may require several stages Similar to high-level decomposition doneby software designers– May be reused, although independentdecomposition is recommended12

Steps Examine each functional unit– Identify parameters Explicit input to the functional unit– Environmental conditions Characteristics of the system’s state Test Cases– Specific values of parameters– And environmental conditionsSteps “Test cases are chosen to maximizechances of finding errors” For each parameter & environmentalcondition– Find categories Major property or characteristic Examples– Browsers, Operating Systems, array size For each category– Find choices» Examples: (IE 5.0, IE 4.5, Netscape 7.0), (WindowsNT, Linux), (100, 0, -1)13

Steps Develop “Formal Test Specification”for each functional unit– List of categories– Lists of choices within each category Constraints Automatically produces a set of“test frames”– Consists of a set of choicesAI Planning Method Key Idea– Input to Command-driven software is asequence of commands– The sequence is like a plan Scenario to test– Initial state– Goal state14

Example VCR command-line software Commands– Rewind If at the end of tape– Play If fully rewound– Eject If at the end of tape– Load If VCR has no tapePreconditions & Effects Rewind– Precondition: If at end of tape– Effects: At beginning of tape Play– Precondition: If at beginning of tape– Effects: At end of tape Eject– Precondition: If at end of tape– Effects: VCR has no tape Load– Precondition: If VCR has no tape– Effects: VCR has tape15

Preconditions & Effects Rewind– Precondition: end of tape– Effects: end of tape Play– Precondition: end of tape– Effects: end of tape Eject– Precondition: end of tape– Effects: has tape Load– Precondition: has tape– Effects: has tapeInitial and Goal States Initial State– end of tape Goal State– end of tape Plan?– Rewind16

Initial and Goal States Initial State– end of tape & has tape Goal State– has tape Plan?– Play– EjectTest Coverage & Adequacy How much testing is enough?When to stop testingTest data selection criteriaTest data adequacy criteria– Stopping rule– Degree of adequacy Test coverage criteria Objective measurement of testquality17

Preliminaries Test data selection– What test cases Test data adequacy criteria– When to stop testing Examples––––Statement CoverageBranch coverageDef-use coveragePath coverageGoodenough & Gerhart [‘75] What is a software test adequacycriterion– Predicate that defines “whatproperties of a program must beexercised to constitute a thoroughtest”, i.e., one whose successfulexecution implies no errors in a testedprogram18

Uses of test adequacy Objectives of testing In terms that can be measured– For example branch coverage Two levels of testing– First as a stopping rule– Then as a guideline for additional testcasesCategories of Criteria Specification based– All-combination criterion choices– Each-choice-used criterion Program based– Statement– Branch Note that in both the above types, thecorrectness of the output must bechecked against the specifications19

Others Random testing Statistical testing Interface based20

Execution-based Testing Generating and Executing Test Cases on the Software Types of Execution-based Testing – Testing to Specifications Black-box Testing – Testing to Code Glass-box (White-box) Testing Black-box Testing

Related Documents:

University Health Center (301) 314-8180 / University Police (non-emergency) (301) 405-3555 Emergency (police, fire, medical) 911 . UMD Parking and Shuttle Bus Washington Area Metro System (Bus and Train) Campus Organizations . Campus Recreation recwell.umd.eduServices

1.1 Software testing This document describes the structured testing methodology for software testing. Software testing is the process of executing software and comparing the observed behavior to the desired behavior. The major goal of software testing

Software testing is considered as one of the key phases in the software-development life cycle (SDLC). The main objective of software testing is to detect the faults either through manual testing or with automated testing approach. The most commonly adopted software testing approach in industries is test case based testing (TCBT) which is .

take your notes, and spend some quiet time to read your Software Testing book! Afterwards you will have a great understanding about Software Testing domain and be prepared to . Software testing is nothing but an art of investigating software to ensure that its quality under test is in line with the requirement of the client. Software testing is

Introduction to Software Testing (Ch. 1) Why Do We Test Software? Brittany Johnson Adapted from slides by Paul Ammann & Jeff Offutt. Testing in the 21stCentury Software defines behavior-network routers, finance, switching networks, etc. . Infrastructure for Software Testing" (2002)-Inadequate software testing cost US alone between 22 and

Context: GUI testing is system testing of a software that has a graphical-user interface (GUI) front-end. Because system testing entails that the entire software system, including the user interface, be tested as a whole, during GUI testing, test cases—modeled as sequences of user input events—are developed

After searching with the keywords: Agile testing software, Scrum agile testing software, Kanban agile testing software, Test Driven Development agile test software, Behavior Driven Development test software, automation test software, the tables 1, 2 and 3 show the number of scientific articles retrieved. This search was made with a restriction

The Baldrige Excellence Framework is an official publication of The National Institute of Standards and Technology (NIST) under the Malcolm Baldrige National Quality Improvement Act. It was developed to help organizations achieve the same Baldrige criteria that award winning well-functioning organizations use. State Policy PY16-04, dated September 30, 2018 identifies it as a recognized .