Pairwise Test Suite Generator Tool Based On Harmony Search . - Core

1y ago
9 Views
1 Downloads
710.46 KB
24 Pages
Last View : 1m ago
Last Download : 5m ago
Upload by : Luis Waller
Transcription

PERPUSTAKAAN UMP ID DII IDID11111 DD 1111 00001 03246 PAIRWISE TEST SUITE GENERATOR TOOL BASED ON HARMONY SEARCH ALGORITHM (HS-PTSGT) LAI Yl XIANG BACHELOR OF sCIwir (SOFTWARE ENGINEERING) UNIVERSITI MALAYSIA PAHANG

V ABSTRACT In the era of technology, software systems are around us in our daily life. Thus, a good system is needed to ensure the well function of gadget and devices with software embedded. However, exhaustive testing that required lot of time and financial resources is normally impossible. This thesis is about the research on developing a pairwise test suite generator tool based on Harmony Search Algorithm (HS-PTSGT) to generate optimum test suite. Test suite which considers all interaction of at most two factors will be generated from two-valued input parameters that are not more than 500. Users are allowed to enter parameter and values through the graphical user interface, and test cases will be generated through the execution of Harmony Search Algorithm involved random initialization of harmony memory and harmony memory improvisation. From the comparison with other existing pairwise testing tools in term of size, it is shown that the strategy used in HS-PTSGT can effectively generate optimum test suite with minimized test data and performed better on certain parameter size.

II ABSTRAK Pada era teknologi mi, system perisian berada di sekeliling kita. Jadi, sistem perisian yang berkualiti amat diperlukan untuk memastikan peralatan yang terbenam perisian sentiasa berfungsi dengan baik. Zalimnya, ia adalah mustahil untuk kita menguji sesuatu sistem dengan sepenuhnya memandangkan masa dan wang yang banyak diperlukan. Tesis mi akan membincang tentang uji kaji untuk menciptapairwise test suite generator tool based on Harmony Search Algorithm (HS-PTSGT) bagi menghasilkan koleksi set pengujian yang optimum. Koleksi set pengujian yang mempertimbangkan semua interaksi antara dua faktor tidak melebihi 500 parameter akan dihasilkan. Pengguna boleh masukkan parameter berserta dengan nilainya ke dalam HS-PTSGT, dan koleksi set pengujian akan dihasilkan setelah operasi Harmony Search Algorithm yang melibatkan harmony memory initialization dan harmony memory improvisation. Daripada keputusan kajian, HS-PTSGT terbukti mampu menghasilkan koleksi set pengujian yang optimum dengan berkesan dan berprestasi lebih baik daripada pairwise testing tool yang lain di keadaan yang tertentu.

vii TABLE OF CONTENTS Page VERIFIED FORM j DECLARATION SUPERVISOR'S DECLARATION ACKNOWLEDGEMENTS iv ABSTRACT v ABSTRAK vi CONTENTS vii LIST OF TABLES x LIST OF FIGURES xii LIST OF ABBREVIATIONS xv CHAPTER 1 INTRODUCTION 1.1 INTRODUCTION 1 1.2 PROBLEM STATEMENT 3 1.3 GOAL AND OBJECTIVES 4 1.4 RESEARCH SCOPE 4 1.5 METHODOLOGY 4

viii 1.6 THESIS ORGANIZATION 6 1.7 SUMMARY 7 CHAPETR 2 LITERATURE REVIEW 2.1 INTRODUCTION 8 2.2 OVERVIEW 8 2. 2. 1 Pairwise Testing 2. 2. 2 Harmony Search Algorithm 2.3 SURVEY OF EXISTING PAIRWISE STRATEGIES 9 14 15 2. 3. 1 'Genetic Algorithm (GA) 16 2. 3. 2 Ant Colony Algorithm (ACA) 18 2. 3. 3 In-Parameter-Oder (IPO) 19 21 2.4 SUMMARY CHAPTER 3 METHODOLOGY 3.1 INTRODUCTION 22 3.2 METHODOLOGY 23 3. 2. 1 Requirement Planning 24 3.2.2 User Design 25 3. 2. 3 Construction 25 3.2.4 Cutover 25 3.3 HARDWARE AND SOFTWARE SPECIFICATION CHAPTER 4 26 THE DESIGN OF HS-PTSGT 4.1 INTRODUCTION 22 4.2 DESIGN APPROACH 23 4. 2. 1 Non-Deterministic Output 29 4. 2. 2 Automated Parameter Mapping Supports 30 4.3 THE DEVELOPMENT OF HS-PTSGT 32

ix 4. 3. 1 Interaction List Generation Algorithm 34 4. 3. 2 HS-PTSGT Test Suite Generation Algorithm 39 4. 3. 3 Actual Parameter Mapping Algorithm 44 4.4 SUMMARY CHAPTER 5 45 IMPLEMENTATION 5.1 INTRODUCTION 47 5.2 IMPLEMENTATION. OF PAIR COMBINATION 47 5.3 IMPLEMENTATION OF TEST SUITE GENERATION 50 5.4 IMPLEMENTATION OF ACTUAL PARAMETER MAPPING 52 5.5 SUMMARY CHAPTER 6 53 RESULT AND DISCUSSION 6.1 INTRODUCTION 54 6.2 DEMONSTRATION OF IMPLEMENTATION 56 6.3 DEMONSTRATION OF MINIMIZED TEST SUITE SIZE 58 6.4 COMPARISON AGAINST OTHER PAIRWISE TESTING TOOLS 59 6.5 ANALYSIS OF HS-PTSGT THROUGH HMCR & PAR VALUES .70 6.6 SUMMARY CHAPTER 7 72 CONCLUSION 7. 1 CONCLUSION 74 7.2 FUTURE WORK 75 FEFERENCES 77 APPENDICES 79

x LIST OF TABLES Table Number Page 2.1 Input Parameter and Value Illustration 9 2.2 Exhaustive Combination of 4 Inputs 10 2.3 2-Way Combination for AB 10 2.4 2-Way Combination for AC 11 2.5 2-Way Combination for AD 11 2.6 2-Way Combination for BC 12 2.7 2-Way Combination for BD 12 2.8 2-Way Combination for CD 13 3.1 Hardware Specification 26 3.2 Software Specification 27 6.1 iPhone Keyboard Input Specification 56 6.2 Pairwise Coverage for iPhone Keyboard Setting Input Specification 57 6.3 Input Specification for Ti until T8 58 6.4 Input Data Specification for 12 Tests 60 6.5 Result of the Generated Test Case in term of the Test Case Size 61 6.6 Results of Test Size Comparison 69

xi 6.7 Values Specification for HMCR and PAR Analysis 71 6.8 Result of the Generated Test Suite in term of Test Case Size 71

xli LIST OF FIGURES Figure Number Page 1.1 Display Tab in Options Dialog for Micrasoft Word 3 1.2 Rapid Application Development (RAD) Model 5 2.1 Merging of AB, AC, AD, BC, BD and CD 13 2.2 Harmony Search Algorithm Flow Chart 14 2.3 Genetic Algorithm 17 2.4 Ant Colony Algorithm 18 2.5 Search Space 19 2.6 IPO H - Algorithm for Horizontal Growth of a Test Suite by Adding Values for New Parameters 2.7 20 IPO V - Algorithm for Vertical Growth of a Test Suite by Adding Values for New Parameters 20 3.1 Rapid Application Development (RAD) Model 24 4.1 Effect of HMCR and PAR in Generating Optimum Solution 29 4.2 Process of Mapping Parameter 31 4.3 Overview of HS-PTSGT Framework 32 4.4 The General Flow of HS-PTSGT 34 4.5 Pseudo-Code of Interaction List Generation Algorithm 35

xlii 4.6 Number of Pairs Calculation 36 4.7 The Interaction Pairs of AB, AC and AD 37 4.8 Initial uncoveredlnteractionPairsSearch Array 38 4.9 Initial uncoveredlnteractionPairsCOunt Array 39 4.10 Pseudo-Code of HS-PTSGT Test Suites Generation Algorithm 39 4.11 The Evaluation of Covered Interaction Pairs for a Test Case 42 4.12 The Final Data Structure of uncoveredPairsSearch 43 4.13 Pseudo-Code of Actual Parameter Mapping Algorithm 44 4.13 Mapping Symbolic Values with Actual Parameter Values 45 5.1 Input Screen for User-Defined Inputs Before Parameter Values Input 48 5.2 Inputs for Hostel Survey 49 5.3 Example of Input Specification for HS-PTSGT 49 5.4 Parameter Values with Symbolic Notation 50 5.5 Harmony Memory Evaluation for Best Harmony Test Case 51 5.6 The Symbolic Final Test List 51 5.7 Final Test Suite Before and After Mapping Process 52 6.1 The Keyboard Setting for iPhone 55 6.2 Final Test Suite Generated by HS-PTSGT 56 6.3 Exhaustive Test Suite Size against HS-PTSGT Test Suite Size 59 6.4 Test Suite Result of 51 61 6.5 Test Suite Result of S2 62 6.6 Test Suite Result of S3 63

xlv 6.7 Test Suite Result of S4 63 6.8 Test Suite Result of S5 64 6.9 Test Suite Result of S6 65 6.10 Test Suite Result of S7 65 6.11 Test Suite Result of S8 66 6.12 Test Suite Result of S9 67 6.13 Test Suite Result of S 1 67 6.14 Test Suite Result of S 1 68 6.15 Test Suite Result of S12 69 6.16 The Pattern of Test Suite Size with Various HMCR and PAR Values 72

xv LIST OF ABBREVIATIONS HM: Harmony Memory HMS : Harmony Memory Size HMCR : Harmony Memory Considering Rate PAR : Pitch Adjustment Rate GA: Genetic Algorithm ACA : Ant Colony Algorithm IPO : In-Parameter-Order RAD : Rapid Application Development CASE: Computer-Aided Software Engineering tools

I CHAPTER 1 INTRODUCTION 1.1 INTRODUCTION In the era of technology, software systems are around us in our daily life. Electronic gadgets and devices that we're using are embedded with software, any bugs in these systems can cause the gadgets and devices to be malfunction. For critical system, the failure bring by software system may cause the loss of life and huge sum of money. Thus, a good system is needed to ensure the well function of these gadget and devices. Start from the time when the term 'bug' is used to describe flaw in a system by the famous Thomas Alva Edison, defect in software system has become a threat in software development (Meerts, J. and Graham, D., 2013). In software development process, software testing plays an important role and required the most financial resources. Software Testing known as the process of discovers errors in system execution and validating the system against its specification (Myers, G. J., 1979).

During testing phase, test suite that with a collection of test cases are used to explore defects by entering test data as input into the system. In order to have effective testing on system, the generation of an optimum test suite is essential to avoid exhaustive testing which are practically impossible. This research is to develop a pairwise test suite generator tool based on Harmony Search Algorithm (HS-PTSGT). Pairwise testing is a combinatorial technique used to reduce the number of test case inputs to a system, consider all interactions of at most two factors (McCaffrey, J. D., 2009). Pairwise testing normally begins by choosing individual inputs variables using domain partitioning; the values are then permuted to ensure the consideration of all pair interactions (Bach, J. and Schroeder, P. J, 2004). From the studies of Kuhn, D.R. and Reilly, M.J. (2002), more than 70% of bugs caused by two or fewer interaction condition able to be noticed. Harmony search algorithm is a music-based metaheuristic optimization algorithm, divined from the aim to search for prefect state of harmony in jazz musician's improvisation process (Yang, X. S., 2009). Harmony Search algorithm works as follow: stochastic generation of an initial population harmony vectors and stored these vectors in a harmony memory. After that, new candidate harmony will be created based on all solutions in the harmony memory using a memory consideration rule, a pitch adjustment rule and a random re-initialization. Lastly, the new candidate harmony is compared with the worst harmony vector in the harmony memory. The new candidate harmony will replace the worst harmony and update the harmony memory if it is better thanO the worst harmony. This process will repeat until certain termination criterion is met. Harmony Search is a generalized optimization method for continuous, discrete, and constrained optimization and has been applied to numerous types of optimization problems. Further information will be discussed in details in Chapter 2.

3 1.2 PROBLEM STATEMENT Testing is required to find faults in a software product. For a big system, exhaustive testing that required lot of time and financial resources is impossible. Figure 1.1 shows the display tab in the options dialog for Microsoft Word. There are 17 possible options that can take two values which are check and uncheck. Hence, the combinations that need to be tested are 217 or 131,072. If a test case need 5 minutes to be test, to completely test only the display tab will need approximately 15 months, which are practically inefficient. Word Options GenerOl [DiOpint Change how document content in displayed on the screen and when printed. I Proofing Page display options ) Show SShute space between pages In Pant Layout H ew() Shnwglghhlghtnrnnarttø 3 Show document toottipa on honor Save Typography Language Advanced Atwaynohow thow 00000thng n.wtmon the onreen Custonnine Ribbon fl lab rharootert Quick Access Toolbar D Add.int ,Trust Center l Paragraph IStrilt -. I Hidden Smut iTS Optional hyphens ObJectan5hors isle Qptfonal break, Show Ill formatting mark, Printing option, 0 ) Print dyawingi Wetted In Word Print Qacttground colors and Images Print document gropertles Pant hidden toyt In Update tields before printing Update linked data before printing rl Figure 1.1: Display Tab in Options Dialog for Microsoft Word

4 1.3 GOAL AND OBJECTIVES The goal for this research is to develop a pairwise test suite generator tool based on Harmony Search Algorithm. Several objectives that have been set to achieve the goal are as follow: i) To apply Harmony Search Algorithm in generating pairwise test suite. ii) To generate optimum test suite with minimized test data based on interactions of two factors. iii) To evaluate the performance of this pairwise test suite generator tool in term of test size against other existing pairwise testing strategies. 1.4 RESEARCH SCOPE In the process of development, NetBeans IDE 8.0 will be used. The scopes of this research are as follow: i) Pairwise test suite generating based on Harmony Search Algorithm. ii) Pairwise test suite generating with 2-valued input parameters that are not more than 500. iii) Pairwise test suite generator tool developed in Java language. 1.5 METHODOLOGY Software methodology is the activities used to control and ensure the achievement of stated objectives and finally reach the goal of this research. The methodology that going to use in this research are Rapid Application Development model, Figure 1.2 below shows the phases in the model:

5 Phase 1: Requirements Planning o Literature review is performed in this phase to get the idea on the design of HS-PTSGT. The other tools is collected to be used in evaluate the performance of this pairwise test suite generator tool in term of test size against other existing pairwise testing strategies. . Phase 2: User Design o The architecture of HS-PTSGT will be design to implement Harmony Search Algorithm. The interface design of the generator tool will be carrying out to ensure good usability. Phase 3: Construction o After the design of HS-PTSGT, this tool will be developed. The concept of pairwise testing and harmony search algorithm will be implemented in this tool, and test will be carry out to ensure correct implementation and well function of this tool. Phase 4: Cutover o The result of generated test suite of HS-PTSGT will be compared with other existing tools to show efficiency of this tool in term of test size. Study will also carry out to observe the effect of HMCR and PAR on test size. The whole research will then be documented. Future improvement will be suggested to improve this research. Figure 1.2: Rapid Application Development (RAD) Model

rel 1.6 THESIS ORGANIZATION This thesis consists of seven (7) chapters, each chapter discuss about its own issues. Chapter 1 is Introduction. This chapter discusses on introduction about the research that will be done. The introduction, problem statement, goal, objectives, and scopes will be known. Chapter 2 is Literature Review. Existing research and literature review that related to the project will be discussed in this chapter. Chapter 3 is Methodology. In this chapter, overall approach and framework of research will be discussed. The method, technique or approach to be used for the project will be fixed. Chapter 4 is Design of HS-PTSGT. This Chapter will illustrate the framework and model through flow work. Chapter 5 is Implementation. All the processes involved in the research development will be recorded. Chapter 6 is Result and Discussion. The results of pairwise test suite generator tool will be discussed on this chapter to determine the development is success or not. The test suite generated will be compared with other strategies in term of test size. Chapter 7 is Conclusion. In this chapter, summarization of the research being done will be discussed. Besides that, the future suggestion and enhancement of the research topic also proposed in this chapter.

7 1.7 SUMMARY This chapter has discussed the introduction of research on Pairwise Test Suite Generator Tool Based on Harmony Search Algorithm. Included in this chapter are problem statement, goal and objective, research scope, and thesis organization. The next chapter is literature review that discusses about the survey of existing pairwise testing strategy and Harmony Search Algorithm techniques.

8 CHAPTER 2 LITERATURE REVIEW 2.1 INTRODUCTION In Chapter 1, the introduction of this research has been discussed, including problem statement, goal and objective, scope and methodology. Building on the material in Chapter 1, this chapter will focus on relevant literature review survey. In particular, pairwise testing techniques, Harmony Search Algorithm, and existing pairwise testing strategy are elaborated to justify the current work. 2.2 OVERVIEW The main objective of this research is to generate optimum test suite with minimized test data based on interactions of two factors. To achieve this goal, pairwise testing and Harmony Search Algorithm will be implemented into this test suite generator tool.

Oj 2. 2. 1 Pairwise Testing Pairwise testing is a combinatorial technique used to reduce the number of test case inputs to a system, consider all interactions of at most two factors (McCaffrey, J. D., 2009). In pairwise testing test suite that consider all combinations of the selected test data values for each pair of variables is generated. To understand the pairwise testing, following example is illustrated. Table 2.1: Input Parameter and Value Illustration A I False --V B True - I T C False I 1H D False Table 2.1 shows the input for a four plugs socket. In this system, there are 4 inputs available to user with 2 options respectively. Here, the exhaustive combination of all possible test data can be calculated by [number of test data (number of value) number of input parameters 2 4 16]. The exhaustive combinations of this four plugs socket are shown in Table 2.2.

10 Table 2.2: Exhaustive Combination of 4 Inputs 1L I A \Y11IG3 True L False [ - True :1 r B ]C j True True True False False - ] L alse ] True True tTrue True] True 1 L False] True True False True True True False False] True False True True False True . JL False True False False I True False False False False True True True EaLsq .HL .Iriu True False .j False rrue False True False True False False 9 I False False True LTrue T False False True False 7 False False ][True False 11 False --71 False IFalse In order to reduce the test data, pairwise testing technique can be used. In this technique, the 2-way possible combinations are AB, AC, AD, BC, BD, CD. For AB combination the total test size is reduced to 4 as only parameter A and B are considered while parameter C and D are treated as don't care value (see Table 2.3). Thus, C and D can randomly take either value True or False. Table 2.3: 2-Way Combination for AB s le LaJ True True 2-way combination for AB BC True True P j I D True j

I The second combination is AC. For this combination the total test data also reduced to 4 since only parameter A and parameter C are considered. Parameter B and D can take either value of True or False randomly as they are treated as don't care values (see Table 2.4). Table 2.4: 2-Way Combination for AC I Base Value -- combination for AC -- I 1: II 1npjaijiables 1u,r: bilL: True -False True False The third combination is AD. For this combination the total test data also reduced to 4 since only parameter A and parameter D are considered. Parameter B and C can take either value of True or False randomly as they are treated as don't care values (see Table 2.5). Table 2.5: 2-Way Combination for AD MMVdbD [ 4 True True True OJ11iftW1ftiTh False Rig A@-- False - LTci False 1I C True iaJse D True True False True False -1 The fourth combination is BC, the total test data also reduced to 4 since only parameter B and parameter C are considered. Parameter A and D can take either value of True or False randomly as they are treated as don't care values (see Table 2.6). 1

12 Table 2.6: 2-Way Combination for BC In the fifth combination (BD), the total test data again reduced to 4 since only parameter B and parameter D are considered. Parameter A and C can take either value of True or False randomly as they are treated as don't care values (see Table 2.7). Table 2.7: 2-Way Combination for BD Similar to previous combinations, the total test data for CD combination is reduced to 4. This is because only C and D are considered. A and B are then treated s don't care value where they can randomly take either True or False values (see Table 2.8).

a pairwise test suite generator tool based on Harmony Search Algorithm (HS-PTSGT). Pairwise testing is a combinatorial technique used to reduce the number of test case inputs to a system, consider all interactions of at most two factors (McCaffrey, J. D., 2009). Pairwise testing normally begins by choosing individual inputs variables

Related Documents:

bitopological space has been introduced by Pervin[10]. The study of supra topology was . Gowri and Jegadeesan[2] studied the concept of pairwise connectedness in soft biCechˇ closure spaces. The purpose of this article is to introduce and . Supra Pairwise Connected and Pairwise Semi-Connected Spaces

Pairwise testing has become an indispensable tool in a software tester's toolbox. This paper pays special attention to usability of the pairwise testing technique. In this paper, we propose a new test generation strategy for pairwise testing using Genetic Algorithm (GA). We compare the result with the random

Verification tool for Vienna Development Method (BWDM) is a test case generation tool for the VDM specification. The existing BWDM could cause a combinatorial explosion of the generated test cases. To reduce the number of test cases, there is Pairwise Independent Combinatorial Testing Tool (PICT): a pairwise testing tool.

Regression Testing Process and Issues Test-suite maintenance Obsolete test cases Test suite Tval Regression test selection Test suite T' Test-suite prioritization Prioritized Test suite T' T est-suite augmentation Test suite Taug T est-suite minimization Redundant test cases Minimized test suite est-case manipulation

In order to address this issue, a number of pairwise testing (and sampling) strategies have been developed in the literature in the past 15 years. In this paper, we propose and evaluate a novel pairwise strategy called pairwise harmony search algorithm-based . result, many strategies (and their tool implementations) have been designed in the .

Therefore, (ˆ ,/ ,/ ) is pairwise connected. Theorem 3.12. If soft biČech closure space is pairwise disconnected such that ˆ / and let 2 be a pairwise connected soft subset of ˆ then 2 need not to be holds the following conditions ( )2 ( )2 . Proof.

pairwise testing in situations where supposedly independent options might interact In our previous webinars, decision tables, state based methods, and use cases Pairwise techniques allow us to cover combinations in a manageable way These advanced three techniques allow you to perform a wide range of important tests AST-Pairwise Testing www.rbcs .

American Revolution in 1788, when he and his contemporaries were still riding the wave of patriotism emanating from their fresh victory over the British Empire. These histories, marked by American prominence on a global scale, were written into the early 20th century as American patriotism was reinforced by further victory in the War of 1812 and by western expansion. By the latter point, they .