Application Of Orthogonal Experimental Design For The Automatic .
Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013)Application of Orthogonal Experimental Design for the Automatic Software TestingHao WuDepartment of Computer ScienceZhuhai College of Jilin University, JLUZHZhuhai, Chinae-mail: haowu mouse@hotmail.comAbstract—According to inputting different combination ofconditions so as to produce different impacts, software testingdesigns a large number of test cases. If the implementation ofan overall test, due to the limit of the combination of conditions,it is difficult to carry out. In order to generate high quality testcases as early as possible to improve the efficiency of softwaretesting, it is designed a generation tool of the automaticsoftware testing case on orthogonal experimental design. Forthe test data, the use of that tool design test cases. The practiceshows that a small number of test cases are generated, theerror detection ability is strong, and it greatly improves theefficiency of software testing.Keywords- orthogonal experimental design; test case;software testingI.INTRODUCTIONAlong with the more powerful software function and theimprovement of software complexity, software developmentprocess is not easy to be controlled. Software qualityproblem has become an important factor to influence thedevelopment of computer application technology. Softwaretesting is a critical element of software quality assurance andrepresents the ultimate review of specification, design, andcode generation. The design of tests for software and otherengineered products can be as challenging as the initialdesign of the product itself. A rich variety of test case designmethods have evolved for software.There are many applications in which the input domain isrelatively limited. That is, the number of input parameters issmall and the values that each of the parameters may take areclearly bounded. When these numbers are very small, it ispossible to consider every input permutation andexhaustively text processing of the input domain. However,as the number of input values grows and the number ofdiscrete values for each data item increases, exhaustivetesting becomes impractical or impossible.Orthogonal experimental design can be applied toproblems in which the input domain is relatively small buttoo large to accommodate exhaustive testing. The orthogonalexperimental design is particularly useful in finding errorsassociated with region faults-an error category associatedwith faulty logic within a software component.Based on the research of orthogonal experimental designmethod, it considers that to be applied to design test cases.Through designing an automatically generated tool togenerate test cases, it can be effective in reducing the numberof test cases, so as to ensure accomplishing software testingin the lower cost and the lower risk. The ultimate aim is toeffectively improve testing results and the efficiency ofsoftware testing.II.MAIN METHODOLOGYA. Test Case DesignThe design of tests for software and other engineeredproducts can be as challenging as the initial design of theproduct itself. A rich variety of test case design methodshave evolved for software. These methods provide thedeveloper with a systematic approach to testing. Moreimportant, methods provide a mechanism that can help toensure the completeness of tests and provide the highestlikelihood for uncovering errors in software.B. Orthogonal Experimental Design1)rthogonal Experimental Design ConceptsOrthogonal experimental design is an important branchof statistical mathematics, based on the probability theory,the mathematical statistics, and the standardized orthogonaltable to arrange the test plan. According to the modernalgebra of Galois theory, Orthogonal experimental design isa scientific test design method, which selects the rightamount of representative points or uses cases from a largenumber of experimental data, so as to arrange experiments ortests reasonably.Orthogonal experimental design usually determines thestandard of test results pros and cons, which is calledindicators that may affect tests as factors, and the impactfactor as factor levels. In designing the optimal test, it musthave a reasonable indicator and a reasonable reference toselecting factor and the corresponding level, in order tocomplete the specific test purpose.2)rthogonal Experimental Design CharacteristicsOrthogonal experimental design is the study of multifactor and level of design method, through the part of the testto find out the optimal level combination. To complete testrequirements needed less number of experiment. The greaterthe number of factors and levels, the more obvious theadvantage of this method is. The distribution of data points isuniform.It is available to use corresponding range analysis method,variance analysis method and regression analysis method toanalyze the test results, and can lead to many valuablePublished by Atlantis Press, Paris, France. the authors2298OO
Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013)conclusions, better reliability. Orthogonal experimentaldesign, base on the orthogonal table, is efficiency, rapid andeconomic test design method.C. Orthogonal Table1)rthogonal Table ConceptsOrthogonal table is a kind of two-dimensional digitalform with the neat comparability and the balancedcollocation. The neat comparability means that each factor ofeach level is completely the same in the same piece oforthogonal table. Since the test of each factor of each leveland other factors involved in the probability of each level testis exactly the same, it ensures that each factor level is ruledout other factors level of interference in the greatest degree.Therefore, it can be most effective in comparison andprospect, and be easy to find a good test condition. Thebalanced collocation means that any two columns (twofactors) level collocation (horizontal formation of digital for)is exactly the same in the same piece of orthogonal table.This helps to ensure that the test condition is balanceddispersion in the level of combination of factors completely,thus which has a strong representation, and is easy to getgood test conditions.The orthogonal table is the foundation of the orthogonalexperimental design, which forms as follow:Ln（mk）(1)The L is the symbol of the orthogonal table, and itssymbol is expressed as follows: n: The number of trials arranged by the orthogonaltable. That is the number of rows in the orthogonaltable in directly corresponding to the number ofproduction test cases. m: The most number of factors arranged in theorthogonal table. That is the number of columns inthe orthogonal table in directly corresponding to theinput parameters in the software module. k: The each factor level number is to get the largestnumber in each single factor. The orthogonal tablecontains values from 0 to the level number - 1 orfrom 1 to the level number, and each factor levelvalues is corresponding to the input parameter.In the process of the orthogonal experimental design, itcan make use of the existing orthogonal table, and also use amathematical model to generate an orthogonal table. Thestructure of the orthogonal table mainly uses the theory ofthe abstract algebra, and there are many existingstandardized orthogonal tables, such as L4(23) 、 L8(27) 、L9(34) 、 L16(45) , etc. Using these orthogonal tables, thegeneral experimental design can be completed. But for someexperiments, due to the factor and the level of the testingmodules are quite complex, it may not be able to select theproper one from these orthogonal tables. Then it should use amathematical model to construct a new orthogonal table,involving more abstract algebra knowledge and as well asmany orthogonal table structure, and so far that have notbeen resolved.2)rthogonal Table Nature The frequency of different digital in each column isOequal, such as L4(23) that there are two differentfigures in each column, and which appears twice. If two figures are seen as the ordered pairs in anytwo lines, and the occurrence number of eachordered pairs appears equal. For example, there arefour ordered numbers of L4(23) and each of themappear once.Due to the orthogonal nature of the table to arrange theexperiment, the various levels of each factor are collocationbalanced, and the experimental number is small, but there isa strong representation. The intuitive analysis, the varianceanalysis and effect analysis can all analyze the size of theimpact of various factors on the experiment results, and todetermine the best experiment plan. Effect analysis can alsoestimate the trend to the index values, and it can guide theexperiment process conversely.3)rthogonal Experimental Design for Test Cases StrategyBy orthogonal experimental design for test cases strategy,there are simple designs and also complex ones. In the realwork, it often encounters very complicated situations,because of the increase of the system scale.There are two key strategies on designing the orthogonalexperiment:a) To Determine FactorsThat all may affect the program result should be takeninto account. That factors are not interactive with others canbe added after the formation of use cases.b) To Determine LevelsThe limited value level is taken all. The infinite valuelevel is determined by the boundary value analysis, theequivalence class analysis and the error determinationmethod.III.ORTHOGONAL EXPERIMENTAL TESTING CASE DESIGNTo design software test cases by orthogonal experimentaldesign can greatly reduce the number of test cases, so as toimprove the efficiency of software testing. That is the mostsuitable for the software module that a number of inputparameters are together to determine the output results. Asthe software structure illustrated in Figure 1, three inputparameters like 1, 2 and 3 together determine the result of theoutput parameter 1, and three input parameters like 3, 4 and5 together determine the result of the output parameter 2,then to choose the orthogonal table should be respectivelysuitable for three factors as 1, 2, 3 and 3, 4, 5.Published by Atlantis Press, Paris, France. the authors2299OO
Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013)Figure 1. Software modules of Orthogonal Array to Design Test Cases.A. Test Data DescriptionIt illustrates the orthogonal experimental design to designtest cases as example that it sets input interface on thesoftware query module of report issued in the financialmanagement and control project. For example, adding aquery can filter out the corresponding budget statement inthe units of the issued report, through setting the inputparameters on this interface, which together determine thebudget statement.The input parameters are as follows: Budget cycle: Annual budget, Month budget Budget time: 2010、2011、201008、201108 Budget classification: Sale power budget, Purchasepower budget, Transportation power budget. Unit: A provincial grid company, a city, State grid. Report/Issue: Reported query, Issued query. Reported/Issued number: Reported number, Issuednumber.B. Argumentation of Selection Orthogonal ExperimentalDesign1)omprehensive TestComprehensive testing is to put all the parameters anddomains into one test collocation. For example, the test of 4factors and 3 levels is needed to do testing use cases of 3 of 4times square equal to 81.The Advantage of the comprehensive test is that it canachieve the comprehensiveness of the test, analyze the effectof various factors and the interaction, and also choose theoptimal level combination. However, the disadvantage ofthis approach is to require the test too many times and theworkload is relatively large. For the test of m factors and nlevels, the total number of test cases is n of m Times Square,which is difficult to fully achieve in the actual test work insome cases.2)rthogonal Eerimental DsignWhen orthogonal experimental testing occurs, an L9(34)orthogonal array of test cases is created. The L9(34)orthogonal array has a balancing property. That is, test casesare dispersed uniformly throughout the test domain, asillustrated in Figure 2. Test coverage across the input domainis more complete.Figure 2. L9(34) orthogonal arrayTo illustrate the use of the L9(34) orthogonal array,consider the send function for a fax application. Fourparameters, P1, P2, P3, P4, are passed to the send function.Each takes on three discrete values.Given the relatively small number of input parametersand discrete values, exhaustive testing is possible. Thenumber of tests required is 34 81, large, but manageable. Allfaults associated with data item permutation would be found,but the effort required is relatively high.The orthogonal array testing approach enables us toprovide good test coverage with far fewer test cases than theexhaustive strategy. An L9(34) orthogonal array for the faxsend function is illustrated in Table Ⅰ.TABLE I.Test case(n)CL9(34) ORTHOGONAL ARRAYTest 2731328321393321Because of the balanced collocation, a factor of anylevels and other ones are combined with once, and the onlyOonce. It can be conductive to analyze the testing results onaccount of that.C. Designing Test Cases on Orthogonal ExperimentalMethodAccording to the above analysis, it selects the orthogonalexperiment scheme. It designs the test as long as each of theinvestigated factors are optionally corresponding to theorthogonal table column, and then the number of eachPublished by Atlantis Press, Paris, France. the authors2300
Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013)column is corresponding to that of one-to-one factor levels.In this way, the combination of each line of each levelconstitutes a test condition.According to the above analysis, the first step is todetermine the factor. Given that there are total 6 factors inthis module, there are 2, 3, 4 kinds of level. Based on theprinciple to select the orthogonal table, to select anorthogonal table of 6-factor and 5-level, the number of testcases generated is 25. It is clear that the number of test casesgenerated by this table is very large, and then it should cutfactors and corresponding levels.According to the product requirement specification, it cananalysis the correlation between various factors. Because therelevance of the budget time and the budget cycle is close, itcan cut out the factor of the budget time. Similarly thecorrelation of the reported / issued factor and that of numberis also very close, and then can also be cut out. Thus theinput parameters remain four, which are as following: Budget cycle: Annual budget, Monthly budget Budget classification: Sale electricity budget,Purchase electricity budget, Transmission electricitybudget Unit: T grid provincial company, t Grid city, Stategrid Reported / issued number: Reported number, IssuednumberAccording to the above analysis conditions, to determinefour factors, that is the budget cycle, the budget classification,the unit and the reported / issued number, as well as itscorresponding level of 2, 3, 3 and 2. The number ofexperiments is not less than 3 (3-1) 1 7 (s), therefore, itcan be considered to use the orthogonal table L9(34) with 4factors and 3 levels. The factor A, B or C can becorresponding to the orthogonal table L9(34) in any threecolumns arbitrarily, for example, A, B and C are placed in1,2 and 3 column, and then test them line by line and inunlimited order. Mapping the determined factors and levelswith the orthogonal table L9(34), and filling null value, it candraw a set of test case table, such as shown in table 3.TABLE II.THE ORTHOGONAL TABLE CORRESPONDING TO TESTPROGRAM - THE TEST CASE ednumberState ertGridcityReportednumberThe analysis of the above process can get 9 test cases,which can cover the critical points of the demandsubstantially.IV.AUTOMATIC GENERATION TOOL ON THEORTHOGONAL TABLE TEST CASEThere are many advantages in use of the orthogonalexperiment to design test cases. For example, that can savetest time and control the number of test cases generated, andwhich coverage is good. But in process of designingorthogonal experimental test cases, to find the orthogonaltable is very troublesome, so it leads to selecting theorthogonal table with less factor levels. That problem can notbe resolved in the artificial stage. In the actual testing work,it is to use an easier orthogonal table. If it is lack of anautomatic method, the application of the orthogonalexperimental design on test cases is a very complicated anddifficult work, so it is necessary to study and improve themethod in using process carefully.A. Code FrameworkThis is the part of code framework which is designedorthogonal table test cases strategy to the automaticgenerated tool./ / To define a class of orthogonal table:public class UniFormTable{private int m Runs;/ / Rows number is that of rows of the orthogonal table,i.e. the one of tests.private int m FactorLevelCount;// If it equals 1, that is the the level orthogonal tableprivate int[] m Factors;Test conditionTestcase6UnitReported/ ridcityIssuednumberState gridReportednumbertGridcityIssuednumberState gridReportednumber// The factor number is that of columns in the orthogonaltable.private int[] m Levels;// The level number is that any single factor can get themaximum value.private string[] m TableMatrix;// The data of the orthogonal matrix table.private string[] m TableMatrixString;// The block of orthogonal matrix table data. //Case1: The factors number and the levels number arethe same.for (int i 0; i UniFormTableList.Count; i ){if (UniFormTableList[i].FactorLevelCount 1)Published by Atlantis Press, Paris, France. the authors2301
Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013){ //To find the levels and factors number in theorthogonal table is equals to the specified one.if (UniFormTableList[i].Factors[0] MyFactors.Count) // Case2: The levels number is equal, but factors numberis not equal.List UniFormTable FitTableList new List UniFormTable ();for (int i 0; i UniFormTableList.Count; i ){if (UniFormTableList[i].FactorLevelCount 1){if (UniFormTableList[i].Levels[0] MyFactors[0].Levels.Count){ // To find that is more than or equal to the specifiedfactors number and is the most close to the standardorthogonal table. // Case3: Each factors number is not the same.for (int i 0; i UniFormTableList.Count; i ){if (UniFormTableList[i].FactorLevelCount FactorLevelPair.Count) // Outputting test cases:for (int aa 0; aa TestCaseTable.Runs; aa ){for (int bb 0; bb MyFactors.Count; bb a,bb].Trim() " ");}MapOutPut.AppendText("\n");}B. Automation Tools GenerationIt realizes a software, develops a tool of the research andanalyzes on the orthogonal experimental to design test cases.The software provides a solution strategy of threeorthogonal experimental design designing test cases that isthe same as the factor number and the level number, not thesame as one. That realization principle is to read a filecontains a number of orthogonal tables. When the programstarts, it loads the orthogonal table file and analyses eachorthogonal tables, and then finds out factors number andlevels number and stores to the orthogonal table object. Toexplain the inputting contents and to analyses factors andlevels, it can find the matching orthogonal table and productthe corresponding test cases.The main function of the software is to input the factorand the corresponding level, and can automatically analyzethe suitable orthogonal table to product the correspondingtest cases.topic in software testing. The Orthogonal experimentaldesign is a good solution to such problems.Using the Orthogonal experimental design to design testcases is the effective means to reduce test cases, therebyimproving the efficiency of software testing. This method ismost suitable for the software module that contains aplurality of inputting parameters to determine one outputtingresults together. It studies the designing scheme of theorthogonal experimental design to design test cases, and putsforward the solving strategies on multi-factor levels. Toselect the right and strong representative points from a largenumber of test cases, which are more comprehensive andmore objective understanding comprehensive tests, and toselect the most optimal level combinations. This method canavoid one-sidedness and blindness testing, thereby canimprove the efficiency of the software testing and can reducethe cost of it. Practice has proved that such orthogonalexperimental design which is “equilibrium dispersion andneat comparable” is a kind of multi-factor testing andeffective method. If it will put together a number of testcases designing techniques, the effect would be better.With the advancement of technology, the software scaleis much larger and the complexity increases more and muchhigher, which bring the new challenges to testers that mustmaster the scientific test cases designing method to apply tothe software testing in the actual work, and the orthogonaltest method is the best choice by testers undoubtedly.ACKNOWLEDGMENTThank Zhuhai College of Jilin University for supportingthe Focus Professional Development ware testing need analysis and design test cases fromdifferent angles, and thus test the system effectively andscientifically. However, due to the limitation of the testingtime and resources, it is impossible to test the systemcompletely that is limited to test. That how to distributelimited resources to the system scientifically is an important[7]Dashan SUI and Zhenshan CUI, “Application of orthogonalexperimental design and Tikhonov regularization method for theidentification of parameters in the casting solidification process”,Acta Metall. Sin.(Engl. Lett.)Vol.22 No.1 pp13-21 Feb.2009.Fu, Jianping; Lu Minyan; Ruan, Lian; Huang, Baiqiao; “Applicationof experimental design in software reliability test ” , BeijingHangkong Hangtian Daxue Xuebao, vol.34, pp. 1379-1383,December 2008.Lan, B; Feng, P.F. ; Wu, Z. J. ; Yu, D. W. “Determination ofconstitutive equation parameters for orthogonal cutting throughpressure bar tests and FEA method”, Ken Engineering Materials,v499, p56-61, 2012, Anti-Fatigue Deaign and ManufacturingTechnologies.Chan, W.K.; Cheng, M.Y.; Cheung, S.C. ;Tse, T.H. “Automatic goaloriented classification of failure behaviors for testing XML-basedmultimedia software applications: An experimental case study”,Journal of Systems and Software, vol 79, pp.602-612, May 2008.Qiao, Wen-Feng; Zhou, Yong-Min; “Optimizing technologicalparameters of preparation of Bi nanoparticles by orthogonalexperimental method”, Lubrication Engineering, September 2008.Zhou, De-Jian; Huang, Hong-Yan; Peng, Kai-Qiang; “Study onevaluation method for the heat characteristics of IC devices based onthe combination of orthogonal experimental method and responsesurface method”, Proceedings of International Symposium on HighDensity Packaging and Microsystem Integration 2007, HDP'07.Zhang, Qingpia; Zhou, Kan; Gui, Yingchun; Yu, Chengda; Sun, Ning;Li, Shizhong; “Simulation research of laminated solid armature basedon orthogonal experimental method”, 2011 2nd InternationalPublished by Atlantis Press, Paris, France. the authors2302
Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013)Conference on Mechanic Automation and Control Engineering,MACE 2011 – Proceedings, 1616-1618.Published by Atlantis Press, Paris, France. the authors2303
Figure 1. Software modules of Orthogonal Array to Design Test Cases. A. Test Data Description It illustrates the orthogonal experimental design to design test cases as example that it sets input interface on the software query module of report issued in the financial management and control project. For example, adding a
Creating Orthogonal Array. As we will explain shortly, the technique proposed in this paper usually requires dif-ferent orthogonal arrays for di erent problems. e con-struction of orthogonal array is not a trivial task since we do not know whether an orthogonal array of given size exists. Many
6-Pair Orthogonal right angle header connector solution. The connectors enable efficient implementation of Direct-Mate orthogonal and midplane orthogonal architectures. Orthogonal architecture solutions eliminate long, complex traces, via stub effects, simplify signal links and reduce backplane layer count.
Experimental design (DOE) -Design Menu: QCExpert Experimental Design Design Full Factorial Fract Factorial This module designs a two-level multifactorial orthogonal plan 2n–k and perform its analysis. The DOE module has two parts, Design for the experimental design before carrying out experiments which will find optimal combinations of
functions in a separate orthogonal region, as shown in Figure 5.9. Orthogonal regions are a relatively expensive mechanism1 that the current implementation of the QEP event processor does not support. Also, orthogonal regions aren't often the desired solution because they offer little opportunity for reuse. You cannot reuse the
ORTHOGONAL FREQUENCY DIVISION MULTIPLEXING DENGAN MENGGUNAKAN DSK DESIGN AND IMPLEMENTATION ORTHOGONAL FREQUENCY DIVISION MULTIPLEXING SYSTEM 1Dwi Aryanta, 1, 2, 3 Jurusan Teknik Elektro Institut Teknologi Nasional 1dwiaryanta@gmail.com Abstrak OFDM adalah salah satu teknik transmisi yang menggu yang saling tegak lurus (orthogonal
the traditional orthogonal multiple access (OMA) technology. In order to improve user throughput of OMA, user equipments (UEs) of non-orthogonal transmission technology needs to enhance their receivers with interference cancellation capability in order to elim-inate interferences generated by other users. In this thesis, two kinds of non-orthogonal
b. Design of Experiments using Orthogonal Array (OA): Dr. Taguchi has developed a novel experimental method based on Orthogonal Array (OA), which is a method of designing experiments that usually requires only a fraction of the full factorial combination. In the Taguchi design, the array is orthogonal, which means the design is balanced so
– Ossa brevia (tulang pendek): tulangyang ketiga ukurannyakira-kirasama besar, contohnya ossacarpi – Ossa plana (tulang gepeng/pipih): tulangyang ukuranlebarnyaterbesar, contohnyaosparietale – Ossa irregular (tulangtak beraturan), contohnyaos sphenoidale – Ossa pneumatica (tulang beronggaudara), contohnya osmaxilla
