Quality Assurance Best Practice Guide 4

3y ago
61 Views
7 Downloads
1.43 MB
103 Pages
Last View : 1m ago
Last Download : 2m ago
Upload by : Casen Newsome
Transcription

GN3 Quality AssuranceBest Practice Guide 4.0Last updated:Activity:Dissemination Level:Document Code:Authors:07-09-2012SA4PU (Public)GN3-09-184v6Branko Marović (AMRES), Marcin Wrzos (PSNC), Marek Lewandowski (PSNC), Andrzej Bobak (PSNC),Tomasz Krysztofiak (PSNC), Rade Martinović (AMRES), Spass Kostov (BREN), Szymon Kupinski (PSNC),Stephan Kraft (DFN), Ann Harding (SWITCH), Gerard Frankowski (PSNC), Ognjen Blagojević (AMRES),Tihana Žuljević (CARnet), Jelena Đurić (AMRES), Paweł Kędziora (PSNC)

Table of Contents1Introduction12Change Management33Configuration Management54Testing 74.1Test Levels94.2Unit Testing104.3Integration Testing114.4System Testing124.4.1Functional Testing124.4.2Non-Functional Testing134.4.3Security-Oriented Testing164.55Principles of Software Testing174.5.1What to Test For174.5.2How to Test184.5.3Testing Improvement from Bug Fixing204.6Tools Used in Testing204.7Testing of Web Applications214.7.1Performance and Stress Testing214.7.2Automated Functional/Regression Testing254.7.3Web Services Testing284.7.4Tools30Code Analysis325.1Internal Code Reviews325.1.1Review Setup335.1.2Carrying Out a Review345.1.3Handling Review Results375.2Automated Code Quality Checking405.2.1Static Analysis415.2.2Impact of Automated Code Reviews on Team and Developer425.2.3Recommended Configurations for Maven and Jenkins435.2.4Static Code Analysis with Macxim47GN3 Quality AssuranceBest Practice Guide 4.0Document Code:GN3-09-184ii

678Performance Improvement and Optimisation506.1Code Optimisation506.1.1Define Goals and Metrics526.1.2Choose an Appropriate Algorithm536.1.3Design and Use Efficient Structures536.1.4Ensure Data Density and Locality546.1.5Select the Appropriate Array Access Pattern546.1.6Use Compiler Optimisations546.1.7Use Profile Feedback556.1.8Computational Performance Optimisation in Java556.1.9Code Parallelisation566.2Relational Database Optimisation646.3Optimisation of JPA/EJB Usage68Problem Management717.172Bug Tracking778.1Bug Reporting778.1.1Bug Report Principles788.1.2Preliminary Steps788.1.3Tips and Guidelines for Completing a Bug Report788.28.3910Acceptance Criteria Based Software EvaluationDealing with Bugs808.2.1Bug Tracking Software818.2.2Bug Tracking Guidelines82Monitoring the Issue Handling Process83Using QA Testbed Infrastructure869.1QA Testbed Procedures869.2Booking QA Testbed Resources879.3QA Testbed Documentation88Closing SW Projects8910.2Project Closeout Reports9110.2.1 Project Closeout Transition Checklist9110.2.2 Project Closeout Report9210.2.3 Post Implementation Report93Documenting Lessons Learned9310.3GN3 Quality AssuranceBest Practice Guide 4.0Document Code:GN3-09-184iii

References95Glossary98Table of FiguresFigure 3.1: Number of active threads over time22Figure 3.2: Throughput (black) and average response time (blue) in ms23Figure 3.3: Throughput (blue), average response time (green) and errors (red) of systemrecovering from the intense, but short, workload (black)23Figure 3.4: Throughput (blue), average response time (green) and errors (red) of a systemhandling the constant workload (black)24Figure 3.5: Throughput (blue), average response time (green) and errors (red) in a system thatcannot cope with the workload (black)25Figure 5.1: Example of Jenkins build result page45Figure 5.2: Example of CheckStyle warning description on Jenkins46Figure 5.3: Example of Jenkins CheckStyle code viewer46Figure 5.4: Lock ordering deadlock62Figure 7.1: Cumulative created (red) vs resolved (green) issues in JIRA84Figure 8.1: QA Testbed resource requesting workflow87Table of TablesTable 6.1: Acceptance thresholdsGN3 Quality AssuranceBest Practice Guide 4.0Document Code:GN3-09-18476iv

Introduction1Software Quality Assurance (QA) controls the software engineering processes and ensures quality. It does thisby auditing the quality management system in which the software system is created. Software QAencompasses the entire development process, which includes processes such as: Software design. Coding. Documentation. Source code control (revision control). gCode reviews. Testing. Change management. Configuration management. Release management.Software QA controls the quality of software products by reviewing code and documents, and by testingsoftware. The main test types are unit, integration, and system tests.To be able to conduct software QA, the following practices must be in place during all stages of softwaredevelopment (requirements definition, design, review): The QA team must work with the application developers to ensure software QA is applied during therequirement definition phase, when the following are identified: Needs. Architecture. Documentation requirements. Software specifications. Software best practice recommendations should be applied during the design phase, especially in theiteration transition. Standard tools and procedures should also be used. Software QA should be an integrated aspect of software testing. A review process should be designed to support code reviews and audit reports should be issued. An environment for load testing should be designed. Process improvement will include ongoing gathering and evaluation of feedback.GN3 Quality AssuranceBest Practice Guide 4.0Document Code:GN3-09-1841

IntroductionQuality assurance is needed to ensure high standards, consistency and make sure common errors are avoided.Such errors are widely recognised and classified as code smells [SMELL] and anti-patterns [ANTIPATTERN].Common code smells include: duplicate code, large methods or classes, inappropriate intimacy betweenclasses, duplicate methods and so on. Anti-patterns have much wider scope, covering project organisation,methodology, management, analysis, software design, and programming.Anti-patterns differ from usual bad habits or practices by appearing to be beneficial and including documentedsolutions that are well proven and can easily be repeated. The concept of anti-patterns has become so popularthat even the most frequent errors specific to some programming languages such as Java [JAVAANTIP]) werecodified as anti-patterns. While original design patterns [PATTERN] are relevant during design and coding, antipatterns can be used to prevent bad practices in software development and help detect and addressproblematic code during code reviews.Quality assurance tasks are often viewed as boring and time consuming by developers, who do not always seethe direct benefits of following good practices, which are mostly visible in long-term project maintenance. Basedon experience gained during the previous GN2 project, the project’s managers set up a task responsible for acompliance audit of best-practice take-up. Task 2 of Activity SA4 provides reports for the management on anannual basis, to provide an overview of what has been done in each project with regards to software quality.The report will contain results of periodic reviews, recommendations and observed improvement or decline ofquality of software development process.It is not possible for the QA team to deeply understand and review each project. Therefore, the evaluation ofthe quality of projects will be based on information provided by each of the development teams. SA4 Task 2reviews are based on prepared documentation and quality reports, therefore, the QA team uses tools tosupport any software quality assurance tasks.In some cases, developers may not understand some of the quality processes or may not know how toimplement them in their project. Therefore, SA4 T2 offers support for every team interested in putting them intopractice.GN3 Quality AssuranceBest Practice Guide 4.0Document Code:GN3-09-1842

Change Management2Change management is a process of requesting, defining, planning, implementing and evaluating changes to asystem. The ITIL (Information Technology Infrastructure Library) defines this process as the standardisation ofmethods and procedures that are used for efficient handling of changes. A well-defined and comprehensivechange policy provides safe and effective introduction and implementation of new or modified requirements.Such a policy should specify the following change process activities: Change request creation. Technical review. Assessment of importance and effort of change. Authorisation. Implementation. Testing. Releasing and deployment. Change review and closing. Remediation – handling of failed changes.Requests for changes are submitted in writing and entered into a requests or issue-tracking system. They canbe grouped into several general categories: Changes related to the project roadmap. New requirements requested by the client. Reported bugs. Comments from support. Developer proposals. Minor features.If the impact of a change on usage or operation is not obvious, it should also be described. Feature requestsshould not be added or features changed without review and approval by both development team and users.The technical review discusses the aspects and consequences of requested changes, including an analysis ofrelated errors' root causes and an implementation proposal. Dependencies with other features, including userand operational documentation, training materials and the operating environment also need to be described.GN3 Quality AssuranceBest Practice Guide 4.0Document Code:GN3-09-1843

Change ManagementDuring the assessment and evaluation of the change's importance and effort needed, the lead developerdecides whether the request is accepted or rejected based on user input and the technical review. In case ofthe latter, the reason for rejection should be explained. If the request is accepted, appropriate priority should beassigned. The more important the issue, the faster the solution must be delivered. However, the cost of thechange also has to be evaluated. For a more detailed discussion of defect severities and prioritisation isprovided, see Acceptance Criteria Based Software Evaluation on page 72. Change feasibility and its impact oncost and schedule can be expressed by the priority given to individual requests. This represents overallimportance and how much effort and resources dealing with the request would require.Cost depends not only on the immediate effects of the issue and its anticipated solution, but also on thenumber of changes in dependent modules. A change does not only affect the cost and time of the development,but often also extends the system’s functionality, which can affect expected milestones, demo scenarios anddeliverables. If new functionality has been requested, the potential gain should be calculated. If the gain doesnot balance the cost and the change significance is minor or trivial, it can be rejected. If a major change requestis accepted, the implementation plan should be reviewed.Change authorisation includes a dependency check, planning, scheduling, grouping of changes and defining oftimeframe. Although the person who is operationally responsible for this is the project or release manager, it isadvisable to establish a change management board or Change Advisory Board (ITIL: CAB) that comprises allkey stakeholders responsible for authorising major changes.The implementation and delivery of changes is strictly related to release and deployment management. Flexiblescheduling of the changes is possible, if a project is managed using an agile methodology. Not only do thesemethodologies allow for the immediate scheduling of requested changes, they also enable their smoothdelivery, verification and testing. OpenUP, one of the most popular agile methodologies, assumes frequent (e.g.monthly) releases. A new wave of change requests usually occurs after a version of the software is released.Again, these requests are analysed, and an appropriate priority assigned. Issues that are marked as criticalshould be solved as soon as possible, as part of software release maintenance. Other requests can beassigned to later releases. Any time an issue is closed (and a version with the implemented solution isreleased), the requester should test the resolution and verify if expectations have been met.At the highest operational level, change management is expressed through creating, maintaining, updating, andcommunicating the project roadmap. It should contain milestones defining future versions, along withorientation dates and planned key features. The major changes should be placed on a larger timescale, whilethe more detailed content only needs to be specified for the next few releases. The moment reserved in therelease cycle schedule for updating the revision plan is the most appropriate moment to update the projectroadmap. Therefore, these two live documents may be merged into one.The above described normal change management process should be used in common situations and for themost of changes. However, an alternative procedure can be also preauthorised and applied to emergencychanges, which may be caused by some issues, most likely some of those marked as critical.To be effective, change management requires comprehensive configuration management, which allows easytracking and accessing of changes. Its effectiveness can be evaluated using Key Performance Indicators (KPIs)associated with ITIL Service Transition [KPI].GN3 Quality AssuranceBest Practice Guide 4.0Document Code:GN3-09-1844

3Configuration ManagementAccording to the IEEE Std-729-1983 glossary, Configuration Management (CM), is the process of identifyingand defining the items in the system, controlling the change of these items throughout their life cycle, recordingand reporting the status of items and change requests, and verifying the completeness and correctness ofitems (see also IEEE Std-610.12-1990 [IEEE610]. According to ITIL, configuration management is thediscipline of identifying, tracking, and controlling the various components of the IT environment in order toenable decision making, and it is implemented through the deployment of a Configuration Management System(CMS).Software Configuration Management (SCM) is the aspect of CM that is primarily relevant for softwaredevelopers, but should also track hardware and the software environment of software configuration items. Morespecifically, it is a set of procedures for tracking and documenting software throughout its lifecycle to ensurethat all changes are recorded and that the current state of the software is known and reproducible. There arealso some related standards, such as the IEEE 828 Standard (2005) for Software Configuration ManagementPlans, CMMI (Capability Maturity Model Integration), and others.It is important to stay in control of configuration items and the ways they change. Manual configurationmanagement often proves to be problematic – it is better to use technology to help discover, record, andmaintain configuration information. Implementing configuration management enables change, incident, problem,and release management. Configuration management is not the most obvious or easiest function to implementin controlling the software development and maintenance environment. However, its integrating perspective is avery powerful tool for ensuring proper control of available resources. It allows understanding the resources andhow they relate to one another, thus permitting informed decisions to be made. It can be implemented bydeploying and using a comprehensive CMS, but may also start with simple tracking of configuration changes ina versioned document, that is then gradually replaced with a coordinated set of tools controlling changes todifferent elements of the IT environment. It is also good practice to maintain a change log section in functionalspecifications and other important documents.SCM is more specific, as it targets the management of software development projects in which multipledevelopers work on the same code simultaneously, targeting multiple platforms and supporting multiplesoftware versions. It deals with the management of complex software systems that integrate separatecomponents (such as databases, application servers, and participating services). It also deals with softwarerelated assets and infrastructure, which includes configuring software instances at distinct locations, and is,therefore, closely related to change and release management. Elements tracked through SCM include: codeversions, internals documentation and manuals, plans, schedules, state of hardware, network, and softwaresettings.GN3 Quality AssuranceBest Practice Guide 4.0Document Code:GN3-09-1845

Configuration ManagementA comprehensive CMS must enable the team to reproduce a specific instance or its earlier configuration whenneeded. This provides baseline data for reviewing proposed changes and making effective changemanagement decisions.SCM is implemented by combining tools and techniques that control the software development process,including software configuration and usage. These tools address the problems of maintenance andsimultaneous updates of shared data, which are related to the concurrent work of developers on the samecodebase, parallel actions made by infrastructure and service administrators, and release and deploymentprocesses.While SCM specifically deals with the propagation of changes in common code (for example, libraries),including bug fixes and configuration changes to all affected supported versions of the software, other closelyrelated configuration management activities deal with the management of the local assets used by the software.This includes various aspects of the execution environment, such as: local software configuration, hardware,installed operating system, middleware, security and network infrastructure and their configurations. Where thedividing line for the software development team’s responsibilities is drawn depends on the role in the operationof the service provided by the software. If the software is not productised, if few of its instances are in use, or ifthe customers do not want to take control of the environment in which the software is running, it is highly likelythat a software development team has to include the latter aspects into its CMS.In order to deal with these problems, the CMS used should provide sufficient control and information for allelements of software configuration (version, functional status, dependencies and other relationships) bysupporting software and configuration changes, tracking changes and notifying all affected by these changes.SCM therefore revolves around four key aspects [KEY]: Version control. Build management. Release management. Process control.The first three are aspects of process control and described in the Software Developer Guide [SDG]. This guidemaps the processes defined for the software development team to their actual implementation with CMS tools.A more detailed overview of the SCM process, related activities and tasks can be found in ITIL literature onService Asset and Configuration Management [ITIL Service Transition] and at [CMJournal].There are also various SCM best practices [SCMBP]. However, all of them are already present incorresponding [SDG] recommendations. Two key recommendations related to all aspects of SCM are to: Name an owner for each process, policy, document, resource, product, component, package, branch,and task. Describe implemented policies and procedures in ‘live’ documents that are readily available and subjectto continuous update, as required.GN3 Quality AssuranceBest Practice Guide 4.0Document Code:GN3-09-1846

Testing4The purpose of testing is to verify that major design items meet specific functional, performance and reliabilityrequirements. A primary purpose for software testing is to detect software failures, so that defects may beuncovered and corrected. However, testing cannot establish if a product functions properly under all conditions,it can only establish that it does not function properly under specific conditions.Software testing often includes examining and executing code in different environmen

GN3 Quality Assurance Best Practice Guide 4.0 Last updated: 07-09-2012 . 4.4.2 Non-Functional Testing 13 4.4.3 Security-Oriented Testing 16 4.5 Principles of Software Testing 17 4.5.1 What to Test For 17 4.5.2 How to Test 18 4.5.3 Testing Improvement from Bug Fixing 20 4.6 Tools Used in Testing 20 4.7 Testing of Web Applications 21 4.7.1 Performance and Stress Testing 21 4.7.2 Automated .

Related Documents:

critical issues the University has established a Quality Assurance Directorate, which is mandated to develop a Quality Assurance Framework and a Quality Assurance Policy. The Quality Assurance Framework would clearly spell out the Principles, Guidelines and Procedures for implementing institutional quality assurance processes.

Quality Assurance and Improvement Framework Guidance 2 Contents Section 1: Quality Assurance and Improvement Framework 1.1 Overview 1.1.1 Quality Assurance (QA) 1.1.2 Quality Improvement (QI) 1.1.3 Access 1.2 Funding Section 2: Quality Assurance 2.1 General information on indicators 2.1.1 Disease registers 2.1.2 Verification

Software Quality Assurance Plan (SQAP) for the SRR-CWDA-2010-00080 H-Area Tank Farm (HTF) Performance Revision 0 Assessment (PA) Probabilistic Model August 2010 Page 5 of 15 1.0 SCOPE This Software Quality Assurance Plan (SQAP) was developed in accordance with the 1Q Quality Assurance Manual, Quality Assurance Procedure (QAP) 20-1, Rev. 11.

This quality assurance manual specifies the methods to prepare and submit Quality Assurance Process Design Diagram for products and parts to be supplied to NSK by suppliers. 2. Purpose Each supplier should prepare quality assurance process design diagram clearly showing the quality assurance methods used in each products and parts production .

Quality Assurance Representative. The Site Manager will appoint a member of the Project Office to control all Quality Assurance issues including - Assisting the Site Manager to co-ordinate and develop the Quality Assurance Plan. Advise Engineers, General Foremen, Foremen and Chargehands in all matters of Quality Assurance.

Quality assurance or software quality assurance is an integral part of the development process and is used in the IT industry by quality assurance professionals as well as testers. Quality assurance is associated with the concept of dependability. Dependability is, first, a guarantee of increased cybersecurity, reliability and

Quality Assurance and Quality Control (QA/QC) policy. Quality assurance/quality control measures for water treatment utilities refer to a set of activities that are to be undertaken to ensure compliance and above all, ensure that the water is safe for public consumption in a sustainable manner. In general, quality assurance (QA) refers to the

Quality Assurance and Quality Control Chapter 8 8.4 IPCC Good Practice Guidance and Uncertainty Management in National Greenhouse Gas Inventories 8 QUALITY ASSURANCE AND QUALITY CONTROL 8.1 INTRODUCTION An important goal of IPCC good practice guidance is to support the development of national greenhouse gas inventories that can be readily assessed in terms of quality and completeness.