Staying Competitive By Evolving Your FPGA Verification .

Staying competitive byevolving your FPGAverification methodologiesAlex GroveEuropean Application EngineerMentor GraphicsOctober 2016

Agenda Industry Trends and Challenges for FPGA Development— What’s the state of FPGA development within industry— What’s driving the need for better verification practices Verification to the Rescue— Improving quality with SystemVerilog and UVM— Lowering the barrier to entry for UVM— Improve productivity with Questa Verification IP— Where are other FPGA users going with their verificationpractices Final Thoughts Mentor Graphics Corp.Staying competitive by evolving your FPGA verification methodologieswww.mentor.comCompany Confidential

INDUSTRY TRENDS ANDCHALLENGES FOR FPGADEVELOPMENT Mentor Graphics Corp.www.mentor.comCompany Confidential

FPGA Completion to Original ScheduleMajority of FPGA Projects Miss Schedule40%2012: 67% behind schedule2014: 59% behind schedule2016: 65% behind schedule35%Design Projects30%20122014201625%20%15%10%5%0%More than 10%EARLY10% EARLYON-SCHEDULE10% BEHINDSCHEDULE20%30%40%50%Actual FPGA design completion compared to project's original scheduleAhead of scheduleSource: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study4H Foster, WRG Functional Verification Study, November 2014Staying competitive by evolving your FPGA verification methodologiesBehind Schedule Mentor Graphics Corp.www.mentor.comCompany Confidential 50% BEHINDSCHEDULE

FPGA Verification Project TimeFPGA Verification Consumes Majority of Project Time25%Design Projects20%2012: Average 43%2014: Average 46%2016: Average 0%51%-60%61%-70%71%-80%Percentage of FPGA Project Time Spent in VerificationSource: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study5H Foster, WRG Functional Verification Study, July 2016Staying competitive by evolving your FPGA verification methodologies Mentor Graphics Corp.www.mentor.comCompany Confidential 80%

Number of FPGA Iterations in the LabBugs found in lab are 10x more expensive to resolve than if found in simulation45%40%87% of FPGA design projectsrequire multiple iterations in the labDesign Projects35%30%25%20%2014201615%10%5%0%1234567 or moreNumber of FPGA iterations in the labSource: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study6H Foster, WRG Functional Verification Study, July 2016Staying competitive by evolving your FPGA verification methodologies Mentor Graphics Corp.www.mentor.comCompany Confidential

Number of FPGA Bug Escapes to ProductionBugs found late in the development cycle are exponentially more expensive35%78% of FPGA design projectshave non-trivial bugs thatescape into productions30%Design Projects25%20%15%10%5%0%0123452016 FPGA Non-Trivial Bug Escapes to ProductionSource: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study7H Foster, WRG Functional Verification Study, July 2016Staying competitive by evolving your FPGA verification methodologies Mentor Graphics Corp.www.mentor.comCompany Confidential6 or More

Driving the Need for Improved VerificationBusiness needs are pushing improved quality effortsSafety CriticalMission Critical Safety integrity High reliability High security High reliability High security High cost of failureBusiness Critical Time to market Quality affects bottom line Reducing development costDemands better verification practices Mentor Graphics Corp.Staying competitive by evolving your FPGA verification methodologieswww.mentor.comCompany Confidential

Biggest FPGA Verification ChallengesSufficient testing and improving debug efficiency are the biggest challengesOther2014Biggest Verification ChallengesDEFINING APPROPRIATE COVERAGE METRICS2016TIME TO DISCOVER THE NEXT BUGTIME TO ISOLATE AND RESOLVE A BUGMANAGING THE VERIFICATION PROCESSKNOWING MY VERIFICATION COVERAGECREATING SUFFICIENT TESTS TO VERIFY THE DESIGN(Coverage Closure)0%5%10%Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study (Preliminary Results)Staying competitive by evolving your FPGA verification methodologies15%20%25%Study Participants Mentor Graphics Corp.www.mentor.com30%35%Company Confidential40%45%

FPGA Productivity GapFPGA vendors continue with design innovations————— Capacity & Complexity Hardened blocks – ReuseIP - ReuseSOC – ReuseGraphical system builders - AutomationHLS – ity“Design Productivity” alone faster “Time to Market”— Designs must also work and be of high quality to gain adoption— TTM depends on maximizing design and verification productivity Verification must similarly evolve to keep pace— Maximize reuse— Improve automation— Raise the level of abstraction Mentor Graphics Corp.Staying competitive by evolving your FPGA verification tion must evolve to keep pace with design innovationsCompany Confidential

Protocol IP Enables Faster DevelopmentMajority of trafficon/off today’sFPGA is throughprotocol IPBut how do you verify your custom application w doCustomI test myFFTcustomCustomapplication in thissea DMAPCIeGTCustom LogicStandard Protocol Mentor Graphics Corp.Staying competitive by evolving your FPGA verification methodologieswww.mentor.comCompany Confidential

What Makes Functional Verification Difficult?CompressedAudio Single, sequential data streams—————Floating point unitGraphics shading unitDSP convolution unitMPEG decode.Sequential data streams1x number of bugs Multiple, concurrent data streams—————Cross barBus traffic controllerDMA controllerStandard I/F (e.g., PCIe).Concurrent data streams10x number of bugs Mentor Graphics Corp.Staying competitive by evolving your FPGA verification methodologieswww.mentor.comCompany Confidential

Outdated FPGA Verification PracticesAre They Now Impacting You?65% of FPGAprojects are behindschedule78% of designshave productionbugsLate tomarketPoor qualityStrugglingwith safetycriticalHighdevelopmentcosts50% of all designswill be safetycriticalBugs found in labare 10x more costlythan simulationLab testing is slow and ineffectiveDirected testing is not sufficientNeed reuse to stay productiveLack complex IP protocol expertise Mentor Graphics Corp.Staying competitive by evolving your FPGA verification methodologieswww.mentor.comCompany Confidential

VERIFICATION TO THERESCUE Mentor Graphics Corp.www.mentor.comCompany Confidential

Improving By Evolving Verification ethodologyMore testingFormalCDCExhaustiveTestingVerification IPAcceleratedDevelopmentTime Reducing Easy StimulusAppsProtocolAssuranceImproved qualityFind bugs earlyGreater insightFaster debugGreater reuseShorter cyclesReuse Mentor Graphics Corp.Staying competitive by evolving your FPGA verification methodologieswww.mentor.comCompany Confidential

How Do You Improve Verification Productivity?Where do spend your time for FPGA verification?Maximize DebugEfficiencyMore simulations more testingTest Planning43%Testbench DevelopmentCreating Test and Running Simulation21%3%Minimize Time onNon-Value AddDevelopment20%13%DebugOtherSource: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study Mentor Graphics Corp.Staying competitive by evolving your FPGA verification methodologieswww.mentor.comCompany Confidential

Benefits of SystemVerilog and UVMImproving Quality and Schedule with SV and UVMSystemVerilogUVMConstrained randomEnables Benefits of SystemVerilog “Automatic” stimulus accelerates closure Tests things you didn’t think of Reduces scope of SV Enhances SV functionalityFunctional coverageProvides Methodology Provides insight into what is being tested Measures testing completeness and quality Guides how to build a testbench Industry standardAssertionsPromotes Reuse Improves debug efficiency Designers can write to augment verification Reuse within and across projects Reduces TB development effort Mentor Graphics Corp.Staying competitive by evolving your FPGA verification methodologieswww.mentor.comCompany Confidential

FPGA Verification Language Adoption TrendsSystemVerilog is now the leading FPGA verification language70%2012Design Projects60%2014201650%Next Year40%30%20%10%0%VHDLVerilogSynopsys VeraSystemCSystemVerilogSpecman eLanguages Used for Verification (Testbenches)Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study18H Foster, WRG Functional Verification Study, July 2016Staying competitive by evolving your FPGA verification methodologies Mentor Graphics Corp.www.mentor.comC/C OTHERTestbench* Multiple answers possibleCompany Confidential

FPGA Testbench Methodology Adoption TrendsUVM is the clear leader in FPGA testbench methodologies60%201220142016Next YearDesign Projects50%40%30%20%10%0%Accellera UVMOVMMentor AVMSynopsys VMM Synopsys RVMCadence eRMFPGA Methodologies and Testbench Base-Class LibrariesSource: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study19H Foster, WRG Functional Verification Study, July 2016Staying competitive by evolving your FPGA verification methodologies Mentor Graphics Corp.www.mentor.comCadence URMNone/Other* Multiple answers possibleCompany Confidential

FPGA Dynamic Verification Adoption TrendsAdoption of advanced verification techniques continue to grow for FPGA verificationCode coverageAssertions201220142016Functional coverageConstrained-Random Simulation0%10%20%30%40%50%60%FPGA Design ProjectsSource: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study20H Foster, WRG Functional Verification Study, July 2016Staying competitive by evolving your FPGA verification methodologies Mentor Graphics Corp.www.mentor.comCompany Confidential70%

Mentor UVM Framework (UVMF) for Easier UVMMaking the transition to UVM possibleUVM Jumpstart Immediately productive while learning UVM Layer on top of UVM that hides UVM details Allows team to focus on verifying product features Promotes reuse increasing productivityUVM Testbench Generators Code generators to create a UVM testbench in minutes Saves 3-4 weeks of effort on every project UVMF is open source and no cost – Delivered in Questa 10.5Helped over 30 companies adopt UVM 75% are FPGA customers Over half are in mil-aero industry Majority use VHDL for design and have no SystemVerilog experience Mentor Graphics Corp.Staying competitive by evolving your FPGA verification methodologieswww.mentor.comCompany Confidential

The UVM Framework Contents Class library— Defines reuse methodology— Component base classes— Package structure for reuse UVMF generatorScripts— Auto generation of components and test bench— Makefiles with common tool flow operations Examples— Block and chip level benches— Technology integrations Documentation— Users Guide Mentor Graphics Corp.Staying competitive by evolving your FPGA verification methodologieswww.mentor.comCompany Confidential

Number of Designs Projects Completed per YearUVM promotes verification reuse which improves productivity across projects30%Design Projects25%20%15%10%5%0%1234Number of FPGA Designs Completed by Project per YearSource: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study23H Foster, WRG Functional Verification Study, July 2016Staying competitive by evolving your FPGA verification methodologies Mentor Graphics Corp.www.mentor.com5Company Confidential6 or more