SoC Verification Methodology - 國立中正大學資工系
SoC Verification MethodologyProf. Chien-Nan LiuTEL: 03-4227151 ext:4534Email: jimmy@ee.ncu.edu.tw1
OutlinellllVerification OverviewVerification StrategiesTools for VerificationSoC Verification Flow2
What is Verification ?lllA process used to demonstrate the functionalcorrectness of a designTo making sure your are verifying that you areindeed implementing what you wantTo ensure that the result of some transformationis as listSource : “Writing Test Benches – Functional Verification of HDL Models” by Janick Bergeron, KAP, 2000.3
Verification ProblemsllllllWas the spec correct ?Did the design team understand the spec?Was the blocks implemented correctly?Were the interfaces between the blockscorrect?Does it implement the desiredfunctionality? 4
Testing v.s. VerificationlTesting verifies manufacturing– Verify that the design was manufactured correctlyHW tTestingSiliconSource : “Writing Test Benches – Functional Verification of HDL Models” by Janick Bergeron, KAP, 2000.5
SoC Design VerificationlUsing pre-defined and pre-verifiedbuilding block can effectively reduce theproductivity gap– Block (IP) based design approach– Platform based design approachlBut 60 % to 80 % of design effort is nowdedicated to verification6
An Industrial ExampleExtended SimulationSystem SimulationEquivalence CheckingASIC TestbenchesEmulation SupportEmulation SoftwareVerificationDesignBEH ModelBottleneck !!High Level DesignTimingAnalysisDFTRTL and Block TestSynthesisSource : “Functional Verification on Large ASICs” by Adrian Evans, etc., 35th DAC, June 1998.7
Verification ComplexitylFor a single flip-flop:– Number of states 2– Number of test patterns required 4lFor a Z80 microprocessor ( 5K gates)– Has 208 register bits and 13 primary inputs– Possible state transitions 2bits inputs 2221– At 1M IPS would take 1053 years to simulate alltransitionslFor a chip with 20M gates– ?*IPS Instruction Per Second8
When is Verification Complete ?lSome answers from real designers:– When we run out of time or money– When we need to ship the product– When we have exercised each line of the HDL code– When we have tested for a week and not found anew bug– We have no idea!!lDesigns are often too complex to ensure fullfunctional coverage– The number of possible vectors greatly exceeds thetime available for test9
Typical Verification weekWeeks10
Error-Free Design ?lAs the number of errors left to be founddecreases, the time and cost to identifythem increaseslVerification can only show the presenceof errors, not their absencelTwo important questions to be solved:– How much is enough?– When will it be done?11
Reference BooklSystem-on-a-Chip VerificationMethodology and TechniqueslbyPrakash RashinkarPeter PatersonLeena SinghCadence Design Systems Inc., USAlpublished byKluwer Academic Publishers, 200112
OutlinellllVerification OverviewVerification StrategiesTools for VerificationSoC Verification Flow13
Verification ApproacheslTop-down verification approach– From system to individual componentslBottom-up verification approach– From individual components to systemlPlatform-based verification approach– Verify the developed IPs in an existing platformlSystem interface-based verification approach– Model each block at the interface level– Suitable for final integration verification14
Advs. of Bottom-up ApproachlLocalitylCatching bugs is easier and faster withfoundational IPs (sub-blocks)lDesign the SoC chip with these highlyconfidence “bug-free” IPs15
Verification EnvironmentTestbenchInput ponse)DUV16
TerminologylVerification environment– Commonly referred as testbench (environment)lDefinition of a testbench– A verification environment containing a set ofcomponents [such as bus functional models (BFMs),bus monitors, memory modules] and theinterconnect of such components with the designunder-verification (DUV)lVerification (test) suites (stimuli, patterns,vectors)– Test signals and the expected response under giventestbenches17
Testbench DesignlllAuto or semi-auto stimulus generator is preferredAutomatic response checking is highlyrecommendedMay be designed with the following techniques– Testbench in HDL– Tesebench in programming language interface (PLI)– Waveform-based– Transaction-based– Specification-based18
Types of Verification Tests (1/2)lRandom testing– Try to create scenarios that engineers donot anticipatelFunctional testing– User-provided functional patternslllCompliances testingCorner case testingReal code testing (application SW)– Avoid misunderstanding the spec.19
Types of Verification Tests (2/2)lRegression testing– Ensure that fixing a bug will not introduceanother bug(s)– Regression test system should be automated Add new tests Check results and generate report Distribute simulation over multiple computer– Time-consuming process when verificationsuites become large20
Bug TrackinglA central database collecting knownbugs and fixeslAvoid debugging the same bug multipletimeslGood bug report system helpsknowledge accumulation21
Bug Rate TrackinglllBug rate usually follow a well-definedcurveThe position on the curve decides themost-effective verification approachHelp determine whether the SoC isready to tape-out22
# of bugs reportedBug Rate Tracking Example1009080706050403020100024681012Time23
Adversarial TestinglFor original designers– Focus on proving the design works correctlylSeparate verification team– Focus on trying to prove the design is broken– Keep up with the latest tools andmethodologieslThe balanced combination of these twogives the best results24
Verification PlanllVerification plan is a part of the design reportsContents– Test strategy for both blocks and top-level module– Testbench components – BFM, bus monitors, .– Required verification tools and flows– Simulation environment including block diagram– Key features needed to be verified in both levels– Regression test environment and procedure– Clear criteria to determine whether the verificationis successfully complete25
Benefits of Verification PlanlVerification plan enables– Developing the testbench environment early– Developing the test suites early– Developing the verification environment in parallelwith the design task by a separate team– Focusing the verification effort to meet theproduct shipment criteria– Forcing designers to think through the time-consuming activities before performing them26
OutlinellllVerification OverviewVerification StrategiesTools for VerificationSoC Verification Flow27
Tools for Verification (1/4)lSimulation– Event-driven: Timing accurate– Cycle-based: Faster simulation time (5x – 100x)– Transaction-based: Require bus functional model (BFM) of eachdesign Only check the transactions between components Have faster speed by raising the level ofabstraction28
Event-Driven SimulationlllTiming accurateGood debugging environmentSimulation speed is slowerScheduleNewEventsMore eventsfor this agateEventsNNTime wheelemptyYFinish29
Cycle-Based SimulationlPerform evaluations justbefore the clock edge– Repeatedly triggered eventsare evaluated only oncein a clock cyclellllClockInputOutputFaster simulation time (5x – 100x)Only works for synchronous designsOnly cycle-accurateRequire other tools (ex: STA) to check timingproblems30
Simulation-Based VerificationlStill the primary approach for functional verification– In both gate-level and register-transfer level (RTL)lTest cases– User-provided (often)– Randomly generatedlHard to gauge how well a design has been tested– Often results in a huge test bench to test large designslNear-term improvements– Faster simulators Compiled code, cycle-based, emulation, – Testbench tools Make the generation of pseudo-random patterns better/easierlIncremental improvements won’t be enough31
Tools for Verification (2/4)lCode coverage– Qualify a particular test suite when applied to aspecific design– Verification Navigator, CoverMeter, lTestbench (TB) automation– A platform (language) providing powerful constructsfor generating stimulus and checking response– VERA, Specman Elite, lAnalog/mixed-signal (AMS) simulation– Build behavior model of analog circuits for systemsimulation– Verilog-A, VHDL-A, 32
Coverage-Driven VerificationlCoverage reports can indicate how much ofthe design has been exercised– Point out what areas need additional verificationlOptimize regression suite runs– Redundancy removal (to minimize the test suites)– Minimizes the use of simulation resourcesllQuantitative sign-off (the end of verificationprocess) criterionVerify more but simulate less33
The Rate of Bug DetectionFunctionalTestingpurgatoryreleaseRate ofdugs foundwith coveragewithout coverageTimesource : “Verification Methodology Manual For Code Coverage In HDL Designs” by Dempster and Stuart34
Coverage AnalysislDedicated tools are required besides the simulatorlSeveral commercial tools for measuring Verilog andVHDL code coverage are available– VCS (Synopsys)– NC-Sim (Cadence)– Verification navigator (TransEDA)lBasic idea is to monitor the actions during simulationlRequire supports from the simulator– PLI (programming language interface)– VCD (value change dump) files35
Analysis ResultsnVerification Navigator (TransEDA)Untested code line will be highlighted36
Testbench AutomationllRequire both generator and predictor in anintegrated environmentGenerator: constrained random patterns– Ex: keep A in [10 100]; keep A B 120;– Pure random data is useless– Variations can be directed by weighting options– Ex: 60% fetch, 30% data read, 10% writelPredictor: generate the estimated outputs– Require a behavioral model of the system– Not designed by same designers to avoidcontaining the same errors37
Analog Behavioral ModelinglA mathematical model written in HardwareDescription LanguagelEmulate circuit block functionality by sensingand responding to circuit conditionslAvailable Analog/Mixed-Signal HDL:– Verilog-A– VHDL-A– Verilog-AMS– VHDL-AMS38
Mixed Signal SimulationlAvailable simulators:CadenceAntrimMentorSynopsys 39
Tools for Verification (3/4)lStatic technologies– Lint checking: A static check of the design code Identify simple errors in the early design cycle– Static timing analysis: Check timing-related problems without input patterns Faster and more complete if applicable– Formal verification: Check functionality only Theoretically promise 100% coverage but designsize is often limited due to high resource requirement40
HDL LinterlFast static RTL code checker– Preprocessor of the synthesizer– RTL purification (RTL DRC) Syntax, semantics, simulation– Check for built-in or user-specified rules Testability checks Reusability checks – Shorten design cycle Avoid error code that increases design iterations41
InspectionllFor designers, finding bugs by carefulinspection is often faster then that by simulationInspection process– Design (specification, architecture) review– Code (implementation) review Line-by-line fashion At the sub-block levellLint-liked tools can help spot defects withoutsimulation– Nova ExploreRTL, VN-Check, ProVerilog, 42
What is STA ?lllSTA static timing analysisSTA is a method for determining if a circuit meetstiming constraints without having to simulateNo input patterns are required– 100% coverage if applicableADQFF1DQZFF2CLKReference :Synopsys43
Formal VerificationllEnsure the consistency with specification forall possible inputs (100% coverage)Primary applications– Equivalence Checking– Model CheckinglllSolve the completeness problem in simulationbased methodsCannot handle large designs due to its highcomplexityValuable, but not a general solution44
Equivalence CheckingSynthesisRTL or NetlistRTL or NetlistEquivalenceCheckinglGate-level to gate-level– Ensure that some netlist post-processing did not change thefunctionality of the circuitlRTL to gate-level– Verify that the netlist correctly implements the original RTL codelRTL to RTL– Verify that two RTL descriptions are logically identical45
Equivalence CheckingllCompare two descriptions to check theirequivalenceGaining acceptance in practice– Abstract, Avant!, Cadence, Synopsys, Verplex,Verysys, llBut the hard bugs are usually in bothdescriptionsTarget implementation errors, not design errors– Similar to check C v.s. assembly language46
Model CheckingRTL delCheckingFormally prove or disprove some assertions(properties) of the designThe assertions of the design should be providedby users first– Described using a formal language47
Model CheckingllEnumerates all states in the STG of thedesign to check those assertionsGaining acceptance, but not widely used– Abstract, Chrysalis, IBM, Lucent, Verplex,Verysys, lBarrier: low capacity ( 100 register bits)– Require extraction (of FSM controllers) orabstraction (of the design)– Both tend to cause costly false errors48
New ApproacheslThe two primary verification methods both havetheir drawbacks and limitations– Simulation: time consuming– Formal verification: memory explosionlWe need alternate approaches to fill theproductivity gap– Verification bottleneck is getting worselSemi-formal approaches may be the solution– Combine the two existing approaches– Completeness (formal) lower complexity (simulation)49
Tools for Verification (4/4)lHardware modeling– Pre-developed simulation models for otherhardware in a system– Smart Model (Synopsys)lEmulation– Test the system in a hardware-liked fashionlRapid prototyping– FPGA– ASIC test chip50
Assistant HardwarelRule of thumb– 10 cycles/s for software simulator– 1K cycles/s for hardware accelerator– 100K cycles/s for ASIC emulatorlHardware accelerator– To speed up logic simulation by mappinggate level netlist into specific hardware– Examples: IKOS, Axis, .51
EmulationlEmulation– Verify designs using real hardware (like FPGA?)– Better throughput in handling complex designs– Inputs should be the gate-level netlist– Synthesizable testbenches required– Require expensive emulators– Software-driven verification Verify HW using SW Verify SW using HW– Interfaced with real HW components– Examples: Aptix, Quicktum, Mentor’s AVS .52
PrototypinglFPGA– Performance degradation– Limited design capacity (utilization)– Partitioning and routing problemslEmulation system– FPGA Logic modeler– Automatic partitioning and routing underEDA SW control– More expensive53
Limited ProductionlllEven after robust verification process andprototyping, it’s still not guaranteed to bebug-freeEngineering samplesA limited production for new macro isnecessary– 1 to 4 customers– Small volume– Reducing the risk of supporting problemslSame as real cases but more expensive54
Comparing Verification OptionsSource : “System-on-a-chip Verification – Methodology and Techniques” by P. Rashinkar, etc., KAP, 2001.55
OutlinellllVerification OverviewVerification StrategiesTools for VerificationSoC Verification Flow56
SOC Verification MethodologySource : “System-on-a-chip Verification – Methodology and Techniques” by P. Rashinkar, etc., KAP, 2001.57
System Verification StepslVerify the leaf IPslVerify the interface among IPslRun a set of complex applicationslPrototype the full chip and run theapplication softwarelDecide when to release for massproduction58
Interesting Observationsl90% of ASICs work at the first silicon but only50% work in the target system– Do not perform system level verification (withsoftware)lIf a SoC design consisting of 10 blocks– P(work) .910 .35lIf a SoC design consisting of 2 new blocksand 8 pre-verified robust blocks– P(work) .92 * .988 .69lTo achieve 90% of first-silicon success SoC– P(work) .9910 .9059
Interface VerificationlInter-block interfaces– Point-to-point– On-chip bus (OCB)– Simplified models are used BFM, bus monitors, bus checkers60
Check System Functionality (1/2)lVerify the whole system by using fullfunctional models– Test the system as it will be used in thereal worldlRunning real application codes (such asboot OS) for higher design confidence– RTL simulation is not fast enough toexecute real applications61
Check System Functionality (2/2)lSolutions– Move to a higher level of abstraction forsystem functional verification– Formal verification– Use assistant hardware: Hardware acceleratorASIC emulatorRapid-prototyping(FPGA)Logic modeler 62
HW/SW Co-SimulationllCouple a software execution environment witha hardware simulatorSimulate the system at higher levels– Software normally executed on an Instruction SetSimulator (ISS)– A Bus Interface Model (BIM) converts softwareoperations into detailed pin operationslllAllows two engineering groups to talk togetherAllows earlier integrationProvide a significant performance improvementfor system verification– Have gained more and more popularity63
System-Level TestbenchlllAll functionality stated in thespec. should be coveredThe success and failureconditions must be definedfor each testPay particular attention to:– Corner cases– Boundary conditions– Design discontinuities– Error conditions– Exception handlingSource : “System-on-a-chip Verification – Methodology and Techniques” by P. Rashinkar, etc., KAP, 2001.64
Timing VerificationlSTA (static timing analysis) is the fastestapproach for synchronous designs– Avoid any timing exceptions is extremely importantlGate-level simulation (dynamic timing analysis)– Most useful in verifying timing of asynchronouslogic, multi-cycle paths and false paths– Much slower run-time performance– Gate-level sign-off65
Design Sign-offllllSign-off is the final step in thedesign processIt determines whether the design isready to be taped out for fabricationNo corrections can be made afterthis stepThe design team needs to beconfident that the design is 100%correct– Many items need to be checkedSource : “System-on-a-chip Verification – Methodology and Techniques” by P. Rashinkar, etc., KAP, 2001.66
Traditional Sign-offlTraditional sign-off simulation– Gate level simulation with precise timing under agiven parallel verification vectors– Verify functionality and timing at the same time(dynamic timing analysis, DTA)– Parallel verification vectors also serve as themanufacturing test patternslProblems– Infeasible for million gates design (take too muchsimulation time)– Fault coverage is low (60% typically)– Critical timing paths may not be exercised67
SoC Sign-off SchemelFormal verification used to verifyfunctionalitylSTA for timing verificationlGate level simulation– Supplement for formal verification and STAlFull scan BIST for manufacturing test68
31 Simulation-Based Verification l Still the primary approach for functional verification –In both gate-level and register-transfer level (RTL) l Test cases –User-provided (often) –Randomly generated l Hard to gauge how well a design has been tested –Often results in a huge test bench to test large designs l Near-term improvements –Faster simulators .
SOC/G&WS 200 Intro to LGBTQ Studies SOC 210 Survey of Sociology SOC/C&E SOC 211 The Sociological Enterprise SOC/C&E SOC/G&WS 215 Gender & Work in Rural Am SOC/ASIAN AM 220 Ethnic Movements in the US SOC/C&E SOC 222 Food, Culture, and Society x Any SOC course with a Social Sciences breadth will satisfy this prerequisite.
LLP. About SSAE 16 Professionals, LLP SSAE 16 Professionals, LLP is a leading provider that specializes solely in SSAE 16 (SOC 1) and SOC 2 readiness assessments, SSAE 16 (SOC 1) and SOC 2 Reports, and other IT audit and compliance reports. Each of our prof
Requisites: Completion of introductory Sociology course (SOC/C&E SOC 140, SOC 181, SOC/C&E SOC 210, or SOC/C&E SOC 211) . be reading close to 100 pages per week. If you are unable or unwilling to do this much reading, you . Each quiz is due by 12:30 PM on the day we will discuss the reading; late .
casa mia ed. soc. soc.coop in pe casa mia ed. soc. soc.coop in pe casa mia ed. soc. soc.coop in pe fall.to salumificio rugiada snc fallimento la maiolica s.r.l. in l ballotti sistemi srl fallimento borghi lorenzo costruzioni fai . bernardi maria teresa geosaving srl fallimento . 5707 2012 uni
complex system chip design, especially for NoC-based SoC, which has been increasingly applied to complex system design. This paper proposes a collaborative verification and testing platform for NoC-based SoC. Based on VMM verification methodology, a hierarchical NoC validation platform is constructed and assigned to the function verification of NoC
Static Technologies “Lint” Checking – Syntactic correctness – Identifies simple errors Static Timing Verification – Setup, hold, delay timing requirements – Challenging: multiple sources SoC Design - ICS, Fall 2010 November 13, 2010 J. A. Abraham Verification of SoC Designs
SOC Policy Applies” elsewhere in this section for additional information.) Reversing SOC Transaction To reverse SOC transactions, providers enter the same information as for a clearance but specify that the entry is a reversal transaction. After the SOC file is updated, provi
Introduction 4 Order Number: 329866-001US 1 Introduction The Intel Quark SoC X1000 processor is the next generation secure, low-power Intel Architecture (IA) SoC for deeply embedded applications. The SoC integrates the Intel Quark SoC X1000 Core plus all the required hardware components to run off- the-shelf operating
