UFResearchComputing: Overview Programmingtools#

4/2/15What  Is  Computer  ProgramUF  Research  Computing:OverviewProgramming  toolswww.rc.ufl.edu} A sequence of instructions} Written in a computer language} Perform specified tasks Code C, Fortran, Java, or Python AlgorithmYing g  Environment} www.it.ufl.eduSerial  VS  ParallelSoftware setup for writing, compiling andexecuting programs} Instructions are executed sequentially On a single processor Compilers C, C , Fortran, etc} InterpreterSerial ComputingParallel Computing A problem is broken into independent componentsand can be solved concurrently Each Components execute simultaneously ondifferent processor or threads Matlab, Perl, Python, PHP, etc. Other tools Debuggers Profilers Performance analyzerswww.it.ufl.eduProgramming  Tools  at  RC} } } ce profilers} Compilers available Intel Compiler Suite Online cles/new-user-compiler-basic-usage Modules module spider intel GNU Compiler Collection Online 7.4/gcc/https://gcc.gnu.org/fortran/ Modules Module spider gccwww.it.ufl.eduwww.it.ufl.edu1

4/2/15Intel  Compiler  Suite} Intel  Compiler  SuiteFor serial programs} } For MPI parallel programs Modules module load intel/2013 module load intel/2013 openmpi/1.6.5Usage Usage Compile C code: icc Compile C code: mpicc e.g.: icc –O1 mytest.c –o mytest e.g.: mpicc –O1 mytest.c –o mytest Compile C code: icpc Compile C code: mpicxx e.g.: mpicxx -g mytest.cpp –o mytest e.g.: icpc -g mytest.cpp –o mytest Compile Fortran code: mpif77, mpif90 Compile Fortran code: ifort e.g.: mpif90 –O2 mytest.f90 –o mytest e.g.: ifort –O2 mytest.f90 –o mytest} For OpenMP programs module load intel/2013 icc/icpc/ifort –openmp mytest.c/.cpp/.f90 –o mytestwww.it.ufl.eduIntel  Compiler  Suite} www.it.ufl.eduIntel  Compiler  SuiteBasic compiler options -g: debugging -O#: optimization -O0: no optimization-O1: optimize for speed without increase code size-O2: default, optimize for speed, eg. vectorization-O3: high-level optimizer, prefetch, loop unrolling Support for target: SSE2, SSE3, SSE4.1, SSE4.2, AVX -xtarget: for Intel architecture only -mtarget: for any architecture supporting target,recommended on HiPerGatorSSE: Streaming SIMD Extensions; AVX: Advanced Vector Extensionwww.it.ufl.eduGNU  Compiler  Collection  (GCC)} For serial programswww.it.ufl.eduGNU  Compiler  Collection} For MPI parallel programs Modules Module module load gcc/4.7.2 openmpi/1.6.5 module load gcc/4.7.2 Usage Usage Compile C code: mpicc Compile C code: gcc e.g.: mpicc –O1 mytest.c –o mytest Compile C code: mpicxx e.g.: gcc –O1 mytest.c –o mytest e.g.: mpicxx -g mytest.cpp –o mytest Compile C code: g Compile Fortran code: mpif77, mpif90 e.g.: g -g mytest.cpp –o mytest e.g.: mpif90 –O2 mytest.f90 –o mytest Compile Fortran code: gfortran} e.g.: gfortran –O2 mytest.f90 –o mytestFor OpenMP programs Module module load intel/2013 Usage gcc/g /gfortran –fopenmp mytest.c/.cpp/.f90 –o mytestwww.it.ufl.eduwww.it.ufl.edu2

4/2/15GNU  Compiler  Collection} DebuggersBasic compiler options} Debugging -g: debugging -O#: optimization} Debugger available on HiPerGator Locate and remove errors or abnormalities-O0: default, reduce compilation time-O1: reduce code size and execution time-O2: default, optimize for speed-O3: O2 prefetching, loop unrolling, vectorization, etc.-Ofast: O3 ffast-math and other Fortran relatedoptimization Allinea DDT Intel Inspector XE Valgrind Support for: SSE2, SSE3, SSE4.1, SSE4.2, AVX -mtune cpu-type: generic, native, corei7, core-avx-I -msse, -msse2, -msse3, -msse4.1, -mavx www.it.ufl.eduAllinea  DDT} } Allinea  DDTA graphical distributed debugging tool www.it.ufl.edu} Serial codeMultithreaded codeMultiprocess codeC, C , Fortran, CUDAFlow control Control program progress Static analysis at thread and process level Identify and fix problems} Module requiredData monitoring module load ddt Latest version: 5.0Track variablesDetect memory errorsCheck data calculationTrace pointswww.it.ufl.eduAllinea  DDTwww.it.ufl.eduAllinea  DDTMemory usage analysisFlow control and data monitoringSource: amples.pdfSource: amples.pdfwww.it.ufl.eduwww.it.ufl.edu3

4/2/15Allinea  DDTValgrindMulti-Dimensional Array Viewer} } A memory mismanagement detectorDetect Memory leakDeallocation errorsUse uninitialized memoryRead/Write freed memoryMismatched allocation/deallocation and syntax malloc/new/new[] vs free/delete/delete[]} Cons No bound checking for static arrays Execution specificSource: riablesAndData.htmlwww.it.ufl.eduValgrind} www.it.ufl.eduValgrindRequired modules} Example} Valgrind output module spider valgrind For serial programs module load gcc/4.7.2 valgrind For parallel programs module load intel/2013 openmpi/1.6.5 valgrind module load gcc/4.7.2 openmpi/1.6.5 valgrind} Usage For serial programs, e.g.: valgrind --leak-check yes myprog input For parallel programs e.g.: mpirun –np 4 valgrind --leak-check yes myprog inputSource: uick-start.preparewww.it.ufl.eduPerformance  Profiling  and  Evaluation} Dynamic program analysis that revealsfeatures in the program and interactionsbetween software and hardware } The frequency and duration of function calls & loopsCalculation speedMemory usageCommunication patternMultithreading analysisLoad balance and efficiencywww.it.ufl.eduPerformance  Profiling  &  Evaluation  Tools} Allinea MAP} TAU module load ddt module load intel/2013 tau module load intel/2013 openmpi/1.6.5 tau} Intel Toolkit VTune Amplifier XE Advisor XEGoal: Code optimizationwww.it.ufl.eduwww.it.ufl.edu4

4/2/15Allinea  MAP} Allinea  MAPA graphical distributed profiler Detect bottlenecks at source level Support Serial, multithreaded, and multiprocess code C, C , Fortran 90 Performance metrics Memory usage Floating Point calculations MPI call performance Easy to use, no instrumentation or recompilationSource: -profiling/MAP/www.it.ufl.eduTAU} } www.it.ufl.eduTAUTuning and Analysis UtilitiesIntegrated Performance Toolkit} Performance analysis Time consumption – identify bottlenecks Performance profiling and tracing At source level: loops, functions calls I/O, communication, computation Both serial and parallel programs Source code instrumentation Memory usage Adds probes to collect performance data Manual or automatic Allocation/deallocation, cache misses Floating point operations Comprehensive data analysis and management FLOP rate Include hardware performance counters via PAPI Cross process comparisons Performance data visualization Load balancing and scaling Paraprof Open source .it.ufl.eduTAU} www.it.ufl.eduTAUIdentify potential bottlenecks} Source: Sameer Shende, TAU Performance System TutorialHow much time is spent in each routine?Source: Sameer Shende, TAU Performance System Tutorialwww.it.ufl.eduwww.it.ufl.edu5

4/2/15TAU} TAU  ParaProf  3D  Profile  BrowserProgram call graphSource: Sameer Shende, TAU Performance System TutorialSource: Sameer Shende, TAU Performance System Tutorialwww.it.ufl.eduIntel  Parallel  Studiowww.it.ufl.eduIntel  Inspector  XE} } Graphical software “Correctness Analyzer”Memory Error } Memory t memory usage,Illegal memory accessUninitialized memory readThreading error Data races DeadlocksSource: Intel Parallel Studio Webinarwww.it.ufl.eduIntel  Inspector  XE} www.it.ufl.eduIntel  Inspector  XEDouble click on Problem tonavigate to source (nextslide)Key featuresPowerful filtrationfeature!Code locationsgrouped into Problemsto simplify resultsmanagementSource: Intel Parallel Studio WebinarSource: Intel Parallel Studio Webinarwww.it.ufl.eduwww.it.ufl.edu6

4/2/15Intel  VTune  Amplifier  XE} Intel  VTune  Amplifier  XEPerformance analyzer FunctionCPU timeBottleneck identificationSource level performance dataThread profilingHardware event based samplingSupport FunctionhotspotThreadtimelineC/C , Fortran, Java, .NETGCC, Intel Compiler, WindowsSerial and parallel programsLinux and WindowsCall stackSource: Intel Parallel Studio Webinarwww.it.ufl.eduIntel  VTune  Amplifier  XEwww.it.ufl.eduIntel  Advisor  XESource view – identify hotspots} Threading design and prototyping Analyze, design, tune the threading design inmultitheaded program Identify the code location for parallelism Identify synchronization errors Predict threading errors and scaling Separate design and implementation} Support C, C , Fortran, C# Linux, WindowsSource: Intel Parallel Studio Webinarwww.it.ufl.eduIntel  Advisor  XE} Intel  Advisor  XE1. SurveyTypical workflow www.it.ufl.eduSurveyAdd annotationsModel performance suitabilityCheck correctnessAdd parallel frameworkSource: Intel Parallel Studio Webinarwww.it.ufl.eduwww.it.ufl.edu7

4/2/15Intel  Advisor  XEIntel  Advisor  XE2. Add Annotation3. Add AnnotationEstimated OverallSpeed-upRecommendedImprovementScalability GraphSource: Intel Parallel Studio WebinarSource: Intel Parallel Studio Webinarwww.it.ufl.eduIntel  Advisor  XEwww.it.ufl.eduIntel  Advisor  XE4. Check correctness5. Add parallel framework4 Memory reuseconditions found!Observations helpidentify problemAnalyze your design for errorsSource: Intel Parallel Studio WebinarSource: Intel Parallel Studio Webinarwww.it.ufl.eduwww.it.ufl.eduUF  Research  Computing} Help and Support https://my.it.ufl.edu For any kind of question or help requests http://wiki.rc.ufl.edu Documents on hardware and software resources Various user guides Many sample submission scripts http://rc.ufl.edu Frequently Asked Questions Account set up and maintenancewww.it.ufl.edu8

Source: Intel Parallel Studio Webinar 4 Memory reuse conditions found! Observations help identify problem Analyze your design for errors www.it.ufl.edu IntelAdvisorXE 5. Add parallel framework Source: Intel Parallel Studio Webinar www.it.ufl.edu ! Help and Support https://my.it.ufl.edu ! For any kind of question or help requests