AST Review April 12-13, 2018 - University Of Florida .

2y ago
13 Views
2 Downloads
7.13 MB
172 Pages
Last View : 1m ago
Last Download : 5m ago
Upload by : Jamie Paz
Transcription

AST ReviewApril 12-13, 2018

Center for Compressible Multi-Phase Turbulence1180 Center DriveP.O. Box 116135Gainesville, FL 32611Phone: (352)294-2829Fax: (352) 846-1196Agenda AST Site Visit April 12-13, 2018Thursday April 12, 20187:45Van pick up at University Hilton8:00-9:00Full Breakfast(Review Team and other NNSA personnel will meet in small conference room)9:00-9:05Introductions and opening remarks9:05-10:15Overview, Y5 Plans, Y4 Accomplishments (Jackson, Balachandar)10:15-10:30Coffee break10:30-11:00Integration of experiments and simulations (Rollin)11:00-11:50CMT-nek (Jason Hackl, David Zwick)11:50-1:00Lunch (RT will meet in small conference room)1:00-1:50Exascale (Lam, Stitt)1:50-2:40CS (Ranka, Banerjee)2:45-3:00Coffee break3:00-3:50V&V and UQ (Haftka, Park, Kim)4:00-5:15Lightning Round of Students’ Posters/ Poster Session (light refreshments served)5:15-6:30RT Caucus6:30-8:00Dinner (Faculty, Staff and Visitors; transportation will be provided for all visitorsto the University Hilton)

Friday April 13, 20187:45Van pickup at University Hilton8:00-9:00Full Breakfast(Review Team and other NNSA personnel will meet in small conference room)9:00-10:30Student Presentations – IGiselle Fernandez (15 mins)Chandler Moore (15 mins)Goran Marjanovic (15 mins)Kyle Hughes (15 mins)Heather Zunino (ASU; 30mins)10:30-10:45Coffee Break10:45-11:15Student Presentations – IIKeke Zhai (15 mins)Sai Chenna (15 mins)11:15-12:15Y6 Plans and Center Response to RT Questions (PI Team)12:15-4:00Lunch (RT will meet in small conference room)Private RT deliberations (small conference room)Discussions between Center Management and AST as appropriate (largeconference room)4:00-4:30RT Summary for Center Management (large conf. room)4:30Review ends

Center for Compressible MultiphaseTurbulenceAST Review April 12-13, 2018 Attendee ListFacultyS. Balachandar “Bala”Rafi HaftkaNam-Ho KimHerman LamSanjay RankaGreg StittTom JacksonSiddharth Thakur “ST”Bertrand RollinJu ZhangUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaEmbry-RiddleFlorida Institute of Review TeamSam Schofield (Chair)Brian Carnes (V&V/UQ)Robert Clay (CS)Gabe Rockefeller (CS)Kambiz Salari (Physics)Erik Vold (Physics)Jeremiah Wilke ina MacalusoBob VoigtJohn FeddemaFernando GrinsteinAna Alan KuhlBrian TaylorGreg WeirsLLNLEglin weirs@sandia.govResearch StaffTania BanerjeeKei FujisawaJason HacklNguyen Tri NguyenChanyoung ParkUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of Floridatmishra@cise.ufl.edufujisawa ocean il.comcy.park@ufl.edu

Center for Compressible MultiphaseTurbulenceStudentsRyan BlanchardSai ChennaPaul CrittendenBrad DurantGiselle FernandezMohamed GadouJoshua GarnoTrokon JohnsonTanner JonesKyle HughesRahul KoneruTadbhagya KumarAdeesha MalaviGoran MarjanovicYash MehtaChandler MooreAravind NeelakantanSamaun NiliBrandon OsborneFrederick OuelletCarlo PascoeRaj RajagoplanBen ReynoldsChad SaundersShirly SpathPrashanth SridharanCameron StewartKeke ZhaiYiming ZhangHeather ZuninoDavid ZwickUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaUniversity of FloridaArizona State UniversityUniversity of Administration StaffHollie StarrUniversity of Floridahstarr@ufl.edu

Center for Compressible Multiphase TurbulenceCCMTCCMTOverview & IntegrationY4 AccomplishmentsY5 PlansT.L. JacksonS. BalachandarCCMTAST Meeting AgendaThursday Overview, Integration, Y5 Plans, Y4 Accomplishments (Jackson, Balachandar) Integrated Simulations: Transitioning from RocFlu to CMT-nek (Rollin) CMT-nek (Hackl, Zwick) Lunch BE Results at Scale (Lam, Stitt) CMT-nek: CS Updates (Ranka) V&V and UQ (Park) Student Lightning Round Poster Session RT Caucus Dinner 6:30-8:00 (University Hilton; Dogwood Room)Friday Student Presentations (7) Y6 Plans and Response to RT Questions (PI Team) Lunch RT Deliberations/SummaryCCMT2Page 1 of 167

Center for Compressible Multiphase TurbulenceLeadershipPhysics and Code DevelopmentS. (Bala)SiddharthBalachandar Thakur ldAdrianCharlesJenkinsUQ and ryStittScottParkerUF members in redCCMT3Research Staff & Senior PhD StudentsTaniaBanerjeeCCMTNguyen T.NguyenAngelaDiggs(Eglin Page 2 of 167

Center for Compressible Multiphase TurbulenceCurrent Students (Undergraduate & Li y nOsborneHeatherZunino(ASU)DavidZwick5Internship Program – Completed (16) Heather ZuninoLANLMay-Aug, 2014Dr. Kathy Prestridge Kevin ChengLLNLMay-Aug, 2014Dr. Maya Gokhale Nalini KumarSandiaMarch-Aug, 2015Dr. James Ang Christopher HajasLLNLMay-Aug, 2015Dr. Maya Gokhale Christopher NealLLNLJune-Aug, 2015Dr. Kambiz Salari Carlo PascoeLLNLJune-Aug, 2015Dr. Maya Gokhale Giselle FernandezSandiaOct-Dec, 2015Drs. Gregory Weirs &Vincent Mousseau Justin MathewLANLMay-Aug, 2015Dr. Nick Hengartner David ZwickSandiaMay-Aug, 2016Drs. John Pott &Kevin RuggirelloCCMT6Page 3 of 167

Center for Compressible Multiphase TurbulenceInternship Program – Completed (16) Goran MarjanovicSandiaAug-Nov, 2016Drs. Paul Crozier &Stefan Domino Georges AkikiLANLMay-Aug, 2016Dr. Marianne Francois Paul CrittendenLLNLSpring, 2017Drs. Kambiz Salari &Sam Schofield Mohamed GadouLANLSummer, 2017Dr. Galen Shipman Trokon JohnsonLANLSummer, 2017Drs. Cristina Garcia- Cardona,Brendt Wohlberg, Erik West Yash MehtaLLNLSummer, 2017Dr. Kambiz Salari Kyle HughesLANLFall, 2017Dr. Kathy PrestridgeCCMT7Internship Program – Not Completed Summer 2018 Prashanth Sridharan (MAE, Physics); LANL Fred Ouellet, PhD (MAE, Physics and UQ); LANL Brad Durant, PhD (MAE, Physics and UQ) Joshua Garno, PhD (MAE, Physics and UQ) Chandler Moore, PhD (MAE, Physics) (NSF Fellowship)CCMT8Page 4 of 167

Center for Compressible Multiphase TurbulenceGraduated Students & Postdocs Kevin Cheng, MS (2014), Dr. Alan George, ECE Hugh Miles, BS (2015), Dr. Greg Stitt, ECE Chris Hajas, MS (2015), Dr. Herman Lam, ECE Angela Diggs, PhD (2015), Dr. S. Balachandar, MAE Bertrand Rollin, Postdoc in thru August 2014 Currently employed at Eglin AFB and working with centerAssistant Professor, Embry Riddle, Daytona Beach FL Mrugesh Shringarpure, Postdoc in thru January 2016 Subbu Annamalai, PhD (2015), Dr. S. Balachandar, MAE; Postdoc in thru March 2017 Parth Shah, MS (2016), Drs. H. Lam and G. Stitt Georges Akiki, PhD (2016), Dr. S. Balachandar, MAE; Postdoc thru March 2017 Research Engineer, ExxonMobil, Houston TXSenior Systems Engineer, Optym, Gainesville FLPostdoctoral Associate, LANLCCMT9Graduated Students & Postdocs Nalini Kumar, PhD (2017), Dr. H. Lam, ECE Ajay Ramaswamy, MS (2017), Drs. H. Lam and G. Stitt Justin Matthew, MS (2017), Drs. Haftka and Kim, MAE Yiming Zhang, PhD (2018), Drs. Haftka and Kim,MAE Intel, Santa Clara CAProctor & Gamble, Cincinnati OHGE Global Research, Niskayuna, NYCCMT10Page 5 of 167

Center for Compressible Multiphase TurbulencePhD Students Expected Graduation Dates 2018 Paul CrittendenGiselle FernandezMohamed GadouKyle HughesGoran MarjanovicYash MehtaCarlo PascoePrashanth SridharanHeather Zunino2019 2020 Brad DurantJosh Garno2021 Sai ChennaTrokon JohnsonAdeesha MalavirChandler MooreAravind NeelakantanRaj RajagoplanRahul KoneruSamaun NiliFred OuelletKeke ZhaiDavid ZwickCCMT11Additional Information Additional Graduate Program Announcements David Zwick – NSF Fellowship Graduate Program (Aug 2016) Georges Akiki - MAE Best Dissertation Award (TSFD; May 2017) Chandler Moore – NSF Fellowship Graduate Program (Aug 2017)Other metrics (Y1 – Y4) Publications: 134 Presentations: 94Deep Dive Workshops Exascale & CS Issues, Feb 3-4, 2015, University of Florida Multiphase Physics, Oct 13-14, 2016, Tampa FL CMT-nek/nek5000, April 17-18, 2018, Tampa FL Multiphase Deep Dive 2; Stanford lead with Florida co-lead; Fall 2018Center Webpage http://www.eng.ufl.edu/ccmt/CCMT12Page 6 of 167

Center for Compressible Multiphase TurbulenceAdditional Information CMT-nek/nek5000 April 17-18, 2018, Tampa FL 24 talks 4 from Florida – expanding nek500033 participants DOE laboratories represented 3 Countries (US, Canada, Sweden)CCMT13Additional Information Internal Workshops “Dakota - Tutorial” –organized by Chanyoung Park, February 19, 2015 "Good Software Engineering Practices and Beyond" – organized by BertrandRollin, Feb 19, 2015. “A Boot Camp on CMT-nek” – organized by B. Rollin and J. Hackl, November29, 2017CMT-nek Work Camp, 2017CCMT14Page 7 of 167

Center for Compressible Multiphase TurbulenceEducational ProgramsInstitute for Computational Science (ICE) Course in Verification, Validation and Uncertainty Quantification taught every third semester (N. Kim, R. Haftka)Yearly a specialized course for HPC for computational scientists (as part of the Computational Engineering Certificate) (S. Ranka) Fall, 2016, 2018 – new graduate course on multiphase flows (S. Balachandar) Discusses exascale challenges and the NGEE work in the reconfigurable computingcourse (EEL5721/4720) and digital design (EEL4712) (H. Lam, G. Stitt) Uses the CCMT center as a motivational example in Introduction to Electrical andComputer Engineering (EEL3000) (H. Lam, G. Stitt) EEL6763 (Parallel Computer Architecture) (Ian Troxel)CCMT15Management: Tasks and TeamsThe Center is organized by physics‐based tasks and cross‐cutting teams, ratherthan by faculty and their research groupsHour XXXXUQX CCMT Weekly interactions (black); Regular interactions (red)Teams include students, staff, and facultyAll staff and large number of graduate students located on 2nd floor of PS&T BuildingConstruction in PS&T Building to add 6 new office spaces for studentsAll meetings held in PS&T Building16Page 8 of 167

Center for Compressible Multiphase Turbulence4th Annual CCMT Picnic (March, 2018)CCMT17Outline Background Scope of the center Y4 accomplishments Y5 plans RocFlu to CMT-nek transition Integration and timelineCCMT18Page 9 of 167

Center for Compressible Multiphase TurbulenceDemonstration ProblemCCMT19Prediction MetricsPM‐1: Blast WaveLocationPM‐2: Particle FrontLocationPM‐3: Number of InstabilityWavesPM‐4: Amplitude ofInstability WavesCCMT20Page 10 of 167

Center for Compressible Multiphase TurbulenceSequence of EventsCompaction/collision phaseMetalparticlesExplosivematerialHot, dense,high pr gasShock waveDispersion phaseDetonation phaseCCMT21Physical Models – Sources of ErrorT8:Deformation modelCompaction/collision phaseT4:Interaction modelT5:Compaction modelMetalparticlesExplosivematerialHot, dense,Shock wavehigh pressure gasDispersion phaseDetonation phaseT1:Detonation modelT2:Multiphase turbulence modelT3:Thermodynamic & transport modelT6:Point particle force modelT7:Point particle heat transfer modelCCMT22Page 11 of 167

Center for Compressible Multiphase TurbulenceMultiscale Integration StrategyCCMT23Updated Scope Our focus will be on– Rayleigh‐Taylor (RT) and Richtmeyer‐Meshkov (RM) instabilities– Gas‐particle coupling– Mixing at the rapidly expanding material front– Self‐assembly of explosive‐driven particles– Reduced emphasis on initial compaction Will avoid the following complications– Free‐shear and wall turbulence (stay away from boundaries)– Detonation physics (use simple, well‐studied explosives)– Fragmentation or atomization physics (avoid casing, liquids)– Reactive physics (use non‐reactive metal particles)CCMT24Page 12 of 167

Center for Compressible Multiphase TurbulenceSources of Errors & Uncertainties T1: Detonation modeling T2: Multiphase turbulence modeling T3: Thermodynamics & transport properties T4: Particle-particle interaction modeling T5: Compaction modeling T6: Force coupling modeling T7: Thermal coupling modeling T8: Particle deformation and other complex physics T9: Discretization and numerical approximation errors T10: Experimental and measurement errors & uncertaintiesKey FocusAdvance state-of-the-art- Multiphase turbulence- Force coupling modelCCMT25Uncertainty Budget – Overall aleCharacterization& CalibrationT4T2ASU MesoscaleExperiments&SimulationsT4T5Sandia shockEglin tionsT6Eglin cle BedCharacterizeParticle CurtainT2 – Turbulence modeling T5 – Compaction modeling T6Shock-TubeTrackCharacterizeParticle BedExperimentalError & UncertaintyMacroscale U/E QuantificationCharacterizeParticles AfterDetonationExplosiveTrackCalibration ofExplosionT4 – Particle interaction modeling T6 – Force coupling modeling CCMT26Page 13 of 167

Center for Compressible Multiphase Turbulence4 Micro/Meso Campaigns & Target Models Sandia shock‐tube– T6: Force coupling and T4: Particle‐particle interaction ASU expansion fan– T2: Multiphase turbulence and T4: Particle‐particle interaction Eglin microscale– T6: Force coupling Eglin mesoscale gas‐gun– T5: Compaction Demonstration problem– Yearly hero runCCMT27Uncertainty Reduction WorkflowExperimentsExperimental inputSimulationsInput uncertaintyTarget modelimprovementLarge?CCMTEmpty Success(Small error, butLarge Uncertainty)Target model errorMeasurement / PredictionMeasurement / PredictionMeasured MetricsComputed MetricsUseful Failureemodel eotheremodel 0Control ParameterControl Parameter28Page 14 of 167

Center for Compressible Multiphase TurbulenceUncertainty Budget WorkflowExperimentsExperimental inputSimulationsInput uncertaintyForensicMeasured MetricsSampling uncertaintyMeasurement uncertaintyMeasurement ?Target modelimprovementTarget model errorComputed MetricsPropagated uncertaintyUncertaintyErrorStochastic variabilityDiscretization errorNeglected B Workflow ‐ Experiment Worksheet Experimental input– Shock properties, particle properties, curtain properties, Input uncertainty– Quantified uncertainties in all the above Prediction metrics– PM1: Shock position, PM2: Upstream and downstream curtain Uncertainty & error quantification (UQ)– Error in PM1 and PM2 obtained from Schlieren– Error in X‐ray image particle volume fraction Uncertainty & error reduction (UR)– Perform new experiments without spanwise gapCCMT– Improved measurement to reduce volume fraction uncertainty30Page 15 of 167

Center for Compressible Multiphase TurbulenceHierarchical Error Estimation and UQEglin Macroscale SimulationsVVUUframeworkT?Eglin Macroscale ExperimentsTargetmodelT5T5T2T4T6ASUASU VerticalMesoscaleShocktubeSimulation Exp.ASUASU VerticalMesoscaleShocktubeSimulation Exp.T6T4Eglin MesoscaleSimulationsEglin Gas gunExperimentsSNL MesoscaleSimulationsSNL ShocktubeExperimentsT6EglinEglin in her Simulation Campaigns Microscale simulations of shock contact over structured andrandom array of particles– Testing and improvement of force coupling (T6) Mesoscale simulations of turbulent multiphase jet/plume– Testing and improvement of multiphase LES (T2) Mesoscale simulations of sedimentation– Testing and improvement of particle‐particle interaction model (T4) Mesoscale simulations of controlled instability– Evaluation of PM3 and PM4CCMT32Page 16 of 167

Center for Compressible Multiphase TurbulenceKey Accomplishments Tight integration Empower students and staff PIEP model Machine Learning CMT-nek a versatile multiphase flow code (scalable to O(106) core) Culture of UQ integration BE framework and FPGA Dynamic load balancingCCMT33Y4 Highlights1.Macroscale – Hero Run2.Blastpad & other validation experiments3.CMT-nek development and transition4.Mesoscale – CMT-nek simulation of expansion fan5.Microscale – Shock Contact6.UQ workflow7.Design space exploration with Behavioral Emulation8.Dynamic load balancing of Euler-LagrangeCCMT34Page 17 of 167

Center for Compressible Multiphase Turbulence1: Demonstration Problem (Macroscale)Hero‐3 demonstration simulation (Rocflu)Goal Yearly perform the largest possiblesimulations of the demonstrationproblem and identify improvements tobe made in predictive capabilityYear 4 Used existing code to perform largescalesimulations of the demonstrationproblemQualitative comparison againstexperimental data of Frost (PM1 & PM2)Integrate additional capabilities for thehero runs: real gas EOS, reactive burnPresentation Bertrand RollinCCMT351: Demonstration Problem (Macroscale)CMT‐Hero‐1 demonstration simulationGoal Yearly perform the largest possiblesimulations of the demonstrationproblem and identify improvements tobe made in predictive capabilityYear 4 Used existing code to perform largescalesimulations of the demonstrationproblemQualitative comparison againstexperimental data of Frost (PM1 & PM2)Integrate additional capabilities for thehero runs: real gas EOS, reactive burnPresentation Features: Bertrand Rollin32.4 M Degrees of freedom1 M computational particles0.3 x 0.3 x 0.0015 mtmax 0.3ms32768 MPI ranksCCMT36Page 18 of 167

Center for Compressible Multiphase Turbulence2: Blastpad ExperimentsGoals Obtain validation‐quality experimentalmeasurements of the demonstrationproblemValidation‐quality experiments at microand mesoscalesPerform shock‐tube track micro‐ andmesoscale experimentsYear 4 Blast pad experiments at Eglin AFBDetailed instrumentation for validationSimulation informed experimentsIntegrated UQPresentation Bertrand Rollin, Kyle HughesASUCCMT37Input ParameterSourceExplosive length [mm]AFRL measurementExplosive diameter [mm]AFRL measurement at 5 locationsExplosive density [kg/m3]AFRL calculationExplosive qualityAFRL X-rayParticle diameter [mm]CCMT measurementParticle density [kg/m3]CCMT measurementParticle volume fractionAFRL calculationAmbient pressure [kPa]AFRL weather stationAmbient temperature [C]AFRL weather stationProbe locations [m]CCMT measurementTungsten LinerSEM of singlesteel particle at1000x zoom.SEM of severalsteel particles at100x zoom.Cam 32: Blastpad ExperimentsSteel LinerCCMT38Page 19 of 167

Center for Compressible Multiphase Turbulence3: CMT‐nek DevelopmentGoals Co‐design an exascale code (CMT‐nek) forcompressible multiphase turbulencePerform micro, meso and demonstration‐scale simulationsDevelop & incorporate energy and thermalefficient exascale algorithmsMach 3, ߛ 1.4,Color Pressure, contour temperatureYear 4 Developed and released microscale versionof CMT‐nek for microscale simulationsDeveloped and released mesoscale versionof CMT‐nek for mesoscale simulationsShock capturing with EVMCMT‐nek in nek5000 repositoryPresentation Jason Hackl and David ZwickGoran MarjanovicCCMT39Multiphase SinglephaseFluid-FluidLow-Mach-numberShock wavesTracer -FluidDenseParticle-ParticleMultiphase exascale problems rely on efficient communication andcomputations to maintain both accuracy and scalabilityCCMT40Page 20 of 167

Center for Compressible Multiphase TurbulenceAlgorithmic Scaling Presentation Keke ZhaiCCMT414: Mesoscale Simulations & ExperimentsGoal Perform a hierarchy of mesoscalesimulations to allow rigorous validation,uncertainty quantification andpropagation to the demonstrationproblemYear 4 4‐way coupled simulation with CMT‐nekMesoscale simulations of expansion fanover a bed of particles4‐way coupled simulations with andwithout PIEPBundled simulations for UQPresentation David ZwickCase 1CCMTCase 242Page 21 of 167

Center for Compressible Multiphase Turbulence4: Mesoscale Simulations & ExperimentsGoals Obtain validation‐quality experimentalmeasurements of the demonstrationproblemValidation‐quality experiments at microand mesoscalesPerform shock‐tube track micro‐ andmesoscale experimentsYear 4 Expansion fan experiments at ASUDetailed instrumentation for validationSimulation informed experimentsIntegrated UQPresentation Heather Zunino (ASU)ASUCCMT435: Microscale SimulationsGoals Perform a hierarchy of microscalesimulations at conditions of relevanceDevelop extended point‐particle modelsRigorous validation, uncertaintyquantification and propagationYear 4 Shock propagation over a structured arrayShock propagation over a random arrayShock Contact particlesShock over deformable particlesWith Kambiz Salari (LLNL)CCMT44Page 22 of 167

Center for Compressible Multiphase Turbulence5: Pairwise Interaction Extended Point‐Particle Model CCMTPresentation: Chandler Moore456: UQ WorkflowGoals Identify main uncertainty sources and quantifytheir contributions to the model uncertainty ofthe shock tube simulationReduce uncertainty to focus on model errorUQ and propagation in the context of exascaleemulationJWL mixture‐EOS surrogate for efficientcomputationPresentation Nam‐Ho Kim, Giselle Fernandez, Kyle Hughes1D2DPseudo-turbulence Front Position (mm)Year 4Front Position (mm) Sandia shock tubeUQ WorkflowDevelop UB as the backbone of the CenterUnified application of UB for both physics andexascale emulationFront Position (mm) 3DCCMT46Page 23 of 167

Center for Compressible Multiphase TurbulenceUQ of Eglin MicroscaleU in Density with CI 95%U in Diameter with CI 95%U in Explosive Density with CI 95%22%78%140SimulationParticle location (mm)120100Feb 15 #1Feb 15 #2Oct 14 #1Experiment806040200ExperimentsCCMT050100Time ( sec)150477: Exascale EmulationGoal Nek-5000 on KNL (64 cores)Develop behavioral emulation (BE)methods and tools to support co‐designfor algorithmic design‐space explorationand optimization of key CMT‐bonekernels & applications on futureExascale architecturesYear 4 Enhanced BE methods with networkmodels, interpolation schemes, andbenchmarking for CMT‐bone AppBEOsPerformed large‐scale experiments onDOE platforms with BE‐SST simulatorStarted design space explorationImproved throughput and scalability forFPGAsNek-5000 on KNL (1 core)Presentation Herman Lam, Greg Stitt, Sai ChennaCCMT48Page 24 of 167

Center for Compressible Multiphase TurbulenceDesign Space Exploration: ResultsBE-FPGA SimulationApplication: CMT-nek particle solverArchitecture: Vulcan (512k cores)1 design candidate** – BE-SST* - 12000 minutesEntire space (270K candidates) – BE-FPGA - 100 minutesBE-FPGA Simulations of Vulcan (512k cores)Avg execution time/timestep(seconds) BE-SST Simulation (32k cores)100 runs &100 simulationsCMT-nek Particle-Solver Execution(4k cores)Elements-per-processorCCMT *ran on HiPerGator - 64 cores**100 Monte-Carlo simulations/candidate498: Dynamic Load BalancingGoal Derive computationally intensiveportions of the CMT‐nek code andunderstand its performance, thermaland energy issuesYear 4 Carried out extensive investigation ofperformance and energy issues forCMT‐boneHybrid CPU‐GPU implementation ofCMT‐bone and optimizationThermal aware optimizationDynamic load balancing with particlesPresentation Sanjay Ranka, Keke Zhai67,206 MPI ranks1,867 nodes, 36cores/node900,000 elements1,125,000,000 particlesGrid size: 5x5x5Rarefaction test9.97x improvement inperformanceCCMT50Page 25 of 167

Center for Compressible Multiphase TurbulenceCMT‐nek Transition WorkshopA Boot Camp on CMT-nekNovember 29, 2017Organizers: Bertrand Rollin and Jason HacklAgenda CMT-nek: Anatomy of the Beast Lagrangian Particles in CMT-nekSpeaker: Jason Hackl 1:30 pm – 2:15 pmSpeaker: David Zwick 2:15 pm – 2:45 pmBreak Running CMT-nek Post-Processing and visualization in CMT-nek Lesson Learnt from CCMT’s SimulationsSpeakers: Goran Marjanovic and Brad Durant 3:00 pm – 3:45 pmSpeakers: David Zwick and Brad Durant 3:45 pm – 4:15 pmSpearkers: Fred Ouellet and Yash Mehta 4:15 pm – 4:45 pmCCMT51CMT‐Hero TimelineCMT-Hero 1EVM TuningCMT-Hero 24-wayCouplingCompactionY4-Q1Y4-Q2CMT-Hero 3Real gas EoSOutflow BCLoadBalancingY4-Q3Y4-Q4CMT-Hero 4Reactive BurnSmart StartExact Geom.Better ModelY5-Q1Y5-Q2Y5-Q3Y5-Q4S4a CMT-Hero 1EVM tuningS4b CMT-Hero 2Compaction4-way couplingS4c CMT-Hero 3Real gas EOSOutflow BCLoad BalancingS4d CMT-Hero 4Reactive burnSmart startImproved modelsCCMT52Page 26 of 167

Center for Compressible Multiphase TurbulenceDemonstration Problem RoadmapCCMT53Transition to CMT‐nek – Simulation PlanCCMT54Page 27 of 167

Center for Compressible Multiphase TurbulenceCMT‐nek Micro‐Meso‐Hero TimelineY4-Q1S1Y4-Q2Y4-Q3Y4-Q4Y5-Q1Y5-Q2Y5-Q3Y5-Q4 Goran MarjanovicMicro Sims4-waycouplingLoadBalancingCompactionS2ASU Meso David ZwickOutflow BCImprovedmodelsS3 Josh GarnoEglin MesoEVM tuningSmart startReal gas EOSReactive BurnS4 Rahul Koneru Fred OuelletCMT-HeroCCMT55Major Plans for Y5 Complete CMT-Hero2, CMT-Hero3, and CMT-Hero4 Complete a UB cycle for ASU expansion fan, Eglin mesoscale Propagate uncertainties and perform the UB cycle for the Eglin blastpadexperiment Complete ASU simulation at an unprecedented resolution and detailedcomparison with experimental measurements Finalize load balancing and release it for wide external use Complete the implementation of CMT-nek on CPU-GPU Predict and later test the performance of CMT-nek on “Summit” End-to-end design space exploration of CMT-nek using multiple FPGACCMT56Page 28 of 167

Center for Compressible Multiphase TurbulenceRisks & Mitigation Comprehensive risk assessment and backup plans from each team We have successfully crossed some fundamental risks we started with: Will DGSEMwork for compressible flows with shocks? Will Eglin carry out macroscale experiments? Optimal tuning of EVM for extreme blast conditions Particle compaction is too hard at explosive condition Limit to one or two high fidelity & several lower fidelity simulationsCurrent load balancing strategy may face memory limitation Solve very early time using 2-fluid modelBuilding even a surrogate of the macroscale problem may be too expensive Stay with first order wave-speed viscosity, etc.Experiment domain partitioning strategies that loosely maintain spatial localityBE-SST simulation of beyond 1M MPI ranks Limit to 1M MPI ranksCCMT57CCMTDo you have anyquestions?CCMTPage 29 of 167

Center for Compressible Multiphase TurbulenceCCMTIntegrated Simulations:Transitioning from Rocflu To CMT‐nekBertrand RollinCCMTIntegrated Team MembersBertrand RollinJason HacklCCMTYashMehtaFred OuelletRahul Koneru Joshua GarnoGiselle FernandezNguyen TriNguyenGoranMarjanovicDavid ZwickBrad enChandlerMoore 2Page 30 of 167

Center for Compressible Multiphase TurbulenceFrom Rocflu to CMT‐nekRocfluCMT-nekMethodFinite volumeDiscontinuous GalerkinMeshMixed cells (tetrahedra, hex)Spectral elements (hex)Shock capturingWENO gradient reconstructionArtificial viscosityTime marchingRK3, RK4TVDRK3Point particlemodeling2-way coupled4-way coupledParallel capabilityO(104) MPI ranks, No loadbalancingO(106) MPI ranks, loadbalanced Jason Hackl and David Zwick, 11:00 amCCMT 3CMT‐nek Transition WorkshopA Boot Camp on CMT-nekNovember 29, 2017Organizers: Bertrand Rollin and Jason HacklAgenda CMT-nek: Anatomy of the Beast Lagrangian Particles in CMT-nekSpeaker: Jason Hackl 1:30 pm – 2:15 pmSpeaker: David Zwick 2:15 pm – 2:45 pmBreak Running CMT-nek Post-Processing and visualization in CMT-nek Lesson Learnt from CCMT’s SimulationsSpeakers: Goran Marjanovic and Brad Durant 3:00 pm – 3:45 pmSpeakers: David Zwick and Brad Durant 3:45 pm – 4:15 pmSpearkers: Fred Ouellet and Yash Mehta 4:15 pm – 4:45 pmCCMT 4Page 31 of 167

Center for Compressible Multiphase TurbulenceExplosive‐Driven Particle‐Laden Flows Bring predictive capabilities to particle-laden flow simulations underextreme conditionsCCMT 5Sequence of Events and Physical ModelingT8:Deformation modelCompaction/collision phaseT3: Thermodynamic &transport modelT7: Point particle heattransf

Gainesville, FL 32611 Phone: (352)294-2829 Fax: (352) 846-1196 . “A Boot Camp on CMT-nek” –organized by B. Rollin and J. Hackl, November 29, 2017 CMT-nek Work Camp, 2017 Center for Compressible Multipha

Related Documents:

Mark Fisher, PhD, D(ABMM) Disclosures None. Objectives Discuss current rapid AST methods Evaluate clinical impact of rapid AST Assess future rapid AST technologies. Abbreviations Abx - antibiotics AST –antimicrobial s

AST 2 Reference Guide TP1105 Page 6 Using AST 2 Apple Service Toolkit 2 (AST 2) is a cloud-based diagnostic system to help technicians triage and verify repairs for Mac computers.AST 2 supports iMac (21.5-inch, Mid 2014), all Late 2014 Mac computers, and any subsequently released Mac computers.MacBook Pro (M

AST, AST-L, and AST-PLUS Models 5 Anthony Lifgtates, Inc. 800-482-0003 www.anthonyliftgates.com 1.2 Registration To register your liftgate go to www.anthonyliftgates.com, Customer

B-Band transducers (UST and AST) work with all B-Band A-series preamps. Do not cut, pull, crimp or bend at a sharp angle any B-Band transducers. B-Band UST and AST CANNOT be shortened or altered in any way. This will cause audible hum and void the warranty. Long-time exposure of UST or AST to high temperatures (over 50File Size: 339KB

E.g., tester/timer generation, dependence or optimization specification " Map multiple languages to a single AST ! Input: read in the AST using one syntax ! Output: unparse the AST using

AST T. Rowe Price Natural Resources Portfolio AST Templeton Global Bond Portfolio AST Wellington Management Hedged Equity Portfolio. Join the e-Movement.SM Enroll in e-Delivery TODAY! By enrolling in e-Delivery, you’ll gain secure, online access to important documents with the flexibility to choose which

AST Schroders Multi-Asset World Strategies Portfolio AST T. Rowe Price Asset Allocation Portfolio . AST Goldman Sachs Strategic Income Portfolio 5.9% ASTFIPyramis Quantitative FiveLargestHoldings (%ofNetAssets) Apple, I

3 Presentation A11.N Commercial Space Transportation (AST) RE&D Program Presenter Paul Wilde Summary: Mr. Wilde began with an overview of the AST mission and structure.AST's mission is to ensure the protection of the public, property, national security, and foreign policy interests of the United States during commercial launch and reentry, and to