Parallel Computing With Mathematica UVACSE Short Course
Parallel Computing with MathematicaUVACSE Short CourseE Hall11 Universityof Virginia Alliance for Computational Science and Engineeringuvacse@virginia.eduOctober 8, 2014(UVACSE)October 8, 20141 / 46
Outline1NX Client for Remote Linux Desktop2Parallel Computing with Mathematica3Parallel Computing on the Linux Cluster4References(UVACSE)October 8, 20142 / 46
NX Client for Remote Linux DesktopInstalling and Configuring NX ClientThe NX client provides a Gnome Linux desktop interface to the loginnode of the fir.itc Linux inux-cluster-overview/the-nxclient/(UVACSE)October 8, 20143 / 46
NX Client for Remote Linux DesktopStarting and Configuring NX ClientOnce logged into fir.itc.virginia.edu through NXOpen a terminal from Applications/System Tools/Terminal menuSelect and right-click on Terminal to add to launcherCreate a mathematica directory with mkdir commandStart web browser from icon at top of desktop(UVACSE)October 8, 20144 / 46
NX Client for Remote Linux DesktopDownload Short Course ExamplesDownload the short-course materials atica-at-uva/Follow the links, Parallel Mathematica Parallel Mathematica Short Courseand download 3 files to mathematica directory you create withmkdir commandClassExamples Fa14.zipmathematica-parallel Fa14.pdf(UVACSE)October 8, 20145 / 46
Parallel Computing with MathematicaSolving Big Technical ProblemsComputationally intensive, long-running codesRun tasks in parallel on many processorsTask parallelLarge Data setsLoad data across multiple machines that work in parallelData parallel(UVACSE)October 8, 20146 / 46
Parallel Computing with MathematicaParallel Computing with MathematicaParallel computing in Mathematica is based on launching andcontrolling multiple Mathematica kernel (worker) processes from withina single master Mathematica, providing a distributed-memoryenvironment for parallel programming.Tested on Unix, Linux, Windows, and Macintosh platforms and are wellsuited to working locally on a multi-core machine or in a cluster ofmachines,Parallel computing is now provided as a standard part of Mathematica.(UVACSE)October 8, 20147 / 46
Parallel Computing with MathematicaParallel Computing with MathematicaTo perform computations in parallel, you need to be able to perform thefollowing tasks:start processes and wait for processes to finishschedule processes on available processorsexchange data between processes and synchronize access tocommon resourcesIn the Mathematica environment, the term processor refers to arunning Mathematica kernel, whereas a process is an expression to beevaluated.(UVACSE)October 8, 20148 / 46
Parallel Computing with MathematicaConnection MethodsMathematica can run parallel workers locally on the same machine orremotely on a compute cluster controlled by a resource managementapplication, e.g. PBSPro.Local KernelsThe Local Kernels connection method is used to run parallelworkers on the same computer as the master Mathematica. It issuitable for a multi-core environment, and is the easiest way to getup and running with parallel computation.Cluster IntegrationThe Cluster Integration connection method is used to run parallelworkers on a compute cluster from the master Mathematicaprocess. It integrates with the PBSPro cluster managementsoftware.(UVACSE)October 8, 20149 / 46
Parallel Computing with MathematicaParallel Computing UVACSE)October 8, 201410 / 46
Parallel Computing with MathematicaParallel Computing Functions(UVACSE)October 8, 201411 / 46
Parallel Computing with MathematicaParallel Computing Functions(UVACSE)October 8, 201412 / 46
Parallel Computing with MathematicaParallel Computing FeaturesMathematica supports both task and data parallelism.The main features of parallel computing in Mathematica are:distributed memory, master/slave parallelismwritten in Mathematica, machine independentMathLink communication with remote kernelsexchange of symbolic expressions and programs with remotekernels, not only numbers and arraysvirtual process scheduling or explicit process distribution toavailable processorsvirtual shared memory, synchronization, lockingparallel functional programming and automatic parallelizationsupport(UVACSE)October 8, 201413 / 46
Parallel Computing with MathematicaProgramming Parallel Applications in MathematicaTo demonstrate demonstrate parallel processing in Mathematica, weuse ParallelEvaluate. If this is the first parallel computation, it willlaunch the configured parallel kernels.(UVACSE)October 8, 201414 / 46
Parallel Computing with MathematicaParallel Kernels Status MonitorYou can open the Parallel Kernels Status monitor through theEvaluation Parallel Kernel Status menu selection.(UVACSE)October 8, 201415 / 46
Parallel Computing with MathematicaParallel PreferencesThe default settings of Mathematica automatically configure a numberof parallel kernels to use for parallel computation, as seen through theEvaluation Parallel Kernel Configuration menu selection.(UVACSE)October 8, 201416 / 46
Parallel Computing with MathematicaParallel Preferences(UVACSE)October 8, 201417 / 46
Parallel Computing with MathematicaLaunching and ConnectingMathematica launches parallel kernels automatically as they areneeded, but you can also launch kernels manually with the commandLaunchKernels. This is be useful if you were running in a batchmode.Local KernelsThe Local Kernels connection method supports launching localkernels directly from LaunchKernels by passing it an integer(setting the number of kernels)(UVACSE)October 8, 201418 / 46
Parallel Computing with MathematicaLaunching and ConnectingCluster Integration configuration done through Parallel Preferences(UVACSE)October 8, 201419 / 46
Parallel Computing with MathematicaLaunching and ConnectingCluster Integration configuration done through Parallel Preferences(UVACSE)October 8, 201420 / 46
Parallel Computing with MathematicaLaunching and ConnectingCluster IntegrationThe Cluster Integration connection method is used to run parallelworkers on the compute nodes of a cluster from the masterMathematica.The Cluster Integration connection method supports launchingkernels directly from LaunchKernels. To do so you must firstload the ClusterIntegration‘ package, this is shown below.To launch on a particular cluster you have to pass the name forthat cluster into LaunchKernels.(UVACSE)October 8, 201421 / 46
Parallel Computing with MathematicaLaunching and ConnectingCluster Integration(UVACSE)October 8, 201422 / 46
Parallel Computing with MathematicaSending Commands to Remote KernelsValues of variables defined on the local master kernel are usually notavailable to remote kernels.If you need variable values and definitions carried over to the remotekernels, use DistributeDefinitions or shared variables.(UVACSE)October 8, 201423 / 46
Parallel Computing with MathematicaSending Commands to Remote KernelsRecall that connections to remote kernels, as opened byLaunchKernels, are represented as kernel objects. The commandsbelow take parallel kernels as arguments and use them to carry outcomputations.(UVACSE)October 8, 201424 / 46
Parallel Computing with MathematicaRemote DefinitionsMathematica contains a command DistributeDefinitions thatmakes it easy to transport local variables and function definitions to allparallel kernels.Higher-level parallel commands, such as Parallelize,ParallelTable, ParallelSum, . will automatically distributedefinitions of symbols occurring in their arguments.(UVACSE)October 8, 201425 / 46
Parallel Computing with MathematicaSending Commands to Remote Kernels(UVACSE)October 8, 201426 / 46
Parallel Computing with Mathematica(UVACSE)October 8, 201427 / 46
Parallel Computing with Mathematica(UVACSE)October 8, 201428 / 46
Parallel Computing with Mathematica(UVACSE)October 8, 201429 / 46
Parallel Computing with Mathematica(UVACSE)October 8, 201430 / 46
Parallel Computing with Mathematica(UVACSE)October 8, 201431 / 46
Parallel Computing with MathematicaAutomatically Parallelizing Existing Serial ExpressionsUse Parallelize to have Mathematica decide how to distribute workacross multiple kernels.(UVACSE)October 8, 201432 / 46
Parallel Computing with MathematicaAutomatically Parallelizing Existing Serial ExpressionsThere is a natural trade-off in parallelization between controlling theoverhead of splitting a problem or keeping all the cores busy.(UVACSE)October 8, 201433 / 46
Parallel Computing with MathematicaSending a Command to Multiple KernelsUse ParallelEvaluate to send commands to multiple kernels andwait for completion. Use With to bind locally defined variables beforedistribution.(UVACSE)October 8, 201434 / 46
Parallel Computing with MathematicaImplementing Task-Parallel algorithms(UVACSE)October 8, 201435 / 46
Parallel Computing with MathematicaImplementing Data-Parallel algorithmsParallelMap is a natural way to introduce parallelism using afunctional programming style. When you have a computationallyexpensive function to execute of a large data set, Mathematica canexecute the operations in parallel by splitting the mapping amongmultiple kernels.(UVACSE)October 8, 201436 / 46
Parallel Computing with MathematicaDecomposing a Problem in Parallel Data SetsEstimating Pi Using Monte Carlo Simulation: Serial and Parallel.(UVACSE)October 8, 201437 / 46
Parallel Computing with MathematicaDecomposing a Problem in Parallel Data SetsGraphic of work distribution among the 4 kernels.(UVACSE)October 8, 201438 / 46
Parallel Computing with MathematicaDecomposing a Problem in Parallel Data SetsYou can generate multiple data sets in parallel, then plot or processthem further.(UVACSE)October 8, 201439 / 46
Parallel Computing with MathematicaChoosing an Appropriate Distribution MethodThe parallel primitives Parallelize, ParallelMap,ParallelTable, ParallelDo, ParallelSum, andParallelCombine support an option called MethodIt allows you to specify the granularity of the subdivisions used todistribute the computation across kernels.Use Method "FinestGrained" when the completion time ofeach atomic unit of computation is expected to vary widely.Use Method "CoarseGrained" when the completion time ofeach atomic unit of computation is expected to be uniform.(UVACSE)October 8, 201440 / 46
Parallel Computing with MathematicaVirtual Shared MemoryVirtual shared memory is a programming model that allows processorson a distributed-memory machine to be programmed as if they hadshared memory. A software layer takes care of the necessarycommunication in a transparent way.Mathematica provides functions that implement virtual shared memoryfor these remote kernels.The drawback of a shared variable is that every access for read orwrite requires communication over the network, so it is slower thanaccess to a local unshared variable.Declaring shared variables and functions.(UVACSE)October 8, 201441 / 46
Parallel Computing with MathematicaMathematica Parallel WorkflowThe toolbox enables application prototyping on the desktop with up to16 local workers (left), and with the Mathematica Cluster Integrationpackage(right), applications can be scaled to multiple computers on acluster (subsitute Mathematica for Matlab in figure below).(UVACSE)October 8, 201442 / 46
Parallel Computing on the Linux ClusterScaling Up from the DesktopMathematica’s parallel computing provides the ability to use up to 16local kernels on a multicore or multiprocessor computer.When used together with Cluster Integration package, you can scaleup your application to use any number of kernels running on anynumber of computers.ITS Linux cluster allows for 128 kernels.Alternatively, you can run up to 16 kernels on a single multi-corecompute node of the cluster.(UVACSE)October 8, 201443 / 46
Parallel Computing on the Linux ClusterRunning Mathematica on Cluster Front-end NodeMathematica Parallel Computing jobs can be submitted to the ITCLinux cluster by first logging onto the cluster front-end nodefir.itc.virginia.edu using the NX client.Start up Mathematica from a Linux desktop terminal window.Parallel Mathematica jobs can be submitted from with the Mathematicanotebook interface as well as using PBS command files and theexample scripts show how to setup and submit the jobsDocumentation: Submitting Mathematica Parallel Jobs(UVACSE)October 8, 201444 / 46
Parallel Computing on the Linux ClusterExample Mathematica Scripts(UVACSE)October 8, 201445 / 46
ReferencesReferences1Parallel Computing Tools User ls/tutorial/Overview.html2Parallel Computing: Mathematica de/ParallelComputing.html3Mathematica Cookbook, by Sal Mangano, O’Reilly Press.Need further help? Email uvacse@virginia.edu.(UVACSE)October 8, 201446 / 46
Start up Mathematica from a Linux desktop terminal window. Parallel Mathematica jobs can be submitted from with the Mathematica notebook interface as well as using PBS command files and the example scripts show how to setup and submit the jobs Documentation:Submitting Mathematica Parallel Jobs (UVACSE) October 8, 2014 44 / 46
1 Mathematica Basics This chapter is an introduction to Mathematica.We briefly describe many of the most important and basic elements of Mathematica and discuss a few of the more common technical issues related to using Mathematica.Since our primary goal is to use Mathematica to help us understand calculus, you should not initially spend a great amount of time pouring
Introduction.NET/Link Welcome to .NET/Link, a product that integrates Mathematica and Microsoft's .NET platform.NET/Link lets you call .NET from Mathematica in a completely transparent way, and allows you to use and control the Mathematica kernel from a .NET program. For Mathematica users,.NET/Link makes the entire .NET world an automatic extension to the Mathematica environ-
Dec 09, 2005 · Beginner’s Mathematica Tutorial Introduction This document is designed to act as a tutorial for an individual who has had no prior experience with Mathematica. For a more advanced tutorial, walk through the Mathematica built in tutorial located at Help Tutorial on the Mathematica Task Bar.
Online Help from Mathematica 2 Mathematica Tutorial.nb. Introduction Mathematica is a system for doing mathematics on the computer.It can do numerics,symbolics,graphics and is also a programming language.Mathematica has infinite precision.It can plot functions of a single variable; make
mathematica Remarks This section provides an overview of what wolfram-mathematica is, and why a developer might want to use it. It should also mention any large subjects within wolfram-mathematica, and link out to the related topics. Since the Documentation for wolfram-mathematica is new, you may need to create initial versions of those related .
PROGRAMMING IN MATHEMATICA, A PROBLEM-CENTRED APPROACH 7 1.3. Algebraic computations. One of the abilities of Mathematica is to handle symbolic com-putations. Consider the expression (x 1)2. One can use Mathematica to expand this expression: Expand[(x 1) 2] 1 2x x2 Mathematica can also do the inverse of this task, namely to factorize an expression:
Mathematica. It does not assume any prior knowledge of advanced data analysis or statistical techniques. Familiarity with a programming language may prove to be useful, but it is not essential. For beginners in Mathematica, Chapter 2, Dissecting Data Using Mathematica, provides a short primer on the essentials of Mathematica programming.
wesley long hospital . womack army medical center . 3 overview of goals the student should spend the majority of their time in the inpatient pharmacy learning the roles and responsibilities of every staff member. this is a medication use process and systems based experience, not a clinical experience. students should be encouraged to work with pharmacy technicians, including those .
