Python For Scientists - University Of Delaware

PrefaceI have used computers as an aid to scientific research for over 40 years. During thattime, hardware has become cheap, fast and powerful. However, software relevant to theworking scientist has become progressively more complicated. My favourite textbookson Fortran90 and C run to 1200 and 1600 pages respectively. And then we need documentation on mathematics libraries and graphics packages. A newcomer going downthis route is going to have to invest significant amounts of time and energy in order towrite useful programmes. This has led to the emergence of “scientific packages” such asMatlab or Mathematica which avoid the complications of compiled languages, separatemathematics libraries and graphics packages. I have used them and found them veryconvenient for executing the tasks envisaged by their developers. However, I also foundthem very difficult to extend beyond these boundaries, and so I looked for alternativeapproaches.Some years ago, a computer science colleague suggested that I should take a look atPython. At that time, it was clear that Python had great potential but a very flaky implementation. It was however free and open-source, and was attracting what has turned outto be a very effective army of developers. More recently, their efforts have coordinatedto produce a formidable package consisting of a small core language surrounded by awealth of add-on libraries or modules. A select group of these can and do replicate thefacilities of the conventional scientific packages. More importantly an informed, intelligent user of Python and its modules can carry out major projects usually entrustedto dedicated programmers using Fortran, C etc. There is a marginal loss of executionspeed, but this is more than compensated for by the vastly telescoped development time.The purpose of this book is to explain to working scientists the utility of this relativelyunknown resource.Most scientists will have some computer familiarity and programming awareness, although not necessarily with Python and I shall take advantage of this. Therefore, unlikemany books which set out to “teach” a language, this one is not just a brisk trot throughthe reference manuals. Python has many powerful but unfamiliar facets, and these needmore explanation than the familiar ones. In particular, if you encounter in this text areference to the “beginner” or the “unwary”, it signifies a point which is not made clearin the documentation, and has caught out this author at least once.The first six chapters, plus Appendix A, cover almost everything the working scientist needs to know in order to get started in using Python effectively. My editor andsome referees suggested that I should devote the second half of the book to problems inDownloaded from Cambridge Books Online by IP 132.174.254.72 on Sat Apr 09 18:22:05 BST 1Cambridge Books Online Cambridge University Press, 2016

xiiPrefacea particular field. This would have led to a series of books, “Python for Biochemists”,“Python for Crystallographers”,. . . , all with a common first half. Instead I have chosen to cover just three topics, which however, should be far more widely applicable inmany different fields. Chapter 7 covers four radically different types of ordinary differential equations and shows how to use the various relevant black boxes, which are oftenPython wrappers around tried and trusted Fortran codes. The next chapter while ostensibly about pseudospectral approaches to evolutionary partial differential equations,actually covers a topic of great utility to many scientists, namely how to reuse legacycode, usually written in Fortran77 within Python at Fortran-like speeds, without understanding Fortran. The final chapter about solving very large linear systems via multigridis also a case history in how to use object-oriented programming meaningfully in a scientific context. If readers look carefully and critically at these later chapters, they shouldgain the practical expertise to handle problems in their own field.Acknowledgments are due to the many Python developers who have produced anddocumented a very useful tool, and also to the very many who have published codesnippets on the web, a great aid to the tyro, such as this author. Many of my colleagueshave offered valuable advice. Des Higham generously consented to my borrowing hisideas for the last quarter of Chapter 7. I am especially grateful to Oliver Rinne who readcarefully and critically an early draft. At Cambridge University Press, my ProductionEditor, Jennifer Murphy and my Copy Editor, Anne Rix have exhibited their customaryexpertise. Last but not least I thank the Department of Applied Mathematics and Theoretical Physics, Cambridge for continuing to offer me office space after my retirement,which has facilitated the production of this book.Writing a serious book is not a trivial task and so I am rather more than deeplygrateful for the near-infinite patience of Mary, the “Python-widow”, which made thisbook possible!Downloaded from Cambridge Books Online by IP 132.174.254.72 on Sat Apr 09 18:22:05 BST 1Cambridge Books Online Cambridge University Press, 2016

1IntroductionThe title of this book is “Python for Scientists”, but what does that mean? The dictionary defines “Python” as either (a) a non-venomous snake from Asia or Saharan Africaor (b) a computer scripting language, and it is the second option which is intended here.(What exactly this second definition means will be explained later.) By “scientist”, Imean anyone who uses quantitative models either to obtain conclusions by processingpre-collected experimental data or to model potentially observable results from a moreabstract theory, and who asks “what if?”. What if I analyze the data in a different way?What if I change the model? Thus the term also includes economists, engineers, mathematicians among others, as well as the usual concept of scientists. Given the volume ofpotential data or the complexity (non-linearity) of many theoretical models, the use ofcomputers to answer these questions is fast becoming mandatory.Advances in computer hardware mean that immense amounts of data or evermorecomplex models can be processed at increasingly rapid speeds. These advances alsomean reduced costs so that today virtually every scientist has access to a “personalcomputer”, either a desktop work station or a laptop, and the distinction between thesetwo is narrowing quickly. It might seem to be a given that suitable software will also beavailable so that the “what if” questions can be answered readily. However, this turnsout not always to be the case. A quick pragmatic reason is that while there is a hugemarket for hardware improvements, scientists form a very small fraction of it and sothere is little financial incentive to improve scientific software. But for scientists, thisissue is important and we need to examine it in more detail.1.1Scientific softwareBefore we discuss what is available, it is important to note that all computer softwarecomes in one of two types: proprietary and open-source. The first is supplied by a commercial firm. Such organizations have both to pay wages and taxes and to provide areturn for their shareholders. Therefore, they have to charge real money for their products, and, in order to protect their assets from their competitors, they do not tell thecustomer how their software works. Thus, the end-users have therefore little chance ofbeing able to adapt or optimize the product for their own use. Since wages and taxesare recurrent expenditure, the company needs to issue frequent charged-for updates andimprovements (the Danegeld effect). Open-source software is available for free or atDownloaded from Cambridge Books Online by IP 132.174.254.72 on Sat Apr 09 18:28:46 BST 2Cambridge Books Online Cambridge University Press, 2016

2Introductionnominal cost (media, postage etc.). It is usually developed by computer literate individuals, often working for universities or similar organizations, who provide the servicefor their colleagues. It is distributed subject to anti-copyright licences, which give nobody the right to copyright it or to use it for commercial gain. Conventional economicsmight suggest that the gamut of open-source software should be inferior to its proprietary counterpart, or else the commercial organizations would lose their market. As weshall see, this is not necessarily the case.Next we need to differentiate between two different types of scientific software. Computers operate according to a very limited and obscure set of instructions. A programming language is a somewhat less limited subset of human language in which sequencesof instructions are written, usually by humans, to be read and understood by computers.The most common languages are capable of expressing very sophisticated mathematical concepts, albeit with a steep learning curve. Only a few language families, e.g., Cand Fortran have been widely accepted, but they come with many different dialects, e.g.,Fortran77, Fortran90, Ansi C, C etc. Compilers then translate code written by humansinto machine code which can be optimized for speed and then processed. As such, theyare rather like Formula 1 racing cars. The best of them are capable of breathtakingly fastperformance, but driving them is not intuitive and requires a great deal of training andexperience. Note that compilers need to be supplemented by libraries of software packages which implement frequently used numerical algorithms, and graphics packageswill usually be needed. Fast versatile library packages are usually expensive, althoughgood public domain packages are starting to appear.A racing car is not usually the best choice for a trip to the supermarket, where speedis not of paramount importance. Similarly, compiled languages are not always idealfor trying out new mathematical ideas. Thus for the intended readers of this book thedirect use of compilers is likely to be unattractive, unless their use is mandatory. Wetherefore look at the other type of software, usually called “scientific packages”. Proprietary packages include Mathematica and Matlab, and open-source equivalents includeMaxima, Octave, R and SciLab. They all operate in a similar fashion. Each provides itsown idiosyncratic programming language in which problems are entered at a user interface. After a coherent group of statements, often just an individual statement, has beentyped, the package writes equivalent core language code and compiles it on the fly. Thuserrors and/or results can be reported immediately back to the user. Such packages arecalled “interpreters”, and older readers may remember, perhaps with mixed feelings, theBASIC language. For small projects, the slow operation compared to a fully compiledcode is masked by the speed of current microprocessors, but it does become apparenton larger jobs.These packages are attractive for at least two reasons. The first is their ability to postprocess data. For example, suppose that x is a real variable and there exists a (possiblyunknown) function y(x). Suppose also that for a set X of discrete instances of x wehave computed a corresponding set Y of instances of y. Then a command similar toplot(X,Y) will display instantly a nicely formatted graph on the screen. Indeed, thosegenerated by Matlab in particular are of publication quality. A second advantage isthe apparent ability of some of the proprietary packages to perform in addition someDownloaded from Cambridge Books Online by IP 132.174.254.72 on Sat Apr 09 18:28:46 BST 2Cambridge Books Online Cambridge University Press, 2016

1.1 Scientific software3algebraic and analytic processes, and to integrate all of them with their numerical andgraphical properties. A disadvantage of all of these packages is the quirky syntax andlimited expressive ability of their command languages. Unlike the compiled languages,it is often extremely difficult to programme a process which was not envisaged by thepackage authors.The best of the proprietary packages are very easy to use with extensive on-line helpand coherent documentation, which has not yet been matched by all of the open-sourcealternatives. However, a major downside of the commercial packages is the extremelyhigh prices charged for their licences. Most of them offer a cut down “student version”at reduced price (but usable only while the student is in full-time education) so as toencourage familiarity with the package. This largesse is paid for by other users.Let us summarize the position. On the one hand, we have the traditional compiledlanguages for numerics which are very general, very fast, very difficult to learn and donot interact readily with graphical or algebraic processes. On the other, we have standardscientific packages which are good at integrating numerics, algebra and graphics, but areslow and limited in scope.What properties should an ideal scientific package have? A short list might contain:1 a programming language which is both easy to understand and which has extensiveexpressive ability,2 integration of algebraic, numerical and graphical functions,3 the ability to generate numerical algorithms running with speeds within an order ofmagnitude of the fastest of those generated by compiled languages,4 a user interface with adequate on-line help, and decent documentation,5 an extensive range of textbooks from which the curious reader can develop greaterunderstanding of the concepts,6 open-source software, freely available,7 implementation on all standard platforms, e.g., Linux, Mac OS X, Unix, Windows.The bad news is that no single package satisfies all of these criteria.The major obstruction here is the requirement of algebraic capability. There are twoopen-source packages, wx-Maxima and Reduce with significant algebraic capabilitiesworthy of consideration, but Reduce fails requirement 4 and both fail criteria 3 and 5.They are however extremely powerful tools in the hands of experienced users. It seemssensible therefore to drop the algebra requirement.1In 1991, Guido van Rossum created Python as a open-source platform-independentgeneral purpose programming language. It is basically a very simple language surrounded by an enormous library of add-on modules, including complete access to theunderlying operating system. This means that it can manage and manipulate programmesbuilt from other complete (even compiled) packages, i.e., a scripting language. This versatility has ensured both its adoption by power users such as Google, and a real army ofdevelopers. It means also that it can be a very powerful tool for the scientist. Of course,1The author had initially intended to write a book covering both numerical and algebraic applications forscientists. However, it turns out that, apart from simple problems, the requirements and approaches areradically different, and so it seems more appropriate to treat them differently.Downloaded from Cambridge Books Online by IP 132.174.254.72 on Sat Apr 09 18:28:46 BST 2Cambridge Books Online Cambridge University Press, 2016

4Introductionthere are other scripting languages, e.g., Java and Perl, but none has the versatility oruser-base to meet criteria 3–5 above.Five years ago it would not have been possible to recommend Python for scientificwork. The size of the army of developers meant that there were several mutually incompatible add-on packages for numerical and scientific applications. Fortunately, reasonhas prevailed and there is now a single numerical add-on package numpy and a singlescientific one scipy around which the developers have united.1.2The plan of this bookThe purpose of this intentionally short book is to show how easy it is for the workingscientist to implement and test non-trivial mathematical algorithms using Python. Wehave quite deliberately preferred brevity and simplicity to encyclopaedic coverage inorder to get the inquisitive reader up and running as soon as possible. We aim to leavethe reader with a well-founded framework to handle many basic, and not so basic, tasks.Obviously, most readers will need to dig further into techniques for their particularresearch needs. But after reading this book, they should have a sound basis for this.This chapter and Appendix A discuss how to set up a scientific Python environment.While the original Python interpreter was pretty basic, its replacement IPython is soeasy to use, powerful and versatile that Chapter 2 is devoted to it.We now describe the subsequent chapters. As each new feature is described, we tryto illustrate it first by essentially trivial examples and, where appropriate, by more extended problems. This author cannot know the mathematical sophistication of potentialreaders, but in later chapters we shall presume some familiarity with basic calculus,e.g., the Taylor series in one dimension. However, for these extended problems we shallsketch the background needed to understand them, and suitable references for furtherreading will be given.Chapter 3 gives a brief but reasonably comprehensive survey of those aspects of thecore Python language likely to be of most interest to scientists. Python is an objectoriented language, which lends itself naturally to object-oriented programming (OOP),which may well be unfamiliar to most scientists. We shall adopt an extremely light touchto this topic, but need to point out that the container objects introduced in Section 3.5 donot all have precise analogues in say C or Fortran. Again the brief introduction to Pythonclasses in Section 3.9 may be unfamiliar to users of those two families of languages.The chapter concludes with two implementations of the sieve of Eratosthenes, whichis a classical problem: enumerate all of the prime numbers2 less than a given integern. A straightforward implementation takes 17 lines of code, but takes inordinately longexecution times once n 105 . However, a few minutes of thought and using alreadydescribed Python features suggests a shorter 13 line programme which runs 3000 timesfaster and runs out of memory (on my laptop) once n 108 . The point of this exercise is2The restriction to integer arithmetic in this chapter is because our exposition of Python has yet to dealwith serious calculations involving real or complex numbers efficiently.Downloaded from Cambridge Books Online by IP 132.174.254.72 on Sat Apr 09 18:28:46 BST 2Cambridge Books Online Cambridge University Press, 2016

1.2 The plan of this book5that choosing the right approach (and Python often offers so many) is the key to successin Python numerics.Chapter 4 extends the core Python language via the add-on module numpy, to givea very efficient treatment of real and complex numbers. In the background, lurk C/C routines to execute repetitive tasks with near-compiled-language speeds. The emphasis is on using structures via vectorized code rather than the traditional for-loops ordo-loops. Vectorized code sounds formidable, but, as we shall show, it is much easier to write than the old-fashioned loop-based approach. Here too we discuss the inputand output of data. First, we look at how numpy can read and write text files, humanreadable data an

3.6 Python if statements 32 3.7 Loop constructs 33 3.7.1 The Python for loop 34 3.7.2 The Python continue statement 35 3.7.3 The Python break statement 35 3.7.4 List comprehensions 36 3.7.5 Python while loops 37 3.8 Functions 37 3.8.1 Syntax and scope 38 3.8.2 Positional arguments 41 3.8.3 Keyword arguments 41 3.8.4 Variable number of .