Mastering Python - Paper.bobylive
Mastering PythonMaster the art of writing beautiful and powerful Pythonby using all of the features that Python 3.5 offersRick van HattemBIRMINGHAM - MUMBAI
Mastering PythonCopyright 2016 Packt PublishingAll rights reserved. No part of this book may be reproduced, stored in a retrievalsystem, or transmitted in any form or by any means, without the prior writtenpermission of the publisher, except in the case of brief quotations embedded incritical articles or reviews.Every effort has been made in the preparation of this book to ensure the accuracyof the information presented. However, the information contained in this book issold without warranty, either express or implied. Neither the author, nor PacktPublishing, and its dealers and distributors will be held liable for any damagescaused or alleged to be caused directly or indirectly by this book.Packt Publishing has endeavored to provide trademark information about all of thecompanies and products mentioned in this book by the appropriate use of capitals.However, Packt Publishing cannot guarantee the accuracy of this information.First published: April 2016Production reference: 1270416Published by Packt Publishing Ltd.Livery Place35 Livery StreetBirmingham B3 2PB, UK.ISBN 978-1-78528-972-9www.packtpub.com
CreditsAuthorRick van HattemReviewersRandall DeggesProject CoordinatorSuzanne CoutinhoProofreaderSafis EditingDave de FijterI. de HoogtCommissioning EditorSarah CroftonAcquisition EditorReshma RamanContent Development EditorArun NadarTechnical EditorsRyan KocheryTanmayee PatilCopy EditorVikrant PhadkeIndexerMariammal ChettiyarProduction CoordinatorNilesh MohiteCover WorkNilesh Mohite
About the AuthorRick van Hattem is an experienced programmer, entrepreneur, andsoftware/database architect with over 20 years of programming experience,including 15 with Python. Additionally, he has a lot of experience withhigh-performance architectures featuring large amounts of concurrent usersand/or data.Rick has founded several start-ups and has done consulting for many companies,including a few Y Combinator start-ups and several large companies. One of thestartups he founded, Fashiolista.com, is one of the largest social networks for fashionin the world, featuring millions of users and the performance challengesto accompany those.Rick was one of the reviewers on the book PostgreSQL Server Programming,Packt Publishing.Thanks to my family, in particular Marloes, who supported me everystep of the way; and my mother and sister, who have always beenthere for me.
About the ReviewersRandall Degges is a happy programmer, speaker, author, and amateurbodybuilder living in California.Growing up in Los Angeles, he was intensely interested in building commandline programs and writing quality software. His love of programming eventuallypropelled him into a successful career in software development.Randall has been a life-long open source developer and has contributed tohundreds of popular projects in Python, Node.js, and Go. He's also the author ofseveral popular libraries, which you can find on his public GitHub accountat https://github.com/rdegges.At 23, he cofounded an extremely popular API service in the telephony industry:OpenCNAM (https://www.opencnam.com). At 25, he joined Stormpath(https://stormpath.com) as the head of developer evangelism, whereby hewrites open source security libraries full time and travels the world giving technicaltalks about building secure software.In his free time, Randall writes and edits technical books, runs a security podcastcalled Stormcast (https://www.stormca.st), posts blogs on his personal website(https://www.rdegges.com), and tries to spend time with his high-schoolsweetheart, Samantha.
Dave de Fijter is a Python developer from the Netherlands. He always knew hewould end up "doing something with computers." At a young age, he went to thelibrary to read books about them even though he had no computer at that time.This obsession never really ended. In 2001, aged 14, he started his first part-time job,creating dynamic websites in PHP for a local web development company, and therehe found his calling.In 2007, he finished his bachelor's degree in ICT while already working full time as aPHP developer for over a year. In 2008, he switched from PHP to Python and Djangofor web development and loved this new technology stack so much that he neverlooked back.After working as a Python developer for various start-ups and establishedcompanies, Dave used this experience to start his own business called Indentity(https://indentity.nl) in 2010, focusing on Python/Django development andadvice. Up until now, he runs this company and mainly spends his time helping outstart-ups with designing and building technologically advanced web applicationsfrom the ground up as an interim CTO/technical cofounder.I. de Hoogt, with some basic experience wrought from university assignments in thefield of modeling of multi-phase flows, got himself started in software development.His main experience in programming in Python stems from an internship at acompany dealing in 3D printing software, where a package resulting in optimizedobject orientation and guaranteed mathematical mesh validity was created.Other projects that he's been involved with have dealt with control systems such asself-parking cars, multi-legged robots, and quadcopters, but his current job is in thefield of data analysis.
www.PacktPub.comeBooks, discount offers, and moreDid you know that Packt offers eBook versions of every book published, with PDFand ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy.Get in touch with us at customercare@packtpub.com for more details.At www.PacktPub.com, you can also read a collection of free technical articles, signup for a range of free newsletters and receive exclusive discounts and offers on Packtbooks and ion/packtlibDo you need instant solutions to your IT questions? PacktLib is Packt's online digitalbook library. Here, you can search, access, and read Packt's entire library of books.Why subscribe? Fully searchable across every book published by Packt Copy and paste, print, and bookmark content On demand and accessible via a web browser
Table of ContentsPrefaceChapter 1: Getting Started – One Environment per ProjectCreating a virtual Python environment using venvCreating your first venvvenv argumentsDifferences between virtualenv and venvBootstrapping pip using ensurepipensurepip usageManual pip installInstalling C/C packagesDebian and UbuntuRed Hat, CentOS, and FedoraOS XWindowsSummaryChapter 2: Pythonic Syntax, Common Pitfalls, and Style GuideCode style – or what is Pythonic code?Formatting strings – printf-style or str.format?PEP20, the Zen of PythonBeautiful is better than uglyExplicit is better than implicitSimple is better than complexFlat is better than nestedSparse is better than denseReadability countsPracticality beats purityErrors should never pass silentlyIn the face of ambiguity, refuse the temptation to guessOne obvious way to do itNow is better than 242425
Table of ContentsHard to explain, easy to explainNamespaces are one honking great ideaConclusion252526Explaining PEP827Verifying code quality, pep8, pyflakes, and more32Duck typingDifferences between value and identity comparisonsLoopsMaximum line lengthflake8Pylint272930313235Common pitfallsScope matters!3536Overwriting and/or creating extra built-insModifying while iteratingCatching exceptions – differences between Python 2 and 3Late binding – be careful with closuresCircular importsImport collisionsSummary39414244454748Function argumentsClass propertiesModifying variables in the global scopeChapter 3: Containers and Collections – Storing Data theRight WayTime complexity – the big O notationCore collectionslist – a mutable list of itemsdict – unsorted but a fast map of itemsset – like a dict without valuestuple – the immutable listAdvanced collectionsChainMap – the list of dictionariescounter – keeping track of the most occurring elementsdeque – the double ended queuedefaultdict – dictionary with a default valuenamedtuple – tuples with field namesenum – a group of constantsOrderedDict – a dictionary where the insertion order mattersheapq – the ordered listbisect – the sorted listSummary[ ii ]363738495051525557596262646668717274757679
Table of ContentsChapter 4: Functional Programming – Readability Versus BrevityFunctional programminglist comprehensionsdict comprehensionsset comprehensionslambda functionsThe Y combinatorfunctoolspartial – no need to repeat all arguments every timereduce – combining pairs into a single resultImplementing a factorial functionProcessing treesitertoolsaccumulate – reduce with intermediate resultschain – combining multiple resultscombinations – combinatorics in Pythonpermutations – combinations where the order matterscompress – selecting items using a list of Booleansdropwhile/takewhile – selecting items using a functioncount – infinite range with decimal stepsgroupby – grouping your sorted iterableislice – slicing any iterableSummaryChapter 5: Decorators – Enabling Code Reuse by DecoratingDecorating functionsWhy functools.wraps is importantHow are decorators useful?Memoization using decoratorsDecorators with (optional) argumentsCreating decorators using classesDecorating class functionsSkipping the instance – classmethod and staticmethodProperties – smart descriptor usageDecorating classesSingletons – classes with a single instanceTotal ordering – sortable classes the easy wayUseful decoratorsSingle dispatch – polymorphism in PythonContextmanager, with statements made easyValidation, type checks, and conversions[ iii 5
Table of ContentsUseless warnings – how to ignore themSummary138140Chapter 6: Generators and Coroutines – Infinity, One Stepat a Time141Chapter 7: Async IO – Multithreading without Threads167What are generators?Advantages and disadvantages of generatorsPipelines – an effective use of generatorstee – using an output multiple timesGenerating from generatorsContext managersCoroutinesA basic examplePrimingClosing and throwing exceptionsBidirectional pipelinesUsing the stateSummaryIntroducing the asyncio libraryThe async and await statementsPython 3.4Python 3.5Choosing between the 3.4 and 3.5 8169169170A simple example of single-threaded parallel processingConcepts of asyncio171172Asynchronous servers and clients185Futures and tasksEvent loopsProcessesBasic echo server172174180185SummaryChapter 8: Metaclasses – Making Classes (Not Instances)SmarterDynamically creating classesA basic metaclassArguments to metaclassesAccessing metaclass attributes through classesAbstract classes using collections.abcInternal workings of the abstract classesCustom type checksUsing abc.ABC before Python 3.4[ iv ]188189190191193193194195199201
Table of ContentsAutomatically registering a plugin systemImporting plugins on-demandImporting plugins through configurationImporting plugins through the file systemOrder of operations when instantiating classesFinding the metaclassPreparing the namespaceExecuting the class bodyCreating the class object (not instance)Executing the class decoratorsCreating the class instanceExampleStoring class attributes in definition orderThe classic solution without metaclassesUsing metaclasses to get a sorted namespaceSummaryChapter 9: Documentation – How to Use Sphinx andreStructuredTextThe reStructuredText syntaxGetting started with reStructuredTextInline markupHeadersListsEnumerated listBulleted listOption listDefinition listNested listsLinks, references, and labelsImagesSubstitutionsBlocks, code, math, comments, and quotesConclusionThe Sphinx documentation generatorGetting started with SphinxUsing sphinx-quickstartUsing 231232233233234234239Sphinx directives243Sphinx roles247The table of contents tree directive (toctree)Autodoc, documenting Python modules, classes, and functions[v]243244
Table of ContentsDocumenting codeDocumenting a class with the Sphinx styleDocumenting a class with the Google styleDocumenting a class with the NumPy styleWhich style to chooseSummaryChapter 10: Testing and Logging – Preparing for BugsUsing examples as tests with doctestA simple doctest exampleWriting doctestsTesting with pure documentationThe doctest flagsTrue and False versus 1 and 0Normalizing whitespaceEllipsisDoctest 0271Testing dictionariesTesting floating-point numbersTimes and durations271273273Testing with py.testThe difference between the unittest and py.test outputThe difference between unittest and py.test tests274275280Mock objectsUsing unittest.mockUsing py.test lifying assertionsParameterizing testsAutomatic arguments using fixturesPrint statements and loggingPluginsBasic logging configurationDictionary configurationJSON configurationIni file configurationThe network 14Summary317Usage315Chapter 11: Debugging – Solving the BugsNon-interactive debuggingInspecting your script using trace[ vi ]319320321
Table of ContentsDebugging using loggingShowing call stack without exceptionsDebugging asyncioHandling crashes using faulthandlerInteractive debuggingConsole on demandDebugging using pdb325327329331332332333Debugging using ipdbOther debuggersDebugging servicesSummary341343343344BreakpointsCatching exceptionsCommandsChapter 12: Performance – Tracking and Reducing YourMemory and CPU UsageWhat is performance?Timeit – comparing code snippet performancecProfile – finding the slowest componentsFirst profiling runCalibrating your profilerSelective profiling using decoratorsUsing profile statisticsLine profilerImproving performanceUsing the right algorithmGlobal interpreter lockTry versus ifLists versus generatorsString concatenationAddition versus generatorsMap versus generators and list comprehensionsCachingLazy importsUsing optimized librariesJust-in-time compilingConverting parts of your code to CMemory usageTracemallocMemory profilerMemory leaks[ vii 64364365366366367367368368369369370372
Table of ContentsReducing memory usageGenerators versus listsRecreating collections versus removing itemsUsing slotsPerformance monitoringSummaryChapter 13: Multiprocessing – When a Single CPU Core Isnot EnoughMultithreading versus multiprocessingHyper-threading versus physical CPU coresCreating a pool of workersSharing data between processesRemote processesDistributed processing using multiprocessingDistributed processing using IPyparallelipython config.pyipython kernel config.pyipcontroller config.pyipengine config.pyipcluster config.pySummaryChapter 14: Extensions in C/C , System Calls, andC/C LibrariesIntroductionDo you need C/C modules?WindowsOS XLinux/UnixCalling C/C with ctypesPlatform-specific librariesWindowsLinux/UnixOS XMaking it easyCalling functions and native typesComplex data structuresArraysGotchas with memory managementCFFIComplex data structuresArraysABI or API?[ viii 410410412413414415415
Table of ContentsCFFI or ctypes?Native C/C extensionsA basic exampleC is not Python – size mattersThe example explained415416416419421C is not Python – errors are silent or lethalCalling Python from C – handling complex typesSummary424425427staticPyObject*Parsing arguments421422422Chapter 15: Packaging – Creating Your Own Librariesor Applications429Index451Installing packagesSetup parametersPackagesEntry pointsCreating global commandsCustom setup.py commandsPackage dataTesting packagesUnittestpy.testNosetestsC/C extensionsRegular extensionsCython extensionsWheels – the new eggsDistributing to the Python Package IndexSummary[ ix 49
PrefacePython is a language that is easy to learn and both powerful and convenient from thestart. Mastering Python, however, is a completely different question.Every programming problem you will encounter has at least several possiblesolutions and/or paradigms to apply within the vast possibilities of Python. Thisbook will not only illustrate a range of different and new techniques but also explainwhere and when a method should be applied.This book is not a beginner's guide to Python 3. It is a book that can teach you aboutthe more advanced techniques possible within Python. Specifically targeting Python3.5 and up, it also demonstrates several Python 3.5-only features such as async defand await statements.As a Python programmer with many years of experience, I will attempt to rationalizethe choices made in this book with relevant background information. Theserationalizations are in no way strict guidelines, however. Several of these cases boildown to personal style in the end. Just know that they stem from experience and are,in many cases, the solutions recommended by the Python community.Some of the references in this book might not be obvious to you if you are not a fanof Monty Python. This book extensively uses spam and eggs instead of foo and bar incode samples. To provide some background information, I recommend watching the"Spam" sketch by Monty Python. It is positively silly!What this book coversChapter 1, Getting Started – One Environment per Project, introduces virtual Pythonenvironments using virtualenv or venv to isolate the packages in your Python projects.Chapter 2, Pythonic Syntax, Common Pitfalls, and Style Guide, explains what Pythoniccode is and how to write code that is Pythonic and adheres to the Python philosophy.[ xi ]
PrefaceChapter 3, Containers and Collections – Storing Data the Right Way, is where we usethe many containers and collections bundled with Python to create code that is fastand readable.Chapter 4, Functional Programming – Readability Versus Brevity, covers functionalprogramming techniques such as list/dict/set comprehensions and lambdastatements that are available in Python. Additionally, it illustrates their similaritieswith the mathematical principles involved.Chapter 5, Decorators – Enabling Code Reuse by Decorating, explains not only how tocreate your own function/class decorators, but also how internal decorators such asproperty, staticmethod, and classmethod work.Chapter 6, Generators and Coroutines – Infinity, One Step at a Time, shows howgenerators and coroutines can be used to lazily evaluate structures of infinite size.Chapter 7, Async IO – Multithreading without Threads, demonstrates the usage ofasynchronous functions using async def and await so that external resources nolonger stall your Python processes.Chapter 8, Metaclasses – Making Classes (Not Instances) Smarter, goes deeper into thecreation of classes and how class behavior can be completely modified.Chapter 9, Documentation – How to Use Sphinx and reStructuredText, shows howyou can make Sphinx automatically document your code with very little effort.Additionally, it shows how the Napoleon syntax can be used to document functionarguments in a way that is legible both in the code and the documentation.Chapter 10, Testing and Logging – Preparing for Bugs, explains how code can be testedand how logging can be added to enable easy debugging in case bugs occur at alater time.Chapter 11, Debugging – Solving the Bugs, demonstrates several methods of huntingdown bugs with the use of tracing, logging, and interactive debugging.Chapter 12, Performance – Tracking and Reduc
Mastering Python Master the art of writing beautiful and powerful Python by using all of the features that Python 3.5 offers Rick van Hattem BIRMINGHAM - MUMBAI. . PHP developer for over a year. In 2008, he switched from PHP to Python and Django for web development and loved thi
Python Programming for the Absolute Beginner Second Edition. CONTENTS CHAPTER 1 GETTING STARTED: THE GAME OVER PROGRAM 1 Examining the Game Over Program 2 Introducing Python 3 Python Is Easy to Use 3 Python Is Powerful 3 Python Is Object Oriented 4 Python Is a "Glue" Language 4 Python Runs Everywhere 4 Python Has a Strong Community 4 Python Is Free and Open Source 5 Setting Up Python on .
Python 2 versus Python 3 - the great debate Installing Python Setting up the Python interpreter About virtualenv Your first virtual environment Your friend, the console How you can run a Python program Running Python scripts Running the Python interactive shell Running Python as a service Running Python as a GUI application How is Python code .
Python is readable 5 Python is complete—"batteries included" 6 Python is cross-platform 6 Python is free 6 1.3 What Python doesn't do as well 7 Python is not the fastest language 7 Python doesn't have the most libraries 8 Python doesn't check variable types at compile time 8 1.4 Why learn Python 3? 8 1.5 Summary 9
3. Mastering Tips 3.1 what is mastering? 3.2 typical mastering tools and effects 3.3 what can (and should) be fixed/adjusted 3.4 mastering EQ tips 3.5 mastering compressor tips 3.6 multi-band compressor / dynamic EQ 3.7 brickwall limiter 3.8 no problem, the mastering engineer will fix that!
site "Python 2.x is legacy, Python 3.x is the present and future of the language". In addition, "Python 3 eliminates many quirks that can unnecessarily trip up beginning programmers". However, note that Python 2 is currently still rather widely used. Python 2 and 3 are about 90% similar. Hence if you learn Python 3, you will likely
There are currently two versions of Python in use; Python 2 and Python 3. Python 3 is not backward compatible with Python 2. A lot of the imported modules were only available in Python 2 for quite some time, leading to a slow adoption of Python 3. However, this not really an issue anymore. Support for Python 2 will end in 2020.
A Python Book A Python Book: Beginning Python, Advanced Python, and Python Exercises Author: Dave Kuhlman Contact: dkuhlman@davekuhlman.org
ASTM INTERNATIONAL Helping our world work better Standards Catalog 2016 www.astm.org Highlights in this issue: 24 ook of B Standards 2 uilding Codes B 9 nline TrainingO 6 MNL 43 - 3rd 13 Proficiency Testing Standards Books Journals and Software Training Laboratory QA Programs. What’s New from ASTM International ASTM Compass Your Portal for Standards, Testing, Learning & More Give your .
