The Quick Python Book, Second Edition - Shen@uiuc

xvCONTENTSworks 25020.320.4 Implementing full list functionalityGiving an object full list capability 252Subclassing from built-in types 254Subclassing list20.520.620.7251254 Subclassing UserList 255When to use special method attributesMetaclasses 256Abstract base classes 258256Using abstract base classes for type checking 259 Creatingabstract base classes 260 Using the @abstractmethod and@abstractproperty decorators 260 20.8Summary262PART 4 WHERE CAN YOU GO FROM HERE? . 26321Testing your code made easy(-er) 26521.121.2Why you need to have testsThe assert statement 266Python’s debug variable21.3265266Tests in docstrings: doctests267Avoiding doctest traps 269 Tweaking doctests withdirectives 269 Pros and cons of doctests 270 21.4Using unit tests to test everything, every time270Setting up and running a single test case 270 Running thetest 272 Running multiple tests 272 Unit tests vs.doctests 273 21.522Summary 273Moving from Python 2 to Python 322.1Porting from 2 to 3274274Steps in porting from Python 2.x to 3.x22.222.322.422.5275Testing with Python 2.6 and -3 276Using 2to3 to convert the code 277Testing and common problems 279Using the same code for 2 and 3 280Using Python 2.5 or earlier 280converting back 281 Writing for Python 3.x andLicensed to Jay Vee metacyclist@gmail.com

xviCONTENTS22.623Summary281Using Python libraries 28223.1“Batteries included”—the standard library282Managing various data types 283 Manipulating files andstorage 284 Accessing operating system services 285 Usinginternet protocols and formats 286 Development and debuggingtools and runtime services 286 23.223.323.4Moving beyond the standard library 287Adding more Python libraries 287Installing Python libraries using setup.py 288Installing under the home schemeoptions 28923.523.624PyPI, a.k.a. “the Cheese Shop”Summary 289288Other installation 289Network, web, and database programming 29024.1Accessing databases in Python291Using the sqlite3 database 29124.2Network programming in Python 293Creating an instant HTTP serverclient 29424.3293Creating a Python web application Writing an HTTP295Using the web server gateway interface 295 Using the wsgilibrary to create a basic web app 295 Using frameworks to createadvanced web apps 296 24.4Sample project—creating a message wall 297Creating the database 297 Creating an applicationobject 298 Adding a form and retrieving its contents298 Saving the form’s contents 299 Parsing the URL andretrieving messages 300 Adding an HTML wrapper 303 24.5Summary304appendix 305index 323Licensed to Jay Vee metacyclist@gmail.com

prefaceI’ve been coding in Python for a number of years, longer than any other language I’veever used. I use Python for system administration, for web applications, for databasemanagement, and sometimes just to help myself think clearly.To be honest, I’m sometimes a little surprised that Python has worn so well. Basedon my earlier experience, I would have expected that by now some other languagewould have come along that was faster, cooler, sexier, whatever. Indeed, other languages have come along, but none that helped me do what I needed to do quite aseffectively as Python. In fact, the more I use Python and the more I understand it, themore I feel the quality of my programming improve and mature.This is a second edition, and my mantra in updating has been, “If it ain’t broke,don’t fix it.” Much of the content has been freshened for Python 3 but is largely aswritten in the first edition. Of course, the world of Python has changed since Python1.5, so in several places I’ve had to make significant changes or add new material. Onthose occasions I’ve done my best to make the new material compatible with the clearand low-key style of the original.For me, the aim of this book is to share the positive experiences I’ve gotten fromcoding in Python by introducing people to Python 3, the latest and, in my opinion,the best version of Python to date. May your journey be as satisfying as mine has been.xviiLicensed to Jay Vee metacyclist@gmail.com

acknowledgmentsI want to thank David Fugate of LaunchBooks for getting me into this book in the firstplace and for all of the support and advice he has provided over the years. I can’timagine having a better agent and friend. I also need to thank Michael Stephens ofManning for pushing the idea of doing a second edition of this book and supportingme in my efforts to make it as good as the first. Also at Manning, many thanks to everyperson who worked on this project, with special thanks to Marjan Bace for his support,Tara Walsh for guidance in the development phases, Mary Piergies for getting thebook (and me) through the production process, Linda Recktenwald for her patiencein copy editing, and Tiffany Taylor for proofreading. I also owe a huge debt to WillKahn-Greene for all of the astute advice he gave both as a technical reviewer and indoing the technical proofing. Thanks, Will, you saved me from myself more timesthan I can count. Likewise, hearty thanks to the many reviewers whose insights andfeedback were of immense help: Nick Lo, Michele Galli, Andy Dingley, MohamedLamkadem, Robby O'Connor, Amos Bannister, Joshua Miller, Christian Marquardt,Andrew Rhine, Anthony Briggs, Carlton Gibson, Craig Smith, Daniel McKinnon,David McWhirter, Edmon Begoli, Elliot Winard, Horaci Macias, Massimo Perga,Munch Paulson, Nathan R. Yergler, Rick Wagner, Sumit Pal, and Tyson S. Maxwell.Because this is a second edition, I have to thank the authors of the first edition,Daryl Harms and Kenneth MacDonald, for two things: first, for writing a book so soundthat it has remained in print well beyond the average lifespan of most tech books, andsecond, for being otherwise occupied, thereby giving me a chance to update it. I hopethis version carries on the successful and long-lived tradition of the first.xviiiLicensed to Jay Vee metacyclist@gmail.com

ACKNOWLEDGMENTSxixThanks to my canine associates, Molly, Riker, and Aeryn, who got fewer walks,training sessions, and games of ball than they should have but still curled up besidemy chair and kept me company and helped me keep my sense of perspective as Iworked. You’ll get those walks now, guys, I promise.Most important, thanks to my wife, Becky, who both encouraged me to take on thisproject and had to put up with the most in the course of its completion—particularlyan often-grumpy and preoccupied spouse. I really couldn’t have done it without you.Licensed to Jay Vee metacyclist@gmail.com

about this bookWho should read this bookThis book is intended for people who already have experience in one or more programming languages and want to learn the basics of Python 3 as quickly and directlyas possible. Although some basic concepts are covered, there’s no attempt to teachbasic programming skills in this book, and the basic concepts of flow control, OOP,file access, exception handling, and the like are assumed. This book may also be ofuse to users of earlier versions of Python who want a concise reference for Python 3.How to use this bookPart 1 introduces Python and explains how to download and install it on your system.It also includes a very general survey of the language, which will be most useful forexperienced programmers looking for a high-level view of Python.Part 2 is the heart of the book. It covers the ingredients necessary for obtaining aworking knowledge of Python as a general-purpose programming language. Thechapters are designed to allow readers who are beginning to learn Python to worktheir way through sequentially, picking up knowledge of the key points of the language. These chapters also contain some more advanced sections, allowing you toreturn to find in one place all the necessary information about a construct or topic.Part 3 introduces advanced language features of Python, elements of the languagethat aren’t essential to its use but that can certainly be a great help to a serious Pythonprogrammer.xxLicensed to Jay Vee metacyclist@gmail.com

ABOUT THIS BOOKxxiPart 4 describes more advanced or specialized topics that are beyond the strict syntax of the language. You may read these chapters or not, depending on your needs.A suggested plan if you’re new to Python is to start by reading chapter 3 to obtainan overall perspective and then work through the chapters in part 2 that are applicable. Enter in the interactive examples as they are introduced. This will immediatelyreinforce the concepts. You can also easily go beyond the examples in the text toanswer questions about anything that may be unclear. This has the potential toamplify the speed of your learning and the level of your comprehension. If you aren’tfamiliar with OOP or don’t need it for your application, skip most of chapter 15. If youaren’t interested in developing a GUI, skip chapter 16.Those familiar with Python should also start with chapter 3. It will be a good reviewand will introduce differences between Python 3 and what may be more familiar. It’s areasonable test of whether you’re ready to move on to the advanced chapters in parts3 and 4 of this book.It’s possible that some readers, although new to Python, will have enough experience with other programming languages to be able to pick up the bulk of what theyneed to get going from chapter 3 and by browsing the Python standard library moduleslisted in chapter 23 and the Python Library Reference in the Python documentation.RoadmapChapter 1 discusses the strengths and weaknesses of Python and shows why Python 3 isa good choice of programming language for many situations.Chapter 2 covers downloading, installing, and starting up the Python interpreterand IDLE, its integrated development environment.Chapter 3 is a short overview of the Python language. It provides a basic idea of thephilosophy, syntax, semantics, and capabilities of the language.Chapter 4 starts with the basics of Python. It introduces Python variables, expressions, strings, and numbers. It also introduces Python’s block-structured syntax.Chapters 5, 6, and 7 describe the five powerful built-in Python data types: lists,tuples, sets, strings, and dictionaries.Chapter 8 introduces Python’s control flow syntax and use (loops and if-elsestatements).Chapter 9 describes function definition in Python along with its flexible parameter-passing capabilities.Chapter 10 describes Python modules. They provide an easy mechanism for segmenting the program namespace.Chapter 11 covers creating standalone Python programs, or scripts, and runningthem on Windows, Mac OS X, and Linux platforms. The chapter also covers the support available for command-line options, arguments, and I/O redirection.Licensed to Jay Vee metacyclist@gmail.com

xxiiABOUT THIS BOOKChapter 12 describes how to work and navigate through the files and directories ofthe filesystem. It shows how to write code that’s as independent as possible of theactual operating system you’re working on.Chapter 13 introduces the mechanisms for reading and writing files in Python.These include the basic capability to read and write strings (or byte streams), themechanism available for reading binary records, and the ability to read and write arbitrary Python objects.Chapter 14 discusses the use of exceptions, the error-handling mechanism used byPython. It doesn’t assume that you have any prior knowledge of exceptions, althoughif you’ve previously used them in C or Java, you’ll find them familiar.Chapter 15 introduces Python’s support for writing object-oriented programs.Chapter 16 focuses on the available Tkinter interface and ends with an introduction to some of the other options available for developing GUIs.Chapter 17 discusses the regular-expression capabilities available for Python.Chapter 18 introduces the package concept in Python for structuring the code oflarge projects.Chapter 19 covers the simple mechanisms available to dynamically discover andwork with data types in Python.Chapter 20 introduces more advanced OOP techniques, including the use ofPython’s special method-attributes mechanism, metaclasses, and abstract base classes.Chapter 21 covers two strategies that Python offers for testing your code: doctestsand unit testing.Chapter 22 surveys the process, issues, and tools involved in porting code from earlier versions of Python to Python 3.Chapter 23 is a brief survey of the standard library and also includes a discussion ofwhere to find other modules and how to install them.Chapter 24 is a brief introduction to using Python for database and web programming. A small web application is developed to illustrate the principles involved.The appendix contains a comprehensive guide to obtaining and accessingPython’s full documentation, the Pythonic style guide, PEP 8, and “The Zen ofPython,” a slightly wry summary of the philosophy behind Python.Code conventionsThe code samples in this book, and their output, appear in a fixed-width font andare often accompanied by annotations. The code samples are deliberately kept as simple as possible, because they aren’t intended to be reusable parts that can pluggedinto your code. Instead, the code samples are stripped down so that you can focus onthe principle being illustrated.In keeping with the idea of simplicity, the code examples are presented as interactive shell sessions where possible; you should enter and experiment with these samplesas much as you can. In interactive code samples, the commands that need to beentered are on lines that begin with the prompt, and the visible results of thatcode (if any) are on the line below.Licensed to Jay Vee metacyclist@gmail.com

ABOUT THIS BOOKxxiiiIn some cases a longer code sample is needed, and these are identified in the textas file listings. You should save these as files with names matching those used in thetext and run them as standalone scripts.Source code downloadsThe source code for the samples in this book is available from the publisher’s websiteat stem requirementsThe samples and code in this book have been written with Windows (XP through Windows 7), Mac OS X, and Linux in mind. Because Python is a cross-platform language,they should work on other platforms for the most part, except for platform-specificissues, like the handling of files, paths, and GUIs.Software requirementsThis book is based on Python 3.1, and all examples should work on any subsequentversion of Python 3. The examples also work on Python 3.0, but I strongly recommendusing 3.1—there are no advantages to the earlier version, and 3.1 has several subtleimprovements. Note that Python 3 is required and that an earlier version of Python willnot work with the code in this book.Author OnlineThe purchase of The Quick Python Book, Second Edition includes free access to a privateweb forum run by Manning Publications, where you can make comments about thebook, ask technical questions, and receive help from the author and from other users.To access the forum and subscribe to it, point your web browser to www.manning.com/TheQuickPythonBookSecondEdition. This page provides information abouthow to get on the forum once you’re registered, what kind of help is available, and therules of conduct on the forum.Manning’s commitment to our readers is to provide a venue where a meaningfuldialogue between individual readers and between readers and the author can takeplace. It’s not a commitment to any specific amount of participation on the part of theauthor, whose contribution to the book’s forum remains voluntary (and unpaid). Wesuggest you try asking him some challenging questions, lest his interest stray!The Author Online forum and the archives of previous discussions will be accessible from the publisher’s website as long as the book is in print.About the authorSecond edition author Naomi Ceder has been programming in various languages forover 20 years and has been a Linux system administrator since 2000. She started usingPython for a variety of projects in 2001. She is currently IT Director at Zoro Tools, Inc., inBuffalo Grove, Illinois, and an organizer and teacher of the Chicago Python Workshops.Licensed to Jay Vee metacyclist@gmail.com

xxivABOUT THIS BOOKAbout the cover illustrationThe illustration on the cover of The Quick Python Book, Second Edition is taken from alate 18th century edition of Sylvain Maréchal’s four-volume compendium of regionaldress customs published in France. Each illustration is finely drawn and colored byhand. The rich variety of Maréchal’s collection reminds us vividly of how culturallyapart the world’s towns and regions were just 200 years ago. Isolated from each other,people spoke different dialects and languages. In the streets or in the countryside, itwas easy to identify where they lived and what their trade or station in life was just bywhat they were wearing.Dress codes have changed since then and the diversity by region, so rich at thetime, has faded away. It is now hard to tell apart the inhabitants of different continents, let alone different towns or regions. Perhaps we have traded cultural diversityfor a more varied personal life—certainly for a more varied and fast-paced technological life.At a time when it is hard to tell one computer book from another, Manning celebrates the inventiveness and initiative of the computer business with book coversbased on the rich diversity of regional life of two centuries ago, brought back to life byMaréchal’s pictures.Licensed to Jay Vee metacyclist@gmail.com

Part 1Starting outThis section will tell you a little bit about Python, its strengths and weaknesses, and why you should consider learning P

Python is readable 5 Python is complete—"batteries included" 6 Python is cross-platform 6 Python is free 6 1.3 What Python doesn't do as well 7 Python is not the fastest language 7 Python doesn't have the most libraries 8 Python doesn't check variable types at compile time 8 1.4 Why learn Python 3? 8 1.5 Summary 9