GAWK: Effective AWK Programming - IIT Bombay
GAWK: Effective AWK ProgrammingA User’s Guide for GNU AwkEdition 4.1May, 2013Arnold D. Robbins
“To boldly go where no man has gone before” is a Registered Trademark of ParamountPictures Corporation.Published by:Free Software Foundation51 Franklin Street, Fifth FloorBoston, MA 02110-1301 USAPhone: 1-617-542-5942Fax: 1-617-542-2652Email: gnu@gnu.orgURL: http://www.gnu.org/ISBN 1-882114-28-0Copyright c 1989, 1991, 1992, 1993, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,2005, 2007, 2009, 2010, 2011, 2012, 2013 Free Software Foundation, Inc.This is Edition 4.1 of GAWK: Effective AWK Programming: A User’s Guide for GNU Awk,for the 4.1.0 (or later) version of the GNU implementation of AWK.Permission is granted to copy, distribute and/or modify this document under the terms ofthe GNU Free Documentation License, Version 1.3 or any later version published by theFree Software Foundation; with the Invariant Sections being “GNU General Public License”,the Front-Cover texts being (a) (see below), and with the Back-Cover Texts being (b) (seebelow). A copy of the license is included in the section entitled “GNU Free DocumentationLicense”.a. “A GNU Manual”b. “You have the freedom to copy and modify this GNU manual. Buying copies from theFSF supports it in developing GNU and promoting software freedom.”
To Miriam, for making me complete.To Chana, for the joy you bring us.To Rivka, for the exponential increase.To Nachum, for the added dimension.To Malka, for the new beginning.
iShort ContentsForeword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3123456789Part I: The awk LanguageGetting Started with awk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Running awk and gawk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41Reading Input Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53Printing Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95Patterns, Actions, and Variables . . . . . . . . . . . . . . . . . . . . . . . 117Arrays in awk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157Part II: Problem Solving With awk10 A Library of awk Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 19911 Practical awk Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229Part III: Moving Beyond Standard awk With gawkAdvanced Features of gawk . . . . . . . . . . . . . . . . . . . . . . . . . . .Internationalization with gawk . . . . . . . . . . . . . . . . . . . . . . . . .Debugging awk Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Arithmetic and Arbitrary Precision Arithmetic with gawk . . .Writing Extensions for gawk . . . . . . . . . . . . . . . . . . . . . . . . . . .275289299315331Part IV: AppendicesA The Evolution of the awk Language . . . . . . . . . . . . . . . . . . . . .B Installing gawk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .C Implementation Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .D Basic Programming Concepts . . . . . . . . . . . . . . . . . . . . . . . . .Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .GNU General Public License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .GNU Free Documentation License . . . . . . . . . . . . . . . . . . . . . . . . .Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3853954114214254354474551213141516
iiiTable of ContentsForeword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3History of awk and gawk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .A Rose by Any Other Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Using This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Typographical Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .The GNU Project and This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .How to Contribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4457899Part I: The awk Language1Getting Started with awk . . . . . . . . . . . . . . . . . . . . . 131.1How to Run awk Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.1.1 One-Shot Throwaway awk Programs . . . . . . . . . . . . . . . . . . . . . .1.1.2 Running awk Without Input Files . . . . . . . . . . . . . . . . . . . . . . . .1.1.3 Running Long Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.1.4 Executable awk Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.1.5 Comments in awk Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.1.6 Shell-Quoting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.1.6.1 Quoting in MS-Windows Batch Files. . . . . . . . . . . . . . . . .1.2 Data Files for the Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.3 Some Simple Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.4 An Example with Two Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.5 A More Complex Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.6 awk Statements Versus Lines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.7 Other Features of awk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.8 When to Use awk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2131314141516171818192122232425Running awk and gawk . . . . . . . . . . . . . . . . . . . . . . . . . 272.12.22.32.42.5Invoking awk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Command-Line Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Other Command-Line Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Naming Standard Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .The Environment Variables gawk Uses . . . . . . . . . . . . . . . . . . . . . . . . .2.5.1 The AWKPATH Environment Variable . . . . . . . . . . . . . . . . . . . . . .2.5.2 The AWKLIBPATH Environment Variable . . . . . . . . . . . . . . . . . . .2.5.3 Other Environment Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.6 gawk’s Exit Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.7 Including Other Files Into Your Program . . . . . . . . . . . . . . . . . . . . . .2.8 Loading Shared Libraries Into Your Program . . . . . . . . . . . . . . . . . .2727333434343535363638
iv GAWK: Effective AWK Programming2.9 Obsolete Options and/or Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382.10 Undocumented Options and Features . . . . . . . . . . . . . . . . . . . . . . . . . 393Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.1 How to Use Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.2 Escape Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.3 Regular Expression Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.4 Using Bracket Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.5 gawk-Specific Regexp Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.6 Case Sensitivity in Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.7 How Much Text Matches? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.8 Using Dynamic Regexps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44142444748495051Reading Input Files . . . . . . . . . . . . . . . . . . . . . . . . . . . 534.14.24.34.44.5How Input Is Split into Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Examining Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Nonconstant Field Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Changing the Contents of a Field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Specifying How Fields Are Separated . . . . . . . . . . . . . . . . . . . . . . . . . .4.5.1 Whitespace Normally Separates Fields . . . . . . . . . . . . . . . . . . . .4.5.2 Using Regular Expressions to Separate Fields . . . . . . . . . . . . .4.5.3 Making Each Character a Separate Field . . . . . . . . . . . . . . . . .4.5.4 Setting FS from the Command Line . . . . . . . . . . . . . . . . . . . . . .4.5.5 Field-Splitting Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.6 Reading Fixed-Width Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.7 Defining Fields By Content. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.8 Multiple-Line Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.9 Explicit Input with getline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.9.1 Using getline with No Arguments . . . . . . . . . . . . . . . . . . . . . . .4.9.2 Using getline into a Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.9.3 Using getline from a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.9.4 Using getline into a Variable from a File . . . . . . . . . . . . . . . .4.9.5 Using getline from a Pipe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.9.6 Using getline into a Variable from a Pipe . . . . . . . . . . . . . . .4.9.7 Using getline from a Coprocess . . . . . . . . . . . . . . . . . . . . . . . . .4.9.8 Using getline into a Variable from a Coprocess . . . . . . . . . .4.9.9 Points to Remember About getline . . . . . . . . . . . . . . . . . . . . .4.9.10 Summary of getline Variants . . . . . . . . . . . . . . . . . . . . . . . . . . .4.10 Reading Input With A Timeout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.11 Directories On The Command Line. . . . . . . . . . . . . . . . . . . . . . . . . . 778
v5Printing Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 795.1 The print Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5.2 print Statement Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5.3 Output Separators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5.4 Controlling Numeric Output with print . . . . . . . . . . . . . . . . . . . . . . .5.5 Using printf Statements for Fancier Printing . . . . . . . . . . . . . . . . .5.5.1 Introduction to the printf Statement . . . . . . . . . . . . . . . . . . . .5.5.2 Format-Control Letters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5.5.3 Modifiers for printf Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . .5.5.4 Examples Using printf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5.6 Redirecting Output of print and printf . . . . . . . . . . . . . . . . . . . . . .5.7 Special File Names in gawk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5.7.1 Special Files for Standard Descriptors . . . . . . . . . . . . . . . . . . . .5.7.2 Special Files for Network Communications . . . . . . . . . . . . . . . .5.7.3 Special File Name Caveats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5.8 Closing Input and Output Redirections . . . . . . . . . . . . . . . . . . . . . . . .6797981818282828486879090919192Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 956.1Constants, Variables and Conversions . . . . . . . . . . . . . . . . . . . . . . . . . 956.1.1 Constant Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 956.1.1.1 Numeric and String Constants . . . . . . . . . . . . . . . . . . . . . . . 956.1.1.2 Octal and Hexadecimal Numbers . . . . . . . . . . . . . . . . . . . . 956.1.1.3 Regular Expression Constants . . . . . . . . . . . . . . . . . . . . . . . 966.1.2 Using Regular Expression Constants . . . . . . . . . . . . . . . . . . . . . . 976.1.3 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 986.1.3.1 Using Variables in a Program . . . . . . . . . . . . . . . . . . . . . . . . 986.1.3.2 Assigning Variables on the Command Line . . . . . . . . . . . 986.1.4 Conversion of Strings and Numbers . . . . . . . . . . . . . . . . . . . . . . . 996.2 Operators: Doing Something With Values . . . . . . . . . . . . . . . . . . . . 1016.2.1 Arithmetic Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1016.2.2 String Concatenation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1026.2.3 Assignment Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1046.2.4 Increment and Decrement Operators . . . . . . . . . . . . . . . . . . . . 1066.3 Truth Values and Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1076.3.1 True and False in awk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1076.3.2 Variable Typing and Comparison Expressions. . . . . . . . . . . . 1086.3.2.1 String Type Versus Numeric Type . . . . . . . . . . . . . . . . . . 1086.3.2.2 Comparison Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1096.3.2.3 String Comparison With POSIX Rules . . . . . . . . . . . . . 1116.3.3 Boolean Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1116.3.4 Conditional Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1136.4 Function Calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1136.5 Operator Precedence (How Operators Nest) . . . . . . . . . . . . . . . . . . 1156.6 Where You Are Makes A Difference . . . . . . . . . . . . . . . . . . . . . . . . . . 116
vi GAWK: Effective AWK Programming7Patterns, Actions, and Variables . . . . . . . . . . . . 1177.1Pattern Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.1.1 Regular Expressions as Patterns . . . . . . . . . . . . . . . . . . . . . . . . .7.1.2 Expressions as Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.1.3 Specifying Record Ranges with Patterns . . . . . . . . . . . . . . . . .7.1.4 The BEGIN and END Special Patterns . . . . . . . . . . . . . . . . . . . . .7.1.4.1 Startup and Cleanup Actions . . . . . . . . . . . . . . . . . . . . . . .7.1.4.2 Input/Output from BEGIN and END Rules . . . . . . . . . . .7.1.5 The BEGINFILE and ENDFILE Special Patterns . . . . . . . . . . .7.1.6 The Empty Pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.2 Using Shell Variables in Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.3 Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.4 Control Statements in Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.4.1 The if-else Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.4.2 The while Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.4.3 The do-while Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.4.4 The for Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.4.5 The switch Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.4.6 The break Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.4.7 The continue Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.4.8 The next Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.4.9 The nextfile Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.4.10 The exit Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.5 Built-in Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7.5.1 Built-in Variables That Control awk . . . . . . . . . . . . . . . . . . . . .7.5.2 Built-in Variables That Convey Information . . . . . . . . . . . . .7.5.3 Using ARGC and ARGV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127128129130131132132133135141Arrays in awk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1438.1The Basics of Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8.1.1 Introduction to Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8.1.2 Referring to an Array Element . . . . . . . . . . . . . . . . . . . . . . . . . .8.1.3 Assigning Array Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8.1.4 Basic Array Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8.1.5 Scanning All Elements of an Array . . . . . . . . . . . . . . . . . . . . . .8.1.6 Using Predefined Array Scanning Orders . . . . . . . . . . . . . . . .8.2 The delete Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8.3 Using Numbers to Subscript Arrays . . . . . . . . . . . . . . . . . . . . . . . . . .8.4 Using Uninitialized Variables as Subscripts . . . . . . . . . . . . . . . . . . .8.5 Multidimensional Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8.5.1 Scanning Multidimensional Arrays. . . . . . . . . . . . . . . . . . . . . . .8.6 Arrays of Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .143143144145145146147149151151152153154
vii9Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1579.1Built-in Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1579.1.1 Calling Built-in Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1579.1.2 Numeric Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1579.1.3 String-Manipulation Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 1599.1.3.1 More About ‘\’ and ‘&’ with sub(), gsub(), andgensub() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1689.1.4 Input/Output Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1719.1.5 Time Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1749.1.6 Bit-Manipulation Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1799.1.7 Getting Type Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1819.1.8 String-Translation Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1819.2 User-Defined Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1829.2.1 Function Definition Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1829.2.2 Function Definition Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 1839.2.3 Calling User-Defined Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 1849.2.3.1 Writing A Function Call . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1859.2.3.2 Controlling Variable Scope . . . . . . . . . . . . . . . . . . . . . . . . . 1859.2.3.3 Passing Function Arguments By Value Or By Reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1879.2.4 The return Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1899.2.5 Functions and Their Effects on Variable Typing . . . . . . . . . 1909.3 Indirect Function Calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190Part II: Problem Solving With awk10A Library of awk Functions . . . . . . . . . . . . . . . . . 19910.1 Naming Library Function Global Variables . . . . . . . . . . . . . . . . . .10.2 General Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.2.1 Converting Strings To Numbers . . . . . . . . . . . . . . . . . . . . . . . .10.2.2 Assertions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.2.3 Rounding Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.2.4 The Cliff Random Number Generator . . . . . . . . . . . . . . . . . .10.2.5 Translating Between Characters and Numbers . . . . . . . . . .10.2.6 Merging an Array into a String . . . . . . . . . . . . . . . . . . . . . . . . .10.2.7 Managing the Time of Day . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.3 Data File Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.3.1 Noting Data File Boundaries . . . . . . . . . . . . . . . . . . . . . . . . . . .10.3.2 Rereading the Current File . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.3.3 Checking for Readable Data Files . . . . . . . . . . . . . . . . . . . . . .10.3.4 Checking For Zero-length Files . . . . . . . . . . . . . . . . . . . . . . . . .10.3.5 Treating Assignments as File Names. . . . . . . . . . . . . . . . . . . .10.4 Processing Command-Line Options . . . . . . . . . . . . . . . . . . . . . . . . .10.5 Reading the User Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.6 Reading the Group Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.7 Traversing Arrays of Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18222226
viii GAWK: Effective AWK Programming11Practical awk Programs . . . . . . . . . . . . . . . . . . . . . 22911.1 Running the Example Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.2 Reinventing Wheels for Fun and Profit . . . . . . . . . . . . . . . . . . . . . .11.2.1 Cutting out Fields and Columns . . . . . . . . . . . . . . . . . . . . . . .11.2.2 Searching for Regular Expressions in Files . . . . . . . . . . . . . .11.2.3 Printing out User Information . . . . . . . . . . . . . . . . . . . . . . . . . .11.2.4 Splitting a Large File into Pieces . . . . . . . . . . . . . . . . . . . . . . .11.2.5 Duplicating Output into Multiple Files . . . . . . . . . . . . . . . . .11.2.6 Printing Nonduplicated Lines of Text. . . . . . . . . . . . . . . . . . .11.2.7 Counting Things . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.3 A Grab Bag of awk Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.3.1 Finding Duplicated Words in a Document . . . . . . . . . . . . . .11.3.2 An Alarm Clock Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.3.3 Transliterating Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.3.4 Printing Mailing Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.3.5 Generating Word-Usage Counts . . . . . . . . . . . . . . . . . . . . . . . .11.3.6 Removing Duplicates from Unsorted Text . . . . . . . . . . . . . .11.3.7 Extracting Programs from Texinfo Source Files . . . . . . . . .11.3.8 A Simple Stream Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.3.9 An Easy Way to Use Library Functions . . . . . . . . . . . . . . . .11.3.10 Finding Anagrams From A Dictionary. . . . . . . . . . . . . . . . .11.3.11 And Now For Something Completely Different. . . . . . . . 59262264270272Part III: Moving Beyond Standard awk Withgawk12Advanced Features of gawk . . . . . . . . . . . . . . . . . 27512.1 Allowing Nondecimal Input Data. . . . . . . . . . . . . . . . . . . . . . . . . . . .12.2 Controlling Array Traversal and Array Sorting . . . . . . . . . . . . . .12.2.1 Controlling Array Traversal . . . . . . . . . . . . . . . . . . . . . . . . . . . .12.2.2 Sorting Array Values and Indices with gawk . . . . . . . . . . . .12.3 Two-Way Communications with Another Process . . . . . . . . . . . .12.4 Using gawk for Network Programming . . . . . . . . . . . . . . . . . . . . . . .12.5 Profiling Your awk Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13275276276280281283285Internationalization with gawk . . . . . . . . . . . . . 28913.1 Internationalization and Localization . . . . . . . . . . . . . . . . . . . . . . . .13.2 GNU gettext . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13.3 Internationalizing awk Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . .13.4 Translating awk Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13.4.1 Extracting Marked Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13.4.2 Rearranging printf Arguments . . . . . . . . . . . . . . . . . . . . . . . .13.4.3 awk Portability Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13.5 A Simple Internationalization Example . . . . . . . . . . . . . . . . . . . . . .13.6 gawk Can Speak Your Language. . . . . . . . . . . . . . . . . . . . . . . . . . . . .289289291293293293294295297
ix14Debugging awk Programs . . . . . . . . . . . . . . . . . . . 29914.1 Introduction to gawk Debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29914.1.1 Debugging in General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29914.1.2 Additional Debugging Concepts . . . . . . . . . . . . . . . . . . . . . . . . 29914.1.3 Awk Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30014.2 Sample Debugging Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30014.2.1 How to Start the Debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30114.2.2 Finding the Bug . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30114.3 Main Debugger Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30414.3.1 Control of Breakpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30414.3.2 Control of Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30614.3.3 Viewing and Changing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 30714.3.4 Dealing with the Stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30814.3.5 Obtaining Information about the Program and the DebuggerState . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30914.3.6 Miscellaneous Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31014.4 Readline Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31214.5 Limitations and Future Plans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31315Arithmetic and Arbitrary PrecisionArithmetic with gawk . . . . . . . . . . . . . . . . . . . . . . . 31515.1 A General Description of Computer Arithmetic . . . . . . . . . . . . . . 31515.1.1 Floating-Point Number Caveats . . . . . . . . . . . . . . . . . . . . . . . . 31515.1.1.1 The String Value Can Lie . . . . . . . . . . . . . . . . . . . . . . . . . 31615.1.1.2 Floating Point Numbers Are Not Abstract Numbers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31615.1.1.3 Standards Versus Existing Practice . . . . . . . . . . . . . . . . 31715.1.2 Mixing Integers And Floating-point . . . . . . . . . . . . . . . . . . . . 31815.2 Understanding Floating-point Programming . . . . . . . . . . . . . . . . . 31915.2.1 Binary Floating-point Representation . . . . . . . . . . . . . . . . . . 32115.2.2 Floating-point Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32215.2.3 Floating-point Rounding Mode . . . . . . . . . . . . . . . . . . . . . . . . . 32215.3 gawk MPFR Powerful Arithmetic . . . . . . . . . . . . . . . . . . . . . . 32415.4 Arbitrary Precision Floating-point Arithmetic with gawk . . . . 32415.4.1 Setting the Working Precision . . . . . . . . . . . . . . . . . . . . . . . . . . 32515.4.2 Setting the Rounding Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32615.4.3 Representing Floating-point Constants . . . . . . . . . . . . . . . . . 32615.4.4 Changing the Precision of a Number . . . . . . . . . . . . . . . . . . . 32715.4.5 Exact Arithmetic with Floa
This is Edition 4.1 of GAWK: Effective AWK Programming: A User's Guide for GNU Awk, for the 4.1.0 (or later) version of the GNU implementation of AWK. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the
This is Edition 3 of GAWK: Effective AWK Programming: A User's Guide for GNU Awk, for the 3.1.4 (or later) version of the GNU implementation of AWK. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the
-This makes awk one of the most powerful of the Unix utilities awk processes fields while sed only processes lines nawk (new awk) is the new standard for awk -Designed to facilitate large awk programs -gawk is a free nawk clone from GNU awk gets it's input from -files -redirection and pipes -directly from standard input
This is Edition 5.1 of GAWK: Effective AWK Programming: A User’s Guide for GNU Awk, for the 5.1.0 (or later) version of the GNU implementation of AWK. Permission is granted to cop
This is Edition 4.1 of GAWK: Effective AWK Programming: A User's Guide for GNU Awk, for the 4.1.4 (or later) version of the GNU implementation of AWK. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by
This is Edition 4.1 of GAWK: Effective AWK Programming: A User's Guide for GNU Awk, for the 4.1.1 (or later) version of the GNU implementation of AWK. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the
This is Edition 4.2 of GAWK: Effective AWK Programming: A User's Guide for GNU Awk, for the 4.2.0 (or later) version of the GNU implementation of AWK. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by
This is Edition 5.0 of GAWK: Effective AWK Programming: A User's Guide for GNU Awk, for the 5.0.0 (or later) version of the GNU implementation of AWK. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by
Principles of Animal Nutrition Applied Animal Science Research Techniques for Bioscientists Principles of Animal Health and Disease 1 Optional Physiology of Electrically Excitable Tissues Animal Behaviour Applied Agricultural and Food Marketing Economic Analysis for Agricultural and Environmental Sciences Physiology and Biotechnology option Core Endocrine Control Systems Reproductive .
