UNIX Systems Programming I
ISystemsProgrammingUNIX Systems Programming IShort Course NotesAlan Dix mmingUNIXISystemsProgramming
CourseOutlineUNIXISystemsProgrammingAlan Dixhttp://www.hcibook.com/alan/Session 1UNIX basicsthe UNIX API, system callsand manualsSession 2file I/O and filtersstandard input and output, read,write and openSession 3makefiles andargumentsmake, argc & argv andenvironment variablesSession 4file manipulationcreating and deleting files anddirectories, links and symboliclinksSession 5terminal I/Osimilarities to file I/O, ttyÊdrivers,stty & gtty, termcap and cursesSession 6signals and timecatching signals and problemsthat arise, delays and finding thetimeSession 7mixing C and scriptsshell scripts, getting informationbetween C and shell, puttingthem togetherUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/i
UNIXIReadingSystemsProgramming The Unix V Environment,Stephen R. Bourne,Wiley, 1987, ISBN 0 201 18484 2The author of the Borne Shell! A 'classic' which deals with systemcalls, the shell and other aspects of UNIX. Unix For Programmers and Users,Graham Glass,Prentice-Hall, 1993, ISBN 0 13 061771 7Slightly more recent book also covering shell and C programming.Ì BEWARE ÐUNIX systems differ in details,check on-line documentation UNIX manual pages:man createtc.Most of the system calls and functions are in section 2 and 3 of themanual. The pages are useful once you get used to reading them! The include files themselves/usr/include/time.hetc.Takes even more getting used to, but the ultimate reference tostructures and definitions on your system.UNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/ii
UNIXISystemsProgrammingSession 1UNIX basics the nature of UNIX the UNIX API system calls and library calls system call conventions how they work UNIXISystemsProgrammingUNIXmanuals and other infoShort Course NotesAlan Dix 1996I/1
UNIXUNIX is an operating systemUNIXfile storenetworks etc.programsIt manages: files and data running programs networks and other resourcesIt is defined by its behaviour (on files etc.) its application programmers interface Ð APIUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/2
UNIX API– the system calls ultimately everything works through system e storeUNIXISystemsProgrammingnetworks etc.Short Course NotesprogramsAlan Dix 1996I/3
first system call – exitvoid exit(int status) program ends! its exit code is set to status available to shell: ? statusÐÐBourne shellC shell actually not a real system call! does some tidying upreal system call is exit example: does some tidying upprogram test-exit.c:main(){exit(3);} run it: cc -o test-exit test-exit.c test-exit echo ?3 UNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/4
system calls and library calls system calls executed by the operating systemperform simple single operations library calls executed in the user programmay perform several tasksmay call system calls distinction blurs often a thin layercompatability with older UNIX calls(e.g. pipe) kinds of library: UNIXISystemsProgrammingfunctions Ð layers and O/S utilitiesstdio & ANSI C librariesÐ platform independent functionsother librariesÐ for specialist purposes (e.g. NAG)UNIXShort Course NotesAlan Dix 1996I/5
system call conventions library functions often return pointersFILE * fp fopen("harry","r"); NULL return for failuresystem calls usually return an integerint res sys call(some args) return value res 0res 0ÐÐOKfailureopposite way round! cannot use as Boolean:if ( sys call(some args) ) { . wrong see the global variable errno for more info many system calls take simple arguments but some take special structuresUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/6
how they work①program gets to the system call in the userÕs codeint res sys call(some parameters)②③puts the parameters on the stackperforms a system ÔtrapÕ Ð hardware dependent now in system mode ④⑤operating system code may copy large datastructures into system memorystarts operationif the operation cannot be completed immediatelyUNIX may run other processes at this point⑥⑦operation complete!if necessary copies result data structures back touser programÕs memory⑧ return to user mode ⑨⑩user program puts return code into resprogram recommences UNIX tries to make it as cheap and fast as possiblebut system calls are still ÔexpensiveÕ(compared to ordinary functions)UNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/7
finding out donÕt expect to remember everything. . . I donÕt! even if you did versions differ places to look: manualsÐÐpaper or using man commandmay need to give man the section:e.g. man 2 stat aproposÐespecially to find the right man pagee.g. apropos directory look at the source!Ðread the include fileÐfind the right include file:fgrep dirent /usr/include/*.hfgrep dirent /usr/include/sys/*.hUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/8
UNIX manuals divided into sections1Ðshell commandse.g. mv, ls, cat2Ðsystem callse.g. exit, read, write3Ðlibrary calls4Ðdevice and network specific info(including stdio)e.g. exit, printfe.g. mv, ls, cat5Ðfile formatse.g. passwd, termcap6Ðgames and demose.g. fortune, worms7Ðmiscellaneouse.g. troff macros, ascii character map8Ðadmin functionse.g. fsck, network daemons UNIX manual reading . . . . . a bit of an artUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/9
UNIXISystemsProgrammingSession 2file I/O and filters standard input and output filters using read and write opening and closing files low-level I/O vs. stdio mixing them using itUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/10
input and outputeach running program has numbered input/outputs:0Ðstandard input often used as input if no file is given default input from the user terminal1Ðstandard output simple program's output goes here default output to the user terminal2Ðstandard error error messages from user default output to the user terminalthese numbers are called file descriptors used by system calls to refer to ort Course NotesAlan Dix 1996I/11
redirecting from the shelldefault input/output is user's terminalredirection to or from files: commandÐ fredstandard input from file 'fred''fred'output012errors commandÐ harrystandard output goes to file 'harry''harry'input012errorsÐN.B. commandÐUNIXISystemsProgrammingfile is created if it doesn't existC shell prevents overwriting harrysimilar, but appends to end of 'harry'Short Course NotesAlan Dix 1996I/12
redirection of standard error command 2 errlogÐstandard error goes to file 'errlog'inputoutput012'errlog' command 2 errlogÐ standard error appends to end of 'errlog'command 2 &1Ðstandard error goes to currentdestination of standard hort Course NotesAlan Dix 1996I/13
filters some commands only work on namedfiles:e.g. copyingÐcp from-fileto-file many take standard input as defaultcat, head, tail, cut, paste, etc. these are called filtersÐ very useful as part of pipes also very easy to program! donÕt have to worry about command line arguments opening or closing files just read-process-writeUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/14
read & writeret �a file descriptor (int), open for readingbuffer in which to put charsmaximum number of bytes to readreturns actual number read ret is 0 at end of file, negative for error buff is not NULL terminatedleave room if you need to add Ô\0Õ!ret Ða file descriptor (int), open for writingbuffer from which to get charsnumber of bytes to writereturns actual number written ret is negative for error, 0 means Òend of fileÓret may be less than len e.g. if OS buffers full* should really check and repeat until all gone * buff need not be NULL terminatedif buff is a C string, use strlen to get its lengthN.B. Both may return negative after interrupt (signal)UNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/15
example – file translation a Macintosh UNIX text file conversion programmain() {char buf[256];for(;;) {int iint n read(0,buf,256);if ( n 0 ) exit(-n);for ( i 0; i n; i )if ( buff[i] ‘\r’ )buff[i] ead from file descriptor 0 Ð standard inputbuffer size is 256 bytesnumber of bytes read goes into n②end of file (0) or error (n 0) both exitthe -n means that end of file gives a zero exit code③Macintosh uses carriage return '\r' to end linesUNIX uses newline '\n'④writing to 1 Ð standard outputremember only write n bytes not 256!UNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/16
opening and closing files#include fcntl.h int fd open(name,flags)char *name Ðint flags Ðint fdÐname of the file to openread/write/append etc.returns a file descriptor in simple use flags is one of:O RDONLYÐread onlyO WRONLYÐwrite onlyO RDWRÐread and writebut can ÔorÕ in other options(0)(1)(2) negative result on failure file doesnÕt exist do not have permissions closing files is simpler!int res close(fd)intintUNIXISystemsProgrammingfdretÐÐan open file descriptorreturns:0 OK-1failedShort Course NotesAlan Dix 1996I/17
low-level I/O vs. stdio why use low-level I/O? less layers more efficient more control Ð e.g. random access sometimes have to Ð e.g. networks can you mix low-level I/O and stdio? yes with care! different files/devices no problem same file stdio is bufferedprintf("hello ");write(1,"there ",6);printf("world\n"); output:there hello worldUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/18
Hands on write a simple cypher filter: cypher.c the filter should copy standard input to standardoutput adding 1 to each byte:a b, b c, c d, d e, etc. it will be similar to the Mac UNIX translatorexcept that the loop should add 1 rather thanreplace carriage returns with line feeds to make a better cypher, you could add a differentnumber or have a 256 byte array to act as the cypher compile either with ÔccÕ Ð the old K&R C compiler:cc -o cypher cypher.c or with ÔaccÕ Ð the ANSI C compiler:cc -o cypher cypher.cUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/19
UNIXISystemsProgrammingSession 3makefiles andarguments make argv and argc environment variables using itUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/20
makeÔmakeÕ is a UNIX command which: automates program construction and linking tracks dependencies keeps things up-to-date after changesto use it: construct a file with rules in ityou can call it anything, but ÔmakefileÕ is the default run ÔmakeÕ itselfmake target–(uses the default makefile)make -f myfile target–(uses the rule file myfile)either rebuilds the program ÔtargetÕ if necessary each makefile consists of: definitions rules rules say how one thing depends on anotherthey are either: specificÐ e.g. to make mail-client do this . genericÐ e.g. to make any Ô.oÕ from its Ô.cÕ .make is also available in many other programming environmentsUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/21
makefile formatDefinitions general form:variable value example:SDIR tcpMYLIBS (SDIR)/libN.B. one variable used in another's definition make variables are referred to later using e.g. (SDIR), (MYLIBS) expanded like #defines or shell variables(some versions of make will expand shell variables also)Rules(just specific rules) general form:target : dependent1 dependent2 .command-line N.B. this must be a tab example:myprog: myprog.o another.occ -o myprog myprog.o another.o (MYLIBS)this says:to make myprog you need myprog.o and another.oif either of them is newer than myprog rebuild it using thethen rebuild it using the command: Òcc -o myprog .ÓUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/22
argc & argvint main( int argc, char **argv ) .int main( int argc, char *argv[ ] ) .or: one of the ways to get information into a C program in UNIX you type:myprog a "b c" dthe program gets:argcargv[0]argv[1]argv[2]argv[3]argv[4]N.B. 4 "myprog" "a" "b c" "d" NULLÐlength of argvÐprogram nameÐsingle second argumentÐterminatorDOS is identical (except argv[0] is NULL early versions)argc is one less than the number of arguments! other ways to get information in (UNIX & DOS): configuration file (known name) standard input environment variables using getenv()or (UNIX only) third arg to main:main(int argc, char **argv, char **envp)UNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/23
environment variables set of name value mappings most created during start-up (.profile, .login etc.)setting a variable from the shell:myvar hellovar2 " a value with spaces needs to be quoted"export myvar no spaces before or after ' ' sign variables need to be exported to be seenby other programs run from the shell in C shell: "set name val" and no exportlisting all variables from the shell: setHOME /home/staff2/alanmyvar helloPATH /local/bin:/bin:/local/X/binUSER alan. . . UNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/24
environment variables – 2 accessing from a program Ð 3 methods①extra argument to mainmain(int argc,char **argv,char **envp)②global variableextern char **environ③system functionchar *value getenv(name); both ① and ② give you a structure similar to argv a null terminated array of strings but environment strings are of the formname value the getenv function ③ rather different fetches the value associated with name returns NULL if name not defined also a putenv function only available to this process and its children not visible to the shell that invoked itUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/25
Hands on write a program to produce a list of arguments as inthe 'argc & argv' slide the core of your program will look something like:for(i 0; i argc; i )printf("argv[%d] %s\n",argv[i]); if you use ÔccÕ then the ÔmainÕ program begins:main(argc,argv)intargc;char **argv;the slide shows the ANSI C declaration do a similar program for environment variablesUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/26
UNIXISystemsProgrammingSession 4file manipulation creating new files ÔdeletingÕ files linking to existing files symbolic links renaming files creating/removing directories using itUNIXISystemsProgramming\Short Course NotesAlan Dix 1996I/27
creating filesint fd creat(path,mode)char *pathint modeint fd ÐÐÐthe name/path of the (new) filepermissions for the new filereturns a file descriptorfile is created if it doesnÕt existif it already exists, acts like an open for writingnegative return on failuremode sets the initial permissions, e.g.mode S RWXUSR S IRGRP S IXGRP S IXOTHÐ read/write/execute for user (S RWXUSR)Ð read/execute for group (S IRGRP S IXGRP)Ð execute only for others (S IXOTH) when created, file descriptor is open for writing even if permissions do not include write access alternative Ð use open with special flags:int fd open( path, O WRONLY O CREAT O TRUNC, mode ) O CREAT flag says Òcreate if it doesnÕt existÓ note extra mode parameterUNIXISystemsProgramming\Short Course NotesAlan Dix 1996I/28
deleting files UNIXrm command ÔdeletesÕ files it uses the unlink system callint res unlink(path)char *pathint modeint resÐÐÐthe name/path of the file to removepermissions for the new filereturns an integer0 Ð OK-1 Ð fail doesnÕt necessarily delete file! but neither does rm UNIX filenames are pointers to the file there may be more than one pointerUNIXISystemsProgramming\Short Course NotesAlan Dix 1996I/29
hard links linking to an existing file: shellÐ ln tom fred system call Ð link("tom","fred") file tom must already exist fred points to the same file as tomfredtom unlink simply removes a pointer file destroyed only when last link goesUNIXISystemsProgramming\Short Course NotesAlan Dix 1996I/30
symbolic links ÔhardÕ links are limited: cannot link to a different disk only one link to a directory(actually not quite true as there are all the Ò.Ó links) symbolic links are more flexible shellÐ ln -s tom fred system call Ð sym link("tom","fred") tom need not exist fred points to the name ÔtomÕ Ð an aliasfredUNIXISystemsProgrammingtom\Short Course NotesAlan Dix 1996I/31
links and updatescp fred tomfredtomln fred tomfredtomln -s fred tomfredtomfredfredtomfredtomfred? update file tomfredtomtom delete file tom Ð unlink("tom")fredtomfredtom what is in fred?fredUNIXISystemsProgrammingfred\Short Course NotesAlan Dix 1996I/32
renaming filesint res rename(path1,path2)char *pathint fdÐÐthe name/path of the (new) filereturns a file descriptor system call used by UNIX mv command only possible within a file system① path2 is unlinked② path2 is linked to the file pointed to by path1③ path1 is unlinked result: path2 points to the file path1 used to point ingtom\Short Course NotesfredtomAlan Dix 1996I/33
directories special system callsto create and remove directoriesint res mkdir(path,mode)char *pathint modeint resÐÐÐthe path of the new directorypermissions for the new directoryreturns an integer0 Ð OK-1 Ð fail mkdir rather like creatint res rmdir(path)char *pathint resÐÐthe path of the directory to removereturns an integer0 Ð OK-1 Ð fail unlike unlink does delete directory! but only when emptyUNIXISystemsProgramming\Short Course NotesAlan Dix 1996I/34
Hands on rm has various options so it is hard to delete files with strange namessuch as Ô-bÕ Ð I know I got one once! write a program raw-rm.c which has one commandline argument, a file to delete, and performs anunlink system call on it modify raw-rm so that it can take a list of files:raw-rm tom dick harry write a version of mv that doesnÕt use the renamesystem call, but only link and unlinkN.B.UNIXISystemsProgrammingif you get the order wrong youÕll loose the file!\Short Course NotesAlan Dix 1996I/35
UNIXISystemsProgrammingSession 5terminal I/O terminals are just files? tty drivers stty and gtty handling input the termcap database toolkits using itUNIXISystemsProgrammingShort Course NotesAlan Dix 1996I/36
terminals are easy? terminal is default input/output read and write just like any file use read/write system callsor stdio interactive programs Ð a doddleprintf("what is your name? ");gets(buff);printf("hello %s how are you today\n",buff); get line editing for free paper-roll model of interaction only see user input in whole lines terminals not quite like other files: UNIXISystemsProgrammingwrite anywhere on screencursor movementspecial effects (flashing, colours etc.)non-standard keys: ctrl, alt, F3 etc.Short Course NotesAlan Dix 1996I/37
tty drivers never connected directly to terminal tty driver sits between does line editinghandles break keys and flow control(ctrl-C, ctrl-\, ctrl-S/ctrl-Q) translates delete/end-of-line chars not always wanted!yourprogramUNIX kernalUNIXISystemsProgrammingtty driverShort Course NotesAlan Dix 1996I/38
stty command control the tty driver from the shell stty everything stty -echo type something Ð no echo reset stty differs between UNIXs! can control UNIXISystemsProgrammingechoingparity, stop bits etc. for modemscarriage return / newline mappingdelete key mapping (delete/backspace)break key activityline editing. . . and moreShort Course NotesAlan Dix 1996I/39
gtty & stty functions you can do the same from a program#include sgtty.h int echo off(tty fd)int tty fd;{struct sgttyb buf;gtty(tty f
Unix For Programmers and Users, Graham Glass, Prentice-Hall, 1993, ISBN 0 13 061771 7 Slightly more recent book also covering shell and C programming. Ì BEWARE — UNIX systems differ in details, check on-line documentation UNIX manual pages: man creat etc.
Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction to UNIX (i.e. Unix for Windows Users) The cold hard truth · this course is NOT sponsored by the CS dept. · you will not receive any credit at all introduce ourselv
UNIX and POSIX APIs: The POSIX APIs, The UNIX and POSIX Development Environment, API Common Characteristics. UNIT – 2 6 Hours UNIX Files: File Types, The UNIX and POSIX File System, The UNIX and POSIX File Attributes, Inodes in UNIX
Texts of Wow Rosh Hashana II 5780 - Congregation Shearith Israel, Atlanta Georgia Wow ׳ג ׳א:׳א תישארב (א) ׃ץרֶָֽאָּהָּ תאֵֵ֥וְּ םִימִַׁ֖שַָּה תאֵֵ֥ םיקִִ֑לֹאֱ ארָָּ֣ Îָּ תישִִׁ֖ארֵ Îְּ(ב) חַורְָּ֣ו ם
UNIX Files: File Types, The UNIX and POSIX File System, The UNIX and POSIX File Attributes, Inodes in UNIX System V, Application Program Interface to Files, UNIX Kernel Support for Files, Relationship of C Stream Pointers and File Descriptors, Directory Files, Hard and Symbolic Links. UNIT – 3 7 Hours
UNIX system services UNIX kernel in C computer SH The shell (sh) is a program (written in C) that interprets commands typed to it, and carries out the desired actions. The shell is that part of Unix that most users see. Therefore there is a mistaken belief that sh is Unix. sh is an applications program running under Unix
UNIX Network Programming, Richard Stevens. Fall 1999 CSC209: Software Tools & Systems Programming Slide 12 Course Content Why UNIX? History UNIX Basics: Processes, Login Shells: command processing, running programs, shell programming I/O: file descriptors vs. streams
UNIX operating system, we will try to place our observations in a wider context thanjustthe UNIXsystem or one particular version of the UNIX system. UNIX system security is neither better nor worse than that of other systems. Any system that provides the same facilities as the UNIX system will necessarily have similar hazards.
Unix was originally developed in 1969 by a group of AT&T employees Ken Thompson, Dennis Ritchie, Douglas McIlroy, and Joe Ossanna at Bell Labs. There are various Unix variants available in the market. Solaris Unix, AIX, HP Unix and BSD are a few examples. Linux is also a flavor of Unix which is freely available.
