BGGN 213 - GitHub Pages

2y ago
49 Views
2 Downloads
3.72 MB
64 Pages
Last View : 13d ago
Last Download : 2m ago
Upload by : Roy Essex
Transcription

BGGN 213Introduction to UNIXBarry Granthttp://thegrantlab.org/bggn213

Recap From Last Time: Substitution matrices: Where our alignment match and mis-matchscores typically come from Comparing methods: The trade-off between sensitivity, selectivityand performance Sequence motifs and patterns: Finding functional cues fromconservation patterns Sequence profiles and position specific scoring matrices (PSSMs),Building and searching with profiles, Their advantages andlimitations PSI-BLAST algorithm: Application of iterative PSSM searching toimprove BLAST sensitivity Hidden Markov models (HMMs): More versatile probabilistic modelfor detection of remote similarities

Todays MenuTimeTopicsI9:00-9:15 AMSetup and MotivationII9:15-10:00 AMBeginning Unix10:00-10:40 AMHands on sessionIII10:40-11:00 AMWorking with UnixIV11:00-11:20 PMHow to Get Working11:20-11:45 PMHands on sessionhttps://bioboot.github.io/bggn213 f17/setup/

Prmleobhttps://bioboot.github.io/bggn213 f17/setup/Mac: Terminal or PC: MoblXtermPC: MoblXterm CygUtils plugin (core UNIX tools)Downloaded our class specific jetstream keyfile(See email: This is required for connecting tojetstream virtual machines with your terminal.)Example data downloaded: bggn213 01 unix.zips?Setup Checklist

DourYoitself!Lets get started MacTerminalPCMobaXterm

SideNote: Terminal vs ShellShell: A command-line interface that allows a user toUpinteract with the operating system by typing commands. ingTerminal [emulator]: A graphical interface to the shellusers: open(i.e.a Terminalthe window you get when you launch MobaXterm). dows users: install MobaXterm and then open aminalShell prompt

Introduction ToShell

MotivationWhy do we use Unix?

ModularityCore programs are modular and workwell with othersProgrammabilityBest software developmentenvironmentInfrastructureAccess to existing tools and cuttingedge methodsReliabilityUnparalleled uptime and stabilityUnix PhilosophyEncourages open standards

ModularityCore programs are modular and workwell with othersProgrammabilityBest software developmentenvironmentInfrastructureAccess to existing tools and cuttingedge methodsReliabilityUnparalleled uptime and stabilityUnix PhilosophyEncourages open standards

ModularityThe Unix shell was designed to allow users to easily buildcomplex workflows by interfacing smaller modularprograms together.wgetawkgrepsortuniqplotAn alternative approach is to write a single complexprogram that takes raw data as input, and after hours ofdata processing, outputs publication figures and a finaltable of results.All-in-one custom ‘Monster’ program

Which would you prefer and why?ModularvsCustom

Advantages/DisadvantagesThe ‘monster approach’ is customized to a particular projectbut results in massive, fragile and difficult to modify (thereforeinflexible, untransferable, and error prone) code.With modular workflows, it’s easier to: Spot errors and figure out where they’re occurring byinspecting intermediate results. Experiment with alternative methods by swapping outcomponents. Tackle novel problems by remixing existing modulartools.

Unix ‘Philosophy’“Write programs that do onething and do it well. Writeprograms to work together andthat encourage open standards.Write programs to handle textstreams, because that is auniversal interface.”— Doug McIlory

Unix family tree [1969-2010]LINUXMacOS XSource: https://commons.wikimedia.org/wiki/File:Unix history-simple.svg

BasicsFile ControlViewing spwdmkdirtailwcsedkillmanrmnanocurlsudoCrl-cssh (pipe)touchsourcegitCrl-zscp (write to file)catRbg (read from lated

BasicsFile ControlViewing spwdmkdirtailwcsedkillmanrmnanocurlsudoCrl-cssh (pipe)touchsourcegitCrl-zscp (write to file)catRbg (read from lated

DourYoitself!Lets get started MacTerminalPCMobaXterm

Beginning UnixGetting started with basic Unix commandsDownload the example data and mv to your Desktopbggn213 01 unix.zip

File System Structure Information in the file system is stored in files, which are stored indirectories (folders). Directories can also store other directories,which forms a directory tree. The forward slash character ‘/’ is used to represent the rootdirectory of the whole file system, and is also used to separatedirectory names. E.g. /home/jono/work/bggn213 notes.txt

Basics: Using the filesystemlsList files and directoriescdChange directory (i.e. move to a different ‘folder’)pwdPrint working directory (which folder are you in)mkdir MaKe a new DIRectoriescpCoPy a file or directory to somewhere elsemvMoVe a file or directory (basically rename)rmReMove a file or directory

Side Note: File Paths An absolute path specifies a location from the root ofthe file system. E.g. /home/jono/work/bggn213 notes.txt A relative path specifies a location starting from thecurrent location. E.g. ./bggn213 notes.txt.Single dot ‘.’ (for current directory).Double dot ’.’ (for parent directory) Tilda ‘ ’ (for your home directory)[Tab] Pressing the tab key can autocomplete names

Finding the Right Hammer(man and apropos) You can access the manual (i.e. userdocumentation) on a command with man, e.g: man pwd The man page is only helpful if you know thename of the command you’re looking for.apropos will search the man pages for keywords. apropos "working directory"

Inspecting text files less - visualize a text file: use arrow keys page down/page up with “space”/“b” keys search by typing "/" quit by typing "q" Also see: head, tail, cat, more

Creating text filesCreating files can be done in a few ways: With a text editor (such as nano, emacs, or vi) With the touch command ( touch a file) From the command line with cat or echo andredirection (more on this later) nano is a simple text editor that isrecommended for first-time users. Other texteditors have more powerful features but alsosteep learning curves

DourYoit nano yourfilename.txt There are many other text file editors (e.g. vim,emacs and sublime text, etc.)lf!In the terminal type:seCreating and editingtext files with nano

Connecting to remotemachines (with ssh) Most high-performance computing (HPC)resources can only be accessed by ssh(Secure SHell) ssh [user@host.address]For example: ssh barry@bio3d.ucsd.edu ssh tb170077@IP ADDRESS

Connecting to jetstream(with ssh) First we have to login online and fire-up a new“virtual machine” Full instructions here:https://bioboot.github.io/bggn213 f17/jetstream/boot/ We will go through this process in the lastsection today!

Copying to and from remotemachines (scp) The scp (Secure CoPy) command can be used tocopy files and directories from one computer toanother. scp [file] [user@host]:[destination] scp localfile.txt bgrant@bigcomputer.net:/remotedir/.

BasicsFile ControlViewing spwdmkdirtailwcsedkillmanrmnanocurlsudoCrl-cssh (pipe)touchsourcegitCrl-zscp (write to file)catRbg (read from lated

Process refers to a running instance of a programtopProvides a real-time view of all running processespsReport a snapshot of the current processeskillTerminate a process (the “force quit” of the unix world)Crl-c Stop a jobCrl-zSuspend a jobbgResume a suspended job in the backgroundfgResume a suspended job in the foreground&Start a job in the background

DourYoitself!Hands-on timeSections 1 to 3 of software carpentry UNIX lessonhttps://swcarpentry.github.io/shell-novice/ 20 mins

Working with UnixHow do we actually use Unix?

BasicsFile ControlViewing spwdmkdirtailwcsedkillmanrmnanocurlsudoCrl-cssh (pipe)touchsourcegitCrl-zcatRbgpythonfg (write to file) (read from file)Misc.usefulPowercommandsProcessrelated

Combining Utilities withRedirection ( , ) and Pipes ( ) The power of the shell lies in the ability tocombine simple utilities (i.e. commands) intomore complex algorithms very quickly. A key element of this is the ability to send theoutput from one command into a file or to pass itdirectly to another program. This is the job of , and

Side-Note: Standard Input andStandard Output streamsTwo very important concepts that unpin Unixworkflows: Standard Output (stdout) - default destination ofa program's output. It is generally the terminalscreen. Standard Input (stdin) - default source of aprogram's input. It is generally the commandline.

DourYoitOutput redirection and pipingself! ls /usr/bin # stdin is “/usr/bin”; stdout to screen

Output redirection and piping ls /usr/bin # stdin is “/usr/bin”; stdout to screen ls /usr/bin binlist.txt # stdout redirected to file ls /usr/bin less # sdout piped to less (no file created)

Output redirection and piping -arg ls /usr/bin # stdin is “/usr/bin”; stdout to screen ls /usr/bin binlist.txt # stdout redirected to file ls /usr/bin less # sdout piped to less (no file created) ls -l /usr/bin # extra optional input argument “-l”

Output redirection and piping ls /usr/bin # stdin is “/usr/bin”; stdout to screen ls /usr/bin binlist.txt # stdout redirected to file ls /usr/bin less # sdout piped to less (no file created) ls /nodirexists/ # stderr to screen

DourYoitOutput redirection and pipingself! ls /usr/bin # stdin is “/usr/bin”; stdout to screen ls /usr/bin binlist.txt # stdout redirected to file ls /usr/bin less # sdout piped to less (no file created) ls /nodirexists/ binlist.txt # stderr to screen

DourYoitOutput redirection and pipingse2 ls /usr/bin # stdin is “/usr/bin”; stdout to screen ls /usr/bin binlist.txt # stdout redirected to file ls /usr/bin less # sdout piped to less (no file created) ls /nodirexists/ 2 binlist.txt # stderr to filelf!

Output redirection summary -arg2 2

ls -l

ls -l list of files

ls -l grep partial name list of filesWe have piped ( ) the stdout of one commandinto the stdin of another command!

DourYoitls -l /usr/bin/ grep “tree” list of filesself!grep: prints lines containing a string.Also searches for strings in text files.

BasicsFile ControlViewing spwdmkdirtailwcsedkillmanrmnanocurlsudoCrl-cssh (pipe)touchsourcegitCrl-zcatRbgpythonfg (write to file) (read from file)Misc.usefulPowercommandsProcessrelated

DourYoitSide-Note: grep ‘power command’self! grep - prints lines containing a string pattern. Also searchesfor strings in text files, e.g. grep --color "GESGKS" KIIHEAGYSEEECKQYK grep is a ‘power tool’ that is often used with pipes as itaccepts regular expressions as input (e.g. “G.GK[ST]”)and has lots of useful options - see the man page for details.

DourYoitsegrep example usingregular expressionslf! Suppose a program that you are working with complains thatyour input sequence file contains non-nucleotide characters.You can eye-ball your file or grep -v " " seqdump.fasta grep --color "[ ATGC]"Exercises:(1). Use “man grep” to find out whatthe -v argument option is doing!(2). How could we also show line numberfor each match along with the output?(tip you can grep the output of“man grep” for ‘line number’)

DourYoitsegrep example usingregular expressionslf! Suppose a program that you are working with complains thatyour input sequence file contains non-nucleotide characters.You can eye-ball your file or grep -v " " seqdump.fasta grep --color -n "[ ATGC]" First we remove (with -v option) lines that start with a “ ”character (these are sequence identifiers). Next we find characters that are not A, T, C or G. To do this weuse symbols second meaning: match anything but thepattern in square brackets. We also print line number (with -noption) and color output (with --color option).

Key Point: Pipes and redirectsavoid unnecessary i/o Disc i/o is often a bottleneck in data processing! Pipes prevent unnecessary disc i/o operations byconnecting the stdout of one process to the stdin ofanother (these are frequently called “streams”) program1 input.txt 2 program1.stderr \program2 2 program2.stderr results.txt Pipes and redirects allow us to build solutions frommodular parts that work with stdin and stdoutstreams.

Unix ‘Philosophy’ Revisited“Write programs that do onething and do it well. Writeprograms to work together andthat encourage open standards.Write programs to handle textstreams, because that is auniversal interface.”— Doug McIlory

Pipes provide speed, flexibility andsometimes simplicity In 1986 “Communications of the ACM magazine” asked famouscomputer scientist Donald Knuth to write a simple program tocount and print the k most common words in a file alongside theircounts, in descending order. Kunth wrote a literate programming solution that was 7 pageslong, and also highly customized to this problem (e.g. Kunthimplemented a custom data structure for counting English words). Doug McIlroy replied with one line: cat input.txt tr A-Z a-z sort uniq -c sort -rn sed 10q

Key Point:You can chain any number of programstogether to achieve your goal!This allows you to build up fairly complexworkflows within one command-line.

DourYoitself!Hands-on timeSection 4 of software carpentry UNIX lessonhttps://swcarpentry.github.io/shell-novice/ 15 mins

DourYoitseShell scriptinglf!#!/bin/bash# This is a very simple hello world script.echo "Hello, world!”Exercise: Create a "Hello world"-like script using command line toolsand execute it. Copy and alter your script to redirect output to a file using along with a list of files in your home directory. Alter your script to use instead of . What effect doesthis have on its behavior?

DourYoitseVariables in shell scriptslf!#!/bin/bash# Another simple hello world scriptmessage 'Hello World!'echo message “message” - is a variable to which the string 'HelloWorld!' is assigned echo - prints to screen the contents of the variable" message"

Side-Note: Environment Variables

PATH ‘special’environment variable What is the output of this command? echo PATH Note the structure: path1 : path2 : path3 PATH is an environmental variable which Bashuses to search for commands typed on thecommand line without a full path. Exercise: Use the command env to discover more.

Summary Built-in unix shell commands allow for easy datamanipulation (e.g. sort, grep, etc.) Commands can be easily combined to generateflexible solutions to data manipulation tasks. The unix shell allows users to automate repetitivetasks through the use of shell scripts that promotereproducibility and easy troubleshooting Introduced the 21 key unix commands that you willuse during 95% of your future unix work

BasicsFile ControlViewing spwdmkdirtailwcsedkillmanrmnanocurlsudoCrl-cssh (pipe)touchsourcegitCrl-zcatRbgpythonfg (write to file) (read from file)Misc.usefulPowercommandsProcessrelated

How to Get Working1) Connecting to jetstream2) Best practices for organizing yourcomputational biology projects

Read: Noble PLoS Comp Biol (2009)- “A Quick Guide to Organizing Computational Biology icle?id 10.1371/journal.pcbi.1000424All files and directories used in your project should live in a singleproject directory. Use sub-directories to divide your project into sub-projects. Do not use spaces in file and directory names!Document your methods and workflows with plain text README files Also document the origin of all data in your project directory Also document the versions of the software that you ran and theoptions you used. Consider using Markdown for your documentation.Use version control and backup to multiple destinations!Be lity-guide/sections/introduction/

tmux python fg. Basics File Control Viewing & Editing Files Misc. useful Power commands Process related ls mv less chmod grep top cd cp head echo find ps . fg Resume a suspended job in the foreground & Start

Related Documents:

034 3-313 (213) 628-7734 Judge Efrain M. Aceves 035 3-311 (213) 628-7735 Judge Dorothy B. Reyes 036 3-307 (213) 628-7736 Judge Enrique Monguia 037 3-305 (213) 628-7737 Judge Gustavo N. Sztraicher 038 3-303 (213) 628-7738 Judge Alison M. Estrada 040 5-309 (213) 628-7740 Judge Wendy Segall 041 5-312 (213) 628-7741 Judge Michael E. Pastor

contents page 2 fuel consumption pages 3-6 fiat 500 pages 7-10 fiat 500c pages 11-13 fiat 500 dolcevita pages 14-16 fiat 500 120th anniversary pages 17-21 fiat 500x pages 22-24 fiat 500x 120th anniversary pages 25-27 fiat 500x s-design pages 28-31 fiat 500l pages 32-35 fiat 500l 120th anniversary pages 36-39 tipo hatchback pages 40-43 tipo station wagon pages 44-47 tipo s-design

Pipe Fittings. pages 32-37. Unpolished Fittings. pages 74-80. Polished Fittings. pages 64-73. European Fittings. pages 81-85. Filters / Strainers. pages 111-117. Custom Fabrications. pages 109. Swivels. pages 140-141. Instrumentation. pages 118-133. Air Fittings. pages 162-170. High Press. Quick Disc. pages 171-179. Check Valves. pages 214-222 .

32 Matthews John J 100 32 Mcdonald Pascal A 100 32 Mcgoorty Joseph J 100 32 Mcguire Collin J 100 32 Mcmahon Patrick D 100 32 Merenda Matthew J 100. . 213 Anderson Peter J 95 213 Andrade Frank D 95 213 Andree Christopher 95 213 Andriano John IV 95 213 Angebrandt

District Nursing Services (213) 202-7580 Office of Communications and Media District (213) 241-6766 Service Center (213)241-1000 Office of EmergencyServices 3889 Division of District Operations (213) 241-5337 Office of Environmental Health and Safety (213) 241-3199

5 Final Year 33 124 158 213 5.1 New Project Plan Development 33 124 158 213 5.2 Renewal Quantity Management Reward 33 124 158 213 5.3 Procurement Process 33 125 158 213 5.4 Hand Back of Assets 33 128 158 213 6 Close-out 34 130 159 214 6.1 Contract End 34 130 159 214 6.2 Defects Liability 34 130 159 214

Blood Typing Lab pages 23-29 binder pages 4-6 Fingerprinting Lab pages 30-31 binder page 7 Blood Spatter Lab pages 32-43 binder pages 8-13 Shoe Impressions pages 44- 45 binder page 14 Pathology pages 46-48 binder pages 15-18 ****DNA pages 49-50 binder pages *****must be done last

ANATOMI & FISIOLOGI SISTEM LIMFATIK DAN KONSEP IMUN Atika Dalili Akhmad, M. Sc., Apt . PENDAHULUAN 20 L cairan plasma difiltrasi keluar menuju bagian interstisial, 17 L direabsorpsi oleh pembuluh darah, BAGAIMANA 3 L SISANYA ? Sistem Limfatik sistem yang terdiri dari pembuluh, sel, dan organ yang membawa kelebihan cairan insterstisial ke dalam aliran darah dan filter patogen dari darah. FUNGSI .