3
<title>Introduction To Bash Shell Scripting (2004.11.06)</title>
6
<h1>Introduction To Bash Shell Scripting</h1>
9
<li>Learning the Bash Shell, Cameron Newham & Bill Rosenblatt (O'Reilly & Associates, Inc)
10
<li>Linux in a Nutshell, Jessica Perry Hekman (O'Reilly & Associates, Inc)
11
<li>The UNIX Programming Environment, Brian W Kernigham & Rob Pike (Prenice Hall)
12
<li>Unix Power Tools, Jerry Peek, Tim O'Relly, & Mike Lookides (O'Reilly & Associates,Inc)
13
<li><a href="http://www.tldp.org/LDP/abs/html">Advance Bash Scripting Guide</a> : http://www.tldp.org/LDP/abs/html
14
<br>Appendix B has some good Reference Cards
18
<li><h3>Getting Help</h3>
20
<li>man Get off information about commands
21
<li>man -k Get off information via keyword
22
<li>info Read info documents
23
<li>help Show help on shell built-in commands
24
<li>command --help Help on command .i.e. cp --help Will give help on cp
25
<li>bash -c "help" Short help on bash
26
<li>bash -c "help set" Short help on bash options
28
<li><h3>Special Characters</h3>
30
<li>Space Argument separator
31
<li>\ Quote a single character
32
A \ followed by a carriage return extend the current line
33
<li>' Quote rext with spaces in it i.e. 'Hello world'
34
<li>$' Quote allows string expansion, backslash-escaped characters
35
<li>" Quote rext, expands variables and command substitution
36
<li>` Command substitution i.e. echo "The date is `date`"
37
<li>$ Denote a shell variable
38
<li># Comments the rest of the line
39
<li>; Commands separator
40
<li>(commands) Run multiple commands in a subshell
41
<li>Control-C Interrupt a command
42
<li>Control-D Sends End of File from terminal
43
<li>Control-U Erases the entire command line
44
<li>Control-\ Is a stronger terminate than Control-c
45
<li>Control-Z Suspend process
47
<li><h3>Redirection, Pipes and Filters</h3>
49
<li>- Standard In for some commands
53
<li>>2&1 Redirect standard error to standard out
54
<li>&> Redirect standard error & standard out
57
<li>>> Concatenate output to file
58
<li><< Here Is File for scripts
60
<li>tee Command write to file and standard out
61
<li>tee -a FILE #Allow appending to FILE
62
<li>xargs Build command from standardin
64
<li><h3>Wildcards</h3>
66
<li>* Wildcard for any character(s)
67
<li>? Wildcard for single character
68
<li>[set] Wildcard for character in set
69
<li>[^set ] Wildcard for not the character in the set
70
<li>[!set ] Wildcard for not the character in the set
71
<li>{ab,dc} Wildcard for alternate between commas
72
<li>All wildcard work with existing files
73
<li>Only {} alternate work to create files
76
<li><h3>Process Control</h3>
78
<li>(<i><b>command1</b></i>; <i><b>command</b></i>) Run command1, then command2 in subshell
79
<li><i><b>command1&&command</b></i> Run command1, then command2 if command1 successes
80
<li><i><b>command1||command</b></i> Run command1, then command2 if command1 fails
82
<li><h3>Regular Expressions</h3>
83
<ul><pre><li>Definition: Text pattern of text character and meta characters
84
<li>Some Meta characters<ul>
85
<li>Escape character \
86
<li>Single Character Meta characters<ul>
87
<li>. Matches any one character
88
<li>[...] Matches any one character in a set
89
<li>[^...] Matches any one character not in the set
92
<li>* Matches the previous character zero or more times
93
<li>\{n\} Matches the previous character n times
94
<li>\{n,m\} Matches the previous character at least n & at most m
95
<li>\{n,\} Matches the previous character n or more times</ul>
97
<li>^ Matching at the start the line
98
<li>$ Matching at the end of line</ul>
101
<li>Commands that use regular expressions<ul><pre>
102
<li>awk Pattern scanning and text processing language
103
<li>ed Line-oriented text editor
104
<li>egrep extended grep
105
<li>emacs Emacs full screen text editor
106
<li>ex Line-oriented text editor
107
<li>expr Command evaluates an expression
108
<li>fgrep Grep from patterns in a file
109
<li>gawk GNU pattern scanning and processing language
110
<li>grep Searches file for pattern (also see fgrep & egrep)
111
<ul>grep [OPTIONS] PATTERN [INPUT-FILE...]
114
<li>-e pattern (for multiple pattern on line)
117
<li>-l Only list files containing pattern
118
<li>-q Quit (No output, only Return Code)
119
<li>-v Invert sense mode</ul>
120
<li>perl Perl scripting
121
<li>python Python scripting
122
<li>sed Applies a set of user-specified editing command to a file
123
<ul>sed [OPTIONS] 'sed_command' [INPUT_FILE...]
124
<li>-n Suppress automatic printing
125
<li>-e expression - sed_command
126
<li>substitute other_text for some_text
127
sed 's/some_text/other_text/g' FILE > NEWFILE
129
sed -e 's@abc@def@g' -e 's@xyz@mno@g' FILE
130
<li>print out line with faq in them
132
<li>change Page ### to (Page ###) at end of line
133
sed 's/Page [0-9]+$/(&)/' file # & replace the match
134
<li>delete blank lines
135
sed '/^[ \t]*$/d</ul>
136
<li>tcl Tool command language
137
<li>vi Full screen text editor
140
<h2>Shell Variables</h2>
142
<li><h3>Built-in Shell Variables</h3>
144
<li>CDPATH Path of shortcuts for cd (like PATH)
145
<li>COLUMNS Numbers of columns on the display
146
<li>EDITOR Path for editor
147
<li>HISTSIZE Number of commands in command history (default 500)
148
<li>IFS Input Field Separator
149
<li>LINES Numbers of lines on the display
150
<li>OFS Output Field Separator
151
<li>SECONDS Seconds that this shell is running
152
<li>SHELLOPT Colon separate list of shell options
154
<li><h3>Environment Variables</h3>
156
<li>export var Will make a variable an environment variable
157
<li>HOME User's home directory
158
<li>LOGNAME User's name
159
<li>MAIL Name of user's mailbox
160
<li>PATH List of directories to be search by the shell
161
<br> to find programs whose names are type as commands
162
<li>PS1 String that is used by the shell prompt
163
<li>PWD Name of current directory
164
<li>SHELL Name of current shell
165
<li>TERM The kind of terminal being used
166
<li>Environment variable are global to shell and subshells
168
<li><h3>User Variables</h3>
170
<li>Can be either upper or lower case
171
<li>var=value Define a variable
172
<li>var="" Define a variable as null
173
<li>local var Define a variable local to its scope
175
<li><h3>Positional Variables</h3>
177
<li>$0 Name of function or script being called
178
<li>$1 ... $9 Replace by arguments to shell or function
179
<li>${n} Replace by n-th arguments to shell or function
180
required if number of argument is over 9
182
<li><h3>Special Variables</h3>
184
<li>$? Exit status or return code of last command
185
<li>$# Number of arguments
186
<li>$@ Argument 1 thru n with Input Field Separator
187
<li>$* "$1" $2" ... $n
188
<li>$! Process id of last background process
189
<li>$$ Process id of shell running this script
190
<li>$- The current shell flags
192
<h2>Some Command Useful With Scripts</h2>
194
<li>basename Strip directory and option suffix
195
<li>declare Built-in command declares variable
196
<li>dirname Strip non-directory part
197
<li>echo Built-in command display message to standard out
198
<li>echo -n Built-in command display message to standard out without newline
199
<li>echo -e Builtin Command display message to standout with escape sequences
200
<li>enable Built-in command to enable/disable (-n) built-in commands
201
<li>env Built-in command displays environment variables
202
<li>eval Built-in command evaluate arguments before executing results
203
<li>exec Built-in command runs command as the shell process
204
<li>exit Built-in command exits shell
205
<li>expr Evaluates an expression and output its value
206
<li>false Built-in command always return false condition
207
<li>local Built-in command make variable local
208
<li>read Built-in command reads data into variable from standard in
209
<li>seq Print a sequence of numbers<ul>
210
<li> seq [OPTION]... LAST
211
<li> seq [OPTION]... FIRST LAST
212
<li> seq [OPTION]... FIRST INCREMENT LAST
213
<li> -w equalize width by padding with leading zeroes</ul>
214
<li>sleep Command causes execution to stop for a
215
specified number of seconds
216
<li>test Built-in command tests for various conditions, such as
217
existence of a file, useful for controlling
218
conditional script execution
219
<li>time Built-in command times commands
220
<li>times Print accumulated user and system times.
221
<li>true Built-in command always return true condition
222
<li>type Built-in command show what word is
223
<li>wait wait for the completion of background
224
processing; is used to ensure that critical
225
processing is complete before proceeding in a script
227
<h2>Making Scripts Colorful</h2>
229
<li>tput sgr0 #Reset text attributes to normal without clearing screen
230
<li>Escape sequence to change colors
231
#\e[${forground};${background}m
232
example white on black sequence is \e[37;40m
233
#Forground colors Backgrounfd Color
237
yellow=33; bgyellow=43
239
magenta=35; bgmagenta=45
242
<li>Turn text attribute off \e[0m
243
<li>Change text attribute bold \e[1m
244
<li>Change text attribute underline \e[4m
245
<li>Change text attributer everse \e[7m
247
<h2>Shell Functions and Scripts</h2>
249
<li><h3>Functions</h3>
251
<li>Function must be sourced just like .bashrc
252
<li>type <i><b>function</b></i> will list the function
253
<li>function <i><b>functionname</b></i> {
257
<li><i><b>functionname</b></i> () {
260
<li>Example - a helpful function
261
function givehelp { exec $1 --help | more; }
266
<li>#!/bin/bash Script name run script is run if execute bit is set
267
<li>#!/bin/bash -x As above, but script lines are displayed
268
<li>bash -x script As above
269
<li>/usr/bin/env bash Use the environment bash for the script
271
<li><h3>Script Debugging options</h3><pre><ul>
272
set -o OPTION Command line Action
273
<li>set -o noexec sh -n Don't run command, just check for syntax
274
<li>set -o verbose sh -v Echo commands before running them
275
<li>set -o xtrace sh -x Echo commands after running them
276
<li>set +o OPTION Turns OPTION off
280
<h2>Operation With Variables</h2>
282
<li><h3>Variable Usage</h3>
284
<li>$var Value of variable var
285
<li>"$var" Is null if variable is undefined
286
avoids some shell syntax errors if variable is undefined
287
<li>${var} Value of variable var
288
avoids confusion when concatenating with text
289
<li>${#var} Gives the length of the string contained in var
290
<li>${var:FIRST:N} Extract string from var starting at FIRST position
291
and continuing for N-1 characters. Note FIRST starts at 0.
293
<li><h3>Passing an variable to a program or script</h3>
295
<li>echo $var | command
299
<li>declare -a name Declare an array
300
<li>name[index]=value Just assigning a value defines it
301
<li>Index starts at zero
303
<li>Need not be contiguous
305
<li><h3>Setting Variable From Execution of Command</h3>
307
<li>var=`command` var is set to output of command
308
<li>var=$(command) same as above
310
<li><h3>Arithmetic Operation</h3>
312
<li>$(( expression )) Almost the same as 'expr expression'
313
<li>$[ expression ] Same as above
314
<li>( expression ) Groups expression within $(( ... ))
315
<li>expr expression Note must quote *
317
<li><h3>Arithmetic operator</h3>
324
<li># number conversion (Only in bash not expr)
327
<li><h3>Test Operation</h3>
329
<li>[ -option arg ] Same as 'test -option arg
330
<li>[[ -option arg ]] Same as 'test -option arg
331
<li>[ arg1 -option arg2 ] Same as 'test arg1 -option arg2'
332
<li>[[ arg1 -option arg2 ]] Same as 'test arg1 -option arg2'
333
<li>See test command for test complete list of options
335
<li>string1 = string2 String 1 equals string 2
336
<li>string1 !=string2 String 1 not equals string 2
337
<li>-n string String is not zero length
338
<li>-z string String is zero length
339
<li>-d FILE File is a directory
340
<li>-e FILE File exists
341
<li>-f FILE File exists and is a regular file
342
<li>-r FILE File exists and is readable
343
<li>-s FILE File exists and has length greater than zero
344
<li>-w FILE File exists and is writable
345
<li>-x FILE File exists and is executable
346
<li>num1 -eq num2 Number 1 equals number 2
347
<li>num1 -ne num2 Number 1 not equals number 2
348
<li>num1 -lt num2 Number 1 less than number 2
349
<li>num1 -le num2 Number 1 less than or equals number 2
350
<li>num1 -gt num2 Number 1 greater than number 2
351
<li>num1 -ge num2 Number 1 greater than or equals number 2</ul>
353
<li><h3>String Operation</h3>
355
<li>${varname:-word} Return var if exists and is not null, else word
356
<li>${varname:+word} Return word if var exists and is not null, else null
357
<li>${varname:?mess} Return var if exists and is not null, else
358
display mess and return from script with error
360
<li><h3>Pattern Operations</h3>
362
<li>${variable#pattern} If the pattern matches the beginning of the
363
variable value, delete the shortest part that
364
matches and return the rest
365
<li>${variable##pattern} If the pattern matches the beginning the of
366
variable value, delete the longest part that
367
matches and return the rest
368
<li>${variable%pattern} If the pattern matches the end of the variable
369
value, delete the shortest part that matches and
371
<li>${variable%%pattern} If the pattern matches the end of the variable
372
value, delete the longest part that matches and
375
<h2>The handy 'expr'</h2>
377
<li>expr <i><b>string : regrep</b></i> Anchored pattern match
378
<li>expr match <i><b>string pattern</b></i> Same as above
379
<li>expr substr <i><b>string pos {len]</b></i> Substring beginning at pos of length len
380
<li>expr length <i><b>string</b></i> Length of string
382
<h2>Flow Control</h2>
384
<li><h3>if - general information</h3>
386
if <i><b>command</b></i> Test return code of command
387
if <i><b>command1</b></i> && <i><b>command2</b></i> Test return code of command1 and command2
388
if <i><b>command1</b></i> || <i><b>command2</b></i> Test return code of command1 or command2
389
<i><b>command</b></i> can be condition i.e. [ $# -eq 0 ]</ul>
391
<li><h3>if/then/else or if/then/elif..</h3>
393
if <i><b>condition</b></i>
394
then <i><b>statements...</b><i>
395
[else <i><b>statements...</b></i>]
398
if <i><b>condition</b></i>
399
then <i><b>statements...</b></i>
400
[elif <i><b>condition</b></i>
401
then <i><b>statements...]</b></i>
402
[else] <i><b>statements...</b></i>]
405
<li><h3>return from Function</h3>
407
return [ <i><b>numeric expression or variable</b></i> ] This is the return code
411
for <i><b>name</b></i> [ in <i><b>list</b></i> ] do
412
<i><b>statements</b></i>
415
for <i><b>variable</b></i> = <i><b>start</b></i> to <i><b>end</b></i> do
416
<i><b>statements</b></i>
419
<li><h3>while/until</h3>
421
while <i><b>condition</b></i> do
422
<i><b>statements</b></i>
425
until <i><b>condition</b></i> do
426
<i><b>statements</b></i>
429
<li><h3>break/continue</h3>
431
<li>break [<i><b>n</b></i>] Break out of loop & select
432
<li>continue [<i><b>n</b></i>] Continue next iteration of loop
436
case <i><b>expression</b></i> in<i><b>
437
pattern1[|pattern11] } statements ;;
438
pattern2[|pattern21] } statements ;;
442
<h2>User Interfaces</h2>
445
select <i><b>name</b></i> [in <i><b>list</b></i>] do
446
<i><b>statements</b></i> that can use <i><b>$name</b></i>
448
<li>Generate a menu of each item in the <i><b>list</b></i> formatted with
449
a number for each choice
450
<li>Prompt the user for the number
451
<li>Store the selected choice in <i><b>name</b></i>. The number is stored in REPLY
452
<li>execute the statements in the do
453
<li>Repeat the process again
456
<h3>Command Line Options</h3>
458
<li>shift Shift 1st argument form the argument list
459
<li>getopts Used to process command line options
460
<li>OPTIND Variable contains number of options
461
<li>OPTARG Default variable for option
463
while getopts ":ab:c" opt; do
465
a) process_option_a;;
467
$OPTARG is the option's argument;;
468
c) process_option_c;;
471
shift $(($OPTIND - 1))
473
<h2>I/O in Scripts</h2>
475
<li>printf Built-in command for formatted output
476
<li>read Built-in command to read one line into variable(s)
477
read #Everything entered goes to REPLY
478
read var #Everything is read into var
479
read a b #Read 1st word in to a and rest into b
480
read -t 300 var #Read with 300 second timeout
481
<li>Example how to use read from $file
482
while read <i><b>line</b></i>; do
483
<i><b>do_something_with_line</b></i>
484
done <<i><b>$file</b></i>
486
<h2>Process Handling</h2>
490
Name Number Control Character
491
<li>EXIT 0 Used to trap exiting script
495
<li>KILL 9 can not be ignored or trapped
496
<li>TERM 15 Default kill
497
<li>TSTP 24 Control-Z
501
<li>trap "" <i><b>signal-list</b></i> Ignore signal
502
<li>trap "<i><b>cmds</b></i>" <i><b>signal-list</b></i> Execute commands if signal is caught
503
<li>trap <i><b>signal-list</b></i> Reset signal to original condition
504
<li>trap : <i><b>signal-list</b></i> (undocumented) ignore signal, pass to child
505
Signal are normally not passed to subprocesses
507
trap 'rm tmpfile; exit' 0 1 2 #remove tmpfile on exit, logout, interrupt
509
trap "echo 'You hit Control-C'" INT
513
<li>Example parent child process
516
trap 'echo parent exiting; exit' 0
522
echo child started. pid is $$
523
trap 'echo child got signal 2; exit' INT
527
<h2>Examples Useful Scripts or Function</h2>
530
<li># Function top5 Example how to set defaults
531
# Usage top5 {n} #list n processes
533
ps -ef | head -${1:-5}
536
<li># Function hereis Example of HERE IS FILE and handling arguments
537
# Usage hereis word1 word2 ...
542
This is an example of an HERE IS FILE.
543
One argument is ${name}.
549
<li># Function pick Return selected items by user
550
# Usage: .e.g var=`pick *`
552
for name in $@ ; do #for each item in argument list
553
echo -n "$name (y/n/q)?" >/dev/tty #ask user to select
554
read ans #read answer from standard in
555
case $ans in #Check choices
556
y*) echo $name;; #selected
557
q*) break;; #skip rest of arguments
558
*) continue;; #skip item
563
<li># Function acal Display a nicer calendar
564
# but will accept Alphabetic month
568
0) cal; return;; #no arguments
569
1) m=$1; y=`date +%Y`;; #1 argument
570
2) m=$1; y=$2;; #2 arguments
586
[1-9]|1[0-2] ) ;; #numeric month
592
<li>## Function selectex - Example select
594
function selectex () {
595
choices="/bin /usr /home"
596
select selection in $choices; do
597
if [ $selection ]; then
601
echo 'Invalid selection'
606
<li># Function fwhich Which command in $PATH is executed
610
then cat << EndOfHelp 1>&2; return 2
611
Usage fwhich command #Example of parsing the $PATH
614
Return 2 - No arguments
617
for path in `echo $PATH | sed 's/^:/.:/
622
if [[ -x $path/$1 ]] ; # does executable file exists here?
623
then echo $path/$1 # found it
630
<li># Name: overwrite Copy standard input to output after EOF
633
then echo "Usage: overwrite file command [args]" 1>&2; return 2
637
new=/tmp/overwrite1$$; old=/tmp/overwrite2$$
638
trap 'rm -f $new $old; return 1' 1 2 15 # clean up files
640
if [[ $? -eq 0 ]] ; # collect output
641
then # command completed successfully
642
cp $file $old # save original file
643
trap '' 1 2 15 # we are committed; ignore signals
644
cp $new $file # copy new file into file
645
rm -f $new $old # remove temp files
647
echo "overwrite: $1 failed, $file unchanged" 1>$2
653
# Purpose: caseless grep of gzip files
654
# Usage: zgrep text files.gz
657
if [ $# -eq 0 ] ; then
658
echo "Usage: zgrep grep_text files.gz"
666
gzip -cd $1 | grep -i $text
672
# Purpose: highlighting grep
673
# Usage: hgrep pattern files
676
if [ $# -lt 2 ] ; then
677
echo "Usage: hgrep pattern files"
681
sep=$'\001' #note use of $' ' to create control characters
682
bold=$'\e[1m'; off=$'\e[0m'
683
underline=$'\e[4m'; reverse=$'\e[7m' #other choices of highlighting
684
sed -n "s${sep}${pattern}${sep}${reverse}&${off}${sep}gp" $*
added
removed
Development/languages/bash/shellscript.html
