A Practical Guide to Linux Commands, Editors, and Shell Programming

Preface xxxv



Chapter 1: Welcome to Linux and macOS 1

The History of UNIX and GNU-Linux 3

What Is So Good About Linux? 6

Overview of Linux 11

Additional Features of Linux 16

Chapter Summary 18

Exercises 18





Part I: The Linux and macOS Operating Systems 21



Chapter 2: Getting Started 23



Conventions Used in This Book 24

Logging In from a Terminal (Emulator) 26

Working from the Command Line 28

su/sudo: Curbing Your Power (root Privileges) 32

Where to Find Documentation 33

More About Logging In and Passwords 42

Chapter Summary 46

Exercises 47

Advanced Exercises 48



Chapter 3: The Utilities 49

Special Characters 50

Basic Utilities 51

Working with Files 53

| (Pipeline): Communicates Between Processes 60

Four More Utilities 61

Compressing and Archiving Files 64

Locating Utilities 69

Displaying User and System Information 71

Communicating with Other Users 75

Email 77

Chapter Summary 77

Exercises 80

Advanced Exercises 81



Chapter 4: The Filesystem 83

The Hierarchical Filesystem 84

Directory Files and Ordinary Files 85

Pathnames 90

Working with Directories 92

Access Permissions 100

ACLs: Access Control Lists 106

Links 112

Chapter Summary 122

Exercises 124

Advanced Exercises 126



Chapter 5: The Shell 127

Special Characters 128

Ordinary Files and Directory Files 129

The Command Line 130

Standard Input and Standard Output 137

Running a Command in the Background 150

Filename Generation/Pathname Expansion 152

Builtins 157

Chapter Summary 158

Exercises 159

Advanced Exercises 160





Part II: The Editors 163



Chapter 6: The vim Editor 165



History 166

Tutorial: Using vim to Create and Edit a File 167

Introduction to vim Features 175

Command Mode: Moving the Cursor 181

Input Mode 185

Command Mode: Deleting and Changing Text 186

Searching and Substituting 190

Miscellaneous Commands 197

Copying, Moving, and Deleting Text 197

Reading and Writing Files 200

Setting Parameters 201

Advanced Editing Techniques 206

Units of Measure 210

Chapter Summary 213

Exercises 218

Advanced Exercises 219



Chapter 7: The emacs Editor 221

History 222

Tutorial: Getting Started with emacs 224

Basic Editing Commands 231

Online Help 238

Advanced Editing 240

Major Modes: Language-Sensitive Editing 255

Customizing emacs 265

More Information 270

Chapter Summary 270

Exercises 279

Advanced Exercises 280





Part III: The Shells 283



Chapter 8: The Bourne Again Shell (bash) 285



Background 286

Startup Files 288

Commands That Are Symbols 291

Redirecting Standard Error 292

Writing and Executing a Simple Shell Script 294

Control Operators: Separate and Group Commands 299

Job Control 304

Manipulating the Directory Stack 307

Parameters and Variables 310

Special Characters 325

Locale 326

Time 330

Processes 333

History 336

Aliases 352

Functions 356

Controlling bash: Features and Options 359

Processing the Command Line 364

Chapter Summary 374

Exercises 376

Advanced Exercises 378



Chapter 9: The TC Shell (tcsh) 379

Shell Scripts 380

Entering and Leaving the TC Shell 381

Features Common to the Bourne Again and TC Shells 383

Redirecting Standard Error 389

Working with the Command Line 390

Variables 396

Control Structures 408

Builtins 418

Chapter Summary 422

Exercises 423

Advanced Exercises 425





Part IV: Programming Tools 427



Chapter 10: Programming the Bourne Again Shell (bash) 429



Control Structures 430

File Descriptors 464

Parameters 470

Variables 479

Builtin Commands 489

Expressions 505

Implicit Command-Line Continuation 512

Shell Programs 513

Chapter Summary 523

Exercises 525

Advanced Exercises 527



Chapter 11: The Perl Scripting Language 529

Introduction to Perl 530

Variables 538

Control Structures 545

Working with Files 554

Sort 558

Subroutines 559

Regular Expressions 562

CPAN Modules 568

Examples 570

Chapter Summary 574

Exercises 574

Advanced Exercises 575



Chapter 12: The Python Programming Language 577

Introduction 578

Scalar Variables, Lists, and Dictionaries 582

Control Structures 588

Reading from and Writing to Files 593

Regular Expressions 597

Defining a Function 598

Using Libraries 599

Lambda Functions 603

List Comprehensions 604

Chapter Summary 605

Exercises 606

Advanced Exercises 606



Chapter 13: The MariaDB SQL Database Management System 609

History 610

Notes 611

Installing a MariaDB Server and Client 614

Client Options 615

Setting Up MariaDB 616

Creating a Database 618

Adding a User 619

Examples 620

Chapter Summary 633

Exercises 633



Chapter 14: The AWK Pattern Processing Language 635

Syntax 636

Arguments 636

Options 637

Notes 638

Language Basics 638

Examples 645

Advanced gawk Programming 662

Chapter Summary 667

Exercises 668

Advanced Exercises 668



Chapter 15: The sed Editor 669

Syntax 670

Arguments 670

Options 670

Editor Basics 671

Examples 674

Chapter Summary 685

Exercises 685





Part V: Secure Network Utilities 687



Chapter 16: The rsync Secure Copy Utility 689



Syntax 690

Arguments 690

Options 691

Examples 693

Chapter Summary 700

Exercises 701



Chapter 17: The OpenSSH Secure Communication Utilities 703

Introduction to OpenSSH 704

Running the ssh, scp, and sftp OpenSSH Clients 706

Setting Up an OpenSSH Server (sshd) 717

Troubleshooting 724

Tunneling/Port Forwarding 724

Chapter Summary 727

Exercises 728

Advanced Exercises 728





Part VI: Command Reference 729



Utilities That Display and Manipulate Files 731



Network Utilities 732

Utilities That Display and Alter Status 733

Utilities That Are Programming Tools 734

Miscellaneous Utilities 734

Standard Multiplicative Suffixes 735

Common Options 736

The sample Utility 736

sample: Brief description of what the utility does (macOS) 737

aspell: Checks a file for spelling errors 739

at: Executes commands at a specified time 743

busybox: Implements many standard utilities 747

bzip2: Compresses or decompresses files 750

cal: Displays a calendar 752

cat: Joins and displays files 753

cd: Changes to another working directory 755

chgrp: Changes the group associated with a file 757

chmod: Changes the access mode (permissions) of a file 759

chown: Changes the owner of a file and/or the group the file is associated with 764

cmp: Compares two files 766

comm: Compares sorted files 768

configure: Configures source code automatically 770

cp: Copies files 772

cpio: Creates an archive, restores files from an archive, or copies a directory hierarchy 776

crontab: Maintains crontab files 781

cut: Selects characters or fields from input lines 784

date: Displays or sets the system time and date 787

dd: Converts and copies a file 790

df: Displays disk space usage 793

diff: Displays the differences between two text files 795

diskutil: Checks, modifies, and repairs local volumes (macOS) 800

ditto: Copies files and creates and unpacks archives (macOS) 803

dmesg: Displays kernel messages 805

dscl: Displays and manages Directory Service information (macOS) 806

du: Displays information on disk usage by directory hierarchy and/or file 809

echo: Displays a message 812

expand/unexpand: Converts TABs to SPACEs and SPACEs to TABs 814

expr: Evaluates an expression 816

file: Displays the classification of a file 820

find: Finds files based on criteria 822

finger: Displays information about users 828

fmt: Formats text very simply 831

fsck: Checks and repairs a filesystem 833

ftp: Transfers files over a network 838

gawk: Searches for and processes patterns in a file 845

gcc: Compiles C and C++ programs 846

GetFileInfo: Displays file attributes (macOS) 851

grep: Searches for a pattern in files 853

gzip: Compresses or decompresses files 858

head: Displays the beginning of a file 861

join: Joins lines from two files based on a common field 863

kill: Terminates a process by PID 866

killall: Terminates a process by name 868

launchctl: Controls the launchd daemon (macOS) 870

less: Displays text files, one screen at a time 873

ln: Makes a link to a file 878

lpr: Sends files to printers 881

ls: Displays information about one or more files 884

make: Keeps a set of programs current 892

man: Displays documentation for utilities 898

mc: Manages files in a textual environment (aka Midnight Commander) 902

mkdir: Creates a directory 909

mkfs: Creates a filesystem on a device 911

mv: Renames or moves a file 914

nice: Changes the priority of a command 916

nl: Numbers lines from a file 918

nohup: Runs a command that keeps running after you log out 920

od: Dumps the contents of a file 921

open: Opens files, directories, and URLs (macOS) 926

otool: Displays object, library, and executable files O 928

paste: Joins corresponding lines from files 930

pax: Creates an archive, restores files from an archive, or copies a directory hierarchy 932

plutil: Manipulates property list files (macOS) 938

pr: Paginates files for printing 940

printf: Formats string and numeric data 942

ps: Displays process status 946

renice: Changes the priority of a process 951

rm: Removes a file (deletes a link) 953

rmdir: Removes directories 955

rsync: Securely copies files and directory hierarchies over a network 956

scp: Securely copies one or more files to or from a remote system 957

screen: Manages several textual windows 958

sed: Edits a file noninteractively 964

SetFile: Sets file attributes (macOS) 965

sleep: Creates a process that sleeps for a specified interval 967

sort: Sorts and/or merges files 969

split: Divides a file into sections 978

ssh: Securely runs a program or opens a shell on a remote system 980

sshfs/curlftpfs: Mounts a directory on an OpenSSH or FTP server as a local directory 981

stat: Displays information about files 984

strings: Displays strings of printable characters from files 986

stty: Displays or sets terminal parameters 987

sysctl: Displays and alters kernel variables at runtime 991

tail: Displays the last part (tail) of a file 992

tar: Stores or retrieves files to/from an archive file 995

tee: Copies standard input to standard output and one or more files 1000

telnet: Connects to a remote computer over a network 1001

test: Evaluates an expression 1005

top: Dynamically displays process status 1008

touch: Creates a file or changes a file's access and/or modification time 1012

tr: Replaces specified characters 1014

tty: Displays the terminal pathname 1017

tune2fs: Changes parameters on an ext2, ext3, or ext4 filesystem 1018

umask: Specifies the file-creation permissions mask 1021

uniq: Displays unique lines from a file 1023

w: Displays information about local system users 1025

wc: Displays the number of lines, words, and bytes in one or more files 1027

which: Shows where in PATH a utility is located 1028

who: Displays information about logged-in users 1030

xargs: Converts standard input to command lines 1032





Part VII: Appendixes 1035



Appendix A: Regular Expressions 1037



Characters 1038

Delimiters 1038

Simple Strings 1038

Special Characters 1038

Rules 1041

Bracketing Expressions 1042

The Replacement String 1042

Extended Regular Expressions 1043

Appendix Summary 1045



Appendix B: Help 1047

Solving a Problem 1048

Finding Linux and macOS Related Information 1049

Specifying a Terminal 1050



Appendix C: Keeping the System Up-to-Date 1053

Using dnf 1054

Using apt-get 1060

BitTorrent 1064



Appendix D: macOS Notes 1067

Open Directory 1068

Filesystems 1069

Extended Attributes 1070

Activating the Terminal META Key 1076

Startup Files 1076

Remote Logins 1076

Many Utilities Do Not Respect Apple Human Interface Guidelines 1076

Installing Xcode and MacPorts 1077

macOS Implementation of Linux Features 1078





Glossary 1081

File Tree Index 1135

Utility Index 1137

Main Index 1141