Guides/posix_shell.md

10 KiB
Raw Blame History

Shell Cheatsheet

Introduction

I am only going over POSIX shell things here as there are a bunch of extensions found in some shells / on some systems but not others. The sections below should apply to any UNIX-like system. This cheatsheet is by no means a comprehensive guide to UNIX shell scripting (its a cheatsheet, not a handbook, and I dont know everything) but you can usually find out more information about any given command right there on the shell.

Many systems come with manuals, try man COMMAND or info COMMAND to find out more about most utilities (try man man, for example). Alternatively, if no manual is available, passing the flag -h or --help will usually print some information. I find myself consulting these constantly because nobody is expected to remember everything beyond the basics and its usually faster than googling it. That said, googling for a solution is generally also a viable option.

If you are trying to learn how to use the shell, I recommend you start with the examples section at the end and work your way backwards from there to figure out what they do. I also highly recommend you check the manuals or help information for any command you want to use - especially if it was given to you by a stranger on the internet.

Shell Features

The shells main tasks are providing a text based user interface to an operating system as well as running scripts. To facilitate this, the shell runs the programs the user/script specifies and deals with the output.

Working Directory

A shell (or any process for that matter) is always running inside a specific directory on the file system, the so-called working directory. It is relevant when determining where files specified by relative path are located.

There are two kinds of file paths in the UNIX world: relative and absolute. Absolute paths refer to a fixed location on the file system and start with a /. Relative paths refer to a location relative to the current working directory and cannot start with a /. Relative paths are just the equivalent of an absolute path with the working directory cut off. In many cases, just the name of a file or subdirectory inside the working directory is used.

You can change to another working directory using cd (change directory) like so: cd PATH_TO_DIRECTORY. When no path is specified, cd takes you to the current users home directory (usually...).

Streams

Normally, there are two output streams and one input stream for a running program: Standard output (stdout), standard error (stderr), and standard input (stdin). By default, they are passed to the stdout, stderr, and stdin of the shell, which end up in your terminal when you run a shell interactively. There are two notable exceptions to this which you can specify:

Redirects: You can send the output of a command to a file instead of the standard output/error streams.

  • COMMAND > FILE redirects stdout of COMMAND into FILE.
  • COMMAND 2>FILE redirects stderr into FILE.
  • COMMAND 2>&1 redirects stderr into stdout.

You can have multiple redirects per command though more than two usually dont make sense. The most common example of this is redirecting stderr to stdout and the combined stdout into a file like so: COMMAND > FILE 2>&1.

Pipes: The output of one command can be used as the input of another: COMMAND | OTHER_COMMAND

Variables

There are two kinds of variables that you can deal with on the shell, lets call them shell variables and environment variables. They behave the same when interacting with them on the shell but shell variables are only available in the shell whereas environment variables are handed to a new process when it is spawned.

The shell can set environment variables for its child processes and it itself has variables from the environment it launched with. The environment it launched with is passed on to the processes it spawns.

Shell variables can be set using VARIABLE=VALUE. They can be added to the environment of subsequent commands using export VARIABLE. To alter the environment for just one command, prepend it with the variable like so: VARIABLE=VALUE COMMAND. Variables can be unset using unset VARIABLE.

Unknown or unset variables are ignored.

Control Flow

When a command finishes running, it produces an exit code that gives some rudimentary status information Usually, an exit code of 0 indicates success and anything else an error of some kind.

This can be used in control structures (while loops, conditions) or directly accessed using the special shell variable $?.

If statements are built as follows:

if COMMAND; then
    OTHER
    COMMANDS
    HERE
else
    MORE
    COMMANDS
    HERE
fi

There are also two operators that can be used as short forms of if and else: && and ||. They can be combined and chained as needed. Commands that are chained using && only run if the previous command exited with exit code 0. Commands that are chained using || run if the previous command didnt exit with exit code 0. In many cases, this is used as follows: COMMAND && THEN_COMMAND || ELSE_COMMAND

While loops are built as follows:

while COMMAND; do
    OTHER
    COMMANDS
    HERE
done

The [ ... ] seen in many if statements and while loops is not actually part of the shell syntax but an alias to the test command (see below).

For loops are different in that they act as "for each" instead of checking a condition.

for VARNAME in STRING; do
    COMMANDS
    USING $VARNAME
    HERE
done

You can break out of a loop using break.

You can end the shell session or script using exit. It optionally takes an exit code as argument.

Job Control

You can send a process to the background by adding an & at the end of the command. The jobs command shows programs that are currently running in the background. You can get a process from the background into the foreground using fg. Alternatively, if you just want to wait for all jobs to finish, wait does that.

Miscellaneous

Spaces separate arguments and newlines separate commands. To override this, use quoted strings. Single-quoted strings are taken verbatim, variables in double-quoted strings get replaced.

The POSIX shell is case-sensitive but its not whitespace-sensitive meaning you can put multiple whitespace characters and they count as one or get ignored. You can put multiple commands on the same line using semicolons. This is commonly used for formatting as it makes scripts more readable.

The built-in read command can be used in scripts to read input into a variable.

Command line arguments given to a script end up in the variables $0 through $9. They can be moved to the left using shift. $0 starts out being the name of the script itself.

You can send Ctrl+c to the terminal to send an interrupt to the process running inside it. Note that this is a feature of the terminal, not of the shell itself. Usually, this is used to stop whatever child process of the shell is running in the foreground.

Commands

This section gives you a list of commonly used UNIX utilities. This does not include shell-builtins as I have covered them above already. Some shells have built-in versions of some of these commands but they should mostly work the same as their dedicated counterparts.

  • sh (shell) The shell itself, can also be used to invoke scripts
  • echo output text to standard output
  • pwd (print working directory)
  • ls (list) Get a listing of the specified directory. If none given, the current directory is used.
  • cat (concatenate) Originally created to concatenate files, it is most commonly used to print the contents of a file to the standard output.
  • mkdir (make directory) self-explanatory
  • touch Change the access and modification timestamp of a file. Also commonly used to create empty files.
  • mv (move) files or directories
  • cp (copy) files or directories
  • rm (remove) files or directories
  • test (more commonly known as [ ... ]) used to check conditions, result is passed back using exit code
  • true returns exit code 0
  • false returns non-zero exit code
  • grep search for regular expressions in text
  • tr (text replace) replace all occurrences of a character in text
  • sed (stream editor) edit text on the fly
  • uname (UNIX name) get information about the operating system
  • id (identity) get information about users
  • su (switch user)
  • ps (process status) get information about running processes
  • kill kill a given process
  • find find a file or directoy by specified criteria
  • chmod (change mode) change file permissions
  • chown (change owner) change file ownership
  • df (disk free) show available disk space
  • du (disk usage) show how much space a given file or directoy is using
  • vi unholy abommination of a text editor :)
  • more print only a screen worth of text at a time and wait so you have the time to read it

Commonly Used Special Files

  • . The working directory
  • .. Parent directory of the working directory
  • /dev/null Your personal trash can to redirect all the output you dont care about to. Also known as a black hole or the infinite void. :)
  • /dev/urandom Source of infinite random data
  • /dev/zero Source of infinite null bytes

Honorable Mentions

The following things wont be available on all systems, but are really handy when they are:

  • less better version of more
  • nano nice user-friendly terminal text editor available on many systems
  • htop nice user-friendly terminal task manager
  • which easy way to find out where the binary for a given command is located
  • sudo become root (or any other user) by authenticating the current user instead of the target user given the current user is permitted to do so - useful when root login is disabled
  • free shows information about used/free RAM on Linux systems
  • /dev/stdin, /dev/stdout, and /dev/stderr special files for you guessed what
  • sl a great way to infuriate anyone who happens to type too quickly

Examples

Below are some examples that you can use to better understand above material.

... to be finished another time ...