Linux command line

The Linux command line is a powerful tool for interacting with the operating system and executing commands. It allows users to perform tasks efficiently, automate processes, and manage files and applications. The command line interface (CLI) is often preferred by advanced users for its speed and flexibility compared to graphical user interfaces (GUIs).

The learning curve

The command line can be intimidating at first, but with practice, it becomes a valuable skill. Start with basic commands and gradually explore more advanced features. Learning some basic before using the clusters, can greatly enhance you experience and productivity.

Basics¶

Bash¶

Bash is a powerful scripting language, which can be used directly in a Linux terminal and is often combined with Linux commands. It can simplify many tasks when working on an HPC cluster. Bash is the default shell on Euler.

.bashrc¶

The .bashrc file is executed when you start a new terminal session. You can add aliases, environment variables, and functions here to customize your shell environment. To edit it, use:

nano ~/.bashrc

After making changes, apply them with:

source ~/.bashrc

Editing .bashrc

Please be very careful when editing your .bashrc. A misconfigured .bashrc can result in you not being able to login to Euler any more. And please don't delete your .bashrc file.

For more details, see the bashrc manual

Tab completion¶

Most terminals support tab completion. Press Tab while typing a command or filename to auto-complete or list possible options.

history¶

Use the up/down arrow keys to scroll through previous commands. View history with:

history

For more details, see the history manual

exit¶

Exit the current shell:

exit

For more details, see the exit manual

Environment variables¶

Environment variables store system-wide settings. View all environment variables with:

printenv

To view a specific variable, use:

echo $VARIABLE_NAME

To set an environment variable, use:

export VARIABLE_NAME=value

To unset a variable, use:

unset VARIABLE_NAME

For more details, see the echo manual, export manual, unset manual

echo¶

Print a string to the terminal:

echo "this goes to the standard output"

For more details, see the echo manual

alias¶

Aliases are shortcuts for commands. Create an alias with:

alias shortname='long command'

For example, to create an alias for ls -l:

alias ll='ls -l'

To view all aliases, use:

alias

For more details, see the alias manual

Bash scripts¶

Bash scripts are files containing a series of commands that can be executed together. They help automate repetitive tasks and create reusable workflows.

To create a bash script:

Create a file with a .sh extension
Add the shebang line at the top: #!/bin/bash
Make it executable: chmod +x script.sh
Run it: ./script.sh

Variables¶

Set a variable and print its value:

VAR_A="some string variable"
echo $VAR_A

For more details see: Shell variables

Functions¶

Define a function to group commands:

my_function() {
    echo "This is my function"
}
my_function

For more details, see the bash manual

Comments¶

Comments start with # and are ignored by the shell:

# This is a comment
echo "This will run"
# echo "This will not run"

For more details, see the bash manual

Conditionals¶

Use if statements to execute commands based on conditions:

if [ $VAR_A == "some string variable" ]; then
    echo "Variable matches"
else
    echo "Variable does not match"
fi

For more details, see the bash manual

Loops¶

To repeat a command multiple times, you can use a loop. Here are some examples:

for i in {1..5}; do echo $i; done

For more details, see the bash manual

Files and Directories¶

pwd¶

Pwd (print working directory) prints the current directory you are in:

pwd

For more details, see the pwd manual

ls¶

Ls (list) is used to list files and directories in the current directory.

ls

Use wildcards to match files:

*: Matches any number of characters
?: Matches a single character
[abc]: Matches any character in brackets

ls *.txt

Some options:

-a: Include hidden files
-h: Human-readable sizes, such as KB, MB, GB
-l: Long listing format
-r: Reverse order
-s: Include file sizes
-S: Sort by file size, largest first
-t: Sort by time, newest first

For more options and details, see the ls manual.

Colorized Output

If your terminal supports it, you can use ls --color=auto to get colored output for different file types. This is disabled by default on Euler because it caused issues with the Lustre filesystem.

du¶

Du (disk usage) shows the disk usage of files and directories.

du

Some options:

-h: Human-readable sizes, such as KB, MB, GB
-s: Summary of total size
-c: Include a grand total

For more details, see the du manual

find¶

Search for files and directories: - Find files by name:

find . -name "filename.txt"

Find directories by name:

find . -type d -name "directory_name"

For more details, see the find manual

cd¶

Cd (change directory) is used to navigate between directories in the filesystem.

cd /path/to/directory

To go up one level, use:

cd ..

To return to your home directory, use:

cd ~

To return to the previous directory, use:

cd -

For more details, see the cd manual

mkdir¶

Mkdir (make directory) is used to create new directories.

Create a new directory:
```bash
mkdir new_directory

Create a directory and its parent directories if they do not exist:

mkdir -p /path/to/new_directory

For more details, see the mkdir manual

rmdir¶

Mkdir (remove directory) is used to delete empty directories.

rmdir empty_directory

For more details, see the rmdir manual

touch¶

Touch is used to create an empty file or update the timestamp of an existing file.

touch new_file.txt

Update the timestamp of an existing file:

touch existing_file.txt

For more details, see the touch manual

chmod¶

Chmod (change mode) is used to change the permissions of files and directories.

chmod u+x script.sh

This command adds execute permission for the user on the file script.sh. To change permissions for all users:

chmod a+r file.txt

This command adds read permission for all users on the file file.txt. For more details, see the chmod manual and the Wikipedia's File-system permissions

chown¶

Chown (change owner) is used to change the ownership of files and directories.

chown user:group file.txt

This command changes the owner of file.txt to user and the group to group. To change ownership recursively for a directory and its contents:

chown -R user:group directory_name

For more details, see the chown manual

cp¶

Cp (copy) is used to copy files and directories.

cp source/path/file.txt destination/path/

Copy a directory recursively:

cp -r dir1 dir2

For more details, see the cp manual

mv¶

Mv (move) is used to rename or move files and directories.

mv old_name.txt new_name.txt

Move a file to a different directory:

mv file.txt /path/to/destination/

For more details, see the mv manual

rm¶

Rm (remove) is used to delete files and directories.

rm file.txt

Delete a directory and its contents recursively:

rm -r dir_name

For more details, see the rm manual

ln¶

Ln (link) is used to create links to files. - Create a hard link:

ln source_file.txt link_name.txt

- Create a symbolic link:

ln -s source_file.txt link_name.txt

For more details, see the ln manual

cat¶

Cat (concatenate) is used to display the content of a file, contatenate files, or create new files. To display the content of a file:

cat file.txt

To concatenate multiple files:

cat file1.txt file2.txt > combined.txt

To create a new file and write to it, use:

cat > file.txt

Terminate input with Ctrl+D. For more details, see the cat manual

less¶

View the content of a file with backward navigation:

less file.txt

Use Space to go to the next page, b to go back a page, and q to quit. For more details, see the less manual

more¶

View the content of a file one screen at a time:

more file.txt

Press Space to go to the next page, Enter to go to the next line, and q to quit. For more details, see the more manual

Text editors¶

nano¶

Edit a file with the nano editor:

nano file.txt

Use Ctrl+O to save, Ctrl+X to exit, and Ctrl+K to cut a line. It is user-friendly and suitable for beginners. For more details, see the nano manual

vi¶

Edit a file with the vi editor:

vi file.txt

To enter insert mode, press i. After editing, press Esc to exit insert mode. To save and exit, type :wq. To exit without saving, type :q!. For more details, see the vi manual

vim¶

Edit a file with the vim editor:

vim file.txt

Similar to vi, but with more features. Use i to enter insert mode, Esc to exit insert mode, and :wq to save and exit. To exit without saving, type :q!. For more details, see the vim manual

Redirections¶

Redirect the output of a command to a file:

command > output.txt

Append the output to a file:

command >> output.txt

Redirect both standard output and standard error to a file:

command &> output.txt

Redirect standard error to a file:

command 2> error.txt

Redirect standard output to standard error:

command 1>&2

Redirect standard input from a file:

command < input.txt

Use a pipe to connect the output of one command to the input of another:

command1 | command2

For example, to filer the output of a command and sort it:

command | grep "pattern" | sort

For more details, see the bash manual

Compression and Archiving¶

zip¶

Create a compressed zip archive:

zip archive.zip file1 file2

Extract a zip archive:

unzip archive.zip

For more details, see the zip manual

bzip2¶

Compress a file with bzip2:

bzip2 file.txt
bunzip2 file.txt.bz2

For more details, see the bzip2 manual

gzip¶

Gzip (GNU zip) is a file compression utility. Compress a file with gzip:

gzip file.txt

Decompress a file with gzip:

gunzip file.txt.gz

For more details, see the gzip manual

tar¶

Tar (tape archive) is used to create and extract archive files. Invented for tape backups, it is now widely used for file archiving. Create a tar archive:

tar -cf archive.tar file1 file2

Extract a tar archive:

tar -xf archive.tar

Some options: - -c: Create a new archive - -x: Extract files from an archive - -f: Specify the archive file name - -v: Verbose output (list files being processed) For more details, see the tar manual

System information¶

man¶

Display the manual for a command:

man ls
man grep

For more details, see the man manual

top¶

Real-time view of running processes:

top

For more details, see the top manual

htop¶

Interactive process viewer:

htop

For more details, see the htop homepage

id¶

Show user and group IDs:

id

For more details, see the id manual

whoami¶

Display the current user:

whoami

For more details, see the whoami manual

which¶

Find the path of an executable command:

which ls

For more details, see the which manual

Strings and Text¶

awk¶

Awk is a powerful text processing tool that can be used to manipulate data and generate reports. Example: sum columns in a queue listing:

bqueues | awk 'BEGIN{p=0;r=0} NR>1{p+=$9;r+=$10} END{printf "Pending:%8i\nRunning:%8i\n",p,r}'

For more details, see the awk manual

cut¶

Cut is used to extract sections from each line of input.

cut -d':' -f1 /etc/passwd

This command extracts the first field (username) from the /etc/passwd file, using : as the delimiter. For more details, see the cut manual

grep¶

Grep (global regular expression print) is used to search for patterns

grep pattern file.txt
ls -l | grep test

For more details, see the grep manual

sed¶

Sed (stream editor) is used to perform basic text transformations on an input stream (file or input from a pipeline).

echo "what a beautiful day" | sed s/day/night/

For more details, see the sed manual

sort¶

Sort lines of text files:

sort file.txt

For more details, see the sort manual

Linux command line

Basics¶

Bash¶

.bashrc¶

Tab completion¶

history¶

exit¶

Environment variables¶

echo¶

alias¶

Bash scripts¶

Variables¶

Functions¶

Comments¶

Conditionals¶

Loops¶

Files and Directories¶

pwd¶

ls¶

du¶

find¶

cd¶

mkdir¶

rmdir¶

touch¶

chmod¶

chown¶

cp¶

mv¶

rm¶

ln¶

cat¶

less¶

more¶

Text editors¶

nano¶

vi¶

vim¶

Redirections¶

Compression and Archiving¶

zip¶

bzip2¶

gzip¶

tar¶

System information¶

man¶

top¶

htop¶

id¶

whoami¶

which¶

Strings and Text¶

awk¶

cut¶

grep¶

sed¶

sort¶

More resources¶