Marquette.edu//High Performance Computing//
This guide is written for Raj users who have little to no experience with the Linux command line. This guide will cover some basic vocabulary as well as a handful of basic commands for navigation and file manipulation. Finally, there is quick overview of command line text editors as well as links to additional resources.
When working with a Linux server you will have to get used to the fact that there is no GUI(graphical user interface), and you must navigate and interact with the server with only a keyboard using the CLI(command line interface). This guide was created as short introduction to Linux commands in order to help new users get more acquainted with this unusual type of interface.
The terminal or terminal window is any display which accepts input and displays output from a computer.
The Shell prompt
A shell prompt is the main way to interact with a command line only interface. A typical shell prompt will look something like this:
It displays the user (username) you are logged in as, the server you are logged into (domain) followed by the current directory (or folder) you are in (directory). All this information is followed by a symbol that simply separates the hostname user and directory from the rest of the prompt. Most systems use a $ symbol, but %, #and >symbols are also used. To interact with the prompt, you simply type a command and press enter, and the output of the command is displayed in the terminal window.
Note that you may hear people refer to a shell as bash (also known as bourne again shell). Bash is essentially the second edition of shell. For the most part, they can be treated the same, and unless you are an experienced user, you will probably not be able to tell the difference.
Other less common shell prompts include the c shell (csh), z shell (zsh), Korn shell (ksh) and fish shell. However, since the standard shell and bash are the most common shells that is what we will focus on here.
Path refers to the location of a file or directory on the computer. Fully qualified paths start from the root directory (/ ), and relative paths start from your current directory. An easy way to tell the difference is whether or not the path starts with a / or not.
If you are in your home directory and you want to list the location of a simulation directory called sim1 which is a subdirectory of a directory called simulations. The fully qualified path to that directory is:
The relative path is:
Permissions are an important concept in Linux. They inform a user in what ways the user is allowed to interact with the file or directory. The three types of permissionsin Linux are read, write and execute. These permissionsmean slightly different things for files versus directories. File read permissionsgrant access to view the contents of the file. Directory read permissionsgrant access to view the contents of the directory. File write permissions allow a user to modify the contents of the file. Directory write permissiongrant access to create new files and directories inside the directory. File execute permissions grant access to run binaries and scripts. Directory execute permissions grant access to change into the directory.
Each file or directory has owner (or user) level permissions, group level permissions and everyone else (or other) level permissions. The owner is the user who created the file or directory, group means any individual who is a member of that group has these permissions, and other is anyone who is not the owner, or belong to the group.
The Linux Command
The Linux command has three main parts to it. The first is the command itself (i.e., what do you want the computer to do). This can be followed by a series of options, denoted by a '-' and followed by a letter, to modify that command. Finally there is sometimes an input to that command.
Useful Linux commands
The pwd command stands for print working directory. This command lists the path to the directory you are currently working in.
A typical pwd output should look something like this:
[user@raj-ln1 simulations]$ pwd/mmfs1/home/user/simulations
The ls command stands for list. This command lists all the contents of working directory. Depending on how your shell is configured the output will most likely be color coded. Plain text entries are data files or text documents, blue entries are directories and green entries are executables.
ls -[options] [file]
A typical ls output may look something like this:
[user@raj-ln1 simulations]$ lstest.sh README sim1 sim2 sim3 sim4 test.py test.slurm
The -l option stands for long. Thisoption giveseach item in the directory its own line displays additional information about each entry in the listing.This option gives considerably more information than we will cover in this tutorial, so let's just focus on the parts that matter.Each line starts with a string of 10 characters. Each character in that line represents something specific about the permissions for that file. The first character can be a -, d, or l. The - is a placeholder which means the space is blank (i.e., there is nothing special about this item). The d means the item is a directory, and the l means the item is a link or a symbolic link. We will not discuss what a symbolic link is in this tutorial. The remaining permissions are then grouped into three groups of three. Each character representing a specific permission for a specific group. The first group of three is for the owner of the item, the second group of three is for the group associated with the item, and the last group of three is for everyone else. Each character can either be a letter or a -, with a letter denoting the user, group, or other has that permission, and a - meaning they do not. In this context, an r stands for read permissions, w stands for write permissions, and x stands for executable permissions.
The -l option also lists who owns the file, and the group the file is associated with. Finally, each line contains the time and date of last modification, as well asthe name of the item.
A typical ls -l output will look something like this:
[user@raj-ln1simulations]$ ls -ltotal 32-rwxrwxr-x 1 user domain_users 2691 Jul 22 15:23 test.sh-rw-rw-r-- 1 user domain_users 1054 Jul 22 15:22 READMEdrwxrwx--x 2 user domain_users 4096 Jul 22 15:01 sim1drwxrwx--x 2 user domain_users 4096 Jul 22 15:01 sim2drwxrwx--x 2 user domain_users 4096 Jul 22 15:01 sim3drwxrwx--x 2 user domain_users 4096 Jul 22 15:01 sim4-rw-rw-rw- 1 user domain_users 559 Jun 22 15:10 test.py-rw-rw-rw- 1 user domain_users 274 Jul 22 15:02 test.slurm
Let's break down the first line in this output:
-rwxrwxr-x 1 user domain_users 2691 Jul 22 15:23 test.sh
The first character is a - which means it is a file and not a link or directory. The next three characters are rwx. This means that the owner, user, can read, write and execute the shell script. The next three characters are rwx. This means that anyone in thegroup domain_users can read, write and execute the shell script. The last three characters are r-x. This means that anyone who is not the owner or part of the group domain_users, can read and execute the file but cannot edit its contents.
The -a option stands for all. This means it shows even hidden items in the directory. Hidden items start with a period. A typical output for ls -a looks like this:
[user@raj-ln1 simulations]$ ls -a. .. test.sh README sim1 sim2 sim3 sim4 test.py test.slurm
Note the addition of the . and .. directories. The directory . stands for the current directory. The directory .. stands for the directory one level up. These directories are often implied in an ls listing so they are not displayed without the -a option, however, they do have their uses and need to be included as we will see in later commands.
Now if we want to use both options we can either type ls -l -a, or for options that do not require an input, they can be strung together like so:
Meaningwe want a long listing, -l, of everything in the directory, -a. Note that if the option requires an input you cannot include it in the string.
By default, ls prints the contents of your current working directory. However, you can get a listing of another directory by typing it as an input after your options. For example:
Will list the contents of the sim1 directory in the simulations. The command:
ls -la /mmfs1/home/user/simulations/sim1
will do a long listing of everything in the Documents directory of the users home directory. Note that ~ can be used in place of a user's home directory, thus:
are equivalent commands.
Finally, ls also accepts relative paths as inputs, so assuming you are in the simulations directory and want to get a listing of the contents of sim1, you can use this command:
The chmod commands change mode bits. The mode bits of a file or directory control the permissions of that file or directory.
chmod -[options] [mode] [file]
Say you just wrote a new bash script called run_sims.sh. You attempt to run the scriptand get this:
[user@raj-ln1 simulations]$ ./run_sims.shbash: ./run_sims.sh: Permission denied
This is because, by default, ASCII text files (which bash scripts are) do not have executable permissions. To add the correct permissions, use the commands:
[user@raj-ln1 simulations]$ ls -l run_sims.sh-rw-r--r-- 1userdomain_users 6 Jul 22 15:32 run_sims.sh[user@raj-ln1 simulations]$ chmod u+x run_sims.sh[user@raj-ln1 simulations]$ ls -l run_sims.sh-rwxr--r-- 1user domain _users 6 Jul 22 15:32 run_sims.sh
The first letter of the mode stands for the level of permissions (u for user or owner level permissions, g for group level permissions, o for other or everyone else level permissions, and a for all levels) you want to modify. The second symbol denotes whether you want to add orremove (+ or - respectively) permissions, and the final letter denotes which permission you want to modify (r for read, w for write and x for executable). Here u+x stands for "add executable permissionsfor the user (or owner) of the file." To modify multiple file permissions at once, separatethe modes with a comma like so:
chmod u+x,g+x run_sims.sh
The -R option stands for recursive. This applies the permission changes to the directory specified, as well as every file and directory within the specified directory.
The cd command stands for change directory. It allows you to change from one directory to another. It has no notable options and takes a directory name as an input.
Assume you are in the simulations directory from the earlier example. You can change to the sim1 directory by issuing the following command:
[user@raj-ln1 simulations]$ pwd/mmfs1/home/user/simulations[user@raj-ln1 simulations]$ cd sim1[user@raj-ln1 sim1]$ pwd/mmfs1/home/user/simulations/sim1
Notice that the shell prompt reflects the change is directory by changing from
Also, note that the .. directory will come in handy here as well. So if you want to go back to the simulations directory, you could do this:
[user@raj-ln1 sim1]$ cd ..[user@raj-ln1 simulations]$ pwd/mmfs1/home/user/simulations
Additionally, the command can be given more than one directory in the path so for example, ifyou are in directory sim1 andyou want to be in directory sim2, you can issue the following commands:
[user@raj-ln1 sim1]$ cd ../sim2[user@raj-ln1 sim2]$ pwd/mmfs1/home/user/simulations/sim2
The mkdir command stands for make directory. The command takes the name of the directory to be created as an input.
mkdir -[options] [directory]
Say you were in the simulations directory and you wanted to make a new simulation and keep all the files associated with that simulation in a directory called sim5. You could make the directory like so:
[user@raj-ln1 simulations]$ lstest.sh README sim1 sim2 sim3 sim4 test.py test.slurm[user@raj-ln1 simulations]$ mkdir sim5[user@raj-ln1 simulations]$ lstest.sh README sim1 sim2 sim3 sim4 sim5 test.py test.slurm
The -p option stands for parent. This allows you to create multiple levels of directories. For instance, say you not only wanted to create a directory sim5, but you wanted to create a Results directory inside of sim5. Instead of doing this:
[user@raj-ln1 simulations]$ lstest.sh README sim1 sim2 sim3 sim4 test.py test.slurm[user@raj-ln1 simulations]$ mkdir sim5[user@raj-ln1 simulations]$ lstest.sh README sim1 sim2 sim3 sim4 sim5 test.py test.slurm[user@raj-ln1 simulations]$ cd sim5[user@raj-ln1 sim5]$ ls[user@raj-ln1 sim5]$ mkdir Results[user@raj-ln1 sim5]$ lsResults
You could do this
[user@raj-ln1 simulations]$ lstest.sh README sim1 sim2 sim3 sim4 test.py test.slurm[user@raj-ln1 simulations]$ mkdir -p sim5/Results[user@raj-ln1 simulations]$ lstest.sh README sim1 sim2 sim3 sim4 sim5 test.py test.slurm[user@raj-ln1 simulations]$ cd sim5[user@raj-ln1 sim5]$ lsResults
The cp command stands for copy. Think of this as copy and paste wrapped up into one command. This command will allow you to copy a file or directory from one location to another. It takes two inputs; the source directory and the destination directory.
cp -[options] [src] [dst]
Where src and dst are the source and destinations respectively.
Say you are in the sim1 directory and you want to copy the file in.txt to the newly created sim5 directory. You could do the following:
[user@raj-ln1 sim1]$ ls ../sim5Results[user@raj-ln1 sim1] cp in.txt ../sim5[user@raj-ln1sim1]$ ls ../sim5Results in.txt
If you want to rename the file to in2.txt as you copy it over you could do this:
[user@raj-ln1 sim1]$ ls ../sim5Resultsin.txt[user@raj-ln1 sim1] cp in.txt ../sim5/in2.txt[user@raj-ln1sim1]$ ls ../sim5Resultsin.txt in2.txt
This will copy the contents of the file in.txt into a file called in2.txt in the directory simulations/sim2
The -r option stands for recursive. If you want to copy an entire directory along with all of its contents to another directory, you can use the -r option. Say you want to run a simulation similar to sim1 with a few changes. Instead of using mkdir to make the directory, then copying the input file into the directory, you could do the following:
[user@raj-ln1 simulations]$ lstest.sh README sim1 sim2 sim3 sim4 test.py test.slurm[user@raj-ln1 simulations]$ ls sim1in.txt out.txt[user@raj-ln1 simulations]$ cp -r sim1 sim5[user@raj-ln1 simulations]$ lstest.sh README sim1 sim2 sim3 sim4 sim5 test.py test.slurm[user@raj-ln1 simulations]$ ls sim5in.txt out.txt
Now you can easily make tweaks to the simulation and get new results without losing the original simulation.
The mv command stands for move. Think of this as a cut and paste wrapped up into one command.
mv [src] [dst]
Say you are in the directory sim1 and you want to move the file in.txt to the simulations directory. You would do thefollowing:
[user@raj-ln1 sim1]$ lsin.txt out.txt[user@raj-ln1 sim1]$ ls ..test.sh READMEsim1 sim2 sim3 sim4 sim5 test.py test.slurm[user@raj-ln1 sim1]$ mv in.txt ..[user@raj-ln1 sim1]$ lsout.txt[user@raj-ln1 sim1]$ ls ..test.shREADMEsim1 sim2 sim3 sim4 sim5 test.py test.slurm in.txt
Another useful function of this command is to rename files. Say you want to rename the file in.txt to input.txt. You would use the command
[user@raj-ln1 sim1]$ lsin.txt out.txt[user@raj-ln1 sim1]$ mv in.txt input.txt[user@raj-ln1 sim1]$ lsinput.txt out.txt
Renaming files can also be done while moving a file from one location to another. So say you want to move the file in.txt to the directory simulations, while also renaming it to input.txt, you can use the command
[user@raj-ln1 sim1]$ lsin.txt out.txt[user@raj-ln1 sim1]$ ls ..test.sh READMEsim1 sim2 sim3 sim4 sim5 test.py test.slurm[user@raj-ln1 sim1]$ mv in.txt ../input.txt[user@raj-ln1 sim1]$ lsout.txt[user@raj-ln1 sim1]$ ls ..test.shREADMEsim1 sim2 sim3 sim4 sim5 test.py test.slurm input.txt
Note that, unlike the cp command, the -r option does not need to be used for directory. The mv command will move an entire directory and the contents of that directory without the use of the -r option. For example, if you wanted to rename the directory sim1 to simulation1, you coulddo:
[user@raj-ln1 simulations] lstest.shREADMEsim1 sim2 sim3 sim4 sim5 test.pytest.slurm[user@raj-ln1 simulations] ls sim1in.txt out.txt[user@raj-ln1 simulations] mv sim1 simulation1[user@raj-ln1 simulations] lstest.shREADMEsimulation1 sim2 sim3 sim4 sim5 test.pytest.slurm[user@raj-ln1 simulations] ls simulation1in.txt out.txt
The rm command stands for remove. This command allows you to delete files or directories. Note that rm does not ask for confirmation and there is no recycle bin. Once you hit enter, the file or directory will be deleted, and it will be gone forever so use carefully. This command only takes one input, which is the name of the file or directory to be removed.
rm -[options] [directory]
Say we wanted to remove out.txt from our sim5 directory, we could issue the following commands from inside the sim5 directory.
[user@raj-ln1 sim5] lsin.txt out.txt[user@raj-ln1 sim5]$ rm out.txt[user@raj-ln1 sim5] lsin.txt
The -r option is the most common option and it stands for recursive. So if you have a directory and you want to delete the directory and everything in it than you can use the -r option. For example, say we wanted to remove the sim5 directory altogether we could issue the commands:
[user@raj-ln1 simulations] lstest.shREADMEsim1 sim2 sim3 sim4 sim5 test.pytest.slurm[user@raj-ln1 simulations] rm -r sim5[user@raj-ln1 simulations] lstest.shREADMEsim1 sim2 sim3 sim4 test.pytest.slurm
Command Line Editors
Many users of high performance computing clustersoften set up and test jobs on smaller systems and then copy the files up to the cluster for larger jobs. However, it can be advantageous to be able to edit your files on the go instead of copying the files back and forth from the cluster to a desktop for editing. Two of the most popular command line editors are vim and nano. Full tutorials of these programs are beyond the scope of this tutorial, but brief introductions to each as well as additional resources are included below.
- Easy and intuitive to use
- Has most ofthe basic features desired for a file editor
- Great for beginners
- Lacks some advanced features
- Most shortcuts use Ctrl+Letter so they may conflictwith other shortcuts on your system
To edit a file type the command
This will open up an editor inside your Linux terminal. From here, simplytyping into the terminal edits the file (as we will see this is not the case for vim). There are some helpful shortcuts displayed along the bottom of the terminal window. Note that in many Linux contexts the ^ symbol is short for control. So for example if you want to get help type Ctrl+g. To write out (or save) the file use Ctrl+o. To exit use the command Ctrl+x.
For more information,How-To Geek has a good tutorial on using nano.
- Hasallthe basic features desired for a file editor
- Has lots of advanced features
- Can edit multiple files at a time
- Has a command mode and an edit mode which negates the need to use Crtl for shortcuts
- Not intuitive
- No helpful guide to shortcuts at the bottom of the editor
To open a file for editing typ the command
The file in command mode. To enter insert mode (editing mode) type a lowercase i. Once in insert mode,typing into the terminal edits the file. To exit insert mode, hit Esc. From command mode, you can save the file by typing :w and hitting Enter. To exit vim, type :q while in command mode, then hit Enter. To save and quit at the same time, type :wq and hit Enter.
For more help on how to use vim, you can use the built-in interactive tutorial on Raj by typing vimtutor into the command line. Otherwise, check outHow-To Geek'svim tutorial.
For additional information on the Linux command line, check out one of these in-depth tutorials: