Bash Scripting 101
Rüdiger Voigt | last update: 2017-01-03
A pipeline is a sequence of commands. Thereby the output of each command in the pipeline becomes the input of the following command.
grep "spamkeyword" ./serverlogfile.log | cut -f 2 -d " " | sort | uniq -c | sort -n | tail > top10spammers.txt
grep scans the server logfile line wise for a keyword identifying a spamming attempt. The result is piped to
cut which keeps only the second column of every line (the column delimiter is a space). This gets sorted. Then the command
uniq -c combines adjacent matching lines into one and adds a counter. This is sorted again (the option -n sorts numerical, not by alphabet) and piped to
tail which chops off the last 10 lines which contain the IPs of the 10 most notorious spammers. The result is written into a file.
Further Reading: introduction to named pipes (Linux Journal)
This is just an overview. Please read the manual carefully before you use these commands to avoid data loss.
- A need to know command.
man commandnamedisplays a manual for the named command. Use
qto return to the command line.
cd pathto change the directory.
cd ~is a shortcut to your home directory.
cd ..means one level up.
- Lists your current directory. Followed by a filename it lists information about that file.
lshas a bunch of options. For example
ls -a -h -l) lists all files (including hidden ones) in the current directory, prints file sizes in human readable format (for example MB instead of bytes) and uses the long listing format (shows owner, permissions, ...).
- The command
tail filenameprints the last 10 lines of a file to standard out. This is extremly useful for debugging on the server. When something goes wrong just print the last lines of the error log to what happened.
tail -f filenamedisplays the last 10 lines of the file but shows more than lines get appended. Excellent to have a look at activity on a server.
- Use to copy files and directories.
- Used to move files. During that it is possible to rename them.
- This command removes files or directories. Beware: once executed the files are gone / there is no paperbin.
rmdiris used to remove a directory. However it fails if the directory is not empty. In this case
rm -rwill delete all files in it and the directory recursively.
- Finds file by name. Use
updatedbto update the file database.
User Management and Permissions
- changes the user and/or group ownership of given files
- changes passwords for user accounts.
pwgen -sis very useful for generating passwords.
- Change user. Usually used to switch to an admin account.
- Executes a command as another user. When permissions are set this command allows specified users to run certain commands as superuser without having to know a superuser password.
- Tutorial on using screen by Matt Cutts
- Used to start other commands that will ignore the HUB-signal. This means that the command will run even if the user logs out. See here for possible problems and alternatives.
- Using a ssh connection to transfer files between machines.
- Displays running processes, RAM usage and CPU load. Type
qto quit. On multiprocessor machines CPU load is shown combinend. Press
1to show load for single processing units.