Showing posts with label Command Line Tricks. Show all posts
Showing posts with label Command Line Tricks. Show all posts

2009-07-29

Bash Fork Bomb

Maybe you've heard of the bash fork bomb, maybe you haven't. You may have even seen the command that launches the fork bomb but don't know what it means or just haven't taken the time to think it through. Whatever the case, the bash fork bomb is something that is pretty interesting.

The short explanation of what the fork bomb does is that it recursively and exponentially spawns processes until your system crashes. So, what does this fork bomb look like? Here it is:

$ :(){ :|:& };:

So, that's great, but what exactly does it do? Well, let's break it down to explain what it is

  • :(){ :|:& }
    This defines a function : that does not accept any arguments, thats the () part. So, what does the function do? That's the part inside the curly braces.

  • { :|:& }
    This is the meat of the function, what gets run when you actually call :. When you call the function :, the first thing it does is to call itself and it pipes the output into another call of itself. So, short story here is that : calls itself twice; hence the exponential growth of processes spawned. The thing is that each time : gets called, that's another process running on the system. And, here's the kicker, the second time : is called, it is sent to run in the background with the &, so killing the parent process won't kill it.

  • ;:
    This is where we actually call : for the first time. The ; operator says to run the following command immediately after the previous one has finished. So, what does it do? Immediately after defining the : function this calls it, and then you play the waiting game until your system crashes.

You should probably not run this command on your system. If you're curious, fire up a virtual machine inside virtual box and give it a go there. Maybe I'll do that and post a video.

So, what good is this fork bomb? Well, this is something that sys admins use as a way to test limits on user processes. Once you launch this fork bomb, the only way to end it may be to reboot your system. The reason is that you have to destroy all instances of it to resume normal function.

See you next time.

2009-07-15

Searching files with grep

It is pretty common that you might want to search for text within a file or directory. The best way to do that is with grep. The basic way to use grep is:

$ grep [PATTERN] [FILE]

  • Searching a single file
    A simple example would be if you wanted to see all the different Linux kernels that show up in your /boot/grub/menu.lst file, you could:

    $ grep kernel /boot/grub/menu.lst

  • Searching for multiple strings
    What if you want to search for multiple strings in one file? Say, you want to see all the section headings, identifiers, and drivers in your /etc/X11/xorg.conf file. You could do this by using the -e option, and I'll also use the -i option to ignore case when searching:

    $ grep -i -e section -e identifier -e driver /etc/X11/xorg.conf

  • Searching multiple files
    It is also possible to use grep on all the files in a directory or just on a couple files that you want using either the * wildcard or by identifying files with the -f option. For example, let's say I wanted to search all my blog posts saved on my computer for ones that mention ImageMagick:

    $ grep -i imagemagick ~/blog/*

  • Displaying lines before and/or after match
    Sometimes you want to see the context in which the matched text appears. To do this we can use the following options: -A, for lines after the match; -B, for lines before the match; or -C, for lines on both sides of match. So, repeating the previous ImageMagick search to display 5 lines of text after the match, 5 lines before the match, or 5 lines on both sides of the match, we could use the following, respectively:

    $ grep -iA 5 imagemagick ~/blog/*
    $ grep -iB 5 imagemagick ~/blog/*
    $ grep -iC 5 imagemagick ~/blog/*

  • Displaying lines that do not match
    You might want to find lines that are not commented out in your /etc/apt/sources.list file. To do so, we can use the -v option, and we want to find lines that do not contain the # character. Since the # character is used to comment out lines in Bash, we have to search for this in a way that won't comment out the rest of the command (hint: regular expressions...I plan on doing a post dedicated specifically to the topic of regular expressions in the future), so we can search for this in either of the following two ways:

    $ grep -v "#" /etc/apt/sources.list
    $ grep -v \# /etc/apt/sources.list

  • Displaying only file names that contain matches
    It may be the case that you are really only concerned with the names of the files that contain your search string. To do this, we can use the -l (lower case L) option. I'll return to the ImageMagick example above here:

    $ grep -il imagemagick ~/blog/*

  • Displaying the number of matches per file
    To go one step further from the previous example, you can use the -c option to show you the number of matches you have in each file:

    $ grep -ic imagemagick ~/blog/*

    If you test out the above example, you'll find that it returns all the files that have no matches as well. If you only want to show the number of matches on files that actually have matches, we can use a little bit of redirection to pipe the output of one grep command into the input of another grep command (and a little help with regular expressions):

    $ grep -ic imagemagick ~/blog* | grep -v ":0$"

  • Displaying the line numbers for the matches
    Sometimes it is important to know where in a file the matches are, to help find this, we can use the -n option:

    $ grep -in imagemagick ~/blog/*

Well, that does is for a pretty basic introduction into grep. As usual, there are still some other things that you can do with grep. The biggest thing that you can use with grep that I have not really covered is the use of regular expressions, but that is a topic deserving of its own post.

See you next time

2009-07-12

Re-issue Previous Command as Root

Here's a quick Bash tip. Don't you just hate it when you key in a command at the terminal and press enter only to realize that you didn't preface the command with sudo? Well, have no fear, because the command !! will re-issue the previous command.

Now, I know you're probably thinking, "what good does that do me? I can just press the up arrow and then hit enter to run the last command." And you'd be right; however, what if you wanted to re-issue the previous command, but you wanted to put sudoat the beginning of it? Well, you could simply press the up arrow and then move the cursor to the beginning of the prompt and then key in sudo and press the enter key, or you could just do this:

$ sudo !!

The time that this happens to me most frequently is when I want to edit a file, for example, if I wanted to add a new repository to my /etc/apt/sources.list file, and run the following by mistake:

$ vim /etc/apt/sources.list

This will open up the sources.list file as read only, a fact which becomes perfectly clear once you change to insert mode in vim, with a Warning: changing a readonly file warning across the bottom of the window. At this point, it's easy to close the file, and re-open it as root with:

$ sudo !!

Well, that pretty much does it for tonight. See you next time.

2009-07-09

More on ImageMagick

I've talked about ImageMagick before, and I figured it would be fun to cover some more features available with it. The reason I like ImageMagick is that it is really convenient for when you want to do the same thing to a large group of pictures. You may find yourself with some digital pictures that you want to make look more artsy by changing them to black and white or maybe to a sepia tone.

The first thing I always do before using ImageMagick is to make backup copies of all my pictures I'm going to be editing. Once you're ready to get started, converting pictures to black and white is really easy:

$ mogrify -monochrome input.jpg

It really is that simple. The above will only convert the image named input.jpg. If you wanted to do the same thing to all .jpg images in a directory, you could do this:

$ mogrify -monochrome *.jpg

Sepia tone allows for a little more tweaking, because it allows you to input a threshold as a percent of the intensity, ranging from 0 - 99.9%. This is something that you would want to play with to get the desired result, but from my experience 80% is usually a good starting point. We could convert an image to sepia tone like this:

$ mogrify -sepia-tone 80% input.jpg

Again, you could use the same command on all .jgp images in a directory:

$ mogrify -sepia-tone 80% *.jpg

This still only scratches the surface of what you can do with ImageMagick. I'll be covering more of the features in future posts.

See you next time

2009-06-24

Bash Tutorial: Part 3

We're back at it with more about using Bash. This time, rather than focus on new commands, we'll look at command operators and redirection. Command operators allow you to run multiple Bash commands in one statement. Redirection is not just a useful tool when performing magic, it's also a great way to change where the input and/or output of a command come from and/or go.

Command Operators
  • ;
    To execute multiple commands one after another, we use the ; operator. In other words, the first command specified will run until completion before the second command starts. For example, if you want to update your index for the locate command and then search for the xorg.conf, we would do the following:

    # updatedb ; locate xorg.conf

    The above is essentially the same as running this:

    # updatedb
    # locate xorg.conf

  • &
    Executing commands in rapid succession all on one line is cool, but it can be more useful to run multiple commands all at the same time. For example, if you wanted to write a Bash script to launch firefox, pidgin, and songbird all at once, the important piece would be the following command:

    $ firefox & pidgin & songbird

    Another use for the & operator is if you want to run a command in the background. In other words, if you would like to run a command, but instead of waiting for the command to finish to regain control of the terminal, the control will immediately be returned and your computer will continue executing the command in the background. To do this, you simply end your command with &. If you wanted to update your index for the locate command, but don't want to wait for this to finish, you could do the following:

    # updatedb &

Redirection
  • >
    Sometimes, the output of a command isn't immediately important, but you would like to save a record of it for later. To do that, we can use the > operator to create a new file, or overwrite an existing file, with the output of a command. For example, if you're trying to troubleshoot a networking problem, you might want someone to look at the output of the ifconfig command on your system. To save this output, you can do the following:

    # ifconfig > output.txt

  • >>
    Sometimes you want the output of a command to write to a file, but you don't want to lose the existing contents of that file. In this case, we can use the >> operator to append the output to the end of the specified file. For example, if you want to add the output of the lsmod command to the output file we just created, we could do the following:

    # lsmod >> output.txt

    If you have read some of my other posts, you have seen me use the two above redirection operators in conjunction with the echo command to create or add to files that control the behavior of some parts of the computer.

  • <
    There are also cases where you might want to specify the input of a command or program from a file. One example I can think of is if you have a text file that you would like to sort, you could use this as the input for the sort command like this:

    $ sort < unsorted.txt

    Of course, the above will simply output the text from unsorted.txt to the screen in sorted order. If we wanted to save the sorted version of the text file, we could combine input and output redirection to accomplish this:

    $ sort < unsorted.txt > sorted.txt

  • <<
    The << operator acts basically as a simple line based text editor. If you remember, we used it in the Installing Songbird post as an option for creating the Bash script to launch Songbird. Here's an example of the << operator with the sort command:

    $ sort << EOF
    > 3
    > 1
    > 2
    > EOF

    For another example, I've included the section where we used this in the Songbird post. In this example we use both input and output redirection to create a Bash script without dealing with a real text editor:

    # cat > /usr/share/Songbird/launcher.sh << EOF
    > #!/bin/sh
    > cd /usr/share/Songbird
    > ./songbird
    > EOF

  • |
    Pipes. Saving the best for last here, pipes allow you to redirect the standard output of one command to the standard input of another command. That may sound a little confusing as you read it, but let me give an example, that may clear things up:

    $ ls -al | less

    Using what we have learned about Bash command operators, the above command is basically the same thing as:

    $ ls -al > temp.txt ; less temp.txt ; rm temp.txt

    As you can see, piping commands is powerful and can save a bit of typing. My favorite use of pipes is with the grep command:

    $ ps -ef | grep firefox

    I'll cover both the ps and grep commands in future posts, but the above will give you information about any processes running on your system that have firefox as part of the process name. The above would be the same as the following:

    $ ps -ef > temp.txt ; grep firefox temp.txt ; rm temp.txt

That does it for Part 3 of the introduction to Bash.

See you next time.

2009-06-21

Bash Tutorial: Part 2

For part 2 of my Bash tutorial, I didn't have the time to make it as big as I had wanted, so it's a little shorter. Let's get started.

Displaying file contents
  • cat
    Concatenate files. The cat command can be used to type out the contents of a file or of multiple files. To type out the contets of your /etc/apt/sources.list file, just do:

    $ cat /etc/apt/sources.list

    If you want to type out the contents of multiple files right after each other, just do:

    $ cat file1 file2 file3

  • less
    less is similar to cat for displaying the contents of a single file. However, less is much better than cat for large files. If you use cat on a large file, you will only be able to see the end of the file because the contents would have scrolled out of view and even beyond the scroll buffer. less allows you to scroll through the entire contents of the file with the up and down arrows as well as the page up and page down keys. To use less to display the contents of your /etc/apt/sources.list file, just do:

    $ less /etc/apt/sources.list

That's all for now. Next time we'll be back with part3.

See you next time

2009-06-17

Bash Tutorial: Part 1

I know a lot of people get scared any time the command line comes up when using a computer. Really, the command line shouldn't be scary, and learning how to use the command line in any operating system will only make you better at using your computer. As such, I figured I would do a couple posts as an introduction into the Linux shell I use most: the Bourne-again shell (Bash). Let's get started.

Moving around the shell
  • pwd
    Print working directory. When you first open a shell, it starts you in your home directory, usually /home/[username], also abbreviated ~. If you're ever not sure what directory you're currently working in, the pwd command will tell you:

    $ pwd
    /home/[username]
    $

  • ls
    List files. If you want to see what files and folders are in a directory, ls is the command to do it. If you just want to see the contents of the current directory, just use:

    $ ls

    In Linux, files whose name begin with . are hidden. To list the contents of the current directory and show hidden files, use:

    $ ls -a

    If you want to see more information about the files, such as who has permissions to read, run, and execute files; the owning user and group; the file size; and the last date modified, just use:

    $ ls -l

    The ls command does not simply restrict you to looking at files in your current working directory; you can look inside any directory you want. To see the long listing of all files in the /usr/bin directory, just use:

    $ ls -al /usr/bin

  • cd
    Change directories. Now that we've covered how to see your current working directory and listing files within directories, the next place to go is to change directories. In order to change directories to /etc/X11, just do:

    $ cd /etc/X11

    We can also use .. to move up a level:

    $ pwd
    /lib/modules/2.6.26-2-686/kernel/net/ipv6
    $ cd ..
    $ pwd
    /lib/modules/2.6.26-2-686/kernel/net
    $ cd ../..
    $ pwd
    /lib/modules/2.6.26-2-686
    $ cd ../2.6.26-1-686
    $ pwd
    /lib/modules/2.6.26-1-686

Working with files
  • cp
    Copy. The basic syntax for the cp command is:

    $ cp [source file] [destination file]

    For example, if you wanted to make a backup copy of your .bashrc file before making changes:

    $ cp .bashrc .bashrc-backup

    This is a good for copying single files, but if you want to make a copy of an entire directory, we can use the -r option to do a recursive copy:

    $ cp -r Documents Documents-backup

  • mv
    Move. The mv command works similar to cp:

    $ mv [source file] [destination file or directory]

    If you want to simply rename your .bashrc-backup file to fix edits made to your .bashrc file:

    $ mv .bashrc-backup .bashrc

    If you want to move a file into a different directory:

    $ mv .bahsrc-backup Backup/

    If you need to move or rename a directory, it works exactly the same with the mv command as moving files.

  • mkdir
    Make directory. If you want to create a new directory for all your digital pictures, just do:

    $ mkdir Pictures

    You can also create a any missing parent directories with the -p option:

    $ mkdir -p Pictures/Vacation/2009


  • rm
    Remove. If you want to delete the .bashrc-backup file, just do:

    $ rm .bashrc-backup

    If you want to delete a directory and all the files and subdirectories within it, you can use the -r recursive option:

    $ rm -r /Pictures/Vacation/2009

  • rmdir
    Remove directory. If you want to delete a directory, you can use the rmdir command. The rmdir command requires that the directory that you're trying to delete is empty:

    $ rmdir Pictures/Vacation/2009

    Similar to the recursive option that we've seen with other commands, we have the -p parents option to use with rmdir:

    $ rmdir -p Pictures/Vacation/2009

    The above behaves the same as the following:

    $ rmdir Pictures/Vacation/2009
    $ rmdir Pictures/Vacation
    $ rmdir Pictures

Learning more

We've covered a lot in this post, but there is still more that can be done with the above commands. Here are some resources that are available to learn more of the options available.
  • man
    Manual. The man command will bring up what is called the "man pages" for other commands. The man pages describe the syntax and available options for bash commands and programs. The syntax for man is as follows:

    $ man [command]

    So, if you wanted to see more information about the cp command:

    $ man cp

  • info
    Information. The info command is similar to the man command, but the info pages can be a little easier to learn from if you're not used to reading man pages. If you want to see the info page for the cp command:

    $ info cp

That does it for Part 1 of the introduction to Bash.

See you next time.

2009-06-14

Scheduling Commands with Crontab

A common thing to want to do is to schedule a command to run at a specific time. We can do with with cron using the crontab command. By default, all users have permissions to use crontab; however, this can be changed by the existence of one of two files:
  • /etc/cron.allow - Only users listed in this file can use crontab
  • /etc/cron.deny - Users listed in this file cannot user crontab

To see what you have scheduled via crontab, you can run the following command:

$ crontab -l

If you want to see what is scheduled to be run as a different user, you can use the following:

# crontab -u [username] -l

Where you would replace [username] with the appropriate username. Entries in the crontab file have the following syntax:

[m] [h] [dom] [mon] [dow] [command]

Now, I'll explain the above:
  • [m] - Minute to run; represented as a number: 0-59
  • [h] - Hour to run; represented as a number in: 0-23
  • [dom] - Day of Month to run; represented as a number: 1-31
  • [mon] - Month to run; represented as a number: 1-12
  • [dow] - Day of Week to run; represented as a number: 0-6 (Sunday is 0)
  • [command] - Command to run

All of the timing options above can be replaced by an * where necessary to have your command run for all possibilities of that option. For example, in my post on locating files I mentioned scheduling the updatedb command. Let's say we want to schedule this command to run every morning at 6:30 am, then we would put the following into the root user's crontab file:

30 6 * * * updatedb

You can also use a comma separated list for the timing options to have a command run at multiple timings. For example, I have the bad habit of staying on my computer too late at night when I should be going to bed. To try to correct this, in the past, I have scheduled my computer to shutdown at 11:00 pm on Sunday through Thursday nights:

55 22 * * 0,1,2,3,4 shutdown -h +5

Now that we see the necessary syntax for entries in the crontab file, how do we use the crontab command to edit the file to schedule our commands? We can use the following to create or edit a user's crontab file:

$ crontab -e

The above will open your current user's crontab file in the current user's default text editor. Once here, you simply need to put in the entries you would like using the above syntax to schedule them. There are some other options that you can use with the crontab command:
  • -l [username] - Displays [username]'s crontab file
  • -r [username] - Removes [username]'s crontab file
  • -u [username] - Runs the specified contab command on [username]'s crontab file

That pretty much does it for a basic introduction to scheduling commands with crontab.

See you next time

2009-06-10

Finding Files on Your Computer

It happens to all of us, we forget where files are saved. The good news is that with the locate command is a great way to find files on your computer. Before we can use the locate command, we've got to build an index of the files on our computer, which is easy enough: we just need to use the updatedb command:

# updatedb

It is a good idea to periodically run the updatebd command to keep your index current when you go to search for files. All you need to know to search for files is to know all or part of the file name. For example, if you couldn't remember where the xorg.conf file is saved, you can search for it like this:

$ locate xorg.conf

Here is an example output of the above command:

$ locate xorg.conf
/etc/X11/xorg.conf
/usr/share/man/man5/xorg.conf.5.gz
/var/lib/x11/xorg.conf.md5sum
/var/lib/x11/xorg.conf.roster
$

Alternatively, if you were looking for the xorg.conf file, but could only remember that the filename started with xorg., you could search for it like the following:

$ locate /xorg.
/etc/X11/xorg.conf
/usr/share/X11/xkb/rules/xorg.lst
/usr/share/X11/xkb/rules/xorg.xml
/usr/share/man/man5/xorg.conf.5.gz
/var/lib/dpkg/info/xorg.list
/var/lib/dpkg/info/xorg.md5sums
/var/lib/x11/xorg.conf.md5sum
/var/lib/x11/xorg.conf.roster
$

There is more that can be done with the locate command, but this has covered about 98% of the times I've used it. I'll probably cover some more of these features in a later post. Another thing that is useful is to schedule the updatedb command via cron, which we'll be discussing next time.

See you next time

2009-06-03

Convert Raster Graphics to Vector Graphics

Have you ever wanted to convert a raster image (.jpg, .png, etc.) to a vector image (.svg)? Maybe not yet, but in my post on Installing Songbird, when we added Songbird to the menu, we did not have an icon to set for the menu item. It turns out that this is because we need a .svg image and the Songbird tarball only came with a .png image. No worries, though, we'll go through converting raster to vector and, specifically, how to convert the songbird.png image to a vector image for use on our menu.

In order to do this, we're going to make use of two programs: ImageMagick and Inkscape. ImageMagick is a powerful command line utility for all kinds of graphics manipulations. Inkscape is more widely known as an open source GUI vector graphics editor; however, it also comes with its own set of command line tools. Let's make sure we have these two programs installed:

# apt-get update
# apt-get install imagemagick inkscape

While it is possible to do this though the Inkscape GUI, I find the whole process easier and faster right from the command line. We will use ImageMagick to resize images, if necessary (refer to my first ImageMagick post if you have questions about resizing images with ImageMagick), and we will use Inkscape to do the actual conversion. The command to do the actual conversion is actually very simple:

$ inkscape InputImage.png --export-plain-svg=OutputImage.svg

When you run this command, expect to see a message similar to the following:

(inkscape:30498): GLib-GObject-CRITICAL **: g_object_set_qdata_full: assertion `quark > 0' failed

This is OK, your image still got converted. Now, let's use this to get the songbird.png image in place for use on our menu. Let's start by copying the songbird.png image into out home directory to make things easier:

$ cp /usr/share/Songbird/songbird.png ~/

Now we can use the mogrify command to resize this to a 48x48 pixel image and then use the inkscape command to convert this to songbird.svg:

$ mogrify -resize 48x48 songbird.png
$ inkscape songbird.png --export-plain-svg=songbird.svg

Now, we'll just copy our new songbird.svg image into the same directory as all the other menu icons:

# cp songbird.svg /usr/share/icons/hicolor/scalable/apps/

If we set up Songbird in your menu the way I described before, then all we will need to do is to restart X (Ctrl + Alt + Backspace) and our new songbird.svg image should should show up in our menu.

See you next time

2009-05-31

Batch Image Editing with ImageMagick

If you take a lot of digital pictures, you may find that there are times when you want to do the same thing to a group of images. You could use the GIMP to edit all of your pictures one at a time, but that is a time-consuming process, especially if you just want to make thumbnails of your vacation pictures. The good thing is that there is a simple, yet powerful, command line utility that you can use to edit photos en masse: ImageMagick.

Let's start by making sure that we have ImageMagick installed:

# apt-get update
# apt-get install imagemagick

I'm going to start by going over two of the features of ImageMagick that I use the most frequently: resizing and cropping images. There are a lot more features available in ImageMagick, and I will probably cover more of them future posts.
  • Batch Resizing Images
    Digital SLR cameras are great for taking high quality pictures, the problem is that they typically take very large pictures that are too large for a lot of uses, such as posting on a blog or using in a presentation. Generally, the first thing I will do is open one of my pictures in the gthumb or some other image viewer that allows me to zoom in and out to find the percentage I want to shrink the images to.

    The next thing I do is I will make copies of all my images that I am going to edit so that I can retain the originals; I do this in a separate directory to minimize my own confusion:

    $ cd /[path-to-images]/
    $ mkdir thumb
    $ cp *.jpg thumb/
    $ cd thumb

    Now, we just use the mogrify command with the -resize option to resize our images. Let's say we want to resize them all to 20% of their original size:

    $ mogrify -resize 20% *.jpg

    Depending on the number of images you are resizing, this command may take a little bit to run, but it is a lot faster than editing each image by hand. If you wanted to resize your images to a specific size, say to create background images for your Mini 9, we would specify 1024x600 pixels rather than a percent:

    $ mogrify -resize 1024x600 *.jpg

    That pretty much does it for resizing pictures with ImageMagick.

  • Batch Cropping Images
    For my post on Installing Debian, I ran the installation in VirtualBox and took screen shots of the VirtualBox window to get the images I posted. The problem with this was, there was some of the window showing in the screen shots that I did not want to include in my post as you can see below:

    The good news is that this window border was identical on all of my screen shots, so I just had to figure out what I needed to crop on one image, and I could use that information in ImageMagick to crop all of the screen shots that I took.

    In order to figure out what I needed to crop, I opened one of the images in the GIMP. Using either the rectangle selection or the crop tool, I selected the region that I wanted to crop out of the image. Then looking on the tool window, I was able to see the position of the top-left corner of the selection as well as the dimensions of the selection:

    The Position section, indicated above in red, is what we will use for our offset with mogrify, and the Size section, indicated above in blue, is what we will use to define the size of the area we want to crop. We just need to use the information above with the mogrify command with the -crop option. But first, we'll copy the images to keep a record of the originals:

    $ cd /[path-to-images]/
    $ mkdir cropped-images
    $ cp *.png cropped-images/
    $ cd cropped-images
    $ mogrify -crop 639x480+1+44! *.png

    The exclamation point in the above command is there to make ImageMagick move the viewport for the image to the same as the crop offset. That may sound a little confusing, so I've included a screen shot of what a picture looks like in the GIMP if you don't use the exclamation point in the above command:

    So, you are best to use the exclamation point in the command, and when you use it, you're picture will look like this:

    That's about all there is for batch cropping images with ImageMagick

See you next time.