Command-Line Wizardry
So with the System Administration work I do now I write a lot of convoluted commands. I think there is a kind of poetry involved with it. So here are some commands I’ve used recently and a synopsis of their purpose
grep -liR --exclude='cow' foo /bar/* >> foobar.txt &
Grep searches the content of files to match a pattern. liR tells it to only output the filenames that match, ignore the case of the pattern to match, and to search recursively through files and directories. The exclude pattern does that, anything that is ‘cow’ is ignored. Finally the output is appended to the foobar.txt file. That little & at the end tells it to run in the background. Helpful if you have more work to do.
Next is this: rsync -avvze 'ssh' --stats --exclude-from='foobar.txt' foo.bar:/foo/ /foo/
Now for those who don’t know rsync. It is an amazing program. It’s purpose is to perform network file copies intelligently. Specifically it will only move the differences between files. It can do nearly anything under the sun related to that task.
This command in particular is going to copy everything from the remote server foo.bar to the local directory on the computer we run the command from. Now, along the way it will compare files between the two and only move ones that are different. The –excludes-from will read a file of things to ignore in this process. The -avvze ‘ssh’ does a couple of things. The two v’s increase the output to level 2, the e tells rsync to tunnel everthing through ssh, the z compresses all the data before it sends it, and the a is a shortcut for a lot of other options that make it really great for making backups.
Now, the reason I put those two commands together is simple. You can use the grep command to find files that you don’t want to move, lets say they have sensitive login information. Those files you don’t want rsynced are listed in a text file that you can use on the rsync command so that they will be ignored. So rsync will copy everything but those files, so the sensitive information is preserved.
Clear as mud, right?
February 4th, 2010 at 4:35 pm by rl