Category Archives: Linux Tips

MySQL: How do I import individual table dump files in to MySQL using shell script?

12 Replies

After I wrote the post: How do I dump all tables in a database into separate files? I got emails from couple people asking how to import the individual table files back in to MySQL. First way to import each sql file created by the post is to import each file individually by typing:mysql db_name < table1.sql This will work as long as you are only importing few files. But if you need to import all of the files in the directory, which could be in 100’s, this method does not scale well. To achieve this I wrote a shell script which does the work for me. Of course, there are other ways to do this and I am only showing you one way of doing it. This works for me so here it is:

#!/bin/bash
db=$1
if [ "$db" = "" ]; then
echo "Usage: $0 db_name"
exit 1
fi
mkdir done
clear
for sql_file in *.sql; do
echo "Importing $sql_file";
mysql $db< $sql_file;
mv $sql_file done;
done

Related posts:

————————————-
DISCLAIMER: Please be smart and use code found on internet carefully. Make backups often. And yeah.. last but not least.. I am not responsible for any damage caused by this posting. Use at your own risk.

Linux: How do I mass find and replace text in files under linux using perl?

408 Replies

Few friends have asked me how to do mass find and replace text in files under linux. There are quite a bit of options in linux to achieve mass replacing of text in files. If you are doing it file by file, you can achieve that in vi by opening, running replace and closing and going to next file. But sometimes that can be very tedious and you would rather do mass replacement on all files containing certain extension. We can do this by using sed or perl. Since most people are familiar with perl (at least most system admins and programmers), I will show you a perl way of doing it which you can use with sed as well if you wish. First step is to get perl to do what we want on one file

perl -w -i -p -e "s/search_text/replace_text/g" filename

-w turns warnings on
-i makes Perl operate on files in-place (if you would like to make backups, use -i.bak, this will save filename.bak files)
-p loops over the whole input
-e specifies Perl expression
filename works on one file at a time

Once we get the results we want, we can now pass it multiple files by doing something like:

perl -w -i -p -e "s/search_text/replace_text/g" *.php

This will search and replace within all php files in the directory you are in.

Now, let us say you want to go through your whole web directory and replace every place where you have “Perl is good” to “Perl is great”, you would use following command:

find /www_root -name "*.php"|xargs perl -w -i -p -e "s/Perl is good/perl is great/g"

find will start at /www_root, look for filenames which have .php extension, xargs takes that filename and passes it to perl as an arguement.

————————————-
DISCLAIMER: Please be smart and use code found on internet carefully. Make backups often. And yeah.. last but not least.. I am not responsible for any damage caused by this posting. Use at your own risk.

Linux virus scan: How do I check my linux installation for viruses using ClamAV? (CentOS, Linux, Redhat)

3 Replies

There are quite a bit of antivirus software exist for Linux nowadays. One of the popular antivirus software is ClamAV. We will install Clam AntiVirus software from source and install it to a custom directory. We will also install gmp-devel package which installs GMP library. GMP library is used to verify the digital signature of the virus database.

yum -y install gmp-devel
wget http://freshmeat.net/redir/clamav/29355/url_tgz/clamav-0.91.2.tar.gz
adduser -M -s /bin/false clamav
tar zxf clamav-0.91.2.tar.gz
cd clamav-0.91.2
./configure --prefix=/usr/local/clamav
make install
for binaries in `find /usr/local/clamav/bin/*` ; do ln -s ${binaries} /usr/bin/; done

At this point Clam AntiVirus is installed and ready for use. Edit the configuration file and remove the line which says: Example It is there to ensure. If you want, you can look at other options but we don’t need to change anything else here to make ClamAV work for us.

vi /usr/local/clamav/etc/freshclam.conf #remove Example

Now let us run the freshclam which will download virus database and bring our virus database up to date. We should do this manually and make sure it didn’t give any errors. If this works, you will a lot of “downloading” messages.

/usr/bin/freshclam

If everything checks out, let us add this to our crontab to ensure our virus database is updated hourly. I chose to be updated every 9 minutes in to every hour. You can change to fit your needs or leave it as it is.

crontab -e

9 * * * * /usr/bin/freshclam –quiet

At this point our ClamAV virus database is up to date and now we can scan whichever directory we want. Go to the directory you want to scan and type:

clamscan -r -i

Once it is done scanning, it will display something similar to below.
-r parameter tells clamscan to recurse into directories
-i will print out infected filenames

----------- SCAN SUMMARY -----------
Known viruses: 159855
Engine version: 0.91.2
Scanned directories: 1437
Scanned files: 8836
Infected files: 0
Data scanned: 464.83 MB
Time: 103.678 sec (1 m 43 s)

Happy scanning!
————————————-
DISCLAIMER: Please be smart and use code found on internet carefully. Make backups often. And yeah.. last but not least.. I am not responsible for any damage caused by this posting. Use at your own risk.

Subversion (SVN): How do you set up backup for your Subversion repositories?

18 Replies

Subversion is becoming more and more popular every day. To get quick installation guide, see my post on installing Subversion. Once you have Subversion set up, make sure you have backups of repository setup! This is one of those things most people forget to do. If you followed the installation guide and did source install, you already have a tool available to you. You can find hot-backup.py tool in the tools/backup/ directory of the Subversion source distribution. One thing about this script is that it doesn’t backup all your repositories. Which might be an issue if you create more repositories but forget to create a script to backup that particular repository. Here is a very simple shell script to help you backup all of your subversion repositories. You can edit it to point to different /svn location if you didn’t install to default location. Make sure you copy hot-backup.py to a location which is included in your path so your scripts will work.

mkdir /backups/repos -p
for repostobackup in `ls /svn/` ; do
filename=`basename ${repostobackup}`
if [ ! $filename = "" ]; then
hot-backup.py --archive-type=bz2 /svn/${filename} /backups/repos
fi
done

This will create your backup files and compress them using bzip2. You can also use zip or gz if you prefer. Another thing you should do is to edit the hot-backup.py script and change num_backups to be whatever you think is a good number. I personally use: num_backups = 10

That’s all there is to it. Just add this script to your crons and your backups will be automated. You may want to consider moving your backups to another server by adding ftp, scp or rsync at the end of the script.

Cron: crond will not read/honor new entries in crontab file. (CentOS, Redhat, Fedora)

1 Reply

I ran in to something really odd. I have been using crontab for LONG time and never seen this issue before. Basically I added new entry into my crontab by type crontab -e and as always saved/exited. For some reason that command never ran at the time I specified. I tested few more times and no luck. So I started looking on google to find out why crond will not ready my new entries from crontab. And then while I was looking, I decided to restart crond by issuing: service crond restart and to my surprise, my new schedule started working.

While I was looking around for solution for this problem on Google, I came upon few pages where they talked about if you have empty line before command, that command will not work. For example:

#this is a cron file

* * * * * /do/something

In this example, /do/something will not run since there is a blank line before it. You can fix this by moving that command up or put another comment above it. This is a very odd thing and I have never experienced it myself but I figured if you stumble upon this post looking for help, blank line before command in crontab might be your issue. Please comment if you had to delete a blank line to fix your crontab file. Also mention which OS you are using. To learn more about crond, see man crond and man crontab.