Category Archives: Enterprise level solutions

sshfs: How do you install sshfs and fuse? [CentOS/Linux/Redhat]

25 Replies

One may wonder what is sshfs and why would you want it?  Well simply put, sshfs allows you to mount another server’s filesystem into a folder on your local system which in the background is doing ssh commands and transfers.  As a mounted folder, you are able to move about and copy files back and forth as everything was on local server.  As you can see this makes it very easy for you to work with files on multiple servers.

Note:  you only have to do the following installations on the server where you are doing the mounts on.

Let us download and install the filesystem framework which is a requirement for sshfs called fuse.

wget http://voxel.dl.sourceforge.net/sourceforge/fuse/fuse-2.7.4.tar.gz
tar zxpfv fuse-*.gz
cd fuse*
./configure

If you get the following error, you will either have to point to the location of the kernel source or install it if needed.

checking kernel source directory... Not found
configure: error:
*** Please specify the location of the kernel source with
*** the '--with-kernel=SRCDIR' option
configure: error: ./configure failed for kernel

In our case here, we will be installing the source using yum.

yum -y install kernel-devel

Once installed, you will have to find out the directory it is installed in

ls -l /usr/src/kernels/
total 4.0K
drwxr-xr-x 18 root root 4.0K Oct  7 14:50 2.6.18-92.1.13.el5-x86_64/

./configure --with-kernel=/usr/src/kernels/2.6.18-92.1.13.el5-x86_64
make && make install
cd ..

Now let us get sshfs source and install it.

wget http://voxel.dl.sourceforge.net/sourceforge/fuse/sshfs-fuse-2.1.tar.gz
tar zxpfv sshfs*
cd sshfs-fuse-*
./configure

If you get the following error:

checking for SSHFS... configure: error: The pkg-config script could not be found or is too old.  Make sure it
is in your PATH or set the PKG_CONFIG environment variable to the full
path to pkg-config.

OR

checking for SSHFS... configure: error: Package requirements (fuse >= 2.2 glib-2.0 gthread-2.0) were not met:

No package ‘glib-2.0’ found
No package ‘gthread-2.0’ found

You need to install glib2.  Do the following:

yum install glib2-devel

Once installation is done, continue with configure.

./configure
make && make install

After installation is done, we can move on with testing the installation:

cd /mnt
mkdir test
sshfs 10.0.0.2:/ test

If you get the following error,
sshfs: error while loading shared libraries: libfuse.so.2: cannot open shared object file: No such file or directory
execute this: NOTE: this is for x64 system. If you have 32 bit system, you have to symlink to /lib instead.
ln -s /usr/local/lib/libfuse.so.2 /lib64/
Let us try mounting again:
sshfs 10.0.0.2:/ test
At this point it would be like if you were making a ssh connection to 10.0.0.2 You will have to type in a password to get the mount to happen. You may get the following error: fuse: device not found, try 'modprobe fuse' first

If you do ‘modprobe fuse’, as they tell you to, and you get:
modprobe fuse
FATAL: Module fuse not found.

That means your running kernel is not the same version as the one you compiled with. You have two options here:
1) you can upgrade your kernel by typing: yum update kernel
2) find the source files for the kernel you have running and recompile fuse.

I went with option 1. Once you do the update, reboot and try doing modprobe fuse again.

At this point we can try doing the mount again.
cd /mnt
sshfs 10.0.0.2:/ test

If you do not get any errors, do df -h to see the mount:
...
sshfs#10.0.0.2:/ 1000G 0 1000G 0% /mnt/test
...

At this point you can browse 10.0.0.2 server filesystem as it was local on your server.

————————————-

DISCLAIMER: Please be smart and use code found on internet carefully. Make backups often. And yeah.. last but not least.. I am not responsible for any damage caused by this posting. Use at your own risk.

MySQL: How do you enable sphinxse (Sphinx Storage Engine) in your mysql installation?

8 Replies

As you may know mysql fulltext search is not highly scalable.  One of the options to get around this scalability limitation, which I prefer, is to use Sphinx.  You can use Sphinx with out having to alter your mysql installation.  But, if you would like to use from within mysql and not have to worry about how to pass data between Sphinx and MySQL, you can enable sphinxse (sphinx storage engine).  It is not included with mysql by default so you will have to compile it yourself.

Here are the instructions on how to get sphinxse compiled with your mysql installation on CentOS x64.  I am sure same instructions will work for other flavors but I have not tested it.  I will be compiling the most current version of sphinx (0.9.8) with most current stable version of mysql (5.0.51b) at the time of the writing.  Let’s get the appropriate packages first:

wget http://www.sphinxsearch.com/downloads/sphinx-0.9.8.tar.gz
wget http://dev.mysql.com/get/Downloads/MySQL-5.0/mysql-5.0.51b.tar.gz/from/http://mysql.he.net/
tar zxpf sphinx*
tar zxpf mysql*

You will also need “bison”, “patch”, “automake” and “libtool” installed.  Let us just do a yum install for it.

yum -y install bison patch automake libtool

NOTE:  if you don’t install bison, you will get the following error:
sed '/^#/ s|y\.tab\.c|sql_yacc.cc|' y.tab.c >sql_yacc.cct && mv sql_yacc.cct sql_yacc.cc
sed: can't read y.tab.c: No such file or directory
make[2]: *** [sql_yacc.cc] Error 2

Let us continue with patching mysql source with sphinx storage engine (sphinxse) code and compile/install our new binaries.

cd mysql*
patch -p1 < ../sphinx-0.9.8/mysqlse/sphinx.5.0.37.diff #Make sure everything succeeded.
BUILD/autorun.sh
mkdir sql/sphinx
cp ../sphinx-0.9.8/mysqlse/* sql/sphinx
./configure --prefix=/usr/local/mysql --with-sphinx-storage-engine
make
make install

Now start your mysql installtion and check if engine support is compiled in:

mysql> show engines\G
Engine: SPHINX
Support: YES
Comment: Sphinx storage engine 0.9.8

To read more about how to use Sphinx storage engine, please refer to:  Sphinx documentation for using sphinx storage engine

————————————-
DISCLAIMER: Please be smart and use code found on internet carefully. Make backups often. And yeah.. last but not least.. I am not responsible for any damage caused by this posting. Use at your own risk.

MySQL: How do you use symlinks with MySQL tables?

15 Replies

Creating table symlinks in MySQL is very easy and it is probably one of the features a lot of MySQL users overlook or never think about. You may ask, why would I want to use symlinks? Some of the reasons you would want symlinks is if you are running low on disk space on the partition where your data generally is and/or you want to move one (or more) table(s) on to a different disk/partition for performance reasons.

One of the things worth mentioning is that MySQL documentation states: “Symlinks are fully supported only for MyISAM tables. For files used by tables for other storage engines, you may get strange problems if you try to use symbolic links.” Keeping that in mind, if you have a innodb table and would like to create symlinks for, you should change the engine type to myisam before symlinking. Although, I have heard and personally used symlinks for innodb databases with no side affects.

First let us find out if your MySQL configuration allows symlinks:

mysql> SHOW VARIABLES LIKE 'have_symlink'\G
*************************** 1. row ***************************
Variable_name: have_symlink
Value: YES

I am going to make following assumptions:

  • MySQL installation is in /usr/local/mysql.
  • You want to create new tables under /mnt/another_partition/mysql/var
    1. you must create the dir: mkdir /mnt/another_partition/mysql/var -p
    2. Make it owned by mysql user: chown mysql /mnt/another_partition/mysql/var

We will create some test data. Following are the steps:

mysql> create database db1;
Query OK, 1 row affected (0.05 sec)
mysql> use db1;
Database changed
mysql> CREATE TABLE `users` ( `id` int(11) default NULL, `name` varchar(255) default NULL ) ENGINE=myisam, data directory="/mnt/another_partition/mysql/var", index directory="/mnt/another_partition/mysql/var";
Query OK, 0 rows affected (0.09 sec)
mysql> show tables;
+---------------+
| Tables_in_db1 |
+---------------+
| users |
+---------------+

Drop back to command prompt and “ls” your db1 directory as shown below. Here you can see that users table’s data file (users.MYD) is a symbolic link to the location we specified. You can see that users index file (users.MYI) is symlinked to the location we specified as well.

# ls -al /usr/local/mysql/var/db1/
total 24K
drwx------ 2 mysql mysql 4.0K May 2 00:03 ./
drwx------ 23 mysql mysql 4.0K May 1 23:59 ../
-rw-rw---- 1 mysql mysql 65 May 1 09:33 db.opt
-rw-rw---- 1 mysql mysql 8.4K May 2 00:03 users.frm
lrwxrwxrwx 1 mysql mysql 42 May 2 00:03 users.MYD -> /mnt/another_partition/mysql/var/users.MYD
lrwxrwxrwx 1 mysql mysql 42 May 2 00:03 users.MYI -> /mnt/another_partition/mysql/var/users.MYI

Here are the real files which above symlinks are pointing to:

# ls -al /mnt/another_partition/mysql/var/
total 12K
drwxr-xr-x 2 mysql root 4.0K May 2 00:03 ./
drwxr-xr-x 3 root root 4.0K May 1 23:39 ../
-rw-rw---- 1 mysql mysql 0 May 2 00:03 users.MYD
-rw-rw---- 1 mysql mysql 1.0K May 2 00:03 users.MYI

If you go back in to mysql prompt and insert couple rows, you can see the size of users.MYD increase:

mysql> insert into db1.users values (1,"test1");
Query OK, 1 row affected (0.01 sec)
mysql> insert into db1.users values (2,"test2");
Query OK, 1 row affected (0.01 sec)
# ls -al /mnt/another_partition/mysql/var/
total 16K
drwxr-xr-x 2 mysql root 4.0K May 2 00:03 ./
drwxr-xr-x 3 root root 4.0K May 1 23:39 ../
-rw-rw---- 1 mysql mysql 40 May 2 00:08 users.MYD
-rw-rw---- 1 mysql mysql 1.0K May 2 00:08 users.MYI

If you already have data file and index file you would like to move, I recommend shutting down your MySQL server and moving the files manually and create symbolic links by using “ln -s” command. You can read more about ln by doing man ls

NOTE: Even though MySQL documentation says this is only supported 100% for myisam tables, I have read few posts where people say they have done this with innodb tables without any problems. If anybody else can comment on this, I would greatly appreciate it.

————————————-
DISCLAIMER: Please be smart and use code found on internet carefully. Make backups often. And yeah.. last but not least.. I am not responsible for any damage caused by this posting. Use at your own risk.

MySQL: Solution for ERROR 1442 (HY000): Can’t update table ‘t1’ in stored function/trigger because it is already used by statement which invoked this stored function/trigger.

54 Replies

Here is a sample table you can create to test following problem/solution on:

CREATE TABLE `t1` (
`a` char(1) default NULL,
`b` smallint(6) default NULL
);
insert into t1 values ('y','1');

I have a table t1 which has column a and b, I want column a to be updated to ‘n’ when column b = 0. Here is the first version I created:

DELIMITER |
CREATE TRIGGER trigger1 AFTER UPDATE ON t1
FOR EACH ROW UPDATE t1 SET a= 'n' WHERE b=0;
|
DELIMITER ;

The trigger created successfully but I got this error when I tried to do an update on column b on table t1:
mysql> update t1 set b=0;
ERROR 1442 (HY000): Can't update table 't1' in stored function/trigger because it is already used by statement which invoked this stored function/trigger.

After searching online for a while and trying different solutions, I finally found a way to update the table which has trigger on it:

drop trigger trigger1;
DELIMITER |
CREATE TRIGGER trigger1 BEFORE UPDATE ON t1
FOR EACH ROW
BEGIN
IF NEW.b=0 THEN
SET NEW.a = 'n';
END IF;
END
|
DELIMITER ;

After the new trigger is in, I issued the same update query and “ERROR 1442 (HY000): Can’t update table ‘t1’ in stored function/trigger because it is already used by statement which invoked this stored function/trigger.” didn’t show up and it updated the col a value to “n” as it suppose to.

mysql> update t1 set b=0;
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> select * from t1\G
*************************** 1. row ***************************
a: n
b: 0

Therefore, if you want to create a trigger on the table which will update itself, make sure you use the NEW.column_name to refer to the row after it’s updated and don’t do the full update statement!

However, if you are updating some other table, then you can use the regular update statement:

DELIMITER |
CREATE TRIGGER trigger1 AFTER UPDATE ON t1
FOR EACH ROW UPDATE t2 SET a= 'n' WHERE b=0;
|
DELIMITER ;

————————————-
DISCLAIMER: Please be smart and use code found on internet carefully. Make backups often. And yeah.. last but not least.. I am not responsible for any damage caused by this posting. Use at your own risk.

MySQL: How do you set up master-master replication in MySQL? (CentOS, RHEL, Fedora)

16 Replies

Setting up master-master replication in MySQL is very similar to how we set up master/slave replication. You can read up about how to setup master/slave replication in my previous post: How to set up master/slave replication in MySQL. There is obviously pros and cons about using master/master replication. But this is not a post which discuses advantages and disadvantages for using master/master replication. One of the differences between master/master set up and master/slave is that in master/master set up, you have true redundancy. If one server dies, second server can take all the inserts/selects. In master/slave setup, if master dies, you will have to go through steps to make slave become the master. Master/master set up we are going to set up is essentially master/slave and slave/master. Meaning, if you had two servers, db0 and db1, your setup will be db0(master)/db1(slave) and also db0(slave)/db1(master). Here are some assumptions:

Master1 server ip: 10.0.0.1
Master2 server ip: 10.0.0.2
Slave username: slaveuser
Slave pw: slavepw
Your data directory is: /usr/local/mysql/var/

Let us go through the steps you must take on Master1 to enable it to act as master and slave by using following configuration which goes under [mysqld] section:

# let's make it so auto increment columns behave by having different increments on both servers
auto_increment_increment=2
auto_increment_offset=1
# Replication Master Server
# binary logging is required for replication
log-bin=master1-bin
binlog-ignore-db=mysql
binlog-ignore-db=test
# required unique id between 1 and 2^32 - 1
server-id = 1
#following is the slave settings so this server can connect to master2
master-host = 10.0.0.2
master-user = slaveuser
master-password = slavepw
master-port = 3306

Following is the configuration which goes on master2 under [mysqld] section:

# let's make it so auto increment columns behave by having different increments on both servers
auto_increment_increment=2
auto_increment_offset=2
# Replication Master Server
# binary logging is required for replication
log-bin=master2-bin
binlog-ignore-db=mysql
binlog-ignore-db=test
# required unique id between 1 and 2^32 - 1
server-id = 2
#following is the slave settings so this server can connect to master2
master-host = 10.0.0.1
master-user = slaveuser
master-password = slavepw
master-port = 3306

On master1 server, go to mysql> prompt and add the appropriate user:

mysql> grant replication slave on *.* to slaveuser@'10.0.0.2' identified by 'slavepw';

On master2 server do the same but allow right ip:

mysql> grant replication slave on *.* to slaveuser@'10.0.0.1' identified by 'slavepw';

Restart both of the master servers and check slave status:

mysql> show slave status\G

That is all you have to do to set up the replication. Of course there are a lot more configuration options but this should get your replication going and you can tweak from here on.

————————————-
DISCLAIMER: Please be smart and use code found on internet carefully. Make backups often. And yeah.. last but not least.. I am not responsible for any damage caused by this posting. Use at your own risk.