Category Archives: mysql

MySQL statement based replication with triggers, events, procedures, functions and variables

Posted by on January 6, 2012 No comments

Before starting statement based replication in MySQL database We have to be aware of some specific behaviors for this environment. This kind of replication (statement based) writes each query that modifies data to the Binary Log in order to replicate them on the slave or to use as a point-in-time recovery (PITR). Because of this kind of query logging We should be aware how MySQL replication engine behaves with some special queries like triggers, functions, procedures or events.

Functions

Function calls are logged directly to Binary Log, so If You forget to create on slave any function that is created on master – You will break your replication and probably You’ll see error like below:

Last_Error: Error 'FUNCTION postfix.recount_quota not exist' on query. Default database: 'postfix'. Query: 'UPDATE user_imap SET quota=(recount_quota())'

When promoting slave to master no additional steps according to functions are required - everything is needed is having functions defined in both: master and slave.

Procedures

Procedure calls are not replicated as in functions - this is important to know. Only the queries inside the procedures get logged to the Binary Log, so You don't have to create procedures on slaves.

In order to promote slave to master you should have procedures created on slave - so it is wise to have all the procedures created on both - master and slaves.

Events

Events created on master server get replicated to the slave with the DISABLE ON SLAVE option - that's why those events are not reexecuted on every slave in our MySQL architecture and we have no duplicated and corrupted data. MySQL logs only queries from inside the event so only those queries are replicated via Binary Log.

In order to promote slave to master according to events we have to do some more job. I've created a simple event below (We create it on the master):

mysql> CREATE EVENT mysql_heartbeat
ON SCHEDULE AT CURRENT_TIMESTAMP + INTERVAL 1 MINUTE
DO INSERT UPDATE `mysql_stat`.`heartbeat` SET `last`=CURTIME();

Now it's replicated on slave via Binary Log - below I've placed replication entry for that event from Binary Log:

CREATE DEFINER=`user`@`localhost` EVENT `mysql_heartbeat` ON SCHEDULE AT CURRENT_TIMESTAMP + INTERVAL 1 MINUTE DO INSERT UPDATE `mysql_stat`.`heartbeat` SET `last`=CURTIME();

And how it looks like after replication on the slave:

CREATE DEFINER=`user`@`localhost` EVENT `mysql_heartbeat` ON SCHEDULE AT '2012-01-06 21:12:56' ON COMPLETION NOT PRESERVE DISABLE ON SLAVE DO INSERT UPDATE `mysql_stat`.`heartbeat` SET `last`=CURTIME();

So now with this knowledge a little procedure to promote slave to master using events:

  • Disabling event manager on slave with SET GLOBAL event_scheduler = OFF;
  • Enabling all the events with ALTER EVENT `event_name` ENABLE - We have to do this for each event, so writing a little script is very helpful here.
  • Enabling event manager with SET GLOBAL event_scheduler = ON;

In order to demote back the master to slave You should follow the previous procedure with a little change on ALTER EVENT - here You just need to DISABLE all the events (not ENABLE).

Triggers

In order to have triggers running properly on master and slaves You have to define them in both - master and slave servers. MySQL statement based replication replicates only the original query to the Binary Log - not the subsequent triggered statements.

When promoting slave to master no additional steps according to triggers are required - everything is needed is having triggers defined in both: master and slave.

Mixed triggers / procedures / functions calls

Let's imagine that We have a trigger, that triggers a procedure which uses a function call. How will this behave in statement based replication?

  1. We should have trigger defined on both: master and slave
  2. We don't have to have procedure defined on the slave - only on master is enough
  3. We should have function defined on both: master and slave

Despite of all - my advice is to keep function, triggers, procedures and events defined on all the servers (masters and slaves) - just to be sure, that We can always promote slave to master without any issues.

And one more thing before finishing this post. If You plan to start replication with just copying FRM, MYI, MYD and InnoDB files You should also dump any functions / triggers and stored procedures on master (or slave) and then import those on the new slave. You can do it (for every database) with:

mysqldump --routines --no-create-info --no-data --no-create-db --skip-opt <database> > dumpfile.sql

And recreate those on the new box:

mysql <database> < dumpfile.sql

MySQL tunneling via SSH and error “channel: open failed: connect failed: Connection refused”

Posted by on December 28, 2011 No comments

Lately I wrote a short article about MySQL tunneling via SSH in order to start safe MySQL replication. Afterwards I noticed some problems with creating a new SSH tunnel for MySQL connection on a quite different environment. After creating SSH tunnel and trying to connect via this tunnel to the SSH server I received SSH error on tunnel error-log:

channel 2: open failed: connect failed: Connection refused

or:

channel 3: open failed: connect failed: Connection refused

And below:

ERROR 2013 (HY000): Lost connection to MySQL server during query

in the MySQL terminal.

First of all We have to make sure, that our tunnel is working properly, so We just kill the current tunnel and create new one without "-f" and "-N" options:

ssh -p 2345 mysql_tunnel@mysqlmaster-server.com -L 4406:mysqlmaster-server.com:3306

If everything is ok, then We can assume that tunnel is working fine. We can also try to create another tunnel to some other service on different target port and then just try if this other service is working via the tunnel - just to exclude any problems with SSH tunneling.

My problem was that MySQL was configured in the way it was blocking any connections outside localhost. It is default MySQL configuration - We can achieve it via my.cnf entries:

bind-address = 127.0.0.1

or:

skip-networking

So in order to make our MySQL accessible via our tunnel We have to comment out the skip-networking line and make sure that We are connecting to the correct IP addr in our tunnel. For example If we have in our my.cnf this line:

bind-address = 127.0.0.1

Then our tunnel should look like:

ssh -p 2345 -f mysql_tunnel@mysqlmaster-server.com -L 4406:127.0.0.1:3306 -N

(notice that 127.0.0.1 in the above command).

If We would bind our MySQL to some other IP, like:

bind-address = 192.168.0.12

Then We should change our tunneling parameters:

ssh -p 2345 -f mysql_tunnel@mysqlmaster-server.com -L 4406:192.168.0.12:3306 -N

After commenting out that skip-networking our security depends on IP address We are binding the MySQL to. If it's local IP addres in DMZ, than there is no security breaches here. Unwise would be to bind to the WAN address and leave MySQL port opened without any SSL encryption or without filtering traffic by the client IP addr...

MySQL replication over SSH tunnel

Posted by on December 24, 2011 No comments

Sometimes it is a good decision to replicate between datacenters. It is not for a backup purposes – as replication cannot be used for backups (maybe under some circumstances, but let’s say that for now We’re not thinking about replication as backup solution) – for now We’re using it to just have up2date data in some other datacenter.

General idea to set up this replication is to make a SSH tunnel between those two datacenters and then start transferring data using this secure transport layer. I will call “replication server” – the server that will be slave in our destination and “the master server” will be our master.

I won’t write here how to set up a replication from the scratch. Let’s say that for now there are at least two ways to do it without stopping mysql master (using another slave to take data snapshot or using Percona Xtrabackup).

Firstly we have to start SSH tunnel. We have to ensure, that this tunnel will keep alive trough any connection problems and will not be killed due to an idle (how come when there is replication stream over this?).

Let’s start with ensuring that our tunnell will keep alive. In SSH client configuration (default: /etc/ssh/ssh_config) We should add the following:

ServerAliveInterval 300

With above server maintaining the tunnel will send some keep-alive request every 300 seconds to the master (destination) server.

Now We have to open MySQL port on the master (destination) server on WAN interface. This is not secure unless We filter source IP address trying to connect to this port (let's allow only our slave's server IP addr. to use this port). For maximum security We can use TCP Wrappers on the master (destination) MySQL server, but this will put some overhead to the server functionality as TCP Wrappers always use some DNS resolution. In my opinion filtering MySQL port based on source IP address is enough.

Now We can start our tunnel:

ssh -p 2345 -f mysql_tunnel@mysqlmaster-server.com -L 4406:mysqlmaster-server.com:3306 -N

Let's explain:

  • -p 2345 - port We are using to connect over SSH (default 22, but should be changed to something else for standard security reasons)
  • -n - SSH will go to background just before command execution. Make sure, that You have SSH keys exported to the master (destination) server from the slave server / user and You will not have to enter any passwords during creating the tunnel
  • -L - turns on port forwarding - this is the core of creating SSH tunnel
  • -N - "do not execute a remote command" - just because We are just forwarding ports :)

Now We can test this tunnel. Let's try to connect to MySQL master from the slave server:

mysql -h 127.0.0.1 -p 4406 --user=replication

We should be able to connect to the master server with above command. And If We really did - then We can use this connection to start the replication.

This is very simple method that should be wrapped with some monitoring, scripts that will create SSH tunnels automatically when the original tunnel dies or after server crash. We should also remember, that replication lags can be quite high using this technique - everything depends on connection quality and number of writes on master that will have to be replicated on slave. In order to tune this method of replication It can be good to use statement-based replication - because in many cases this method use a bunch less number of kilobytes to transmit replication data to the slave.