[root@machine .ssh]# ssh -p 56789 user@othermachine.com
ssh_exchange_identification: Connection closed by remote host

This was due to server configuration. Maximum default number of simultaneous connections tries (login attemps) is defined in /etc/ssh/sshd_config:

MaxStartups 10

We can change above value or use new-style format:

MaxStartups 10:30:60

Which stands for:

1. 10 – number of allowed simultaneous connections attempts. Above this number SSHD will start to randomly drop connections with percentage chance of 30%
2. 60 – number of simultaneous connections attempts after which SSHD will drop every new connection

Assuming, that You ave already dig installed (if not – try yum install bind-utils on CentOs or whatever else on different distros) We can start with explaining how to use properly dig.

1. Simple query

Let’s check Gamedesire’s domain www.gamedesire.com:

[docent@docent-desktop ganymede]$ dig www.gamedesire.com

; <<>> DiG 9.8.1-P1-RedHat-9.8.1-3.P1.fc15 <<>> www.gamedesire.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55580
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 4

;; QUESTION SECTION:
;www.gamedesire.com.		IN	A

;; ANSWER SECTION:
www.gamedesire.com.	59985	IN	A	174.123.95.100
www.gamedesire.com.	59985	IN	A	174.123.95.101

;; AUTHORITY SECTION:
gamedesire.com.		52534	IN	NS	ns2.theplanet.com.
gamedesire.com.		52534	IN	NS	ns1.theplanet.com.

;; ADDITIONAL SECTION:
ns1.theplanet.com.	84570	IN	A	207.218.247.135
ns1.theplanet.com.	130	IN	AAAA	2607:f0d0:0:f:6::1
ns2.theplanet.com.	55905	IN	A	207.218.223.162
ns2.theplanet.com.	130	IN	AAAA	2607:f0d0:0:f:6::2

;; Query time: 31 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Thu Jan 12 22:41:46 2012
;; MSG SIZE  rcvd: 202

Above We can see a bunch of data.. "status: NOERROR" is answer status - NOERROR means, that query was resolved properly. ANSWER SECTION contains of information about resolved records (here We can see 2 A records for www.gamedesire.com: 174.123.95.100 and 174.123.95.101). Next We have AUTHORITY SECTION which gives us an answer to the question: "what are the authoritative DNS servers for this domain?". We have also some statistic data (query time, ADDITIONAL SECTION with DNS servers info etc). And remember to check answer flags (in above example: qr rd ra). You can find explanation of those in RFC 1035 §4.1.1..

What happens when domain name is not resolved properly? Let's try with non-existing sysop.gamedesire.com:

[docent@docent-desktop ganymede]$ dig sysop.gamedesire.com

; <<>> DiG 9.8.1-P1-RedHat-9.8.1-3.P1.fc15 <<>> sysop.gamedesire.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 20931
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;sysop.gamedesire.com.		IN	A

;; AUTHORITY SECTION:
gamedesire.com.		10800	IN	SOA	ns1.theplanet.com. root.theplanet.com. 2012010400 7200 600 1728000 43200

;; Query time: 185 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Thu Jan 12 22:50:02 2012
;; MSG SIZE  rcvd: 93

As we can see the answer status is NXDOMAIN here - it means NoneXistingDomain.

2. Querying specific records

There are plenty of DNS record types. Most commonly used are A, MX, TXT and CNAME. Here is the explanation and complete list of those types: http://en.wikipedia.org/wiki/List_of_DNS_record_types. So now - how can We query for a MX domain?

[docent@docent-desktop ganymede]$ dig mx google.com

; <<>> DiG 9.8.1-P1-RedHat-9.8.1-3.P1.fc15 <<>> mx google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30841
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 4, ADDITIONAL: 9

;; QUESTION SECTION:
;google.com.			IN	MX

;; ANSWER SECTION:
google.com.		373	IN	MX	20 alt1.aspmx.l.google.com.
google.com.		373	IN	MX	30 alt2.aspmx.l.google.com.
google.com.		373	IN	MX	40 alt3.aspmx.l.google.com.
google.com.		373	IN	MX	50 alt4.aspmx.l.google.com.
google.com.		373	IN	MX	10 aspmx.l.google.com.

;; AUTHORITY SECTION:
google.com.		273555	IN	NS	ns3.google.com.
google.com.		273555	IN	NS	ns2.google.com.
google.com.		273555	IN	NS	ns4.google.com.
google.com.		273555	IN	NS	ns1.google.com.

;; ADDITIONAL SECTION:
aspmx.l.google.com.	29	IN	A	173.194.67.27
alt1.aspmx.l.google.com. 171	IN	A	209.85.173.27
alt2.aspmx.l.google.com. 50	IN	A	74.125.127.26
alt3.aspmx.l.google.com. 79	IN	A	74.125.127.27
alt4.aspmx.l.google.com. 146	IN	A	209.85.225.26
ns1.google.com.		274504	IN	A	216.239.32.10
ns2.google.com.		274402	IN	A	216.239.34.10
ns3.google.com.		273449	IN	A	216.239.36.10
ns4.google.com.		274772	IN	A	216.239.38.10

;; Query time: 33 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Thu Jan 12 22:54:07 2012
;; MSG SIZE  rcvd: 352

So We see above in ANSWER SECTION all found MX records. We can ask for any record type with this method.

3. Tracing DNS query

Every DNS query takes some hierarchical steps. Knowing those could be very helpful under some circumstances:

[docent@docent-desktop ganymede]$ dig +trace www.wikipedia.org

; <<>> DiG 9.8.1-P1-RedHat-9.8.1-3.P1.fc15 <<>> +trace www.wikipedia.org
;; global options: +cmd
.			209351	IN	NS	k.root-servers.net.
.			209351	IN	NS	m.root-servers.net.
.			209351	IN	NS	j.root-servers.net.
.			209351	IN	NS	l.root-servers.net.
.			209351	IN	NS	b.root-servers.net.
.			209351	IN	NS	f.root-servers.net.
.			209351	IN	NS	h.root-servers.net.
.			209351	IN	NS	g.root-servers.net.
.			209351	IN	NS	c.root-servers.net.
.			209351	IN	NS	e.root-servers.net.
.			209351	IN	NS	d.root-servers.net.
.			209351	IN	NS	i.root-servers.net.
.			209351	IN	NS	a.root-servers.net.
;; Received 512 bytes from 192.168.1.1#53(192.168.1.1) in 693 ms

org.			172800	IN	NS	b2.org.afilias-nst.org.
org.			172800	IN	NS	a0.org.afilias-nst.info.
org.			172800	IN	NS	a2.org.afilias-nst.info.
org.			172800	IN	NS	c0.org.afilias-nst.info.
org.			172800	IN	NS	b0.org.afilias-nst.org.
org.			172800	IN	NS	d0.org.afilias-nst.org.
;; Received 437 bytes from 128.8.10.90#53(128.8.10.90) in 379 ms

wikipedia.org.		86400	IN	NS	ns1.wikimedia.org.
wikipedia.org.		86400	IN	NS	ns0.wikimedia.org.
wikipedia.org.		86400	IN	NS	ns2.wikimedia.org.
;; Received 147 bytes from 199.19.54.1#53(199.19.54.1) in 332 ms

www.wikipedia.org.	3600	IN	CNAME	wikipedia-lb.wikimedia.org.
wikipedia-lb.wikimedia.org. 600	IN	CNAME	wikipedia-lb.esams.wikimedia.org.
wikipedia-lb.esams.wikimedia.org. 3600 IN A	91.198.174.225
;; Received 121 bytes from 208.80.152.142#53(208.80.152.142) in 157 ms

We see above all the DNS query steps - first question to the root servers for a proper TLD DNS server (org), next the question to the TLD org's server for a proper wikipedia.org NS servers, and then the question to the wikipedia's NS servers for a resolution to the name 'wikipedia.org' which appears to be a CNAMEs for something more...

4. Shortening the output

Digging is a quite verbose action - by design - much verbosity is good for debugging purposes. It's good to know how can We reduce output of this command - for example when We would like to wrap dig command with some monitoring script. For this purpose I suggest to get known with:

+nostats
+nocmd
+noquestion
+short

Let's try with the strongest one from above - +short will reduce all the dig's "noise":

[docent@docent-desktop ganymede]$ dig +short www.gamedesire.com
174.123.95.101
174.123.95.100

We can join +short with eg. "+trace":

[docent@docent-desktop ganymede]$ dig +short +trace www.gamedesire.com
NS l.root-servers.net. from server 192.168.1.1 in 33 ms.
NS c.root-servers.net. from server 192.168.1.1 in 33 ms.
NS g.root-servers.net. from server 192.168.1.1 in 33 ms.
NS m.root-servers.net. from server 192.168.1.1 in 33 ms.
NS b.root-servers.net. from server 192.168.1.1 in 33 ms.
NS a.root-servers.net. from server 192.168.1.1 in 33 ms.
NS i.root-servers.net. from server 192.168.1.1 in 33 ms.
NS j.root-servers.net. from server 192.168.1.1 in 33 ms.
NS d.root-servers.net. from server 192.168.1.1 in 33 ms.
NS e.root-servers.net. from server 192.168.1.1 in 33 ms.
NS h.root-servers.net. from server 192.168.1.1 in 33 ms.
NS f.root-servers.net. from server 192.168.1.1 in 33 ms.
NS k.root-servers.net. from server 192.168.1.1 in 33 ms.
A 174.123.95.100 from server 207.218.247.135 in 173 ms.
A 174.123.95.101 from server 207.218.247.135 in 173 ms.

You can try Yourself with other params.

5. Asking specific DNS server

It is very good practice during checking DNS resolutions (especially while transferring domains etc) to ask query to a couple of DNS servers. We can ask particular DNS server (but only when this server allows us to do so). Remember that by default dig uses DNSes listed in Your /etc/resolv.conf file. Let's try to ask google's DNSes first:

[docent@docent-desktop ganymede]$ dig +short @8.8.8.8 www.gamedesire.com
174.123.95.101
174.123.95.100

And now some ThePlanet's:

[docent@docent-desktop ganymede]$ dig +short @ns1.theplanet.com www.gamedesire.com
174.123.95.101
174.123.95.100

6. The authority

In this example:

[docent@docent-desktop ~]$ dig www.gamedesire.com @ns1.theplanet.com

; <<>> DiG 9.8.1-P1-RedHat-9.8.1-3.P1.fc15 <<>> www.gamedesire.com @ns1.theplanet.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18903
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 4
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;www.gamedesire.com.		IN	A

;; ANSWER SECTION:
www.gamedesire.com.	86400	IN	A	174.123.95.101
www.gamedesire.com.	86400	IN	A	174.123.95.100

;; AUTHORITY SECTION:
gamedesire.com.		86400	IN	NS	ns2.theplanet.com.
gamedesire.com.		86400	IN	NS	ns1.theplanet.com.

;; ADDITIONAL SECTION:
ns1.theplanet.com.	86400	IN	A	207.218.247.135
ns1.theplanet.com.	600	IN	AAAA	2607:f0d0:0:f:6::1
ns2.theplanet.com.	86400	IN	A	207.218.223.162
ns2.theplanet.com.	600	IN	AAAA	2607:f0d0:0:f:6::2

;; Query time: 175 msec
;; SERVER: 207.218.247.135#53(207.218.247.135)
;; WHEN: Thu Jan 12 23:43:50 2012
;; MSG SIZE  rcvd: 202

We can see, that AA bit is set here (flags: qr aa rd). So ns1.theplanet.com is authoritative for gamedesire.com domain (as in AUTHORITY section). Now let's try to dig it again using some other DNS server:

[docent@docent-desktop ~]$ dig www.gamedesire.com

; <<>> DiG 9.8.1-P1-RedHat-9.8.1-3.P1.fc15 <<>> www.gamedesire.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63097
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 2

;; QUESTION SECTION:
;www.gamedesire.com.		IN	A

;; ANSWER SECTION:
www.gamedesire.com.	19128	IN	A	174.123.95.101
www.gamedesire.com.	19128	IN	A	174.123.95.100

;; AUTHORITY SECTION:
gamedesire.com.		9373	IN	NS	ns1.theplanet.com.
gamedesire.com.		9373	IN	NS	ns2.theplanet.com.

;; ADDITIONAL SECTION:
ns1.theplanet.com.	84587	IN	A	207.218.247.135
ns2.theplanet.com.	2292	IN	A	207.218.223.162

;; Query time: 29 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Fri Jan 13 10:42:38 2012
;; MSG SIZE  rcvd: 146

Here We can see, that there is no AA bit set, and also TTL values are lower than 86400 for www.gamedesire.com (We can see 19128 value). Why is that? Because this answer is cached somewhere in the middle and our current DNS server is not authorative for gamedesire.com . If We would repeat this query this TTL value would be dropping every each question. We can't tell looking at above dig output where this query was cached - to know this We would have to repeat the query with recursion disabled and step manually through all the DNS tree (but in 9/10 cases It will be You local DNS cache... which is caching answers).

7. Tracing DIG execution

As I wrote above We can set +trace param using dig to trace resolving path in the DNS tree. But how can We trace what exact queries are sent and received? Surely with tcpdump:

[root@docent-desktop ~]# tcpdump -i p2p1 -s1024 udp port domain
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on p2p1, link-type EN10MB (Ethernet), capture size 512 bytes
10:54:17.047260 IP pma.local.40206 > 192.168.1.1.domain: 9836+ A? www.gamedesire.com. (36)
10:54:17.047721 IP pma.local.55428 > 192.168.1.1.domain: 59517+ PTR? 1.1.168.192.in-addr.arpa. (42)
10:54:17.079337 IP 192.168.1.1.domain > pma.local.55428: 59517 NXDomain* 0/1/0 (109)
10:54:17.080473 IP 192.168.1.1.domain > pma.local.40206: 9836 2/2/2 A 174.123.95.101, A 174.123.95.100 (146)
^C
4 packets captured
4 packets received by filter
0 packets dropped by kernel

From the tcpdump(8) manpage: Name server requests are formatted as:

src > dst: id op? flags qtype qclass name (len)

I rather suggest reading this manpages Yourself - just look for "UDP Name Server Requests" section.

[docent@docent-desktop ~]$ rpm -qa | grep -i wacom
xorg-x11-drv-wacom-0.11.1-3.fc15.x86_64

If You have issues with Wacom under Fedora 15 – just try to install above drivers and reboot X.

But I really hate this multi – touch when I’d like to draw something with the pen, so I wanted to turn it off. Also I wanted the pen to work only on the first monitor – not on the both (I use dual monitors with extended desktops). We can set all those things with xwacomset command. First We have to know our Wacom devices:

[docent@docent-desktop ~]$ xsetwacom list dev
Wacom Bamboo Craft Finger touch 	id: 11	type: TOUCH
Wacom Bamboo Craft Finger pad   	id: 12	type: PAD
Wacom Bamboo Craft Pen stylus   	id: 13	type: STYLUS
Wacom Bamboo Craft Pen eraser   	id: 14	type: ERASER

Now when We have those devices’ names We can turn off the multi touch:

[docent@docent-desktop ~]$ xsetwacom set 'Wacom Bamboo Craft Finger touch' touch off

When done with multi touch We can set the working space of our tablet to the first display unit. Firsly we have to know current mapping settings:

[docent@docent-desktop ~]$ xsetwacom get "Wacom Bamboo Craft Pen stylus" Area
0 0 14720 9200

Now If You want to map the pen to the left monitor, just double the third number:

xsetwacom set "Wacom Bamboo Craft Pen stylus" Area "0 0 29440 9200"

Now I had all done and were ready to go with my work :)

In the simplest scenario let’s assume that We have only one mail server (mail.somedomain.com). We’d like to start a backup MX server for this. We can do this in a few simple steps:

Step 1: Backup postfix configuration

On the backup server We should change some postfix configuration in main.cf file. We should add / change relay_domains, set maximal_queue_lifetime, smtpd_recipient_restrictions and We should create relay_recipient_maps:

relay_recipient_maps = hash:/etc/postfix/relay_recipients
maximal_queue_lifetime = 30d
smtpd_recipient_restrictions =
       [...]
       permit_mx_backup
relay_domains = $mydestination somedomain.com
permit_mx_backup_networks = 128.128.128.0/24 201.201.201.0/24

Now let’s explain the following configuration:

• relay_recipient_maps = hash:/etc/postfix/relay_recipients – this is optional, but I advise to use this parameter. It defines a hash table containing valid recipients. If the backup system wouldn’t know all the valid mailboxes it would have to accept all the emails – including spam for non-existing adresses. With knowledge of legal addresses backup server is able to bounce back emails that have invalid recipient set. This does not apply in environment using catchall mailboxes to catch all the emails. I attached a sample relay_recipients file below. Remember to use postmap command after every change in this file: postmap /etc/postfix/relay_recipients
• maximal_queue_lifetime = 30d – default value for Postfix is 5 days. This number sets the time period in which backup server will try to deliver emails to the main server – so this is maximum time of downtime for main server until mails are bounced back to their original senders with an error.
• relay_domains = $mydestination somedomain.com – this parameter will allow postfix to relay emails for somedomain.com
• permit_mx_backup – security, see http://www.postfix.org/postconf.5.html#permit_mx_backup
• permit_mx_backup_networks – security, see http://www.postfix.org/postconf.5.html#permit_mx_backup_networks

And the sample relay_recipients_file:

user1@somedomain.com   any_value
user2@somedomain.com   any_value
user3@somedomain.com   any_value
user4@somedomain.com   any_value
user5@somedomain.com   any_value

So as You see – You should have replicated users addresses on the MX server in the relay_recipients file.

Step 2: DNS configuration

Having only one mail server it is enough to have only one MX record in our DNS zone file:

[user@server ~]# dig mx somedomain.com
;; ANSWER SECTION:
somedomain.com.		86400	IN	MX	10 mail.somedomain.com.

Here We see our only MX record with 10 priority pointing to the A record mail.somedomain.com. In order to create a new record for our backup MX server We should first add a new A record, like:

mail2.somedomain.com.  	86400	IN	A	129.129.129.129

And then We can create a new MX record with lower priority:

mail.somedomain.com.	86400	IN	MX	20 mail2.somedomain.com.

Step 3: Flushing messages

When main MX server is down, and backup server gets some messages to hold those until main server is back – It moves those messages immediately to the flush queue. Now those messages can be delivered via flush daemon, which is run every some time (set in /etc/postfix/master.cf):

flush     unix  n       -       n       1000?   0       flush

Here the “1000?” stands for 1000 seconds every which flush daemon is activated (until it is not already running – this is why we use here question mark after 1000).

Now we can set how often messages should be flushed via the running flush daemon using the fast_flush_refresh_time param (default set to 12h). So every 12h messages that haven’t had redelivery requested are being kicked automaticly.

When our master server is back We could just flush all the messages manually:

postqueue -f

But above command will flush all the messages in the flush queue – this might not be the best solution as the backup MX can be a slave for a bunch of main MX servers – are you sure You would like to flush all those messages from all those servers when only one is back online?

Better solution is to use:

postqueue -s somedomain.com

Above command will flush only the messages from the given domain – and that’s what We would like to do. But We have to know, that We can use this command only when We have this domain configured as “fast_flush_domains”. Again – We’re lucky, because default “fast_flush_domains” value is:

fast_flush_domains = $relay_domains

And If We configured our somedomain.com as “$relay_domain” – then our flush command will work :) If not then We only have to set:

fast_flush_domains = $relay_domains somedomain.com

And when our main MX comes back again – We can flush this domain on the backup MX – it’s good to be wrapped with some script :)

And We’re good to go – from now (after correct DNS entries’ propagation, so in max 72 hours) our backup MX should work and receive emails when master mail server is offline.

Functions

Function calls are logged directly to Binary Log, so If You forget to create on slave any function that is created on master – You will break your replication and probably You’ll see error like below:

Last_Error: Error 'FUNCTION postfix.recount_quota not exist' on query. Default database: 'postfix'. Query: 'UPDATE user_imap SET quota=(recount_quota())'

When promoting slave to master no additional steps according to functions are required - everything is needed is having functions defined in both: master and slave.

Procedures

Procedure calls are not replicated as in functions - this is important to know. Only the queries inside the procedures get logged to the Binary Log, so You don't have to create procedures on slaves.

In order to promote slave to master you should have procedures created on slave - so it is wise to have all the procedures created on both - master and slaves.

Events

Events created on master server get replicated to the slave with the DISABLE ON SLAVE option - that's why those events are not reexecuted on every slave in our MySQL architecture and we have no duplicated and corrupted data. MySQL logs only queries from inside the event so only those queries are replicated via Binary Log.

In order to promote slave to master according to events we have to do some more job. I've created a simple event below (We create it on the master):

mysql> CREATE EVENT mysql_heartbeat
ON SCHEDULE AT CURRENT_TIMESTAMP + INTERVAL 1 MINUTE
DO INSERT UPDATE `mysql_stat`.`heartbeat` SET `last`=CURTIME();

Now it's replicated on slave via Binary Log - below I've placed replication entry for that event from Binary Log:

CREATE DEFINER=`user`@`localhost` EVENT `mysql_heartbeat` ON SCHEDULE AT CURRENT_TIMESTAMP + INTERVAL 1 MINUTE DO INSERT UPDATE `mysql_stat`.`heartbeat` SET `last`=CURTIME();

And how it looks like after replication on the slave:

CREATE DEFINER=`user`@`localhost` EVENT `mysql_heartbeat` ON SCHEDULE AT '2012-01-06 21:12:56' ON COMPLETION NOT PRESERVE DISABLE ON SLAVE DO INSERT UPDATE `mysql_stat`.`heartbeat` SET `last`=CURTIME();

So now with this knowledge a little procedure to promote slave to master using events:

• Disabling event manager on slave with SET GLOBAL event_scheduler = OFF;
• Enabling all the events with ALTER EVENT `event_name` ENABLE - We have to do this for each event, so writing a little script is very helpful here.
• Enabling event manager with SET GLOBAL event_scheduler = ON;

In order to demote back the master to slave You should follow the previous procedure with a little change on ALTER EVENT - here You just need to DISABLE all the events (not ENABLE).

Triggers

In order to have triggers running properly on master and slaves You have to define them in both - master and slave servers. MySQL statement based replication replicates only the original query to the Binary Log - not the subsequent triggered statements.

When promoting slave to master no additional steps according to triggers are required - everything is needed is having triggers defined in both: master and slave.

Mixed triggers / procedures / functions calls

Let's imagine that We have a trigger, that triggers a procedure which uses a function call. How will this behave in statement based replication?

1. We should have trigger defined on both: master and slave
2. We don't have to have procedure defined on the slave - only on master is enough
3. We should have function defined on both: master and slave

Despite of all - my advice is to keep function, triggers, procedures and events defined on all the servers (masters and slaves) - just to be sure, that We can always promote slave to master without any issues.

And one more thing before finishing this post. If You plan to start replication with just copying FRM, MYI, MYD and InnoDB files You should also dump any functions / triggers and stored procedures on master (or slave) and then import those on the new slave. You can do it (for every database) with:

mysqldump --routines --no-create-info --no-data --no-create-db --skip-opt <database> > dumpfile.sql

And recreate those on the new box:

mysql <database> < dumpfile.sql

channel 2: open failed: connect failed: Connection refused

or:

channel 3: open failed: connect failed: Connection refused

And below:

ERROR 2013 (HY000): Lost connection to MySQL server during query

in the MySQL terminal.

First of all We have to make sure, that our tunnel is working properly, so We just kill the current tunnel and create new one without "-f" and "-N" options:

ssh -p 2345 mysql_tunnel@mysqlmaster-server.com -L 4406:mysqlmaster-server.com:3306

If everything is ok, then We can assume that tunnel is working fine. We can also try to create another tunnel to some other service on different target port and then just try if this other service is working via the tunnel - just to exclude any problems with SSH tunneling.

My problem was that MySQL was configured in the way it was blocking any connections outside localhost. It is default MySQL configuration - We can achieve it via my.cnf entries:

bind-address = 127.0.0.1

or:

skip-networking

So in order to make our MySQL accessible via our tunnel We have to comment out the skip-networking line and make sure that We are connecting to the correct IP addr in our tunnel. For example If we have in our my.cnf this line:

bind-address = 127.0.0.1

Then our tunnel should look like:

ssh -p 2345 -f mysql_tunnel@mysqlmaster-server.com -L 4406:127.0.0.1:3306 -N

(notice that 127.0.0.1 in the above command).

If We would bind our MySQL to some other IP, like:

bind-address = 192.168.0.12

Then We should change our tunneling parameters:

ssh -p 2345 -f mysql_tunnel@mysqlmaster-server.com -L 4406:192.168.0.12:3306 -N

After commenting out that skip-networking our security depends on IP address We are binding the MySQL to. If it's local IP addres in DMZ, than there is no security breaches here. Unwise would be to bind to the WAN address and leave MySQL port opened without any SSL encryption or without filtering traffic by the client IP addr...

General idea to set up this replication is to make a SSH tunnel between those two datacenters and then start transferring data using this secure transport layer. I will call “replication server” – the server that will be slave in our destination and “the master server” will be our master.

I won’t write here how to set up a replication from the scratch. Let’s say that for now there are at least two ways to do it without stopping mysql master (using another slave to take data snapshot or using Percona Xtrabackup).

Firstly we have to start SSH tunnel. We have to ensure, that this tunnel will keep alive trough any connection problems and will not be killed due to an idle (how come when there is replication stream over this?).

Let’s start with ensuring that our tunnell will keep alive. In SSH client configuration (default: /etc/ssh/ssh_config) We should add the following:

ServerAliveInterval 300

With above server maintaining the tunnel will send some keep-alive request every 300 seconds to the master (destination) server.

Now We have to open MySQL port on the master (destination) server on WAN interface. This is not secure unless We filter source IP address trying to connect to this port (let's allow only our slave's server IP addr. to use this port). For maximum security We can use TCP Wrappers on the master (destination) MySQL server, but this will put some overhead to the server functionality as TCP Wrappers always use some DNS resolution. In my opinion filtering MySQL port based on source IP address is enough.

Now We can start our tunnel:

ssh -p 2345 -f mysql_tunnel@mysqlmaster-server.com -L 4406:mysqlmaster-server.com:3306 -N

Let's explain:

• -p 2345 - port We are using to connect over SSH (default 22, but should be changed to something else for standard security reasons)
• -n - SSH will go to background just before command execution. Make sure, that You have SSH keys exported to the master (destination) server from the slave server / user and You will not have to enter any passwords during creating the tunnel
• -L - turns on port forwarding - this is the core of creating SSH tunnel
• -N - "do not execute a remote command" - just because We are just forwarding ports :)

Now We can test this tunnel. Let's try to connect to MySQL master from the slave server:

mysql -h 127.0.0.1 -p 4406 --user=replication

We should be able to connect to the master server with above command. And If We really did - then We can use this connection to start the replication.

This is very simple method that should be wrapped with some monitoring, scripts that will create SSH tunnels automatically when the original tunnel dies or after server crash. We should also remember, that replication lags can be quite high using this technique - everything depends on connection quality and number of writes on master that will have to be replicated on slave. In order to tune this method of replication It can be good to use statement-based replication - because in many cases this method use a bunch less number of kilobytes to transmit replication data to the slave.

find /somewhere -name 'some_files*' -exec wc -c '{}' +

Find command is simple here. We use here also a POSIX solution - ending "exec" with a plus-sign not a semicolon. This form groups the finding results into sets and then run commands on whole that set. And that's why we use it here - our wc -c command is run on a whole set of results giving us in the end line the total size of all the files:

[root@docent log]# find . -name 'maillog*' -exec wc -c '{}' +
 6847 ./maillog-20111120
 6137 ./maillog
 6580 ./maillog-20111204
 5028 ./maillog-20111113
 6424 ./maillog-20111127
31016 total

We could shorten that a little:

[root@docent log]# find . -name 'maillog*' -exec wc -c '{}' + | tail -1
31016 total

And If we would like to have only the number:

[root@docent log]# find . -name 'maillog*' -exec wc -c '{}' + | tail -1 | cut -d' ' -f 1
31016

1. Excluding irrelevant words: sometimes We have to grep for some word but We have to exclude some irrelevant string. E.g. let’s grep for ‘index.html’ but let’s also exclude ’404′ from this:

   grep 'index.html' access.log | grep -v 404

2. egrep (extended grep, same as grep -e or grep --regexp=) allows us to do more powerful search including regular expressions with metacharacters like +, ?, | and ()

   egrep "html|cgi" access.log

3. Counting results - If we just want to know the number of lines that matched our query - We would use:

   grep -c 'index.html' access.log

4. Case Insensitive search - by default grep is case sensitive, If we want to make case insensitive search than we use:

   grep -i 'Index' access.log

5. Matching eXact word only - by default grepping for Word will return lines containing SomeWord and . If We would like to find only those lines containing exact word Word We should use:

   grep -x '404' access.log

grep -w could be also useful here.

6. Matching left and right side of the word - to search for instances of string matching Word in the end or start We use \< or \>:

   Below would match any word starting with access, like access_entry:

   grep 'error\>'

   Below would match any word ending with error, like general_error:

   grep '\<access'

7. Showing context results - sometimes We would like to grep for some errors in logs, but we also would like to view the context of that log entry - e.g. grepping for 'Relay access denied' in Postfix logs to see If that error is occurring with some pattern:

   grep --context=3 'Relay access denied' maillog

8. zgrep - this one would grep in the compressed gzip file - just like gunzip -c flog.gz | grep Word:

   zgrep 'Relay access denied' maillog3.gz

9. Coloring matched words - We can highlight our matched words with some color (check man page to see how to set exact color):

   grep --color 'Relay access denied' maillog3.gz

Features:
- menu and keyboard shortcuts to open (checkout), revert, and sync files.
- prompts to open a file for editing when you attempt to change a checked-in file.
- perforce status displayed on the ruler.

Until I reached this:

- tested on Windows operating systems only.

Bloody hell I say.. who the heck makes integration of P4 and VIM outside (and only outside) Lin? :) It made me really amused :)