Case 1 – Making Relay Log Dump possible

Hi All,

This is my first post on this Blog ๐Ÿ™‚

I would like to thank Ramakrishna Nalam , Shashwat Agarwal , Suthirta for inspiring me to take up this new Blog.

I am chief Surgeon at the Flipkart M*A*S*H – which is an internal blog ,ย  well MASH 213 is now here.

At work, we had a use case for MultiSource Replication . We were using Tungsten to achieve this.

topo

We were stuck on a case where fail over would occur – i.e. what would happen if a slave we were replicating from would die, how would we continue using another slave ?

Using MySQL 5.5 – Well, the hard way would be to parse binlog – find the location of last applied transaction in other slave binlog, get its position and continue from there – the format of binlog – statement/ row/ mixed had to be same across all slaves else this could be tough.

Well Since we were using tungsten for MultiSource Replication and tungsten feeds of a binlog, i thought –

hey the Slave Relay Log is = Master Binlog (The slave copies over the binlog of master and keeps it as relay log)

why not replicate from relay log of slave instead of its binlog – All slaves would have similar Relay Logs.

On replication failure from the slave – Tungsten would tell us the last successful position in the log file (in this case the relay log of failed slave) it has applied. Parsing the relaylog using mysqlbinlog utility – we could go to this position & get the Transaction & its Transaction ID (XID)

Since tungsten would replicate from the relay log of the new slave now & the new slave relay log and the failed slave are same, the new slave would have that XID in its relay log , we could now narrow down to the corresponding position and continue replication from there from this slave.

We could even start replicating from the Master Binlog – (i.e. now ask tungsten to feed of binlog instead of relay log) and when the slave is up, we could make tungsten switch to relay log of slave.

I have modified the Percona Server to add Relay Log Dump capability. [Client included for testing this out]

Its available @ https://github.com/jaihind213/mash213

Do visit the code – This is my first patch to MySQL ๐Ÿ™‚ – though not included in oracle/percona source tree.

All in all – my first surgery at this Unit.

Thank you for reading this post – you can reach me for any doubts atย  -> jaihind213@ gmail dot com

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s