Punch Clock for debugging Apache Storm

I have been working a lot with Transactional topologies of Apache Storm these days.

In the course of my work, i have come up with questions like

• Why is the topology stuck ?
• Which part of the topology is the batch stuck at ?

So I came up with an idea based on a punch clock.

To Find out …

1. When did the Person enter / exit the office ?
2. Who is still in office ?

Context Change:

Apply Punch clock to Storm:

Possible Solution:

 

Code:

In the emitBatch method of Partitioned Transactional Spout:

punchCardId = "SPOUT__"+ InetAddress.getLocalHost().getHostAddress()+Thread.currentThread().getId()+"__"+System.currentTimeMillis();
 
PunchClock.getInstance().punchIn(punchCardId); // Punch In 
collector.emit(tuples); // Emit tuple(s) 
PunchClock.getInstance().punchOut(punchCardId); // Punch Out 

—————

Prepare method of Transactional Bolt:

       punchCardId ="Bolt__"+Thread.currentThread().getId()+"__"+System.currentTimeMillis();  //Create Punch Card for txn 

Execute method of Transactional Bolt:

       PunchClock.getInstance().punchIn(punchCardId);      // Punch In

In the finishBatch method of Transactional Bolt:

       PunchClock.getInstance().punchOut(punchCardId);  // Punch Out

—————

Small Demo:

PS: if there are no punch cards available anywhere & topology is stuck, then the problem is probably not your bolts/spout.

Code: https://github.com/jaihind213/storm-punch-clock

Presentation: here

Other Solutions:

(1) logging while entering and exiting

(2) using http://riemann.io/ -> suggested by my friend Angad @Inmobi

Hope you like the idea & hope its useful to you.

Thank you for reading & any feedback is welcome.