Saturday, January 26, 2013
Script - Thread Dump of Multiple Servers located across distributed Machines
Posted by
Thirunavukkarasu Muthuswamy
at
Saturday, January 26, 2013
Labels:
Java,
Oracle Weblogic Application Server,
Script
In this post, we will see how to take thread dump of multiple application servers which is located across distributed machines at one shot.
Scenario:
Consider that you have production code deployed(running) on 9 servers which is distributed 3 servers across to 3 Unix Server Machines.To get a better view about the scenario see the following diagram.

Now we want to take multiple thread dumps of all 9 servers frequently at specified interval.In order to achieve this we are going write a simple bash script, to know about this more keep on reading.
Script Functionalities:
This script have 6 important yet very simple functions for better understanding.
1] Set Environment Variables(set_environment_variables)
2] Create List of Servers (create_servers_list)
3] Get Thread Dump using jstack in Local Machine(get_thread_dump_box1)
4] Get Thread Dump using jstack in Remote Machine(get_thread_dump_box2)
5] Get Thread Dump using jstack in Remote Machine(get_thread_dump_box3)
Note: jstack is java binary executable, to know various other binary executable visit the following link. http://www.industryvertical.co.in/2013/01/java-binary-utilities.html
6] Clean unwanted files(clear_event)
Scripting Part:
Connection across distributed unix servers machines without password is achieved using ssh-rsa and ssh-copy-id command utility. Thanks to Rivest,Shamir, Adleman for this beautiful algorithm. If you have any queries, please leave you reply.
Scenario:
Consider that you have production code deployed(running) on 9 servers which is distributed 3 servers across to 3 Unix Server Machines.To get a better view about the scenario see the following diagram.
Now we want to take multiple thread dumps of all 9 servers frequently at specified interval.In order to achieve this we are going write a simple bash script, to know about this more keep on reading.
Script Functionalities:
This script have 6 important yet very simple functions for better understanding.
1] Set Environment Variables(set_environment_variables)
2] Create List of Servers (create_servers_list)
3] Get Thread Dump using jstack in Local Machine(get_thread_dump_box1)
4] Get Thread Dump using jstack in Remote Machine(get_thread_dump_box2)
5] Get Thread Dump using jstack in Remote Machine(get_thread_dump_box3)
Note: jstack is java binary executable, to know various other binary executable visit the following link. http://www.industryvertical.co.in/2013/01/java-binary-utilities.html
6] Clean unwanted files(clear_event)
Scripting Part:
#!/bin/bash
#Purpose: Script to take thread dump of J2EE application server logs
#Program Name: getThreadDump.sh
clear
set_environment_variables()
{
SERVER_HOME=/optional/Oracle/Middleware/user_projects/domains/base_domain/servers
JAVA_HOME=/optional/Oracle/Middleware/jdk1.6.0_26/
}
create_servers_list()
{
ls ${SERVER_HOME}|grep -v Admin|grep -v logtail|grep -v domain_bak > list_of_servers.txt
}
get_thread_dump_box1()
{
DATE=`date +"%d_%m_%Y_%H%M%S"`
NO_OF_SERVERS=$(cat /optional/Monitoring/list_of_servers.txt|wc -l)
i=1
while [ $i -le $NO_OF_SERVERS ]
do
SERVER_NAME=$(cat list_of_servers.txt|tr '\n' ':'|cut -d":" -f$i)
echo "Getting Process ID of ${SERVER_NAME}"
ps -ef | grep -i Dweblogic.Name=${SERVER_NAME} | grep -v 00:00:00 | awk '{print $2}' > process_id.txt
PID=$(cat process_id.txt|tr '\n' ':'|cut -d":" -f1)
echo "Taking ThreadDump for ${SERVER_NAME} with Process ID ${PID}"
${JAVA_HOME}/bin/jstack $PID > /optional/common/thread_dump/${SERVER_NAME}_${DATE}.out
i=`expr $i + 1`
done
}
get_thread_dump_box2()
{
echo "Checking in 192.168.2.2 Box!!"
ssh 192.168.2.2 /bin/bash getThreadDump.sh
}
get_thread_dump_box3()
{
echo "Checking in 192.168.2.3 Box!!"
ssh 192.168.2.3 /bin/bash getThreadDump.sh
}
clear_event()
{
rm list_of_servers.txt process_id.txt
}
set_environment_variables
create_servers_list
get_thread_dump_box1
get_thread_dump_box2
get_thread_dump_box3
clear_event
Connection across distributed unix servers machines without password is achieved using ssh-rsa and ssh-copy-id command utility. Thanks to Rivest,Shamir, Adleman for this beautiful algorithm. If you have any queries, please leave you reply.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment