To diagnose issues for 100% CPU utilisation or Out of memory
errors it is important to gather thread dumps at the time the problem is
occurring.
The thread dumps will provide information on what process’
are holding onto resources which can then be used to identify the root cause.
In order to identify the processes, multiple thread dumps
over a short period of time are required.
These will then show what threads are present over an extended period of
time, rather than executing and closing as would be expected. It is recommended that 5 consecutive thread
dumps are recorded and supplied.
UNIX/Linux:
Note the process ID number of the Java process
jps –lv
With the Process ID from the previous step, save to a file
and run the following bash script to generate the thread dumps:
#!/bin/sh
#
# Takes the JBoss PID as an argument.
#
# Create thread dumps a specified
number of times and INTERVAL.
# Thread dumps will be in the file
where stdout was redirected
# or in console output.
#
# Usage: sh ./threaddump_linux.sh
<JBOSS_PID>
#
# Number of times to collect data.
LOOP=5
# Interval in seconds between data
points.
INTERVAL=1
for ((i=1; i <= $LOOP; i++))
do
kill -3 $1 > $1_$i.dump
echo "thread dump #" $1 "." $i
if [ $i -lt $LOOP ]; then
echo "Sleeping..."
sleep $INTERVAL
fi
done
Windows:
Create a bat file in the directory called threaddump_win.bat
containing the following (changing folder locations
where appropriate):
@echo off
::
Takes the JBoss PID as an argument.
::
::
Create thread dumps 5 times
::
Thread dumps will be output to the file <PID>_<#>.dump
::
::
Usage: threaddump_win.bat <JBOSS_PID>
::==================================================
::
main
::==================================================
ECHO
==================================================
ECHO JBoss thread dump utility v2
ECHO
==================================================
ECHO.
Set JDK_BIN=C:\Program
Files\Java\jdk1.7.0_07\bin
Set PSTOOLS_DIR=C:\Program Files\PSTools
Set LOG_DIR=C:\ThreadDump\Dumps
::
Set the PID for the threaddumps
CALL :SETPID
::
Check the environment
SET
ENVIRONMENT_OK="TRUE"
CALL
:ENVIRONMENT_CHECK
IF
%ENVIRONMENT_OK%=="TRUE" (
ECHO environment check complete
ECHO.
) ELSE (
ECHO environment check failed, exiting script
GOTO END
)
::
Execute the thread dumps
SET
STAMP=%DATE:/=%_%TIME:~0,2%%TIME:~3,2%%TIME:~6,2%
for %%i in (1 2 3
4 5) do "%PSTOOLS_DIR%\PSExec.exe" -s
"%JDK_BIN%\jstack.exe" %PID% > "%LOG_DIR%\%STAMP: =0%_%PID%_%%i.dump"
::
Exit the script
GOTO END
::==================================================
::
check the environment setup
::==================================================
:ENVIRONMENT_CHECK
ECHO Checking environment
:: Check for PSExec.exe
ECHO ...%PSTOOLS_DIR%\PSExec.exe
CALL :CHECK_EXIST %PSTOOLS_DIR%\PSExec.exe
:: Check for jstack.exe
ECHO ...%JDK_BIN%\jstack.exe
CALL :CHECK_EXIST %JDK_BIN%\jstack.exe
:: Check for output directory
ECHO ...%LOG_DIR%
CALL :CHECK_EXIST %LOG_DIR%
::
Exit the script
GOTO END
:CHECK_EXIST
IF NOT EXIST "%*" (
SET ENVIRONMENT_OK="FALSE"
ECHO.
ECHO **ERROR**
ECHO Could not locate %*
ECHO identify the correct location and
update this script
ECHO.
GOTO :END
)
ECHO ok
ECHO.
GOTO END
::==================================================
::
set the PID
::==================================================
:SETPID
Set PID=%1
IF "%PID%" == "" CALL
:GET_TASKLIST_PID
IF "%PID%" == "" (
ECHO could not find the PID try entering
it as an arguement to the script:
ECHO "threaddump_win.bat
<JBOSS_PID>"
GOTO END
) ELSE (
ECHO Generating threaddumps from %PID%
ECHO.
)
GOTO END
::
Determine the PID from the output of tasklist
:GET_TASKLIST_PID
ECHO No PID specified, attempting to identify
the jboss process automatically
ECHO.
SET /a mem = 0
:: for each java process
FOR /F "tokens=*" %%a in ('TASKLIST
/fi "IMAGENAME eq java.exe"') DO CALL :SET_TASKLIST_PID %%a
GOTO END
:SET_TASKLIST_PID
::skip if the line doesn't contains a memory
value
IF "%5"=="" GOTO END
if %5 GTR %mem% (
:: set the new largest pid and save the
new memory usage
SET /A MEM = %1
SET /A PID = %2
)
GOTO END
::==================================================
:END
Open cmd.exe as administrator
From the command line run threaddump_win.bat optionally using
the pid as a parameter as in the following example:
threaddump_win.bat
1234
