by James Cavazos, Senior Performance Engineer
In Java performance testing, one of the most common and sometimes most frustrating issues is the memory leak. Even the most experienced engineer can slip a memory leak into their code. It is important to know how to spot one and debug the issue. Even if you can’t find the source of the issue without a developer’s help, they will always need data to work with so being able to provide useful information is crucial. Therefore, for a Java memory leak, the first thing you will always be asked for is a heap dump.
What is the heap in Java?
In Java the heap is the portion of memory that the java code uses to store in memory objects. The memory is separate for each java instance and can be configured to meet the needs of the application being run. If not much memory is required, only a small amount can be allocated. If an application is memory intensive a large chunk of memory can be allocated at the start. A heap dump is a file of this heap memory that can be analyzed to see what was in the heap at the time it was taken. It is the Java heap memory dumped into a file.
When the heap is not able to keep up with the demands of the application you will see an error similar to this:
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded at java.lang.AbstractStringBuilder.(Unknown Source) at java.lang.StringBuilder.(Unknown Source) at java.util.UUID.toString(Unknown Source) at examples.MemoryLeak.main(MemoryLeak.java:27)
Seeing this error doesn’t necessarily mean you have a memory leak. Your application may require a larger heap than currently configured or your Garbage Collection settings may need some tweaking. If your current heap size is small – say less than 128 MB then you may just want to try increasing the heap size to see if that resolves the problem. The parameters used to set the heap size are: “-Xms128m -Xmx256m”. This particular example sets the heap to a minimum of 123 MB and a maximum of 256 MB.
So how do you know if you have a memory leak?
The first thing to do in all cases is to add the JVM parameter to create a heap dump on out of memory.
-XX:+HeapDumpOnOutOfMemoryError
The parameter will attempt to create a heap dump whenever there is an out of memory error. Be aware, there are cases where this does not work and it will not always give you what you need.
From here there are a couple of strategies you can take depending on certain conditions. If you are seeing this problem within minutes of starting your application, you will first need to try increasing the heap size. If you are still seeing the application go down rather quickly then Java may have been able to create a heap dump. Otherwise, you will need to try to grab a heap dump with jmap (see below). The heap dump will, by default, be created in the root directory for the application. If you want them to go to a particular location, you can add the parameter:
-XX:HeapDumpPath=/disk2/dumps
If your application is going down in hours or days then you will need to do some monitoring of the active heap. The Java JDK has command line tools that can be used on all platforms. In many performance situations you will not have access to a GUI so these are the standard tools that will be needed to monitor a running application. By monitoring the heap statistics, you should get a better idea of what is happening in the heap and discover if you have a leak.
JMap
The Java JDK tool jmap has many functions for monitoring the heap.
Note: You must use the same version of Java that is running the process you want to monitor. In Unix/Linux when running jmap you must use the same owner:group under which the program or you will not be able to access the running Java process.
Histogram
The first option is the histogram. This prints out a list of current objects in the heap and give you the count and byte size of each. Adding the live setting will only show live objects; meaning it will run a full GC and then print out the remaining objects. It will then give you a total object and byte count. The total byte count is also the amount of used heap.
Example:
$ jmap -histo:live 23128 num #instances #bytes class name ---------------------------------------------- 1: 886808 77893000 [C 2: 886747 21281928 java.lang.String 3: 2150 5344488 [Ljava.lang.Object; 4: 1820 205168 java.lang.Class 5: 463 182960 [B 6: 2553 102120 java.util.TreeMap$Entry 7: 986 86768 java.lang.reflect.Method 8: 1817 43608 java.lang.Long 9: 759 42280 [I 10: 786 37728 java.util.TreeMap 11: 903 28896 java.util.HashMap$Node 12: 1075 24528 [Ljava.lang.Class; 13: 586 23440 java.util.LinkedHashMap$Entry 14: 257 22112 [Ljava.util.HashMap$Node; 15: 502 20080 java.lang.ref.SoftReference 16: 754 18096 javax.management.openmbean.CompositeDataSupport 17: 370 17760 java.util.HashMap 18: 201 16080 java.lang.reflect.Constructor 19: 442 14760 [Ljava.lang.String; 20: 421 13472 java.util.concurrent.ConcurrentHashMap$Node 21: 180 12960 java.lang.reflect.Field 22: 763 12208 java.util.TreeMap$KeySet … … … 725: 1 16 sun.security.provider.NativeSeedGenerator 726: 1 16 sun.util.calendar.Gregorian 727: 1 16 sun.util.locale.provider.AuxLocaleProviderAdapter$NullProvider 728: 1 16 sun.util.locale.provider.SPILocaleProviderAdapter Total 1801171 105779408
We can see, in the output above, that the classes with the most heap usage is character objects followed by String objects. At the bottom, we see that there are about 100 MB in use. Running this at regular intervals will show us if the heap is steadily growing. So what does this tell us? In this example not too much other than the fact that memory usage is growing. Character and String objects are used everywhere so this is not too helpful in determining the cause of the leak. All we can do is monitor the heap and object growth. If a more unique class was in the list then that could help to determine where the leak is coming from. If memory usage is growing at a steady pace then it is safe to assume a memory leak is occurring.
Heap
The heap option gives information on the current heap. Depending on the settings and garbage collection configuration the output may be slightly different.
Example:
$ jmap -heap 2484 Attaching to process ID 2484, please wait... Debugger attached successfully. Server compiler detected. JVM version is 25.92-b15 using thread-local object allocation. Parallel GC with 4 thread(s) Heap Configuration: MinHeapFreeRatio = 0 MaxHeapFreeRatio = 100 MaxHeapSize = 268435456 (256.0MB) NewSize = 44564480 (42.5MB) MaxNewSize = 89128960 (85.0MB) OldSize = 89653248 (85.5MB) NewRatio = 2 SurvivorRatio = 8 MetaspaceSize = 21807104 (20.796875MB) CompressedClassSpaceSize = 1073741824 (1024.0MB) MaxMetaspaceSize = 17592186044415 MB G1HeapRegionSize = 0 (0.0MB) Heap Usage: PS Young Generation Eden Space: capacity = 34078720 (32.5MB) used = 13650576 (13.018203735351562MB) free = 20428144 (19.481796264648438MB) 40.05601149338942% used From Space: capacity = 5242880 (5.0MB) used = 0 (0.0MB) free = 5242880 (5.0MB) 0.0% used To Space: capacity = 5242880 (5.0MB) used = 0 (0.0MB) free = 5242880 (5.0MB) 0.0% used PS Old Generation capacity = 89653248 (85.5MB) used = 0 (0.0MB) free = 89653248 (85.5MB) 0.0% used 4475 interned Strings occupying 346128 bytes.
Dump
This is one of the most important functions of JMap. The dump operation creates a heap dump of the current heap. This is especially useful when you have a slow memory leak in order to get multiple heap dumps during the execution of the Java application. Be aware that the Java application will for the most part be paused while creating the heap dump and can take time depending on the size of the heap.
Example:
$ jmap -dump:live,format=b,file=heap.bin 23128
This will create a file called heap.bin for process with id 23128.
In the next blog we will look into analyzing the heap dump.
Pingback: Java Heap Analysis – Series Part Two |