Memory Management

Lets start from where we left on the earlier post: JVMView

We know each JVM instance is born with a Method Area and a Heap Memory



Heap Memory:

The place where object resides, the Default maximum heap size is 64 Mb, this may be configured with the following VM options as well:

·         -Xmx<size> - to set the maximum Java heap size

·         -Xms<size> - to set the initial Java heap size

Method Area or Non-Heap Area:

This is place where binary code, runtime constant pool, static fields/classes resides. GC will not visit this memory. By default the maximum size of this area is 64 Mb, this may be configured as:

          -XX:MaxPermSize = to set max non heap size

Memory Management:

This is process of managing memory by allocating portion of memory during object creation and reclaiming memory when object is not reachable/referred.

In some programming, reclaiming memory is programmer's responsibility, which is quite complex; developer could spent most of time in debugging. In Java these overhead is taken care automatically by automatic memory management [Garbage Collector]

Garbage Collection Algorithm:

Garbage collection was invented by John McCarthy

Garbage Detection:

As a first step, GC has to detect/identify garbage [objects which are not referred directly/indirectly], for that GC first defines set of roots, from which it will try to reach out to other objects, objects which are not reachable are considered as garbage, because they are no longer required for program execution.

Tracing Approach [All modern JVM prefer this approach]:

Tracing start from root objects and traverse through the whole graph to find out live objects. Objects that are traversed during the trace are marked. After the trace is complete, unmarked objects are known to be unreachable and can be garbage collected. This basic tracing algorithm is called "mark and sweep." 

Yellow are the objects, which are visited by GC during tracing and rest of the objects [dark blue] are considered as non-referred objects and can be garbage collected

So algorithm is quite simple right? Wait!

We all know a real time application is going to have more no. of objects. Expecting GC to traverse through each and every object is going to be time consuming and could degrade the system. Assume a application has some cache objects - which means they are going to be live for long time [if not till the end of the program, at least for some long time] – so GC performance can be improved if it can avoid visiting these kind of long lived objects regularly.

The two important observations about objects:

1. Most objects have very short lives [ex: local objects].
2. Some objects have long lifetimes [ex: Cache objects], and it could refer to any short-lived object as well.

Having these two points as base assumption, GC group’s objects by age, and GC collects younger [short lived] object more often than older [long lived] ones.

Generational Collectors:

In this approach, the heap is divided into two sub-heaps; each serves one generation of objects and garbage will be collected from each heaps separately. 

When an object is created it allocates memory in the younger generation, if the object survives few younger generation garbage collections then the object will get promoted to old generation.

The younger generation is typically small and collected frequently. As most objects are short-lived, only a small percentage of young objects are likely to get promoted to older generation. On the other hand the older generation is typically larger and collected less frequently as it is expected to have long lived objects. 

In order for an object to be retained in younger generation for few cycles, the younger generation is further split in to three areas. 

The Eden: The place where most new objects are allocated, this place is empty after a garbage collection cycle

Two-survivor space: These hold objects, which has survived one garbage collection and have been given another chance to become unreachable before they promoted to old generation. Like shown in the above figure only one survivor space will have object while the other is most of the time empty.

Demonstration of a Minor [Young generation] GC cycle:

• In general all objects will be initially created in Younger generation [Eden space]
• When GC runs in young generation heap[both in eden and from], the object found for garbage are marked as X.  
• Live objects found in the Eden are copied to unused survivor space.
• Live objects in the survivor space [From], which needs to be given another chance, are copied to unused survivor space
• Live objects in the survivor space[From], which are old enough, are promoted to old generation

• Towards the end of the minor GC, the two survivor space swaps space [From <--> To]
• Eden will be empty, one survivor space[From] will have objects and other [To] will be empty

The cycle keeps continuing... At the time of next run: the Eden will be filled by new objects, From will have objects which has survived previous run, and To will be empty as usual. The GC cycle repeats and non-referenced  objects are cleaned up.

With this I assume we have some good understanding how GC deals with short-lived objects.

  

Below are the commonly known garbage collectors:

1. The Serial GC:

For young generation, this GC work in the same way mentioned above

For old generation, it mange by “sliding compacting mark-sweep”

Sliding Compacting Mark-Sweep

After identifying the objects, which are going to live in old generation, it slides them towards the beginning of the heap so that free space will be available as contiguous chunk towards the end. Hence memory allocation for any new object will be easier.

Both Minor and Full GC take place in sequence in a Stop the World fashion; the application will be stopped when GC is in progress.

2. The Parallel GC:

This operates similar to Serial GC [Stop the World Fashion], however both the minor and full garbage collections take place in parallel.

3. The Mostly Concurrent GC:

Also known as Concurrent Mark-Sweep [CMS], manages the young generation in the same way [stop the world fashion], where as the old generation [large part of the heap] is managed concurrently. Unlike other two GC, this does not do the fragmentation, free space are not contiguous as a result allocation into old generation is expensive.

Application, which requires huge heap, tends to opt to this GC. Since Heap is huge the GC is going to take more time, and this does not stop the application while GC is in progress

4. The Garbage-First GC:

This is parallel, concurrent, and compacting low-pause GC.

This does not have physically separate spaces for young and old generation but it is a set of regions which allows this to resize young generation in a flexible way

Detailed description of these GC is beyond the scope of this article.

Notes:

Out Of Memory:When a request is made to create new object, and if heap does not have sufficient space after GC cycle then application crashes with OOM error.

The first thing comes to our mind is to increase the heap size, but before doing that we need to check whether the app really requires huge heap? As the problem might be because of Memory leak, if that is the case then the increased heap might not be sufficient after some period of time. Using Eclipse Memory Analyzer Tool [MAT] we can identify memory leak, there are many such tools available in market. 

Even if you decide to increase heap size, keep in the mind that the huge the heap the more the workload on GC as it has to traverse through the entire heap 

Avoid Object Pooling: Pooled objects are long-lived objects and it will get into Old generation

Sloppy sizing of Array: If Array list is initially sized too small, then its backing array might subsequently needs to be resized several times causing unnecessary allocation

Command Line Tools: -XX:+PrintGCDetails  is command line argument to print the GC information during runtime. This will print Heap [Eden, From, To], Non-Heap memory usage.

References:

http://docs.oracle.com/javase/specs/jvms/se7/html/index.html

http://www.artima.com/insidejvm/ed2/gcP.html

Java Performance by Charlie Hunt & Binu John

http://en.wikipedia.org/wiki/Garbage_collection_(computer_science)

http://www.yourkit.com/docs/kb/sizes.jsp

The Class Loader Subsystem

The Class Loader is one of the key components of Java, All modern day java containers [EJB, Servlet,..] has custom class loaders. An in-depth understanding of class loaders is required for developing, debugging, packaging, deployment and better architecture of J2EE components.

As Wikipedia says, "The Java Classloader is a part of the Java Runtime Environment that dynamically loads Java classes into the Java Virtual Machine". It is as simple as that! 

Now, even before we look into how class loader works, it is important to understand what loading a class means? 

A class has to go through three phases in order to get loaded in to the JVM, they are Loading, Linking and Initialization.

The Java Specifications mandates that the Initialization phase should happen only when the Class gets its first active usage whereas Loading & Linking phases can vary from JVM to JVM. Most of the JVM loading happens much earlier whereas the linking and initializing will be delayed till the first use.

By delaying the Initialization phase, the class loading is delayed until the class is referred/required [using new, static reference, etc.] thus reducing memory usage and improving system response time. 

Irrespective of when the loading, linking or initializing happens, any error occurred during these phases will be captured in the form of Linkage Error and thrown only at its first usage of that class. 

The Class Loader is responsible for doing all these three phases, now let's see how Class Loader functions.

Almost all Class loader subsystem has minimum of three class loaders in it; in addition to that we can write our own class loaders. Almost all java programs/containers will relay on the class loaders delegation hierarchy to load their classes into JVM.

Bootstrap or Primordial Class loader: This is responsible for loading only the core Java API (e.g. classes files from rt.jar).  This is the only class loader, which comes with JVM implementation [Mostly written in C, C++]. This is root of all the class loader hierarchy.

Extension Class loader: Responsible for loading classes from the Java extension directory (i.e. classes or jars in jre/lib/ext of the java installation directory). Bootstrap is the parent of this class loader.

System class loader or Application class loader: Responsible for loading classes from the java class path [class directories, jars from class path]. Extension class loader is the parent of this class loader. This class loader is by default the parent of all the custom/user defined class loaders in a java application.

The Bootstrap class loader is the primordial class loader comes with JVM implementation, rest of the class loaders are just like any other java objects.

Consider the below example: We are trying to find the class loader responsible for loading three classes, where the MyConstant.class is placed in ././ jre/lib/ext directory as well as in local workspace and DemoTest is the main class from local workspace

The output will be something like this

Where:

ExtensionClassLoader  loads MyConstant class from ext directory

ApplicationClassLoader loads DemoTest  class from local workspace 

BootStrapClassLoader loads String class but it is null because it is not a java object

But we know that MyConstant class is there in the ext directory as well as the local workspace then why does the AppClassLoader didn’t load the class?

To understand this we need to be aware of the Class Loader Algorithm, which is based on Delegation Hierarchy Principle:

1. The current class loaders checks whether the class is already loaded, if so return it 
2. If class is not already loaded, it delegates the work of loading classes to their parent. This delegation can go up to BootStrapClassLoader 
3. If the parent classloader cannot find the class, then the current classloader attempt to find them locally.
4. If the class is not found then java.lang.ClassNotFoundException will be thrown

In our case, when
String class is referred - the AppClassLoader delegates the request to its parent ExtClassLoader, which in turn delegates the request to its parent BootStrapClassLoader. The Bootstrap class loader finds the class from java.lang package and returns it back

MyConstant class is referred - the ApplicationClassLoader delegates the request to its parent ExtClassLoader, which in turn delegates the request to BootStrapClassLoader; since MyConstant is not part of java api jars BootStrapClassLoader fails to find the class so the request comes back to ExtensionClassLoader, and since MyConstant class is available in ext directory, the ExtensionClassLoader loads the class.

That’s why the MyConstant was not loaded from local classpath; instead ExtClassLoader loaded it from ext directory.

Every class will be loaded only once by a class loader, there could be chances where a same class might get loaded by different class loaders, which is covered in the Class Loading Errors and Suggestion section.

With this, Assume when we are packaging a war file having its own libraries, say log4j jar [version x] whereas the servlet container [server] has its own library  [log4j jar version y]. 

In this case, we want our application [war] to load your library classes [log4j of version x] and not the container’s. How it is achieved?  

Java EE Delegation Hierarchy:

In general almost all container will have its own class loader to load their libraries.
Also every WAR/EAR has its own class loader as well. 

The Java Servlet specification recommends [does not force] the war classloader should load the class even before it delegate to its parent i.e., container class loader. Container class loader will work in the normal way, it delegates to system, extension and bootstrap classloader.

Most of the application container this recommendation is followed, in container like GlassFish we will be able to configure on which delegation model to be followed.

With this I hope you would have got some basic understanding on class loader. Before we complete I would also want to let you know about few of the class loading error, which you would have seen very often

Class Loading Errors and Suggestions

ClassNotFoundException: This occurs when we try to load the class dynamically [Class.forName() or ClassLoader.loadClass() - where the class name is passed as String] and the class is not found. This is a Checked Exception and caller can perform any corrective action.

NoClassFoundError : This error occurs a class is directly referred [new or ClassName] in the code is not found. This comes under LinkageError, it occur's in the first phase of class loading life cycle.

ClassCastException: This exception occurs in two cases

1. When we try to cast object of two different incompatible types, which can be solved easily
2. When more than one class loader loaded the same class and casting between them [casting of object of same type loaded by different classloader]

Assume there is an EAR package, consisting of a WAR, and its library files [say lib1.jar, lib2.jar]. The WAR package in turn has its own library say [the same lib2.jar]

A java class from WAR package calls a method from lib1.jar say DemoHelper.getDemo() [available in EAR package], and that method returns a object from lib2.jar [say Demo] and it is casted to Demo.  Something similar to below code snippet

Now the class cast exception will be thrown because the Demo class loaded by the WAR class loader is different from the Demo class loaded by EAR Class loader.

Just to reiterate the JavaEEContainer Delegation hierarchy, the WAR/EAR classloader will try to load the required class before it delegate to its parent.

In our case, when Demo is referred in EAR jar, the EAR classloader loads the Demo class from its library, and when Demo class is referred from WAR class [casting], the WAR classloader tries to loads the class from its library and it finds one but it is different from the only returned from the other class loader hence the error occurred. 

Incase the WAR classloader couldn't find  the Demo class [lib2.jar is not there in WAR], then the request would have delegated to its parent class loader, i.e. EAR class loader and which would have returned the already loaded Demo class hence this issue could have avoided.

To Summarize:

1. Class loaders are a powerful mechanism for dynamically loading software components on the Java platform
2. Every loaded class has an instance of the java.lang.Class object in the heap
3. Any class will be loaded only once by a class loader hierarchy
4. In general every class loader delegates the class load requests to its parent before trying to load the class on its own
5. Avoid using duplicated libraries, a good understanding of class loaders is required for packaging, deployment of J2EE components

There are much more on class loading, reloading, Hence requesting the readers to continue exploring. Below are some of the articles on Class Loaders.

http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-5.html

http://www.artima.com/insidejvm/ed2/lifetype.html

http://tutorials.jenkov.com/java-reflection/dynamic-class-loading-reloading.html#dynamicloading

http://geekexplains.blogspot.in/2008/07/loading-linking-initialization-of-types.html

http://zeroturnaround.com/labs/rebel-labs-tutorial-do-you-really-get-classloaders/

http://javapapers.com/core-java/java-class-loader/

http://java.sys-con.com/node/37659

http://developeriq.in/articles/2007/oct/03/java-class-loader-class-apart/

http://onjava.com/pub/a/onjava/2005/01/26/classloading.html

JVM View

Java Virtual Machine

A Java virtual machine is a program, which executes certain other programs, those containing Java bytecode instructions.

The JVM [Interpreter] translates bytecode to platform specific OS call, i.e. it act as the Middleman between bytecode and operating system. This makes Java a Platform Independent language.

JVM knows nothing of the Java programming language, only of a particular binary format, the class file format. Although the JVM is primarily aimed at running compile java programs, any language that can express in terms of valid class file can make use of JVM for execution

Each Platform [Windows, Linux] has a JVM written specifically for it, JVM implementation can come from many vendors [Oracle, IBM, etc..] but it should adhere to the JVM Specification

Once JVM Implementation is installed, we will be able to run/execute java application/class which will inturn give rise to a JVM instance.  

So now we understood the term "JVM" could denote either 

a)     The Abstract Specification

b)     A Concrete Implementation

c)     A Runtime Instance 

     

JVM Instance

When a java application starts, JVM instance is born and it dies when the program ends. i.e., if you are running three java programs, then that mean there are three JVM instances running.

Each JVM  instance is born with a Method Area and one Heap. Both these areas are shared across the threads within the JVM instance. These are not thread safe.

Heap: Memory where all objects, global variables resides

Method Area: This is the area where the bytecodes reside. It has all the instructions which needs to be executed.

When a new thread comes into existence, it gets its own PC register (program counter) and Java stack. No thread can access the PC register or Java stack of another thread. These are thread-safe!

Java Stack: This stores the state of method invocations, where the state includes the in progress methods, local variables [primitive / reference variable], method arguments, and return value

PC Register: This points to some byte in the method area. While a thread is executing a method, the value of the pc register indicates the next instruction to execute.

How does Java Stack track the state of method invocation?

Java Stack is composed of stack frames. A stack frame contains the state of one Java  method invocation. A new frame is created on top of stack each time a method is invoked, and it is removed when the method completes. Only one frame [current executing method] is active at any point in a given thread of control.

Runtime data area

Where

      a) Method Area & Heap are shared across threads

b) Stack and PC Register are created for each thread, no thread can access other’s stack or pc register

c) Stack refer to the object available in heap

d) PC register points to some byte in the method area

e) Highlighted Frame2 is the active frame

JVM Execution Demonstration

Assume Class ExecDemo has two methods [1 & 2], where method1 calls method2, method2 creates a new instance for A which in turn hold reference of B 

Below figure show how the Java Stack, Frames and Heap memory are constructed for the above program, please note that since it is single thread we have only one Stack. 

The top highlighted method2 frame is active frame [current menthod invoked] 

Method1 Frame

a)     Creates local variable in stack

b)     Invoke method2     

Method2 Frame,

a)     Instance of A along with dependent object B will be created in heap

b)     Reference of A will be created in stack

When method2 execution is completed, the method2 frame will be removed from Stack and control will go back to method1 also since Object A [Heap] is no where referenced it will be available for garbage collection

Garbage Collection:

A garbage collector's job is to automatically reclaim the memory used by objects that are no longer referenced by the running application.

Garbage Collection happens only in Heap memory, Java Stack does not require Garbage Collection as the stack frames [holds local variables] will be removed when method is completed

No garbage collection technique is dictated by the Java virtual machine specification. Designers/JVM Implementors can use whatever techniques seem most appropriate for them.

My initial thought was to write an article on Garbage collection but later felt a basic understanding on Heap memory is essential  and hence I ended up with this article. Please watch out for my next article hopefully it will be on garbage collection.

Points to Remember: 

• Instances are created in Heap 
• Stack stores state of the method invocation includes local variables [primitive and reference], arguments, and return type 
• The stack is threadsafe (each thread will have its own stack) but the heap is not threadsafe 
• If the computation in a thread requires a larger Java virtual machine stack than is permitted, the Java virtual machine throws a StackOverflowError. 
• If a computation requires more heap than can be made available by the automatic storage management system, the Java virtual machine throws an OutOfMemoryError.
• Although the same runtime data areas exist in some form in every Java virtual machine implementation, the JVM specification is quite abstract. Many decisions about the structural details of the runtime data areas are left to the designers of individual implementations

References:

The Java™ Virtual Machine Specification

http://docs.oracle.com/javase/specs/jvms/se7/html/index.html 

The Java Virtual Machine by Bill Venners

http://www.artima.com/insidejvm/ed2/jvm.html

http://viralpatel.net/blogs/java-virtual-machine-an-inside-story/

An Exception View

Wikipedia Definition on Exception:

An action that is not part of ordinary operations or standards

From the definition it is clear that exception doesn’t mean error. In our day today life there are certain things which happens occasionally, and we handle it in different way [not in regular way].

For example, a festival is an occasion, which is not like our regular day to office, and we have a different plan for that special day and that is called an Exception.

Somehow we have the perception that Exception means something wrong/unexpected happened but which is not the case always

Exception Handling In Real Life – Human eye:

When an object appears near to your eyes with considerable force, the eyelid close automatically i.e., the Eye Lid is there to protect your eye, as it expects these situations to occur and force an action.

Exception Handling in Java:

The answer is pretty simple, put a try/catch block around the code which you feel is expected to fail / might need a substitute action.

In the above human eye example, the eyelid act as the try/catch block around the eye, which reacts on the unexpected

Example 1a:  A service to withdraw amount from the given account

The above snippet is self-explanatory and you can notice that method has throws InSufficientBalanceException in its signature. 

InSufficientBalanceException is expected to be thrown by the service whenever the withdraw amount is greater than balance amt. So now the caller is forced to handle this exception.

Example 1b:  Exception handling

The above client code calls the service [line 5] in a try / catch block [Line 4 – 10], if you have sufficient fund then transaction happens and the doll is yours [line 6], else the exception will be thrown by the service and it is logged in the catch block [line 9] and corrective action should be taken accordingly here the user can purchase by cash [Line 9]

This is what in java world we call it as Checked Exception / Compile time Exception, there is also another important type of exception called Unchecked Exception / Runtime Exception

 

 

 

 

 

 

 

 

So what is Unchecked Exception / Runtime Exception?

• Is that it cannot/should not have corrective action? – Big NO
• Is that because it occurs at runtime? - All Exceptions are expected to occur at runtime even the checked exception/compile exception so this cannot be the reason.

From the above answers it means, an Unchecked Exception can occur at runtime and it can be rectified/substitute action can also be applied but then this can be achieved with checked exception itself so why does unchecked Exception still exist?

Real world Example – Mobile and its insurance

When we purchase a mobile, it doesn’t come up insurance its up to the owner to decide whether it has to be insured or not.

If something happens for the uninsured mobile, it becomes purely the owner’s risk.

Some users feel the insurance is not required for them because of its low price, or the insurance doesn’t cover all the disabilities.

Similarly in java programming, the caller will be given the rights to handle/not to handle the exception i.e., compiler will not force them to write the try/catch block

If you can revisit Example 1a, you can see that retrieveAvailableBalance() method is called to get the available balance, assume if the account does not exist/not active then the service has to let the caller know about it.

Example 2a: Runtime Exception

You can see that the method do not have throws in its signature, that means the compiler will not force the caller of this method to put try/catch block i.e, eventhough we introduced new exception [InvalidAccountException], Example 1a & 1b still will compile without any change and it will works fine for a valid account

Now, the question what happens to the client/caller [Example 1b] for invalid account?

Yes, it will fail abnormally as the caller didn’t had the try/catch/rethrow when he calls the service, so the client code [Example 1b] should be something like below:


Example 2b: Exception Handling

Now we have the exception handling for invalid account is also in place.

There are many other things in Exception, Hierarchies, Best Practices, etc that are not covered in this article as my primary focus is on Exception and not on the Exception API

I would recommend readers not to stop it here, there are many better-written articles available.

  

  

Below are some of the articles on Exception:

Anders Hejlsberg on checked vs. unchecked exceptions

http://www.artima.com/intv/handcuffs.html

James Gosling on checked exceptions

http://www.artima.com/intv/solid.html

Bill Venners on Exceptions

http://www.artima.com/interfacedesign/exceptions.html

Jacob Jenkov - Exception Handling

http://tutorials.jenkov.com/java-exception-handling/catching-multiple-exceptions.html

An Exceptional Philosophy:

http://www.dlugosz.com/Magazine/WTJ/May96/