Terracotta (The company I work for) recently came out with a beta release for their BigMemory product. Our claim is that if you use Ehcache BigMemory then those GC problems go away. If you don't know what Ehcache or Terracotta is you can find out here.
This is far from just a claim, we didn't simply didn't write BigMemory, ran a few tests, and release the Beta. We took BigMemory and freed our own Terracotta Server for GC constraints that most Java server product have.
So what exactly did we do? In Terracotta we keep a representation of distributed objects in your server. For our clustered Ehcache product we keep a representation of cache segments and cache entries on the server. As the cache grows, we need to keep track of the keys associated with the cache segment, as well as have more cache entries representations on the server. The Terracotta Server has the ability to flush cache entries to disk when we detect memory pressure on our server. and fault in objects when cache entries are needed.
To accommodate the keys on the server, you had two options. One is the increase heap so that more keys could fit on the server. The second is to add additional Terracotta Servers to the cluster so that the segments (and their keys) are distributed to among many servers.
Adding more heap threw us into the classic GC problem. Once you start getting into heap sizes of 5-6GB, you start seeing 5-8 second GC pauses. Adding stripes solves the problem and worked for our customers as well.
But we found a pattern emerging. Customers were purchasing huge boxes (I'm going regret saying huge a few months from now ), 32 GB ram and 16 cores of processing power. They were like "We want to run your servers on this thing." And they did what everyone does for Java Servers, they basically run multiple JVMs (if possible) on the same box and deal with the added complexity and unpredicatability and probably says a prayer or two. I'm sure you all understand what I mean by complexity, but what do I mean by unpredictability. Lets say you run your servers with small heaps and your getting a 2 second GC, but will so many JVMs, how do you know your GCs will not be staggered. Meaning if you had 16 processes running and they GCed one after the other. That is a 32 second pause. Get the picture?
We figured, there must be a better way. So we built BigMemory and put it in our server. Now you can take our pure Java Terracotta Server, have it use that 32 GB ram and still get less than 1 second pauses. Practically No GC.
For anyone who spent countless hours like I have tuning GC is going to love this. We ran and messed around with every knob the Sun (now Oracle) gave use to tune GC. When we put in BigMemory, we just deleted all those settings. Our heap is small enough so that default Java settings are good enough!
End result? Our Terracotta Server with BigMemory enabled can achieve higher density will less servers and best of all you can get predictable latency out of them.
Check out BigMemory and let me know if it works just as well for you.
5 comments:
Hello, the java based terra cotta server seems a mystery to me ... you claim to store the data in memory and still won't get caught by GC pauses... and since its pure java as you said ... i can't believe there are no GC pauses ... or you guys might have made memory management solution like an OS in java ... what so ever ... can you tell more about the off heap store??
Hi Asad, Thanks for your interest.
Yes, It is pure Java, and yes we don't store data on the heap with BigMemory. In Java you can specify off heap memory using this flag for example:
-XX:MaxDirectMemorySize=2G.
This is nothing new and been around for awhile. In fact the nio library uses it. Our secret sauce is how we used this Offheap. Most people who have tried it found it to be way to slow and have common C-like problems like memory segmentation.
you should check out this url for more infromation:
http://ehcache.org/documentation/offheap_store.html
Hope that helps.
Wow.. that is awesome... 32Gigs of RAM with 1 sec GC pause. Unbelievable. is this 32G ram allocated for L2?
Another Question
-XX:MaxDirectMemorySize=2G. does that mean part of the heap uses direct memory?
really interesting
Can I have the big memory plugged into my JVM, instead of through EHCache. I want my JVM to use BigMemory, where can I have im-memory Db like HSQL?
If you’re looking for Managed IT Services in NYC, Etech7 is the right place for you. I’m a dummy in IT sphere, so all the time I need a good company to solve my IT problems. Finally I’ve found Etech7 and when I need IT Consulting, I’ll definitely call them.
Post a Comment