java - Memory spike in Grails + mongodb -

we facing strange issue in our grails application. total memory consumption shoots in short period. runs long time, @ point, memory consumption rises until there no more memory consume(xmx). may rise 4 gb consumption 20 gb within 5 minutes. once complete memory has been consumed, tomcat becomes unresponsive. not heel if left alone. @ point, expect outofmemory exception. never happens if leave application untouched. can see garbage collector keeps on running(we using new relic), still consumed memory not go down.

when noticed garbage collection happening in 1 big blow, changed garbage collector g1gc concurrentmarksweep. has not helped well. analysing non-responsive jvm, took thread dump , found there no deadlocks , lot of our threads in “blocked” state:

thread 6632: (state = blocked)  - sun.misc.unsafe.park(boolean, long) @bci=0 (compiled frame; information may imprecise)  - java.util.concurrent.locks.locksupport.park(java.lang.object) @bci=14, line=186 (compiled frame)  - java.util.concurrent.locks.abstractqueuedsynchronizer$conditionobject.await() @bci=42, line=2043 (compiled frame)  - java.util.concurrent.linkedblockingqueue.take() @bci=29, line=442 (compiled frame)  - org.apache.tomcat.util.threads.taskqueue.take() @bci=36, line=104 (interpreted frame)  - org.apache.tomcat.util.threads.taskqueue.take() @bci=1, line=32 (interpreted frame)  - java.util.concurrent.threadpoolexecutor.gettask() @bci=156, line=1068 (interpreted frame)  - java.util.concurrent.threadpoolexecutor.runworker(java.util.concurrent.threadpoolexecutor$worker) @bci=26, line=1130 (compiled frame)  - java.util.concurrent.threadpoolexecutor$worker.run() @bci=5, line=615 (interpreted frame)  - java.lang.thread.run() @bci=11, line=724 (interpreted frame) 

only few of these threads in “in_native” state.

we took memory dump using jmap, top consuming classes this:

we using grails 2.3.8 mongodb plugin 3.0.1 gorm. using memcached session sharing on tomcat7 , redis caching(spring-cache native redis).

our server details are:

server version: apache tomcat/7.0.35 server built:   may 24 2013 09:52:20 server number:  7.0.35.0 os name:        linux os version:     3.8.0-19-generic architecture:   amd64 jvm version:    1.7.0_25-b15 jvm vendor:     oracle corporation 

we out of ideas on how fix issue. looking help/pointers in resolving issue.