#Sysinternals memory monitor windows#

#Sysinternals memory monitor windows#

Using the same configuration and log files from one of our customers we have started monitoring a test environment where we have set up Logstash on a Windows 2012 instance and started observing the behavior as new files to be processed were automatically generated.Īs reported by the users impacted by this bug, the problem with the high memory consumption of the Logstash process would develop in time, only after several hours of usage this is usually a common pattern when dealing with memory leaks, however as seen in JConsole picture above, no signs of leak were observable within the Java heap.Īt this point, we’ve started looking around for alternative more comprehensive memory monitoring/troubleshooting tools as it became evident after hours of tests that what we were looking for was not something that JConsole could show us. When there are no steps to follow, in order to recreate a problem, the only approach that usually gives results is just to recreate an environment as close as possible, in terms of OS and application configuration and versions, as the one where the problem has been showing its symptoms: every little detail can make the difference. So where to start from to understand where is this memory being used? Reproducing the problem

Interestingly though, the process total memory footprint on Windows would keep growing and growing over time, well beyond the configured JVM limits (1GB). In this case, an analysis of the JVM heap usage over time using JConsole showed no significant anomalies within heap memory consumption, showing the classic healthy “sawtooth”, where garbage collection periodically frees memory:

This all started with several reports of abnormal behaviour in terms of memory consumption have been reported by our users to our support team but also through GitHub issues, see 37. In this blog post, we will go through the story of our investigations to identify a bug which has been impacting a large amount of users and their logstash deployments and tell how we have joined forces between support and development teams in Elastic in order to solve it. Nasty bugs come out now and then and sometime identifying their root cause can be a painful and frustrating exercise, going through debug logs, heap/thread dumps, OS metrics and failed reproduction attempts. At Elastic we stand next to our users to help them designing their deployments, architecting their solutions but also troubleshooting problems that can be encountered in day to day operations.