You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
534 lines
21 KiB
534 lines
21 KiB
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> |
|
<HTML> |
|
|
|
<HEAD> |
|
<link rel="stylesheet" href="designstyle.css"> |
|
<title>Gperftools Heap Leak Checker</title> |
|
</HEAD> |
|
|
|
<BODY> |
|
|
|
<p align=right> |
|
<i>Last modified |
|
<script type=text/javascript> |
|
var lm = new Date(document.lastModified); |
|
document.write(lm.toDateString()); |
|
</script></i> |
|
</p> |
|
|
|
<p>This is the heap checker we use at Google to detect memory leaks in |
|
C++ programs. There are three parts to using it: linking the library |
|
into an application, running the code, and analyzing the output.</p> |
|
|
|
|
|
<H1>Linking in the Library</H1> |
|
|
|
<p>The heap-checker is part of tcmalloc, so to install the heap |
|
checker into your executable, add <code>-ltcmalloc</code> to the |
|
link-time step for your executable. Also, while we don't necessarily |
|
recommend this form of usage, it's possible to add in the profiler at |
|
run-time using <code>LD_PRELOAD</code>:</p> |
|
<pre>% env LD_PRELOAD="/usr/lib/libtcmalloc.so" <binary></pre> |
|
|
|
<p>This does <i>not</i> turn on heap checking; it just inserts the |
|
code. For that reason, it's practical to just always link |
|
<code>-ltcmalloc</code> into a binary while developing; that's what we |
|
do at Google. (However, since any user can turn on the profiler by |
|
setting an environment variable, it's not necessarily recommended to |
|
install heapchecker-linked binaries into a production, running |
|
system.) Note that if you wish to use the heap checker, you must |
|
also use the tcmalloc memory-allocation library. There is no way |
|
currently to use the heap checker separate from tcmalloc.</p> |
|
|
|
|
|
<h1>Running the Code</h1> |
|
|
|
<p>Note: For security reasons, heap profiling will not write to a file |
|
-- and is thus not usable -- for setuid programs.</p> |
|
|
|
<h2><a name="whole_program">Whole-program Heap Leak Checking</a></h2> |
|
|
|
<p>The recommended way to use the heap checker is in "whole program" |
|
mode. In this case, the heap-checker starts tracking memory |
|
allocations before the start of <code>main()</code>, and checks again |
|
at program-exit. If it finds any memory leaks -- that is, any memory |
|
not pointed to by objects that are still "live" at program-exit -- it |
|
aborts the program (via <code>exit(1)</code>) and prints a message |
|
describing how to track down the memory leak (using <A |
|
HREF="heapprofile.html#pprof">pprof</A>).</p> |
|
|
|
<p>The heap-checker records the stack trace for each allocation while |
|
it is active. This causes a significant increase in memory usage, in |
|
addition to slowing your program down.</p> |
|
|
|
<p>Here's how to run a program with whole-program heap checking:</p> |
|
|
|
<ol> |
|
<li> <p>Define the environment variable HEAPCHECK to the <A |
|
HREF="#types">type of heap-checking</A> to do. For instance, |
|
to heap-check |
|
<code>/usr/local/bin/my_binary_compiled_with_tcmalloc</code>:</p> |
|
<pre>% env HEAPCHECK=normal /usr/local/bin/my_binary_compiled_with_tcmalloc</pre> |
|
</ol> |
|
|
|
<p>No other action is required.</p> |
|
|
|
<p>Note that since the heap-checker uses the heap-profiling framework |
|
internally, it is not possible to run both the heap-checker and <A |
|
HREF="heapprofile.html">heap profiler</A> at the same time.</p> |
|
|
|
|
|
<h3><a name="types">Flavors of Heap Checking</a></h3> |
|
|
|
<p>These are the legal values when running a whole-program heap |
|
check:</p> |
|
<ol> |
|
<li> <code>minimal</code> |
|
<li> <code>normal</code> |
|
<li> <code>strict</code> |
|
<li> <code>draconian</code> |
|
</ol> |
|
|
|
<p>"Minimal" heap-checking starts as late as possible in a |
|
initialization, meaning you can leak some memory in your |
|
initialization routines (that run before <code>main()</code>, say), |
|
and not trigger a leak message. If you frequently (and purposefully) |
|
leak data in one-time global initializers, "minimal" mode is useful |
|
for you. Otherwise, you should avoid it for stricter modes.</p> |
|
|
|
<p>"Normal" heap-checking tracks <A HREF="#live">live objects</A> and |
|
reports a leak for any data that is not reachable via a live object |
|
when the program exits.</p> |
|
|
|
<p>"Strict" heap-checking is much like "normal" but has a few extra |
|
checks that memory isn't lost in global destructors. In particular, |
|
if you have a global variable that allocates memory during program |
|
execution, and then "forgets" about the memory in the global |
|
destructor (say, by setting the pointer to it to NULL) without freeing |
|
it, that will prompt a leak message in "strict" mode, though not in |
|
"normal" mode.</p> |
|
|
|
<p>"Draconian" heap-checking is appropriate for those who like to be |
|
very precise about their memory management, and want the heap-checker |
|
to help them enforce it. In "draconian" mode, the heap-checker does |
|
not do "live object" checking at all, so it reports a leak unless |
|
<i>all</i> allocated memory is freed before program exit. (However, |
|
you can use <A HREF="#disable">IgnoreObject()</A> to re-enable |
|
liveness-checking on an object-by-object basis.)</p> |
|
|
|
<p>"Normal" mode, as the name implies, is the one used most often at |
|
Google. It's appropriate for everyday heap-checking use.</p> |
|
|
|
<p>In addition, there are two other possible modes:</p> |
|
<ul> |
|
<li> <code>as-is</code> |
|
<li> <code>local</code> |
|
</ul> |
|
<p><code>as-is</code> is the most flexible mode; it allows you to |
|
specify the various <A HREF="#options">knobs</A> of the heap checker |
|
explicitly. <code>local</code> activates the <A |
|
HREF="#explicit">explicit heap-check instrumentation</A>, but does not |
|
turn on any whole-program leak checking.</p> |
|
|
|
|
|
<h3><A NAME="tweaking">Tweaking whole-program checking</A></h3> |
|
|
|
<p>In some cases you want to check the whole program for memory leaks, |
|
but waiting for after <code>main()</code> exits to do the first |
|
whole-program leak check is waiting too long: e.g. in a long-running |
|
server one might wish to simply periodically check for leaks while the |
|
server is running. In this case, you can call the static method |
|
<code>NoGlobalLeaks()</code>, to verify no global leaks have happened |
|
as of that point in the program.</p> |
|
|
|
<p>Alternately, doing the check after <code>main()</code> exits might |
|
be too late. Perhaps you have some objects that are known not to |
|
clean up properly at exit. You'd like to do the "at exit" check |
|
before those objects are destroyed (since while they're live, any |
|
memory they point to will not be considered a leak). In that case, |
|
you can call <code>NoGlobalLeaks()</code> manually, near the end of |
|
<code>main()</code>, and then call <code>CancelGlobalCheck()</code> to |
|
turn off the automatic post-<code>main()</code> check.</p> |
|
|
|
<p>Finally, there's a helper macro for "strict" and "draconian" modes, |
|
which require all global memory to be freed before program exit. This |
|
freeing can be time-consuming and is often unnecessary, since libc |
|
cleans up all memory at program-exit for you. If you want the |
|
benefits of "strict"/"draconian" modes without the cost of all that |
|
freeing, look at <code>REGISTER_HEAPCHECK_CLEANUP</code> (in |
|
<code>heap-checker.h</code>). This macro allows you to mark specific |
|
cleanup code as active only when the heap-checker is turned on.</p> |
|
|
|
|
|
<h2><a name="explicit">Explicit (Partial-program) Heap Leak Checking</h2> |
|
|
|
<p>Instead of whole-program checking, you can check certain parts of your |
|
code to verify they do not have memory leaks. This check verifies that |
|
between two parts of a program, no memory is allocated without being freed.</p> |
|
<p>To use this kind of checking code, bracket the code you want |
|
checked by creating a <code>HeapLeakChecker</code> object at the |
|
beginning of the code segment, and call |
|
<code>NoLeaks()</code> at the end. These functions, and all others |
|
referred to in this file, are declared in |
|
<code><gperftools/heap-checker.h></code>. |
|
</p> |
|
|
|
<p>Here's an example:</p> |
|
<pre> |
|
HeapLeakChecker heap_checker("test_foo"); |
|
{ |
|
code that exercises some foo functionality; |
|
this code should not leak memory; |
|
} |
|
if (!heap_checker.NoLeaks()) assert(NULL == "heap memory leak"); |
|
</pre> |
|
|
|
<p>Note that adding in the <code>HeapLeakChecker</code> object merely |
|
instruments the code for leak-checking. To actually turn on this |
|
leak-checking on a particular run of the executable, you must still |
|
run with the heap-checker turned on:</p> |
|
<pre>% env HEAPCHECK=local /usr/local/bin/my_binary_compiled_with_tcmalloc</pre> |
|
<p>If you want to do whole-program leak checking in addition to this |
|
manual leak checking, you can run in <code>normal</code> or some other |
|
mode instead: they'll run the "local" checks in addition to the |
|
whole-program check.</p> |
|
|
|
|
|
<h2><a name="disable">Disabling Heap-checking of Known Leaks</a></h2> |
|
|
|
<p>Sometimes your code has leaks that you know about and are willing |
|
to accept. You would like the heap checker to ignore them when |
|
checking your program. You can do this by bracketing the code in |
|
question with an appropriate heap-checking construct:</p> |
|
<pre> |
|
... |
|
{ |
|
HeapLeakChecker::Disabler disabler; |
|
<leaky code> |
|
} |
|
... |
|
</pre> |
|
Any objects allocated by <code>leaky code</code> (including inside any |
|
routines called by <code>leaky code</code>) and any objects reachable |
|
from such objects are not reported as leaks. |
|
|
|
<p>Alternately, you can use <code>IgnoreObject()</code>, which takes a |
|
pointer to an object to ignore. That memory, and everything reachable |
|
from it (by following pointers), is ignored for the purposes of leak |
|
checking. You can call <code>UnIgnoreObject()</code> to undo the |
|
effects of <code>IgnoreObject()</code>.</p> |
|
|
|
|
|
<h2><a name="options">Tuning the Heap Checker</h2> |
|
|
|
<p>The heap leak checker has many options, some that trade off running |
|
time and accuracy, and others that increase the sensitivity at the |
|
risk of returning false positives. For most uses, the range covered |
|
by the <A HREF="#types">heap-check flavors</A> is enough, but in |
|
specialized cases more control can be helpful.</p> |
|
|
|
<p> |
|
These options are specified via environment varaiables. |
|
</p> |
|
|
|
<p>This first set of options controls sensitivity and accuracy. These |
|
options are ignored unless you run the heap checker in <A |
|
HREF="#types">as-is</A> mode. |
|
|
|
<table frame=box rules=sides cellpadding=5 width=100%> |
|
|
|
<tr valign=top> |
|
<td><code>HEAP_CHECK_AFTER_DESTRUCTORS</code></td> |
|
<td>Default: false</td> |
|
<td> |
|
When true, do the final leak check after all other global |
|
destructors have run. When false, do it after all |
|
<code>REGISTER_HEAPCHECK_CLEANUP</code>, typically much earlier in |
|
the global-destructor process. |
|
</td> |
|
</tr> |
|
|
|
<tr valign=top> |
|
<td><code>HEAP_CHECK_IGNORE_THREAD_LIVE</code></td> |
|
<td>Default: true</td> |
|
<td> |
|
If true, ignore objects reachable from thread stacks and registers |
|
(that is, do not report them as leaks). |
|
</td> |
|
</tr> |
|
|
|
<tr valign=top> |
|
<td><code>HEAP_CHECK_IGNORE_GLOBAL_LIVE</code></td> |
|
<td>Default: true</td> |
|
<td> |
|
If true, ignore objects reachable from global variables and data |
|
(that is, do not report them as leaks). |
|
</td> |
|
</tr> |
|
|
|
</table> |
|
|
|
<p>These options modify the behavior of whole-program leak |
|
checking.</p> |
|
|
|
<table frame=box rules=sides cellpadding=5 width=100%> |
|
|
|
<tr valign=top> |
|
<td><code>HEAP_CHECK_MAX_LEAKS</code></td> |
|
<td>Default: 20</td> |
|
<td> |
|
The maximum number of leaks to be printed to stderr (all leaks are still |
|
emitted to file output for pprof to visualize). If negative or zero, |
|
print all the leaks found. |
|
</td> |
|
</tr> |
|
|
|
|
|
</table> |
|
|
|
<p>These options apply to all types of leak checking.</p> |
|
|
|
<table frame=box rules=sides cellpadding=5 width=100%> |
|
|
|
<tr valign=top> |
|
<td><code>HEAP_CHECK_IDENTIFY_LEAKS</code></td> |
|
<td>Default: false</td> |
|
<td> |
|
If true, generate the addresses of the leaked objects in the |
|
generated memory leak profile files. |
|
</td> |
|
</tr> |
|
|
|
<tr valign=top> |
|
<td><code>HEAP_CHECK_TEST_POINTER_ALIGNMENT</code></td> |
|
<td>Default: false</td> |
|
<td> |
|
If true, check all leaks to see if they might be due to the use |
|
of unaligned pointers. |
|
</td> |
|
</tr> |
|
|
|
<tr valign=top> |
|
<td><code>HEAP_CHECK_POINTER_SOURCE_ALIGNMENT</code></td> |
|
<td>Default: sizeof(void*)</td> |
|
<td> |
|
Alignment at which all pointers in memory are supposed to be located. |
|
Use 1 if any alignment is ok. |
|
</td> |
|
</tr> |
|
|
|
<tr valign=top> |
|
<td><code>PPROF_PATH</code></td> |
|
<td>Default: pprof</td> |
|
<td> |
|
The location of the <code>pprof</code> executable. |
|
</td> |
|
</tr> |
|
|
|
<tr valign=top> |
|
<td><code>HEAP_CHECK_DUMP_DIRECTORY</code></td> |
|
<td>Default: /tmp</td> |
|
<td> |
|
Where the heap-profile files are kept while the program is running. |
|
</td> |
|
</tr> |
|
|
|
</table> |
|
|
|
|
|
<h2>Tips for Handling Detected Leaks</h2> |
|
|
|
<p>What do you do when the heap leak checker detects a memory leak? |
|
First, you should run the reported <code>pprof</code> command; |
|
hopefully, that is enough to track down the location where the leak |
|
occurs.</p> |
|
|
|
<p>If the leak is a real leak, you should fix it!</p> |
|
|
|
<p>If you are sure that the reported leaks are not dangerous and there |
|
is no good way to fix them, then you can use |
|
<code>HeapLeakChecker::Disabler</code> and/or |
|
<code>HeapLeakChecker::IgnoreObject()</code> to disable heap-checking |
|
for certain parts of the codebase.</p> |
|
|
|
<p>In "strict" or "draconian" mode, leaks may be due to incomplete |
|
cleanup in the destructors of global variables. If you don't wish to |
|
augment the cleanup routines, but still want to run in "strict" or |
|
"draconian" mode, consider using <A |
|
HREF="#tweaking"><code>REGISTER_HEAPCHECK_CLEANUP</code></A>.</p> |
|
|
|
<h2>Hints for Debugging Detected Leaks</h2> |
|
|
|
<p>Sometimes it can be useful to not only know the exact code that |
|
allocates the leaked objects, but also the addresses of the leaked objects. |
|
Combining this e.g. with additional logging in the program |
|
one can then track which subset of the allocations |
|
made at a certain spot in the code are leaked. |
|
<br/> |
|
To get the addresses of all leaked objects |
|
define the environment variable <code>HEAP_CHECK_IDENTIFY_LEAKS</code> |
|
to be <code>1</code>. |
|
The object addresses will be reported in the form of addresses |
|
of fake immediate callers of the memory allocation routines. |
|
Note that the performance of doing leak-checking in this mode |
|
can be noticeably worse than the default mode. |
|
</p> |
|
|
|
<p>One relatively common class of leaks that don't look real |
|
is the case of multiple initialization. |
|
In such cases the reported leaks are typically things that are |
|
linked from some global objects, |
|
which are initialized and say never modified again. |
|
The non-obvious cause of the leak is frequently the fact that |
|
the initialization code for these objects executes more than once. |
|
<br/> |
|
E.g. if the code of some <code>.cc</code> file is made to be included twice |
|
into the binary, then the constructors for global objects defined in that file |
|
will execute twice thus leaking the things allocated on the first run. |
|
<br/> |
|
Similar problems can occur if object initialization is done more explicitly |
|
e.g. on demand by a slightly buggy code |
|
that does not always ensure only-once initialization. |
|
</p> |
|
|
|
<p> |
|
A more rare but even more puzzling problem can be use of not properly |
|
aligned pointers (maybe inside of not properly aligned objects). |
|
Normally such pointers are not followed by the leak checker, |
|
hence the objects reachable only via such pointers are reported as leaks. |
|
If you suspect this case |
|
define the environment variable <code>HEAP_CHECK_TEST_POINTER_ALIGNMENT</code> |
|
to be <code>1</code> |
|
and then look closely at the generated leak report messages. |
|
</p> |
|
|
|
<h1>How It Works</h1> |
|
|
|
<p>When a <code>HeapLeakChecker</code> object is constructed, it dumps |
|
a memory-usage profile named |
|
<code><prefix>.<name>-beg.heap</code> to a temporary |
|
directory. When <code>NoLeaks()</code> |
|
is called (for whole-program checking, this happens automatically at |
|
program-exit), it dumps another profile, named |
|
<code><prefix>.<name>-end.heap</code>. |
|
(<code><prefix></code> is typically determined automatically, |
|
and <code><name></code> is typically <code>argv[0]</code>.) It |
|
then compares the two profiles. If the second profile shows |
|
more memory use than the first, the |
|
<code>NoLeaks()</code> function will |
|
return false. For "whole program" profiling, this will cause the |
|
executable to abort (via <code>exit(1)</code>). In all cases, it will |
|
print a message on how to process the dumped profiles to locate |
|
leaks.</p> |
|
|
|
<h3><A name=live>Detecting Live Objects</A></h3> |
|
|
|
<p>At any point during a program's execution, all memory that is |
|
accessible at that time is considered "live." This includes global |
|
variables, and also any memory that is reachable by following pointers |
|
from a global variable. It also includes all memory reachable from |
|
the current stack frame and from current CPU registers (this captures |
|
local variables). Finally, it includes the thread equivalents of |
|
these: thread-local storage and thread heaps, memory reachable from |
|
thread-local storage and thread heaps, and memory reachable from |
|
thread CPU registers.</p> |
|
|
|
<p>In all modes except "draconian," live memory is not |
|
considered to be a leak. We detect this by doing a liveness flood, |
|
traversing pointers to heap objects starting from some initial memory |
|
regions we know to potentially contain live pointer data. Note that |
|
this flood might potentially not find some (global) live data region |
|
to start the flood from. If you find such, please file a bug.</p> |
|
|
|
<p>The liveness flood attempts to treat any properly aligned byte |
|
sequences as pointers to heap objects and thinks that it found a good |
|
pointer whenever the current heap memory map contains an object with |
|
the address whose byte representation we found. Some pointers into |
|
not-at-start of object will also work here.</p> |
|
|
|
<p>As a result of this simple approach, it's possible (though |
|
unlikely) for the flood to be inexact and occasionally result in |
|
leaked objects being erroneously determined to be live. For instance, |
|
random bit patterns can happen to look like pointers to leaked heap |
|
objects. More likely, stale pointer data not corresponding to any |
|
live program variables can be still present in memory regions, |
|
especially in thread stacks. For instance, depending on how the local |
|
<code>malloc</code> is implemented, it may reuse a heap object |
|
address:</p> |
|
<pre> |
|
char* p = new char[1]; // new might return 0x80000000, say. |
|
delete p; |
|
new char[1]; // new might return 0x80000000 again |
|
// This last new is a leak, but doesn't seem it: p looks like it points to it |
|
</pre> |
|
|
|
<p>In other words, imprecisions in the liveness flood mean that for |
|
any heap leak check we might miss some memory leaks. This means that |
|
for local leak checks, we might report a memory leak in the local |
|
area, even though the leak actually happened before the |
|
<code>HeapLeakChecker</code> object was constructed. Note that for |
|
whole-program checks, a leak report <i>does</i> always correspond to a |
|
real leak (since there's no "before" to have created a false-live |
|
object).</p> |
|
|
|
<p>While this liveness flood approach is not very portable and not |
|
100% accurate, it works in most cases and saves us from writing a lot |
|
of explicit clean up code and other hassles when dealing with thread |
|
data.</p> |
|
|
|
|
|
<h3>Visualizing Leak with <code>pprof</code></h3> |
|
|
|
<p> |
|
The heap checker automatically prints basic leak info with stack traces of |
|
leaked objects' allocation sites, as well as a pprof command line that can be |
|
used to visualize the call-graph involved in these allocations. |
|
The latter can be much more useful for a human |
|
to see where/why the leaks happened, especially if the leaks are numerous. |
|
</p> |
|
|
|
<h3>Leak-checking and Threads</h3> |
|
|
|
<p>At the time of HeapLeakChecker's construction and during |
|
<code>NoLeaks()</code> calls, we grab a lock |
|
and then pause all other threads so other threads do not interfere |
|
with recording or analyzing the state of the heap.</p> |
|
|
|
<p>In general, leak checking works correctly in the presence of |
|
threads. However, thread stack data liveness determination (via |
|
<code>base/thread_lister.h</code>) does not work when the program is |
|
running under GDB, because the ptrace functionality needed for finding |
|
threads is already hooked to by GDB. Conversely, leak checker's |
|
ptrace attempts might also interfere with GDB. As a result, GDB can |
|
result in potentially false leak reports. For this reason, the |
|
heap-checker turns itself off when running under GDB.</p> |
|
|
|
<p>Also, <code>thread_lister</code> only works for Linux pthreads; |
|
leak checking is unlikely to handle other thread implementations |
|
correctly.</p> |
|
|
|
<p>As mentioned in the discussion of liveness flooding, thread-stack |
|
liveness determination might mis-classify as reachable objects that |
|
very recently became unreachable (leaked). This can happen when the |
|
pointers to now-logically-unreachable objects are present in the |
|
active thread stack frame. In other words, trivial code like the |
|
following might not produce the expected leak checking outcome |
|
depending on how the compiled code works with the stack:</p> |
|
<pre> |
|
int* foo = new int [20]; |
|
HeapLeakChecker check("a_check"); |
|
foo = NULL; |
|
// May fail to trigger. |
|
if (!heap_checker.NoLeaks()) assert(NULL == "heap memory leak"); |
|
</pre> |
|
|
|
|
|
<hr> |
|
<address>Maxim Lifantsev<br> |
|
<!-- Created: Tue Dec 19 10:43:14 PST 2000 --> |
|
<!-- hhmts start --> |
|
Last modified: Fri Jul 13 13:14:33 PDT 2007 |
|
<!-- hhmts end --> |
|
</address> |
|
</body> |
|
</html>
|
|
|