flex: The Default Memory Management
21.1 The Default Memory Management
==================================
Flex allocates dynamic memory during initialization, and once in a while
from within a call to yylex(). Initialization takes place during the
first call to yylex(). Thereafter, flex may reallocate more memory if
it needs to enlarge a buffer. As of version 2.5.9 Flex will clean up
all memory when you call 'yylex_destroy' ⇒faq-memory-leak.
Flex allocates dynamic memory for four purposes, listed below (1)
16kB for the input buffer.
Flex allocates memory for the character buffer used to perform
pattern matching. Flex must read ahead from the input stream and
store it in a large character buffer. This buffer is typically the
largest chunk of dynamic memory flex consumes. This buffer will
grow if necessary, doubling the size each time. Flex frees this
memory when you call yylex_destroy(). The default size of this
buffer (16384 bytes) is almost always too large. The ideal size
for this buffer is the length of the longest token expected, in
bytes, plus a little more. Flex will allocate a few extra bytes
for housekeeping. Currently, to override the size of the input
buffer you must '#define YY_BUF_SIZE' to whatever number of bytes
you want. We don't plan to change this in the near future, but we
reserve the right to do so if we ever add a more robust memory
management API.
64kb for the REJECT state. This will only be allocated if you use REJECT.
The size is large enough to hold the same number of states as
characters in the input buffer. If you override the size of the
input buffer (via 'YY_BUF_SIZE'), then you automatically override
the size of this buffer as well.
100 bytes for the start condition stack.
Flex allocates memory for the start condition stack. This is the
stack used for pushing start states, i.e., with yy_push_state().
It will grow if necessary. Since the states are simply integers,
this stack doesn't consume much memory. This stack is not present
if '%option stack' is not specified. You will rarely need to tune
this buffer. The ideal size for this stack is the maximum depth
expected. The memory for this stack is automatically destroyed
when you call yylex_destroy(). ⇒option-stack.
40 bytes for each YY_BUFFER_STATE.
Flex allocates memory for each YY_BUFFER_STATE. The buffer state
itself is about 40 bytes, plus an additional large character buffer
(described above.) The initial buffer state is created during
initialization, and with each call to yy_create_buffer(). You
can't tune the size of this, but you can tune the character buffer
as described above. Any buffer state that you explicitly create by
calling yy_create_buffer() is _NOT_ destroyed automatically. You
must call yy_delete_buffer() to free the memory. The exception to
this rule is that flex will delete the current buffer automatically
when you call yylex_destroy(). If you delete the current buffer,
be sure to set it to NULL. That way, flex will not try to delete
the buffer a second time (possibly crashing your program!) At the
time of this writing, flex does not provide a growable stack for
the buffer states. You have to manage that yourself. ⇒
Multiple Input Buffers.
84 bytes for the reentrant scanner guts
Flex allocates about 84 bytes for the reentrant scanner structure
when you call yylex_init(). It is destroyed when the user calls
yylex_destroy().
---------- Footnotes ----------
(1) The quantities given here are approximate, and may vary due to
host architecture, compiler configuration, or due to future enhancements
to flex.
Info Catalog
flex: Memory Management
flex: Memory Management
flex: Overriding The Default Memory Management