Skip to content
  • Binbin's avatar
    Allocate Lua VM code with jemalloc instead of libc, and count it used memory (#13133) · 804110a4
    Binbin 创作于
    
    
    ## Background
    1. Currently Lua memory control does not pass through Redis's zmalloc.c.
    Redis maxmemory cannot limit memory problems caused by users abusing lua
    since these lua VM memory is not part of used_memory.
    
    2. Since jemalloc is much better (fragmentation and speed), and also we
    know it and trust it. we are
    going to use jemalloc instead of libc to allocate the Lua VM code and
    count it used memory.
    
    ## Process:
    In this PR, we will use jemalloc in lua. 
    1. Create an arena for all lua vm (script and function), which is
    shared, in order to avoid blocking defragger.
    2. Create a bound tcache for the lua VM, since the lua VM and the main
    thread are by default in the same tcache, and if there is no isolated
    tcache, lua may request memory from the tcache which has just been freed
    by main thread, and vice versa
    On the other hand, since lua vm might be release in bio thread, but
    tcache is not thread-safe, we need to recreate
        the tcache every time we recreate the lua vm.
    3. Remove lua memory statistics from memory fragmentation statistics to
    avoid the effects of lua memory fragmentation
    
    ## Other
    Add the following new fields to `INFO DEBUG` (we may promote them to
    INFO MEMORY some day)
    1. allocator_allocated_lua: total number of bytes allocated of lua arena
    2. allocator_active_lua: total number of bytes in active pages allocated
    in lua arena
    3. allocator_resident_lua: maximum number of bytes in physically
    resident data pages mapped in lua arena
    4. allocator_frag_bytes_lua: fragment bytes in lua arena
    
    This is oranagra's idea, and i got some help from sundb.
    
    This solves the third point in #13102.
    
    ---------
    
    Co-authored-by: default avatardebing.sun <debing.sun@redis.com>
    Co-authored-by: default avatarOran Agra <oran@redislabs.com>
    804110a4