By Don Marti
If you’re writing a program that keeps a cache in memory, you’re probably expecting users to have to set the cache size, which means a configuration setting or command-line argument. And every configuration setting or command-line argument is something that you have to document, or explain to users when they get it wrong.
Thankfully, there’s an easier way.
With OSv, you can ask the OS to let your program know when memory is tight, so that you can manage the cache size on the fly. Less time spent tweaking settings, more items kept in cache, what’s not to like? Just set up a shrinker callback and register it. (There is a mechanism for memory pressure notifications on Linux, but it’s somewhat complex. With OSv it’s just one function to write.)
Defining a shrinker function
OSv notifies your program of a low memory situation by calling a shrinker callback.
A shrinker function takes two arguments:
target amount of memory to free (size_t)
A boolean “hard” argument. This is false if the function is being called for preemptive freeing of memory, and true if the system is under severe pressure.
In most invocations, the argument
hard will be set by OSv to false. This indicates that the system is acting preemptively, and memory pressure is starting to build up. The application is free to defer the freeing of objects to a later stage. Having your shrinker callback called with
hard set to true, however, means that OSv is under severe memory pressure, and may be unable to serve allocations. If not enough memory is freed, the system may be forced to abort.
Now that the shrinker function is defined, you need to register it.
To register a shrinker function involves calling
For example, an application in C can just do:
1 2 3 4 5 6 7 8 9
An extremely basic example of a program that uses the OSv shrinker API is memory-hog. To try it, install Capstan, clone the Git repository, and build an OSv image to run it. Currently memory-hog requires at least 2GB of memory.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38
Shrinker callbacks conceptually work like asynchronous signal handlers, They handle specific events and may be called at any time, from your program’s point of view.
Each cache in your program that the shrinker can release data from, must have its own lock to prevent concurrency issues.
The memory-hog sample program uses a mutex to handle concurrency. To handle soft memory pressure, where your shrinker is called with
hard set to false, you can keep a global
free_memory_please global variable, and set it in the shrinker callback. Then check it in the main loop, and free memory if necessary. However, hard memory pressure must be handled immediately, or the system may hang.
Handling soft pressure
If possible, you should have your shrinker try to release some memory when called with
hard set to false. This will reduce the number of times you have to handle shrinking, and improve performance by making more free memory available to OSv for system tasks. The memory-hog example currently ignores soft pressure.
For more info on memory management and other OSv subjects, please join the osv-dev mailing list. You can get updates on by subscribing to the OSv blog RSS feed or folllowing @CloudiusSystems on Twitter.