Symbols are stored globally and never freed.

[rkaminsk]

Clingo internalizes symbols in global hash tables to only allocate memory for the same symbol once. Currently, symbols are never removed form this hash table. When an application runs for a long time and generates a lot of different symbols, this can lead to memory problems.

Unused symbols should be freed. This could be done via some form of garbage collection or reference counting.

[chenkins]

Workaround for if you have the problem using multiprocessing.pool.Pool in Python: https://stackoverflow.com/questions/38294608/python-multiprocessing-pool-new-process-for-each-variable. Documentation: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool

[rkaminsk]

Workaround for if you have the problem using multiprocessing.pool.Pool in Python: https://stackoverflow.com/questions/38294608/python-multiprocessing-pool-new-process-for-each-variable. Documentation: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool

Just be careful when exchanging data between the workers. The classes in the clingo API do not support pickling (and it would defy the attempt to safe memory). If you restrict yourself to python classes for communication, then these can be passed around easily and they will be freed when no longer referenced.

[0x326]

@rkaminsk Are symbols freed when a clingo Control object is garbage-collected? E.g.:

import clingo


# Abstraction of long-running business logic
while True:
    clingo_control = clingo.Control()
    clingo_control.load(...)
    for _ in range(3):
        clingo_control.ground((
            ...
        ))
        for model in clingo_control.solve(yield_=True, async_=True):
            for symbol in model.symbols(shown=True):
                ...

[rkaminsk]

@rkaminsk Are symbols freed when a clingo Control object is garbage-collected? E.g.:

No, they are never freed and actually shared between multiple control objects.

[0x326]

While a garbage-collection/reference-counting approach of individual symbols is probably the ideal solution, is it possible count references to clingo.Control objects in the meantime and clear the hash table when all clingo.Control objects are garbage-collected by Python?

[rkaminsk]

While a garbage-collection/reference-counting approach of individual symbols is probably the ideal solution, is it possible count references to clingo.Control objects in the meantime and clear the hash table when all clingo.Control objects are garbage-collected by Python?

It is not that easy. The python program might store some other symbol objects. I can have a look if I can provide a function to clear all symbols. But it would be dangerous to use...

[0x326]

Hmm. It's probably better to be memory-safe than to provide a function that could result in a segfault. I think some form of memory management is going to be an essential feature for real-world applications, but I can look into using subprocesses in the meantime.

[kherud]

Is there a way with the C/C++ API to do that? It would be really helpful to at least have a manual option, even if the Python binding doesn't support it.

[rkaminsk]

The next major clingo release will give more control about storing symbols. For now there is no way to delete them - not even on the C++ level.

[sthiele]

@rkaminsk That is really nice to read. You once mentioned that this feature has to be well planned. Can you elaborate on how you plan to do it?