Memory management

Part of the free LogicWiz Python course, Module 13: Python under the hood. The examples and practice run in the app.

In C you must call free() yourself or your program leaks memory. In Python you never free anything, yet Canteen Counter can run all day without filling the RAM. Who is cleaning up behind you?

You'll learn to

• Explain reference counting and when CPython frees an object
• Read `sys.getrefcount` as relative changes, not exact numbers
• Explain reference cycles and how the `gc` module collects them
• Use `weakref` for references that should not keep an object alive

Learn

Reference counting

Every object in CPython keeps a reference count: how many labels and containers point to it right now.

• x = obj, putting it in a list, passing it to a function: the count goes up.
• A name is deleted or moved, a list drops it, a function returns: the count goes down.
• When the count reaches 0, nobody can reach the object any more, so CPython frees it immediately.

Think of Shankar bhaiya clearing a plate the moment no customer is holding its token.

Counting references

import sys

order = ["chai"]
base = sys.getrefcount(order)
alias = order
print(sys.getrefcount(order) - base)
box = [order, order]
print(sys.getrefcount(order) - base)
del alias
print(sys.getrefcount(order) - base)

We print the change from a starting value. The absolute number is not useful: calling getrefcount(order) itself adds a temporary reference, and objects like None or small ints have huge, special counts.

`del` removes a name, not an object

class Plate:
    def __init__(self, name):
        self.name = name
    def __del__(self):
        print("cleared", self.name)

p = Plate("thali")
q = p
del p
print("p deleted")
del q
print("q deleted")

__del__ runs when the object is really freed. After del p the plate still has q, so nothing is cleared. Only when the last label goes does the count hit 0.

Cycles: where counting fails

Suppose a.friend = b and b.friend = a. Even after del a, b, each object is still pointed to by the other, so neither count reaches 0. They are unreachable garbage that counting alone can never free.

CPython has a second tool for this: the cyclic garbage collector in the gc module. Every so often it looks at container objects, finds groups that only point to each other, and frees them. It sorts objects into 3 generations: new objects are checked often; objects that survive move to older, rarely checked generations.

Watching the collector free a cycle

import gc, weakref

class Node:
    pass

gc.disable()
a = Node()
b = Node()
a.friend = b
b.friend = a
watch = weakref.ref(a)
del a, b
print("alive after del:", watch() is not None)
gc.collect()
print("alive after collect:", watch() is not None)
gc.enable()

We switch the automatic collector off so the demo is predictable, then run it by hand with gc.collect(). In real programs leave it on; it runs by itself.

Weak references

A weakref.ref(obj) points to an object without adding to its count. Call it to get the object back, or None if it has already been freed.

Use it for caches and "who is watching" lists that should not keep objects alive on their own: weakref.WeakValueDictionary drops an entry when its value dies, and weakref.WeakSet forgets members that are gone.

Not every object can be weakly referenced: plain int, str, list and dict cannot. Objects of your own classes can.

Do not rely on __del__ to close files or save data. With cycles, or on other Pythons like PyPy, it may run late or never. Use a with block (Module 14) for cleanup that must happen.

Interview favourite: "How does Python manage memory?" Answer in three parts: objects live on a private heap managed by Python (with a fast small-object allocator, pymalloc); reference counting frees most objects the moment they become unused; the generational garbage collector (gc) frees reference cycles.

In one line

CPython frees an object when its reference count hits 0; the gc collector frees reference cycles; weakref points without keeping alive.

Interview corner

How does Python manage memory?

All objects live on a private heap managed by the interpreter. CPython uses reference counting: each object counts its references and is freed as soon as the count reaches 0. A generational cyclic garbage collector (gc module, 3 generations) finds and frees groups of objects that only reference each other.

What is a reference cycle and how is it cleaned up?

A cycle is when objects refer to each other (a.friend = b, b.friend = a), so their counts never reach 0 even when the program cannot reach them. The cyclic GC periodically scans container objects, detects unreachable cycles and frees them. You can trigger it with gc.collect().

What does `del` do?

del x removes the name x from its scope (or an item from a container, like del d[k]) and decreases the object's reference count. The object itself is freed only when no other references remain.

What is a weak reference used for?

A weak reference (weakref.ref) lets you refer to an object without keeping it alive. It is used for caches (WeakValueDictionary), observer lists and parent pointers in trees, so that memory is freed when the real owners let go.