Here is a function that looks harmless. It keeps a chat history for an LLM call, and gives the caller a fresh list when they don't pass one.
def remember(message, history=[]):
history.append(message)
return history
print(remember("hi"))
print(remember("what is RAG?"))
['hi'] ['hi', 'what is RAG?']
The second call "remembers" the first, even though nobody passed a history. In an AI service this is exactly how one user's chat messages leak into another user's prompt. Nothing crashed, so nothing alerts you.
If you read this code with a Java brain, it makes no sense. In Java, a default value would be built fresh on every call. Python doesn't work that way, and the reason is one idea that the rest of this article builds on.
WHY THIS MATTERS
AI engineering code is mostly Python moving big, shared, mutable things around: message lists, config dicts, NumPy arrays, model objects. Almost every "it worked in my test but got weird in production" bug in that code comes from not knowing which object a name points at. Learn the model once and you can predict these bugs before you write them.
Part I · What a variable is
Chapter 1: Names are labels on objects
The problem
In Java you're used to two kinds of variables. A primitive like int x = 5 holds the value directly. An object variable like List<String> xs holds a reference, an address pointing to an object on the heap. Python has only the second kind. Everything is an object, including 5, and every variable is a reference.
Two words we need:
- Object: a thing living in memory. It has a type (what it is), a value (its contents) and an identity (which one it is;
id()returns it as a number). - Name: a word in your code, like
history. It is not storage. It is bound to exactly one object at a time.
MENTAL MODEL
Sticky notes on objects. Objects float in memory. A name is a sticky note stuck on one object. a = b does not copy anything: it peels off nothing and sticks a second note on the same object. Assignment (=) only ever moves a note; it never changes an object.
Details, built on the model
If two notes are on one object, a change made through either note shows up through the other, because there is only one object.
a = [1, 2]
b = a # second note on the same list
b.append(3)
print(a, b, a is b) # 'is' asks: same object?
[1, 2, 3] [1, 2, 3] True
is compares identity (are both notes on one object?). == compares value (do the contents match?). This is Java's == versus .equals(), with the meanings swapped in spelling.
x = [1, 2]
y = [1, 2] # a second, separate list
print(x == y, x is y)
True False
Assigning a new object to a name moves that one note. The other note doesn't follow.
a = [1, 2]
b = a
b = [9] # move note b to a new list
print(a, b)
[1, 2] [9]
Function calls move notes too
Java people ask "is Python pass-by-value or by-reference?". With the model the answer is simple: calling f(x) sticks the parameter's note on the same object the caller's note is on. Mutating it inside is visible outside; reassigning the parameter only moves the function's own note.
def mutate(xs):
xs.append("added") # changes the shared object
def rebind(xs):
xs = ["new list"] # moves only the local note
msgs = ["hi"]
mutate(msgs)
rebind(msgs)
print(msgs)
['hi', 'added']
Java behaves the same way for objects; the difference is that in Python everything is an object, so the rule has no exceptions to memorise.
WHY THIS MATTERS
When you pass a messages list into a helper that "adds the system prompt", you've changed the caller's list. Call it twice and you get two system prompts in one request. Model knowledge tells you to either copy first or return a new list.
MODEL RECAP
Recap: names are sticky notes; = moves a note, it never copies an object.
Chapter 2: When the default list is built
The problem
Back to remember. Why does the default list survive between calls?
A Python def is a statement that runs once, when the module loads, and produces a function object. Default values are evaluated at that moment and stored on the function object, in an attribute called __defaults__.
MENTAL MODEL
The default is one object glued to the function. When def runs, Python builds the default list once and glues it to the function. Every call that omits history sticks its history note on that same glued object. Mutate it once and every future call sees the change.
Details, built on the model
We can look at the glued object directly.
def remember(message, history=[]):
history.append(message)
return history
print(remember.__defaults__)
remember("hi")
print(remember.__defaults__)
([],) (['hi'],)
The fix follows from the model: don't glue a mutable object. Glue None (which can't be mutated) and build a fresh list inside the body, which runs on every call.
def remember(message, history=None):
if history is None:
history = [] # new object, every call
history.append(message)
return history
print(remember("hi"))
print(remember("what is RAG?"))
['hi'] ['what is RAG?']
The same "evaluated once" rule explains a sneakier bug: a default like created=time.time() freezes the timestamp at import time.
WHY THIS MATTERS
Request handlers in FastAPI or a Lambda live for many requests, so module-level and default objects are shared between users. A cache dict as a default argument is a cross-user data leak.
MODEL RECAP
Recap: defaults are built once and glued to the function, so never glue a mutable object.
Part II · What can change
Chapter 3: Mutable and immutable objects
The problem
Some objects can be changed in place (mutable: list, dict, set, most of your own classes). Others can't (immutable: int, float, str, tuple, frozenset). Java has the same split (String is immutable, ArrayList isn't), but in Python it decides what += does.
MENTAL MODEL
Stone objects and clay objects. Clay (mutable) objects can be reshaped where they stand, so every note on them sees the change. Stone (immutable) objects can't be touched: any "change" carves a new stone and moves your one note to it, leaving other notes on the old stone.
Details, built on the model
+= asks the object to reshape itself if it is clay, and carves a new object if it is stone.
a = b = "tok" # stone
a += "en"
print(a, b)
c = d = ["tok"] # clay
c += ["en"]
print(c, d)
token tok ['tok', 'en'] ['tok', 'en']
A tuple is stone, but it can hold notes stuck on clay. The tuple can't be changed, yet the list inside it can.
t = ("cfg", [0.7])
t[1].append(0.9)
print(t)
('cfg', [0.7, 0.9])
Copies: shallow and deep
A shallow copy makes a new outer object whose notes point at the same inner objects. A deep copy recursively copies the inner objects too.
import copy
cfg = {"model": "m", "stop": ["</s>"]}
shallow = copy.copy(cfg)
deep = copy.deepcopy(cfg)
shallow["stop"].append("###")
print(cfg["stop"], deep["stop"])
['</s>', '###'] ['</s>']
Why dict keys must be stone
A dict finds a key by its hash, a number computed from the value. If a key could be reshaped after insertion, its hash would change and the dict could never find it again. So Python only allows hashable (normally immutable) keys.
try:
cache = {["prompt", 1]: "x"}
except TypeError as e:
print("TypeError:", e)
print({("prompt", 1): "x"})
TypeError: unhashable type: 'list'
{('prompt', 1): 'x'}
WHY THIS MATTERS
Caching LLM responses means building a key from the prompt and parameters. Use a tuple or a string (stone), never a list or dict. And when you build per-request configs from a shared template, deep-copy it, or one request's edits reach the next.
MODEL RECAP
Recap: clay changes in place for every note; stone never changes, so "edits" make a new stone.
Part III · Interfaces without interfaces
Chapter 4: Protocols replace interfaces
The problem
In Java, a for loop over your class needs implements Iterable. Python has no implements. Yet you can loop over a file, a dict, a NumPy array or a dataset class you wrote. How?
Two terms:
- Dunder method: a method whose name starts and ends with two underscores, like
__len__. You don't call these yourself; Python calls them for you. - Protocol: an agreed set of dunder methods. Any object that has them "is" that kind of thing. This is called duck typing: if it quacks like a duck, treat it as a duck.
MENTAL MODEL
Syntax is a phone call to a dunder. Every bit of built-in syntax is translated into a call to a named method on the object. len(x) calls x.__len__(), x[i] calls x.__getitem__(i), for calls __iter__, a + b calls a.__add__(b). Nothing checks your class's ancestry. Python just phones the method, and if it answers, it works.
Details, built on the model
A tiny dataset class, the shape PyTorch's Dataset expects:
class Chunks:
def __init__(self, text, size):
self.text, self.size = text, size
def __len__(self):
return -(-len(self.text) // self.size) # ceil
def __getitem__(self, i):
if i >= len(self):
raise IndexError(i)
s = i * self.size
return self.text[s:s + self.size]
ds = Chunks("retrieval augmented", 8)
print(len(ds), ds[1])
print(list(ds))
3 l augmen ['retrieva', 'l augmen', 'ted']
Notice list(ds) worked without __iter__. When __iter__ is missing, Python falls back to calling __getitem__(0), (1), … until IndexError. That's the model again: Python tries the phone numbers it knows.
__repr__ controls how the object prints, and __eq__ controls ==. Without __eq__, == falls back to identity, just like Java's Object.equals.
class Msg:
def __init__(self, role, text):
self.role, self.text = role, text
def __repr__(self):
return f"Msg({self.role!r}, {self.text!r})"
def __eq__(self, other):
return (self.role, self.text) == (other.role, other.text)
print(Msg("user", "hi"), Msg("user", "hi") == Msg("user", "hi"))
Msg('user', 'hi') True
WHY THIS MATTERS
Every AI library leans on protocols. A Hugging Face dataset, a PyTorch DataLoader, a LangChain retriever: they all just phone __len__, __getitem__ or __iter__. Once you see that, "make my data work with library X" becomes "give my class the methods X phones".
MODEL RECAP
Recap: syntax phones a dunder method; any object that answers qualifies, no implements needed.
All the mental models, in one place
MODEL RECAP
- Sticky notes on objects. Names are notes;
=moves a note and never copies.isasks "same object?",==asks "same contents?". - The default is one object glued to the function. It is built once at
deftime, so useNoneand build inside. - Stone and clay. Clay (list, dict) changes in place for every note; stone (int, str, tuple) never changes, so edits make a new object. Dict keys must be stone.
- Syntax is a phone call to a dunder.
len,[],for,+,==each call a named method; implementing it is all it takes.
With those four pictures you can predict the output of every snippet above without running it, which is the real test of having learned it.