🍲 Advanced Iteration Tricks in Python: From Buffets to Conveyor Belts

In Part 1, we met the buffet (iterables) the endless source of food.
In Part 2, we met the waiter (iterators) serving one dish at a time, remembering their place, and shouting StopIteration! when empty-handed.

But restaurants don’t stop there. Behind the kitchen doors, chefs have gadgets, assistants, conveyor belts, and delegation systems that keep food moving smoothly.

In Python, those same tricks exist in the form of itertools, yield from, generator pipelines, and more.

This is Part 3: Advanced Iteration Tricks.
By the end, you’ll be building lazy pipelines like a pro chef designing a menu mixing infinite streams, slicing huge datasets, and cloning iterators without breaking a sweat.

Grab your apron. Let’s dive in. 🍴

🍴 The Kitchen Gadget Drawer: itertools

If iterators are waiters, then itertools is the drawer full of shiny kitchen gadgets. You don’t need them, but once you try them, you’ll never go back.

Example: Endless Breadsticks

import itertools

for bread in itertools.count(1):  # counts forever
    if bread > 5:
        break
    print(bread)

Output:

1
2
3
4
5

📌 Behind the curtain:

• count is just an iterator with an internal counter.

• It doesn’t pre-generate anything. It’s lazy and efficient.

🔥 Real-World Demo: Streaming Logs Lazily

Imagine a gigantic server log. Instead of loading the whole file:

import itertools

def read_logs(filename):
    with open(filename) as f:
        for line in f:
            yield line.strip()

logs = read_logs("server.log")
errors = (l for l in logs if "ERROR" in l)

# Peek at the first 5 errors
for e in itertools.islice(errors, 5):
    print("🚨", e)

👉 Process logs line by line without blowing up memory.
👉 islice = grab a safe bite from an infinite buffet.

🥡 yield from: Delegating to Another Waiter

Sometimes your waiter is juggling too many trays. Instead of handling every dish, they can call an assistant waiter.

Example:

def menu():
    yield 1
    yield 2
    yield from [3, 4]  # delegation

print(list(menu()))  # [1, 2, 3, 4]

📌 yield from iterable = expands into a loop behind the scenes.

🔥 Real-World Demo: Nested Config Loader

def load_base():
    yield "db=sqlite"
    yield "timeout=30"

def load_dev():
    yield from load_base()
    yield "debug=true"

print(list(load_dev()))
# ['db=sqlite', 'timeout=30', 'debug=true']

👉 Cleaner than writing nested loops. Perfect for layered configs, plugin systems, or merging defaults with overrides.

🍝 Generator Delegation: Sous-Chef Pipelines

Instead of one chef doing everything, you split the kitchen: appetizers, mains, desserts.

def appetizer():
    yield "salad 🥗"
    yield "soup 🍜"

def main_course():
    yield "steak 🥩"
    yield "pasta 🍝"

def dessert():
    yield "cake 🍰"

def full_meal():
    yield from appetizer()
    yield from main_course()
    yield from dessert()

print(list(full_meal()))

Output:

['salad 🥗', 'soup 🍜', 'steak 🥩', 'pasta 🍝', 'cake 🍰']

🔥 Real-World Demo: CSV Processing Pipeline

def read_lines(path):
    with open(path) as f:
        for line in f:
            yield line

def parse_csv(lines):
    for line in lines:
        yield line.strip().split(",")

def filter_users(rows):
    for row in rows:
        if int(row[2]) > 30:  # age > 30
            yield row

def users_over_30(path):
    yield from filter_users(parse_csv(read_lines(path)))

for u in users_over_30("users.csv"):
    print(u)

👉 Each generator = one sous-chef. Together, they make a pipeline kitchen that processes data lazily and efficiently.

🥂 itertools.tee: Cloning Waiters

Normally, once one person eats from a waiter’s tray, it’s gone.

But sometimes you need two independent waiters serving the same dishes.

import itertools

orders = (f"Order {i}" for i in range(1, 6))
chef, cashier = itertools.tee(orders)

print("Chef sees:", list(chef))
print("Cashier sees:", list(cashier))

Output:

Chef sees: ['Order 1', 'Order 2', 'Order 3', 'Order 4', 'Order 5']
Cashier sees: ['Order 1', 'Order 2', 'Order 3', 'Order 4', 'Order 5']

📌 Behind the curtain:

• tee buffers results if one consumer lags behind.

• Be careful: memory usage grows if one side falls too far behind.

👉 Real-world: when you need the same stream for both logging and analytics.

🍧 Lazy Pipelines: The Conveyor Belt

The real magic of iterators is building conveyor belts where each stage passes dishes forward, one bite at a time.

import itertools

nums = range(1, 1000000)

pipeline = itertools.islice(
    (n**2 for n in nums if n % 2),  # odd squares
    5
)

print(list(pipeline))  # [1, 9, 25, 49, 81]

📌 Nothing happens until you pull from the pipeline. That’s laziness in action.

🔥 Real-World Demo: Infinite Orders Stream

import itertools

def infinite_orders():
    n = 1
    while True:
        yield f"Pizza #{n}"
        n += 1

orders = itertools.islice(
    (o for o in infinite_orders() if "3" not in o),  # skip unlucky 3s
    5
)

print(list(orders))

Output:

['Pizza #1', 'Pizza #2', 'Pizza #4', 'Pizza #5', 'Pizza #6']

👉 Perfect pattern for real-time feeds, simulations, or never-ending event streams.

🎨 ASCII Mental Model

[ Buffet ] → [ Waiter ] → [ Gadget ] → [ Filter ] → [ Pipeline ] → [ Customer ]

Think of it as a restaurant conveyor belt:

• Each station adds, filters, or transforms a dish.

• Dishes keep flowing lazily.

• No one dumps a giant tray on the customer.

🚨 Common Pitfalls

• Iterator exhaustion

    it = iter([1, 2, 3])
    print(list(it))  # [1, 2, 3]
    print(list(it))  # []
  

  Once eaten, a waiter’s tray is empty.

• Infinite loops
  Always guard count() or cycle() with islice.

• Shared iterators
  If multiple consumers share one iterator, one might eat everything first. Use tee if you really need copies.

🎬 Wrap-Up

In this deep dive, we learned how to turn basic waiters into a full restaurant crew:

• itertools → kitchen gadgets (count, chain, islice).

• yield from → waiter delegation.

• Generator delegation → sous-chef pipelines.

• tee → cloning waiters.

• Lazy pipelines → conveyor belts for infinite or massive data.

👉 Next up (Part 4): Coroutines waiters who not only serve but can also take new orders mid-service (send, throw, async/await).