Python Intermediate_008: Understanding Generators in Python

---

In our previous tutorial (https://codeaddict.tistory.com/entry/Python-Intermediate007-Understanding-property-and-Setter-Decorators-in-Python), we explored how to use @property and @<property_name>.setter to manage class attributes. Today, we’ll dive into generators in Python, a powerful tool for creating iterators in a memory-efficient way. We’ll cover the basics, syntax, and practical examples, and end with some problems for you to solve!

---

1- What Are Generators?

Generators are a special type of function in Python that allow you to iterate over a sequence of values without storing the entire sequence in memory. They are particularly useful when working with large datasets or infinite sequences.

• Generator Function: A function that uses yield to return values one at a time, pausing its state between calls, instead of returning a list or all values at once.
• Generator Expression: A compact, memory-efficient alternative to list comprehensions, using parentheses (…) instead of brackets […].
• Unlike regular functions that return all data at once, generators produce values on-demand, making them ideal for lazy evaluation.

Key Features:

• Lazy Evaluation: Values are generated on-the-fly, one at a time.
• Memory Efficiency: Only one value is stored in memory at a time.
• Use yield instead of return to produce a sequence of values.

---

2- Syntax and Execution Order

2–1 Syntax

def my_generator():
    yield 1
    yield 2
    yield 3

• yield: Pauses the function and returns a value to the caller. The function resumes from where it left off when next called.

Execution Order:

1. When the generator function is called, it returns a generator object (but doesn’t start execution).
2. The generator runs until it hits a yield statement, then pauses and returns the yielded value.
3. On the next call, it resumes execution from where it paused.

---

Example: Simple Generator

def simple_generator():
    yield "Python"
    yield "Generators"
    yield "Are"
    yield "Awesome"

# Create a generator object
gen = simple_generator()
# Iterate through the generator
for word in gen:
    print(word)

Output:

Python
Generators
Are
Awesome

2–2 Execution Order with one example

# Generator Function
def my_generator():
    n = 0
    while n < 3:
        yield n
        n += 1

# Generator Expression
gen_expr = (x * 2 for x in range(5))

# Usage
gen = my_generator()
for value in gen:
    print(value)  # Prints 0, 1, 2

• Generator Function: Uses yield to produce a value and suspend execution until the next call (e.g., via a next() or loop).
• Generator Expression: Similar to list comprehensions but generates values lazily.

Execution Order

1. Function Definition: The generator function is defined, but no code runs until called.
2. Generator Creation: Calling my_generator() returns a generator object without executing the body.
3. Value Generation: Each next(gen) or loop iteration resumes execution at the last yield, producing the next value.
4. Exhaustion: The generator stops when it encounters a return or the end of the function.

3- Simple Problems and Solutions

Problem 1: Create a Generator for Even Numbers

Write a generator function that yields even numbers from 0 to 10. Solution:

def ev_num(n):
    for i in range(n + 1):  # Loop from 0 to n
        if i % 2 == 0:  # Check if the current number (i) is even
            yield i  # Yield the even number

# Specify n outside the function
n = 10
gen1 = ev_num(n)  # Create generator object

# Iterate through the generator
for a in gen1:
    print(a)

Output:

0
2
4
6
8
10

Problem 2: Use a Generator Expression

Create a generator expression to double numbers from 1 to 5 and print them. 

Solution:

gen_expr = (x * 2 for x in range(1, 6))
for value in gen_expr:
    print(value)  # Outputs: 2, 4, 6, 8, 10

4- Challenging Problems for Readers

Now, test your skills with these more complex generator problems. Try solving them before checking the solutions below!

Problem 3: Infinite Fibonacci Sequence

Write a generator function that produces an infinite Fibonacci sequence (0, 1, 1, 2, 3, 5, 8, …). Use it to print the first 10 numbers.

• The infinite Fibonacci sequence is a series of numbers where each number is the sum of the two preceding ones, starting with 0 and 1. It continues indefinitely, producing an unending list of numbers. Here are the first few numbers in the sequence to help you understand:
• 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181, …

- How It Works:

• Start with 0 and 1.
• Add the last two numbers to get the next one:
• 0 + 1 = 1
• 1 + 1 = 2
• 1 + 2 = 3
• 2 + 3 = 5
• 3 + 5 = 8
• And so on…

 

Problem 4: Filtered Prime Number Generator

Create a generator function that yields prime numbers up to a given limit (e.g., 20). A prime number is divisible only by 1 and itself.

4- 1 Solutions to Problems 3&4

Solution to Problem 3: Infinite Fibonacci Sequence

def fibonacci_generator():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

fib = fibonacci_generator()
for _ in range(10):
    print(next(fib))  # Outputs: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34

• Explanation: The generator uses an infinite loop with yield to produce Fibonacci numbers, updating a and b on each iteration.

Solution to Problem 4: Filtered Prime Number Generator

def prime_generator(limit):
    def is_prime(n):
        if n < 2:
            return False
        for i in range(2, int(n ** 0.5) + 1):
            if n % i == 0:
                return False
        return True
    
    for num in range(limit + 1):
        if is_prime(num):
            yield num

primes = prime_generator(20)
for prime in primes:
    print(prime)  # Outputs: 2, 3, 5, 7, 11, 13, 17, 19

• Explanation: The is_prime helper function checks for primality, and the generator yields numbers up to the limit that pass this test.

- Algorithm Summary

Here’s a high-level summary of the algorithm used in the code:

1. Input: A number limit that specifies the upper bound for generating prime numbers.
2. Process:

• For each number num from 0 to limit:
• Check if num is prime using the is_prime(n) function.
• If num is prime, yield it (make it available to the caller).

 

3. Primality Test (is_prime(n)):

• If n < 2, it’s not prime.
• Check if n is divisible by any integer i from 2 to the square root of n.
• If n is divisible by any i, it’s not prime.
• If no divisors are found, n is prime.

4. Output: A sequence of prime numbers, yielded one at a time, which can be consumed by the caller (e.g., printed, stored, or processed further).

- Example Walkthrough for limit = 10

To solidify understanding, let’s walk through what happens when limit = 10:

• num = 0: is_prime(0) returns False (not prime, no yield).
• num = 1: is_prime(1) returns False (not prime, no yield).
• num = 2: is_prime(2) returns True (prime, yield 2).
• num = 3: is_prime(3) returns True (prime, yield 3).
• num = 4: is_prime(4) checks divisors 2 (4 % 2 = 0), returns False (not prime, no yield).
• num = 5: is_prime(5) checks divisors 2 (5 % 2 = 1), returns True (prime, yield 5).
• num = 6: is_prime(6) checks divisors 2 (6 % 2 = 0), returns False (not prime, no yield).
• num = 7: is_prime(7) checks divisors 2 (7 % 2 = 1), returns True (prime, yield 7).
• num = 8: is_prime(8) checks divisors 2 (8 % 2 = 0), returns False (not prime, no yield).
• num = 9: is_prime(9) checks divisors 2 (9 % 2 = 1), 3 (9 % 3 = 0), returns False (not prime, no yield).
• num = 10: is_prime(10) checks divisors 2 (10 % 2 = 0), returns False (not prime, no yield).

 

---

5- Conclusion

Generators are a powerful feature in Python that enable you to work with sequences in a memory-efficient and elegant way. By mastering generators, you can write cleaner and more efficient code, especially when dealing with large datasets or infinite sequences. Try solving the advanced problems above to deepen your understanding!