Python coroutine, from yield/send to async/await


What is a coroutine?

Python is known to be unable to play the role of multi-core parallel computing due to the limitations of the GIL of the global interpretation. Since under GIL, only one thread can be running at a time, then for CPU-intensive programs, the switching overhead between threads becomes a drag, and the program with I/O as the bottleneck is the coroutine. good at
Reference link:

The difference between coroutine and process

The coroutine has its own register context and stack. When the coroutine schedules switching, the register context and stack are saved to other places, and when the switch back, the previously saved register context and stack are restored. Therefore: the coroutine can retain the state of the last call (that is, a specific combination of all local states), each time the process reenters, it is equivalent to the state of the last call, in other words: the last time you left The location of the logic flow.
Reference link:

Advantages of the coroutine:

  1. No process context switching overhead required
  2. No need for atomic locking and synchronization overhead
  3. Easy to switch control flow and simplify programming model
  4. High concurrency + high scalability + low cost: A CPU supporting tens of thousands of coroutines is not a problem. So it is very suitable for high concurrent processing.

Shortcomings of coroutine
can't take advantage of multi-core resources: the essence of coroutine is a single thread, it can't use multiple cores of a single CPU at the same time, coroutine needs to cooperate with process to run on multiple CPUs. Of course Most of the applications we write on a daily basis are not necessary, except for cpu-intensive applications.


Speaking from yield

Ordinary code to create Fibonacci:

def old_fib(n):
	res = [0] * n
	index = 0
	a = 0
	b = 1
	while index < n:
		res[index] = b
		a, b = b, a + b
		index += 1
	return res
 
print('-'*10 + 'test old fib' + '-'*10)
for fib_res in old_fib(20):
	print(fib_res)

But if we only need to get the mth position of Fibonacci, it is a waste of memory.
This time the yield comes in handy.

def fib(n):
	index = 0
	a = 0
	b = 1
	while index < n:
		yield b
		a, b = b, a + b
		index += 1
 
print('-'*10 + 'test yield fib' + '-'*10)
for fib_res in fib(20):
	print(fib_res)

When a function contains a yield statement, python automatically recognizes it as aBuilder. At this time, fib(20) does not actually call the function body, but generates a generator object instance from the function body.

The generator has a delay operation. The so-called delayed operation means that the result is produced when needed, rather than immediately. This is also the main benefit of the generator.
In the middle of each result, suspend the state of the function so that it will continue to execute where it left off next time.

Yield here can keep the calculation site of the fib function, pause the fib calculation and return b. When fib is placed in a for...in loop, next (fib(20)) is called every time, the generator is woken up and executed until the next yield statement until a StopIteration exception is thrown. This exception is caught by the for loop, causing the loop to jump out.


Send is coming

Send can send a return value to yield

import random
import time
def stupid_fib(n):
	index = 0
	a = 0
	b = 1
	while index < n:
		sleep_cnt = yield b
		print('let me think {0} secs'.format(sleep_cnt))
		time.sleep(sleep_cnt)
		a, b = b, a + b
		index += 1
print('-'*10 + 'test yield send' + '-'*10)
N = 20
sfib = stupid_fib(N)
fib_res = next(sfib)   #equivalent to sfib.send(None)
while True:
	print(fib_res)
	try:
		fib_res = sfib.send(random.uniform(0, 0.5))
	except StopIteration:
		break
  1. Next(sfib) is equivalent to sfib.send(None)
  2. Generates a random number of seconds as the return value of yield in the coroutine, and yield returns b to fib_res

We can control the thinking time when the coroutine is calculated from the "main" program, and the coroutine can return to the "main" program to calculate the result.

Yield from introduction

Yield is used to refactor the generator.

def copy_fib(n):
	print('I am copy from fib')
	yield from fib(n)
	print('Copy end')
print('-'*10 + 'test yield from' + '-'*10)
for fib_res in copy_fib(20):
	print(fib_res)

Intelligent Recommendation

Python yield and send and yield from

The next function is very similar to the send function, and you can get the value of the next expression of the generator's next yield. The difference is that the send function can pass arguments to t...

Yield, yield from and send in python

1. Use yield 1) Using yield in the function can make the function a generator. If a function generates an array, it must store the data in memory. If a generator is used, the data is generated when it...

Python asynchronous coroutine async/await/asyncio

Article Directory Asynchronous IO asyncio coroutine Common names of coroutines Create a coroutine Create a task Task Get the return result, callback function future/ directly use Task to get Blocking ...

Python coroutine 2: yield from entry to mastery

This article will introduce: How to return a value when the coroutine terminates The use and semantics of the new syntax of yield At the same time, a few coroutine examples are used to show the usage ...

After Python 3.5 async/await (coroutine) use problem summary

Foreword In recent projects, it is necessary to implement a timer that is executed periodically. The first idea was to use celery, but after looking at it, I found that for this simple requirement, ce...

More Recommendation

Python coroutine implementation--yield, yield from, greenlet, gevent

I have talked about coroutine before, and coroutine is to achieve concurrency in one thread. Pay attention to distinguish it from return to understand In layman's terms, the coroutine will save the ru...

Coroutine-async/await

Introduction Async/await is pythonCoroutineusage Appeared after python3.4 Use async to declare a function as an asynchronous function. The asynchronous function is characterized by being able to hang ...

Unity coroutine and async, await

Coroutine A coroutine is like a function that can suspend execution and return control to Unity, and then continue execution at a specified time. The coroutine is essentially a function declared with ...

Coroutine keyword yield and yield from

yield When the yield keyword appears in a function, then this function is a generator. You can iterate with a for loop or a next() function. Yield can receive variables passed in from the outside worl...

Python Builder Send, Next, Yield from FROM

Builder: The generator is a special iterator, which is required to generate the result yield when generated, rather than returning the result Return. This saves CPU and memory at the same time. The ge...

Copyright  DMCA © 2018-2026 - All Rights Reserved - www.programmersought.com  User Notice

Top