Location>code7788 >text

Go runtime scheduler in a nutshell (X): asynchronous preemption

Popularity：584 ℃/2024-09-19 11:29:44

0. Preface

Previously, runtime overruns and preemption caused by system calls were introduced, and they are both collaborative preemption. This talk will introduce signal-based true preemption scheduling.

Before introducing true preemptive scheduling look at Go's two preemptive schedulers:

Preemptive Scheduler - Go 1.2 so far

  • Collaboration-based preemptive scheduler - Go 1.2 - Go 1.13
    Improvement: Insertion at function call time via compilerPreempting Inspectiondirective that checks whether the current Goroutine has initiated a preemption request at function call time, enabling collaboration-based preemption scheduling.
    Defect: Goroutine may cause program suspension due to garbage collection and loops taking up resources for a long time.
  • Signal-based preemptive scheduler - Go 1.14 so far
    Improvements: realized signal-basedtrue preemption scheduling。
    Defect 1: Garbage collection triggers preemptive scheduling when scanning the stack.
    Deficiency 2: Not enough time points are preempted to cover all edge cases.

（Note: This paragraph is derived frompreemptive scheduler）

Collaborative preemption is accomplished by inserting at function call time thePreempting Inspection The problem with this kind of preemption is that if there is no function call in the goroutine, there is no way to insert thePreempting Inspection, resulting in an inability to seize it. Let's see.Go runtime scheduler in a nutshell (VII): case study Examples of:

//go:nosplit
func gpm() {
	var x int
	for {
		x++
	}
}

func main() {
	var x int
	threads := (0)
	for i := 0; i < threads; i++ {
		go gpm()
	}

	(1 * )
	("x = ", x)
}

Disable asynchronous preemption:

# GODEBUG=asyncpreemptoff=1 go run

The program gets stuck. This is because inserting the//go:nosplit would prohibit function stack expansion, and collaborative preemption cannot be inserted before the function stack callPreempting InspectionThis causes the goroutine to not be able to be preempted.

And signal-based true preemption scheduling can improve this problem.

1. Signal-based true preemption scheduling

By asynchronous preemption here we mean signal-based true preemption scheduling.

Asynchronous preemption is implemented in :

func preemptone(pp *p) bool {
	...
	// Request an async preemption of this P.
	if preemptMSupported && == 0 {
		 = true
		preemptM(mp) // asynchronous preemption
	}

	return true
}

go intopreemptM：

func preemptM(mp *m) {
	...
	if (0, 1) { // update signalPending
		signalM(mp, sigPreempt) // signalM signal a thread
	}
	...
}

// signalM sends a signal to mp.
func signalM(mp *m, sig int) {
	tgkill(getpid(), int(), sig)
}

func tgkill(tgid, tid, sig int)

invocationssignalM Send the sigPreempt (_SIGURG: 23) signal to the thread. The thread receives the signal and processes it accordingly.

1.1 Threads handling preemption signals

How does the thread handle the sigPreempt signal from the OS?

The signal processing of the thread is done in thesighandler：

func sighandler(sig uint32, info *siginfo, ctxt , gp *g) {\
    // The g executing the signal handler. This is almost always
	// . See delayedSignal for an exception.
	gsignal := getg()
	mp := 

    if sig == sigPreempt &&  == 0 && !delayedSignal {
		// Might be a preemption signal.
		doSigPreempt(gp, c)
		// Even if this was definitely a preemption signal, it
		// may have been coalesced with another signal, so we
		// still let it through to the application.
	}
    ...
}

go intodoSigPreempt：

// doSigPreempt handles a preemption signal on gp.
func doSigPreempt(gp *g, ctxt *sigctxt) {
	// Check if this G wants to be preempted and is safe to
	// preempt.
	if wantAsyncPreempt(gp) {
		if ok, newpc := isAsyncSafePoint(gp, (), (), ()); ok {
			// Adjust the PC and inject a call to asyncPreempt.
			(abi.FuncPCABI0(asyncPreempt), newpc)
		}
	}

	// Acknowledge the preemption.
	(1)
	(0)
}

First.doSigPreempt invocationswantAsyncPreempt Determines whether to do asynchronous preemption:

// wantAsyncPreempt returns whether an asynchronous preemption is
// queued for gp.
func wantAsyncPreempt(gp *g) bool {
	// Check both the G and the P.
	return ( ||  != 0 && ().preempt) && readgstatus(gp)&^_Gscan == _Grunning
}

If so, go ahead and callisAsyncSafePoint Determines if the current execution is an asynchronous safe point; threads can only handle asynchronous preemption if they execute to an asynchronous safe point. A safe point is a location where the Go runtime believes it is safe to suspend or preempt a running Goroutine. A safe point for asynchronous preemption ensures that the state of the system is stable and consistent when the Goroutine is paused or switched, and that there are no data contention, deadlocks, or unfinished critical computations.

If it is a safe point for asynchronous preemption. Then the call to(abi.FuncPCABI0(asyncPreempt), newpc) fulfillmentasyncPreempt：

// asyncPreempt saves all user registers and calls asyncPreempt2.
//
// When stack scanning encounters an asyncPreempt frame, it scans that
// frame and its parent frame conservatively.
//
// asyncPreempt is implemented in assembly.
func asyncPreempt()

//go:nosplit
func asyncPreempt2() { // asyncPreempt will call to the asyncPreempt2
	gp := getg()
	 = true
	if {
		mcall(preemptPark) // Type of preemption，If it is preemptStop failing agreement preemptPark seize (the strategic high ground)
	} else {
		mcall(gopreempt_m)
	}
	 = false
}

asyncPreempt call (programming)asyncPreempt2 deal with peacebuilding of the grab. For the non The grabbing that we did in theGo runtime scheduler in a nutshell (viii): runtime overrun preemption The main idea is to put a goroutine that is taking too long to run into a global queue. Then the thread performs scheduling to get the next available goroutine.

1.2 Case Studies

Remember that inGo runtime scheduler in a nutshell (VII): case studies The last thought left in it?

//go:nosplit
func gpm() {
	var x int
	for {
		x++
	}
}

func main() {
	var x int
	threads := (0)
	for i := 0; i < threads; i++ {
		go gpm()
	}

	(1 * )
	("x = ", x)
}

# GODEBUG=asyncpreemptoff=0 go run

Why does the program still get stuck with asynchronous preemption turned on?

From the previous analysis combined with ourdlv debug It was found that at the safe point of judgmentisAsyncSafePoint This always returns false, and you can't access theasyncpreempt The goroutine is preempted. and, since the preemption point check for collaborative preemption was//go:nosplit is disabled, preventing both collaborative and asynchronous preemption of the goroutine.

2. Summary

This talk introduces asynchronous preemption, or signal-based true preemption scheduling. This basically wraps up our Go runtime scheduler primer, which gives a general understanding of what the Go runtime scheduler is doing through ten lectures. In the next talk, we'll take a look at the big picture and tie together what we've talked about.

<Previous Next>