When Go needs to run something after a delay, or repeat an action at fixed intervals, it reaches for timers. The API looks small, but the runtime underneath is doing more work than the surface suggests. The design also changed significantly around Go 1.14, so it is useful to look at both the older bucket-based implementation and the newer per-P implementation to see what was optimized and why.
Some runtime background helps: the GMP scheduler, how goroutines are scheduled, and how a heap works.
Ticker first
Tickers are a good entry point because they expose the timer machinery through a simple channel. A ticker fires at intervals, and the questions worth asking are: if a tick is not consumed in time, is it queued? And what is the difference between NewTicker and Tick?
// A Ticker holds a channel that delivers "ticks" of a clock
// at intervals.
type Ticker struct {
C <-chan Time // The channel on which the ticks are delivered.
r runtimeTimer
}
The structure is intentionally small: a channel and a runtime timer. When the timer fires, it sends a Time value into C.
NewTicker, Stop, and Reset are also straightforward:
// NewTicker returns a new Ticker containing a channel that will send
// the time on the channel after each tick. The period of the ticks is
// specified by the duration argument. The ticker will adjust the time
// interval or drop ticks to make up for slow receivers.
// The duration d must be greater than zero; if not, NewTicker will
// panic. Stop the ticker to release associated resources.
func NewTicker(d Duration) *Ticker {
if d <= 0 {
panic(errors.New("non-positive interval for NewTicker"))
}
// Give the channel a 1-element time buffer.
// If the client falls behind while reading, we drop ticks
// on the floor until the client catches up.
c := make(chan Time, 1)
t := &Ticker{
C: c,
r: runtimeTimer{
when: when(d),
period: int64(d),
f: sendTime,
arg: c,
},
}
startTimer(&t.r)
return t
}
// Stop turns off a ticker. After Stop, no more ticks will be sent.
// Stop does not close the channel, to prevent a concurrent goroutine
// reading from the channel from seeing an erroneous "tick".
func (t *Ticker) Stop() {
stopTimer(&t.r)
}
// Reset stops a ticker and resets its period to the specified duration.
// The next tick will arrive after the new period elapses.
func (t *Ticker) Reset(d Duration) {
if t.r.f == nil {
panic("time: Reset called on uninitialized Ticker")
}
modTimer(&t.r, when(d), int64(d), t.r.f, t.r.arg, t.r.seq)
}
Two details matter. First, the channel buffer is one. If the current tick has not been handled, one tick can wait; after that, ticks are dropped. A ticker does not guarantee that every interval produces a value when the receiver is slow. Second, Stop does not close the channel. Closing could race with a send and cause a panic; resources are released by stopping, so a long-lived ticker needs an explicit Stop.
The Tick function is just a convenience wrapper:
// Tick is a convenience wrapper for NewTicker providing access to the ticking
// channel only. While Tick is useful for clients that have no need to shut down
// the Ticker, be aware that without a way to shut it down the underlying
// Ticker cannot be recovered by the garbage collector; it "leaks".
// Unlike NewTicker, Tick will return nil if d <= 0.
func Tick(d Duration) <-chan Time {
if d <= 0 {
return nil
}
return NewTicker(d).C
}
It returns only the channel, so the caller cannot stop the ticker. That makes Tick convenient but leaky unless the process exits soon after. NewTicker is the safer default.
Timers before Go 1.14
The older implementation is easier to follow. Timers lived in a fixed set of buckets, each protected by a lock. startTimer simply added the timer:
func startTimer(t *timer) {
if raceenabled {
racerelease(unsafe.Pointer(t))
}
addtimer(t)
}
addtimer assigned a bucket and took its lock:
func addtimer(t *timer) {
tb := t.assignBucket()
lock(&tb.lock)
ok := tb.addtimerLocked(t)
unlock(&tb.lock)
if !ok {
badTimer()
}
}
Bucket assignment used the current P's id modulo 64:
const timersLen = 64
func (t *timer) assignBucket() *timersBucket {
id := uint8(getg().m.p.ptr().id) % timersLen
t.tb = &timers[id].timersBucket
return t.tb
}
addtimerLocked appended the timer to the bucket's slice and sifted it up:
// Add a timer to the heap and start or kick timerproc if the new timer is
// earlier than any of the others.
// Timers are locked.
// Returns whether all is well: false if the data structure is corrupt
// due to user-level races.
func (tb *timersBucket) addtimerLocked(t *timer) bool {
// when must never be negative; otherwise timerproc will overflow
// during its delta calculation and never expire other runtime timers.
if t.when < 0 {
t.when = 1<<63 - 1
}
t.i = len(tb.t)
tb.t = append(tb.t, t)
if !siftupTimer(tb.t, t.i) {
return false
}
if t.i == 0 {
// siftup moved to top: new earliest deadline.
if tb.sleeping && tb.sleepUntil > t.when {
tb.sleeping = false
notewakeup(&tb.waitnote)
}
if tb.rescheduling {
tb.rescheduling = false
goready(tb.gp, 0)
}
if !tb.created {
tb.created = true
go timerproc(tb)
}
}
return true
}
The heap is a four-ary heap: siftupTimer finds a parent with (i-1)/4. Starting a timer means lock, append, and heapify.
Stopping removes the timer and heapifies again:
// stopTimer removes t from the timer heap if it is there.
// It returns true if t was removed, false if t wasn't even there.
//go:linkname stopTimer time.stopTimer
func stopTimer(t *timer) bool {
return deltimer(t)
}
// Delete timer t from the heap.
// Do not need to update the timerproc: if it wakes up early, no big deal.
func deltimer(t *timer) bool {
if t.tb == nil {
// t.tb can be nil if the user created a timer
// directly, without invoking startTimer e.g
// time.Ticker{C: c}
// In this case, return early without any deletion.
// See Issue 21874.
return false
}
tb := t.tb
lock(&tb.lock)
removed, ok := tb.deltimerLocked(t)
unlock(&tb.lock)
if !ok {
badTimer()
}
return removed
}
The old timerproc goroutine did the firing. It woke up, checked the heap root, and either ran a timer or slept until the next deadline. For a periodic timer, it adjusted when and sifted down; for a one-shot timer, it removed the root. Then it called the timer function outside the lock. The key snippet:
// Timerproc runs the time-driven events.
// It sleeps until the next event in the tb heap.
// If addtimer inserts a new earlier event, it wakes timerproc early.
func timerproc(tb *timersBucket) {
tb.gp = getg()
for {
lock(&tb.lock)
tb.sleeping = false
now := nanotime()
delta := int64(-1)
for {
if len(tb.t) == 0 {
delta = -1
break
}
// Get the heap root.
t := tb.t[0]
// Check whether it is time to fire.
delta = t.when - now
if delta > 0 {
break
}
ok := true
// period > 0 means this is a periodic timer, i.e. a ticker;
// otherwise it fires once and is removed from the heap.
if t.period > 0 {
// leave in heap but adjust next time to fire
// Change the current element's fire time, then heapify;
// it will move behind by itself.
t.when += t.period * (1 + -delta/t.period)
if !siftdownTimer(tb.t, 0) {
ok = false
}
} else {
// remove from heap
last := len(tb.t) - 1
if last > 0 {
tb.t[0] = tb.t[last]
tb.t[0].i = 0
}
tb.t[last] = nil
tb.t = tb.t[:last]
if last > 0 {
if !siftdownTimer(tb.t, 0) {
ok = false
}
}
t.i = -1 // mark as removed
}
f := t.f
arg := t.arg
seq := t.seq
unlock(&tb.lock)
if !ok {
badTimer()
}
if raceenabled {
raceacquire(unsafe.Pointer(t))
}
// This is where the timer function is actually called.
// For a ticker, it is the initial sendTime function, which
// sends the current time to the channel.
f(arg, seq)
lock(&tb.lock)
}
if delta < 0 || faketime > 0 {
// No timers left - put goroutine to sleep.
tb.rescheduling = true
goparkunlock(&tb.lock, waitReasonTimerGoroutineIdle, traceEvGoBlock, 1)
continue
}
// At least one timer pending. Sleep until then.
// If the heap root has not reached its fire time, sleep for delta.
tb.sleeping = true
tb.sleepUntil = now + delta
noteclear(&tb.waitnote)
unlock(&tb.lock)
notetsleepg(&tb.waitnote, delta)
}
}
The approach is coherent: one goroutine per bucket watches the heap, fires the earliest timer, and sleeps or gets woken when a new earlier timer is added.
Timers after the change
The newer runtime gives each timer an explicit status. The status list is long, but the point is to handle concurrent delete, modify, and fire operations without immediately mutating the heap:
// Values for the timer status field.
const (
// Timer has no status set yet.
timerNoStatus = iota
// Waiting for timer to fire.
// The timer is in some P's heap.
timerWaiting
// Running the timer function.
// A timer will only have this status briefly.
timerRunning
// The timer is deleted and should be removed.
// It should not be run, but it is still in some P's heap.
timerDeleted
// The timer is being removed.
// The timer will only have this status briefly.
timerRemoving
// The timer has been stopped.
// It is not in any P's heap.
timerRemoved
// The timer is being modified.
// The timer will only have this status briefly.
timerModifying
// The timer has been modified to an earlier time.
// The new when value is in the nextwhen field.
// The timer is in some P's heap, possibly in the wrong place.
timerModifiedEarlier
// The timer has been modified to the same or a later time.
// The new when value is in the nextwhen field.
// The timer is in some P's heap, possibly in the wrong place.
timerModifiedLater
// The timer has been modified and is being moved.
// The timer will only have this status briefly.
timerMoving
)
The bigger structural change is that timers are stored on the P, not in global buckets. addtimer gets the current P and uses its timersLock:
// addtimer adds a timer to the current P.
// This should only be called with a newly created timer.
// That avoids the risk of changing the when field of a timer in some P's heap,
// which could cause the heap to become unsorted.
func addtimer(t *timer) {
// .................
when := t.when
// Disable preemption while using pp to avoid changing another P's heap.
mp := acquirem()
// Get the m bound to the current g, then get the bound p.
pp := getg().m.p.ptr()
lock(&pp.timersLock)
// First clean up timers already marked as deleted.
cleantimers(pp)
// Then add the current timer to the current p's timer list.
doaddtimer(pp, t)
unlock(&pp.timersLock)
wakeNetPoller(when)
releasem(mp)
}
doaddTimer appends and heapifies on the P:
// doaddtimer adds t to the current P's heap.
// The caller must have locked the timers for pp.
func doaddtimer(pp *p, t *timer) {
// ...............
t.pp.set(pp)
i := len(pp.timers)
// The operation is similar to before, but the timer is now stored
// on the P rather than in a bucket; after appending, heapify as usual.
pp.timers = append(pp.timers, t)
siftupTimer(pp.timers, i)
if t == pp.timers[0] {
atomic.Store64(&pp.timer0When, uint64(t.when))
}
atomic.Xadd(&pp.numTimers, 1)
}
The P has its own timer heap:
type p struct {
// ..............
// Lock for timers. We normally access the timers while running
// on this P, but the scheduler can also do it from a different P.
timersLock mutex
// P has a dedicated place to store this timer heap.
// Actions to take at some time. This is used to implement the
// standard library's time package.
// Must hold timersLock to access.
timers []*timer
// Number of timers in P's heap.
// Modified using atomic instructions.
numTimers uint32
// Number of timerDeleted timers in P's heap.
// Modified using atomic instructions.
deletedTimers uint32
// ..............
}
Deletion now marks state instead of immediately removing from another P's heap. deltimer walks statuses and may return "already removed" or spin briefly waiting for another operation:
// deltimer deletes the timer t. It may be on some other P, so we can't
// actually remove it from the timers heap. We can only mark it as deleted.
// It will be removed in due course by the P whose heap it is on.
// Reports whether the timer was removed before it was run.
func deltimer(t *timer) bool {
for {
switch s := atomic.Load(&t.status); s {
case timerWaiting, timerModifiedLater:
// Prevent preemption while the timer is in timerModifying.
// This could lead to a self-deadlock. See #38070.
mp := acquirem()
if atomic.Cas(&t.status, s, timerModifying) {
// Must fetch t.pp before changing status,
// as cleantimers in another goroutine
// can clear t.pp of a timerDeleted timer.
tpp := t.pp.ptr()
if !atomic.Cas(&t.status, timerModifying, timerDeleted) {
badTimer()
}
releasem(mp)
atomic.Xadd(&tpp.deletedTimers, 1)
// Timer was not yet run.
return true
} else {
releasem(mp)
}
case timerModifiedEarlier:
// Prevent preemption while the timer is in timerModifying.
// This could lead to a self-deadlock. See #38070.
mp := acquirem()
if atomic.Cas(&t.status, s, timerModifying) {
// Must fetch t.pp before setting status
// to timerDeleted.
tpp := t.pp.ptr()
if !atomic.Cas(&t.status, timerModifying, timerDeleted) {
badTimer()
}
releasem(mp)
atomic.Xadd(&tpp.deletedTimers, 1)
// Timer was not yet run.
return true
} else {
releasem(mp)
}
case timerDeleted, timerRemoving, timerRemoved:
// Timer was already run.
return false
case timerRunning, timerMoving:
// The timer is being run or moved, by a different P.
// Wait for it to complete.
osyield()
case timerNoStatus:
// Removing timer that was never added
// or has already been run. Also see issue 21874.
return false
case timerModifying:
// Simultaneous calls to deltimer and modtimer.
// Wait for the other call to complete.
osyield()
default:
badTimer()
}
}
}
Where does firing happen now?
The trigger point moved into the scheduler. checkTimers is called from schedule, findrunnable, and work-stealing paths. That is why timer checking can appear in scheduling code:
// One round of scheduler: find a runnable goroutine and execute it.
// Never returns.
func schedule() {
_g_ := getg()
// ....................
top:
pp := _g_.m.p.ptr()
pp.preempt = false
if sched.gcwaiting != 0 {
gcstopm()
goto top
}
if pp.runSafePointFn != 0 {
runSafePointFn()
}
// Sanity check: if we are spinning, the run queue should be empty.
// Check this before calling checkTimers, as that might call
// goready to put a ready goroutine on the local run queue.
if _g_.m.spinning && (pp.runnext != 0 || pp.runqhead != pp.runqtail) {
throw("schedule: spinning with local work")
}
checkTimers(pp, 0)
// ..................
}
checkTimers compares timer0When and timerModifiedEarliest. If the next timer is not due, it normally returns early unless the P is local and enough deleted timers have accumulated. When it proceeds, it adjusts the heap, runs due timers, and may clear deleted timers in batches:
func checkTimers(pp *p, now int64) (rnow, pollUntil int64, ran bool) {
// If it's not yet time for the first timer, or the first adjusted
// timer, then there is nothing to do.
next := int64(atomic.Load64(&pp.timer0When))
nextAdj := int64(atomic.Load64(&pp.timerModifiedEarliest))
if next == 0 || (nextAdj != 0 && nextAdj < next) {
next = nextAdj
}
if next == 0 {
// No timers to run or adjust.
return now, 0, false
}
if now == 0 {
now = nanotime()
}
if now < next {
// Next timer is not ready to run, but keep going
// if we would clear deleted timers.
// This corresponds to the condition below where
// we decide whether to call clearDeletedTimers.
// If the next fire time has not arrived, note a small detail:
// when the number of timers to delete is less than 1/4, do nothing
// and return; wait to handle them in a batch.
if pp != getg().m.p.ptr() || int(atomic.Load(&pp.deletedTimers)) <= int(atomic.Load(&pp.numTimers)/4) {
return now, next, false
}
}
lock(&pp.timersLock)
if len(pp.timers) > 0 {
// Adjust the current p's timer heap; this method has many details
// that are not expanded here.
adjusttimers(pp, now)
for len(pp.timers) > 0 {
// Note that runtimer may temporarily unlock
// pp.timersLock.
// If a timer needs to run, call runtimer to execute it.
if tw := runtimer(pp, now); tw != 0 {
if tw > 0 {
pollUntil = tw
}
break
}
ran = true
}
}
// If this is the local P, and there are a lot of deleted timers,
// clear them out. We only do this for the local P to reduce
// lock contention on timersLock.
if pp == getg().m.p.ptr() && int(atomic.Load(&pp.deletedTimers)) > len(pp.timers)/4 {
clearDeletedTimers(pp)
}
unlock(&pp.timersLock)
return now, pollUntil, ran
}
runtimer checks the root's status and calls runOneTimer once the time has arrived:
func runtimer(pp *p, now int64) int64 {
for {
// Get the heap root.
t := pp.timers[0]
if t.pp.ptr() != pp {
throw("runtimer: bad p")
}
switch s := atomic.Load(&t.status); s {
case timerWaiting:
if t.when > now {
// Not ready to run.
return t.when
}
if !atomic.Cas(&t.status, s, timerRunning) {
continue
}
// Note that runOneTimer may temporarily unlock
// pp.timersLock.
// If it is time to fire, run this timer.
runOneTimer(pp, t, now)
return 0
case timerDeleted:
// .....................
case timerModifiedEarlier, timerModifiedLater:
// .....................
case timerModifying:
osyield()
case timerNoStatus, timerRemoved:
badTimer()
case timerRunning, timerRemoving, timerMoving:
badTimer()
default:
badTimer()
}
}
}
runOneTimer mirrors the old timerproc logic: periodic timers stay in the heap and get their next when, one-shot timers are removed, and the function is called outside the lock:
func runOneTimer(pp *p, t *timer, now int64) {
// .....................
f := t.f
arg := t.arg
seq := t.seq
// The logic here matches the old timerproc.
if t.period > 0 {
// Leave in heap but adjust next time to fire.
delta := t.when - now
t.when += t.period * (1 + -delta/t.period)
if t.when < 0 { // check for overflow.
t.when = maxWhen
}
siftdownTimer(pp.timers, 0)
if !atomic.Cas(&t.status, timerRunning, timerWaiting) {
badTimer()
}
updateTimer0When(pp)
} else {
// Remove from heap.
dodeltimer0(pp)
if !atomic.Cas(&t.status, timerRunning, timerNoStatus) {
badTimer()
}
}
// .....................
unlock(&pp.timersLock)
// This is where the timer function is actually executed.
f(arg, seq)
lock(&pp.timersLock)
// .....................
}
There is one more case: when a P is destroyed, its timers cannot simply disappear. They are moved to another P:
func (pp *p) destroy() {
assertLockHeld(&sched.lock)
assertWorldStopped()
// .....................
if len(pp.timers) > 0 {
// Find another P.
plocal := getg().m.p.ptr()
// The world is stopped, but we acquire timersLock to
// protect against sysmon calling timeSleepUntil.
// This is the only case where we hold the timersLock of
// more than one P, so there are no deadlock concerns.
lock(&plocal.timersLock)
lock(&pp.timersLock)
// Move all timers on the current P away.
moveTimers(plocal, pp.timers)
// .....................
}
}
What actually changed between versions
Two changes stand out. Storage moved from global buckets to per-P heaps. Triggering moved from a dedicated timerproc goroutine to scheduler checks. The first reduces lock contention; the second avoids repeated M/P binding and unbinding in the timer path.
In the old design, the goroutine running timerproc had to be scheduled, and the timer path involved binds and unbinds between M and P. With many timers, especially a ticker firing every second, that switching becomes a cost. In the newer design, timers live close to the P and are checked in the scheduling loop. There is no separate timerproc goroutine to maintain, and no repeated bind/unbind just to fire a timer.
The heap itself is still the core idea: the root is the next timer to fire. Deletion and modification are deferred through status flags because another P may own the heap; the owner cleans them up later. This lazy handling is part of why the newer code has more statuses and why checkTimers sometimes waits until enough deleted timers accumulate before doing cleanup.
The implementation is not mysterious once the two layers are separated: a heap orders deadlines, and the scheduler decides when to look at the heap. Older Go used a global bucket layout and a dedicated goroutine; newer Go moved the heap onto P and folded checks into scheduling. The evolution is a good example of a simple heap-based design being reshaped for concurrency and scheduling costs.