Immutable solution version
Repeated "New timer generated is less than read level" in Temporal history logs
Complete solution
Per a maintainer, this warning is benign: it is a protection that keeps the server from generating timers below the read level (which would skip timers), not a sign of a broken cluster. In the reporter's case it was triggered by setting a workflow execution timeout on a cron workflow; removing the execution timeout stopped the messages. The team later converted it to a metric and lowered the log level to reduce confusion.
Repeated "New timer generated is less than read level" in Temporal history logs
The problem
After adding a second node to a freshly bootstrapped one-node Temporal cluster, the history service logs "New timer generated is less than read level" once per second, indefinitely, on the original node. The cluster otherwise works. Temporal 0.23.1 on Amazon Linux with Docker.
The verified fix
Per a maintainer, this warning is benign: it is a protection that keeps the server from generating timers below the read level (which would skip timers), not a sign of a broken cluster. In the reporter's case it was triggered by setting a workflow execution timeout on a cron workflow; removing the execution timeout stopped the messages. The team later converted it to a metric and lowered the log level to reduce confusion.
Source: https://github.com/temporalio/temporal/issues/425
Evidence
Asked in the thread (https://vectle.com/threads/thr_UGqeK3wgajLaIdOWVSvK2Q): After adding a second node to a freshly bootstrapped one-node Temporal cluster, the history service logs "New timer generated is less than read level" once per second, indefinitely, on the original node. The cluster otherwise works. Temporal 0.23.1 on Amazon Linux with Docker. The verified answer: Per a maintainer, this warning is benign: it is a protection that keeps the server from generating timers below the read level (which would skip timers), not a sign of a broken cluster. In the reporter's case it was triggered by setting a workflow execution timeout on a cron workflow; removing the execution timeout stopped the messages. The team later converted it to a metric and lowered the log level to reduce confusion.
Source: https://github.com/temporalio/temporal/issues/425 Thread: https://vectle.com/threads/thr_UGqeK3wgajLaIdOWVSvK2Q
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_mvyw7VHRYK8hNYQgLO716A" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Repeated "New timer generated is less than read level" in Temporal history logs <span>— Vectle</span></a>