Per Stenebo
2017-01-25 05:55:15
2022-02-03 16:44:57
Auto-reboot a hung Raspberry Pi using the on-board watchdog timer
Raspberry Pi OS 11 Bullseye (and 10 Buster) with systemD
Debian manpages: | watchdog.conf | systemd-system.conf.5 |
Tutorial: | Raspberry Pi watchdog made simple |
systemd service
Check that the devices exist:
ls -al /dev/watchdog* it should result in something like:
crw------- 1 root root 10, 130 2 feb 21.03 /dev/watchdog crw------- 1 root root 250, 0 2 feb 21.03 /dev/watchdog0
Check default settings given from systemd: systemctl show | grep -i watchdog
RuntimeWatchdogUSec=0 RebootWatchdogUSec=10min KExecWatchdogUSec=0 ServiceWatchdogs=yes
The line RuntimeWatchdogUSec=0 tells us that the watchdog isn't active. Must be greater than 0.
Enable the watchdog by editing sudo nano /etc/systemd/system.conf
where the different parameters are described here and in my case I choose
RuntimeWatchdogSec=14 ShutdownWatchdogSec=2min
which means that if the watchdog hardware isn't contacted within 14 seconds the reboot process will start and the second parameter ensures that the reboot takes place even if a clean reboot attempt times out.
Please note that the value "RuntimeWatchdogSec" must be <= 15, since the processors can't handle a greater value.
Reboot the device and check statup messages: dmesg | grep watchdog wich should result in something like this:
[ 3.019795] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer [ 4.228181] systemd[1]: Using hardware watchdog 'Broadcom BCM2835 Watchdog timer', version 0, device /dev/watchdog [ 4.231766] systemd[1]: Set hardware watchdog to 15s.
Test
Test watchdog with a fork bomb (less reliable) or cause a kernel panic (if you dare):
sudo su echo 1 > /proc/sys/kernel/sysrq echo "c" > /proc/sysrq-trigger
Should result in: client_loop: send disconnect: Broken pipe
The software watchdog
Install the software watchdog for extended functionality: sudo apt install watchdog
Edit the configuration file: sudo nano /etc/watchdog.conf
Uncomment line and modify to watchdog-timeout=15
The default timeout is 60 s, wich is reported not to work in Raspberry Pi/Raspbian.
Enable some checks, at least uncomment max-load-5
Check status with systemctl status watchdog
RAM: Check number of pages (pagesize): getconf PAGESIZE -> 4096 bytes ~= 4 kB ~= 0.004 MB
Raspbian Stretch (and Jessie) with systemD
Debian manpages: | watchdog.conf |
| forgeeks.com | domoticz.com | raspberrypi.org/forums |
Instructions for systemD systems with kernel module bcm2835-wdt.
Check that the kernel module are enabled:
ls -al /dev/watchdog* it should result in something like:
crw------- 1 root root 10, 130 nov 10 22:13 /dev/watchdog crw------- 1 root root 252, 0 nov 10 22:13 /dev/watchdog0
If not, enable it:
sudo nano /boot/config.txt and add: dtparam=watchdog=on and reboot.
Install the software watchdog: sudo apt install watchdog
Edit the configuration file: sudo nano /etc/watchdog.conf
Enable some checks, at least uncomment max-load-5
and add the line: watchdog-timeout=15
The default timeout is 60 s, wich is reported not to work in Raspberry Pi/Raspbian.
If timeout is not modified, a sudo systemctl status watchdog might give something like this:
cannot set timeout 60 (errno = 22 = 'Invalid argument')
RAM: Check number of pages (pagesize): getconf PAGESIZE -> 4096 bytes ~= 4 kB ~= 0.004 MB
Test watchdog with a fork bomb.
Raspbian Wheezy
| bayerschmidt.com |
Instructions for pre-systemD versions with kernel module bcm2708_wdog, tested on Raspian Wheezy.
Install the watchdog daemon
sudo apt-get install watchdog
For me it installed watchdog_5.12-1_armhf.deb (Wheezy 2017-01-24)
Set the daemon to start at boot
sudo update-rc.d watchdog defaults or sudo chkconfig --add watchdog
Load the bcm2708_wdog kernel module
sudo modprobe bcm2708_wdog
sudo nano /etc/modules
Add: bcm2708_wdog
Configure the watchdog daemon
sudo nano /etc/watchdog.conf
Uncomment the following:
max-load-1 = 24 watchdog-device
Start the watchdog daemon
sudo service watchdog start or sudo /etc/init.d/watchdog start