Automatically Restart MariaDB Galera On Failure Ubuntu

Database replication is a great tool to have in any Linux Server Administrators’ toolbox. The ability to spread data over multiple hosts is the cornerstone of any type of high availability. However, as with any type of technology, problems do happen. For the most part, problems can be resolved by restarting the system that has failed. In Linux, you can restart most services with systemd. By default, MariaDB Galera Hosts do not restart automatically on failure. This can be a problem because if too many of your MariaDB Glaera hosts go offline, that will cause MariaDB Galera to crash on your cluster. In this guide, we will learn how to automatically restart MariaDB Galera on failure or any other type of crash on Ubuntu systems.

MariaDB SystemD Service

In the default Ubuntu MariaDB Galera setup, the service file is located at /etc/systemd/system/mariadb.service. You can check this by running systemctl status mariadb and inspecting the “Loaded” line. That’s line 2 below.

mariadb.service - MariaDB 10.6.16 database server
     Loaded: loaded (/etc/systemd/system/mariadb.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2024-04-19 10:12:44 UTC; 17min ago
       Docs: man:mariadbd(8)
             https://mariadb.com/kb/en/library/systemd/
    Process: 575179 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS)
    Process: 575180 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
    Process: 575182 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`cd /usr/bin/..; /usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP_STAR>
    Process: 575479 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
    Process: 575481 ExecStartPost=/etc/mysql/debian-start (code=exited, status=0/SUCCESS)
   Main PID: 575280 (mariadbd)
     Status: "Taking your SQL requests now..."
      Tasks: 34 (limit: 1023)
     Memory: 148.7M
        CPU: 1.385s
     CGroup: /system.slice/mariadb.service
             └─575280 /usr/sbin/mariadbd --wsrep_start_position=e8eee71f-d664-11ee-8fde-

So we want systemd to automatically restart this MariaDB Galera Host if MariaDB should fail or otherwise crash. First, copy this file to the /etc/systemd/system folder.

cp /etc/systemd/system/mariadb.service /etc/systemd/system

Automatically Restart MariaDB Galera on Failure in Ubuntu

Now, in your NVMe VPS open up the file you just copied.

nano /etc/systemd/system/mariadb.service

Navigate down to the [Service] section and copy to below lines to the service section. If any exist already. Just comment them out with a #.

Restart=always
RestartSec=5s
Type=notify

You can use on-failure instead of always but this won’t catch all scenarios where MariaDB Galera may crash. You can also adjust the RestartSec= if you want to wait a longer period before restarting the failed MariaDB Galera host. Save and close the file then, restart the services daemon and MariaDB.

systemctl daemon-reload && systemctl restart mariadb

Test MariaDB Galera Restarts Automatically on Failure

Finally, test that MariaDB Galera will restart automatically on failure. Kill the MariaDB service with SIGSEGV. If you just kill the service this won’t produce an error. Use SIGSEGV to simulate a MariaDB Galera Host crash.

systemctl kill -s SIGSEGV mariadb.service

If you are using a LoadBalancer like HAProxy you should see MariaDB reporting it is down then back up again. MariaDB Galera automatically restarted after it failed.