Hi all,
I have influxdb 2.7 installed and managed by systemd. Whenever systemd rotates the log files, influxdb is deactivated like other services but it is never scheduled to restart - so the service just goes missing.
In the log below you can see that journald is “Stopped” then “Started”; Mosquitto is “reloaded”; rsyslogd receives a SIGHUP; and nodered gets a “Scheduled restart job”.
Influx however gets “Deactivated successfully” then nothing - and the service is indeed stopped afterwards. Any pointers as to how to resolve this would be really really appreciated!
Apr 23 22:30:01 orangepi4-lts CRON[5244]: pam_unix(cron:session): session opened for user root(uid=0) by (uid=0)
Apr 23 22:30:01 orangepi4-lts CRON[5245]: (root) CMD (/usr/lib/armbian/armbian-truncate-logs)
Apr 23 22:30:04 orangepi4-lts systemd-journald[5056]: Journal stopped
Apr 23 22:30:04 orangepi4-lts systemd-journald[5056]: Received SIGTERM from PID 1 (systemd).
Apr 23 22:30:04 orangepi4-lts systemd[1]: Stopping Journal Service...
Apr 23 22:30:04 orangepi4-lts systemd[1]: systemd-journald.service: Deactivated successfully.
Apr 23 22:30:04 orangepi4-lts systemd[1]: Stopped Journal Service.
Apr 23 22:30:04 orangepi4-lts systemd[1]: Starting Journal Service...
Apr 23 22:30:04 orangepi4-lts systemd-journald[5267]: Journal started
Apr 23 22:30:04 orangepi4-lts systemd-journald[5267]: Runtime Journal (/run/log/journal/dc154932e447442a8f76454c313ff899) is 4.8M, max 38.6M, 33.8M free.
Apr 23 22:30:04 orangepi4-lts systemd[1]: Started Journal Service.
Apr 23 22:30:04 orangepi4-lts systemd-journald[5267]: Time spent on flushing to /var/log/journal/dc154932e447442a8f76454c313ff899 is 61.067ms for 1779 entries.
Apr 23 22:30:04 orangepi4-lts systemd-journald[5267]: System Journal (/var/log/journal/dc154932e447442a8f76454c313ff899) is 6.2M, max 12.9M, 6.6M free.
Apr 23 22:30:04 orangepi4-lts systemd[1]: Reloading Mosquitto MQTT Broker...
Apr 23 22:30:04 orangepi4-lts systemd[1]: Reloaded Mosquitto MQTT Broker.
Apr 23 22:30:04 orangepi4-lts systemd[1]: rsyslog.service: Sent signal SIGHUP to main process 894 (rsyslogd) on client request.
Apr 23 22:30:04 orangepi4-lts CRON[5244]: (CRON) info (No MTA installed, discarding output)
Apr 23 22:30:04 orangepi4-lts CRON[5244]: pam_unix(cron:session): session closed for user root
Apr 23 22:30:33 orangepi4-lts systemd[1]: nodered.service: Main process exited, code=exited, status=1/FAILURE
Apr 23 22:30:33 orangepi4-lts systemd[1]: nodered.service: Failed with result 'exit-code'.
Apr 23 22:30:33 orangepi4-lts systemd[1]: nodered.service: Consumed 12.246s CPU time.
Apr 23 22:30:53 orangepi4-lts systemd[1]: nodered.service: Scheduled restart job, restart counter is at 15.
Apr 23 22:30:53 orangepi4-lts systemd[1]: Stopped Node-RED graphical event wiring tool.
Apr 23 22:30:53 orangepi4-lts systemd[1]: nodered.service: Consumed 12.246s CPU time.
Apr 23 22:34:44 orangepi4-lts systemd[1]: influxdb.service: Deactivated successfully.
Apr 23 22:34:44 orangepi4-lts systemd[1]: influxdb.service: Consumed 1min 1.198s CPU time.
Apr 23 22:34:50 orangepi4-lts systemd[1]: telegraf.service: Deactivated successfully.
Apr 23 22:34:50 orangepi4-lts systemd[1]: telegraf.service: Consumed 25.042s CPU time.
Apr 23 22:34:50 orangepi4-lts systemd[1]: telegraf.service: Scheduled restart job, restart counter is at 15.
Apr 23 22:34:50 orangepi4-lts systemd[1]: Stopped Telegraf.
Apr 23 22:34:50 orangepi4-lts systemd[1]: telegraf.service: Consumed 25.042s CPU time.
Apr 23 22:34:50 orangepi4-lts systemd[1]: Starting Telegraf...
After systemd has deactivated the service the status is:
> systemctl list-units --all --type=service
influxdb.service loaded inactive dead InfluxDB is an open-source, distributed, time s>
After I manually restart influxdb the status is fine:
influxdb.service loaded active running
My service file looks like this:
cat /etc/systemd/system/influxdb.service
# If you modify this, please also make sure to edit init.sh
[Unit]
Description=InfluxDB is an open-source, distributed, time series database
Documentation=https://docs.influxdata.com/influxdb/
After=network-online.target
[Service]
User=influxdb
Group=influxdb
LimitNOFILE=65536
EnvironmentFile=-/etc/default/influxdb2
ExecStart=/usr/lib/influxdb/scripts/influxd-systemd-start.sh
KillMode=control-group
Restart=on-failure
Type=forking
PIDFile=/var/lib/influxdb/influxd.pid
TimeoutSec=900s
[Install]
WantedBy=multi-user.target
Alias=influxd.service
The journald configuration looks like this:
> cat /etc/systemd/journald.conf
[Journal]
Storage=transient
Compress=yes
#Seal=yes
#SplitMode=uid
#SyncIntervalSec=5m
RateLimitIntervalSec=30s
RateLimitBurst=10000
SystemMaxUse=50M
