Install a syslog server on Ubuntu (rsyslog)

careful intermediate #rsyslog#ubuntu#install#linux#udp#tcp#514

When to use this: You want a free, reliable syslog server on Ubuntu that stores each device's logs in its own daily file.

Be careful with this one. You are opening a network port. Allow only your senders' IPs in the firewall.

Before you start

Steps

  1. Install rsyslog (it is usually preinstalled)

    terminal
    sudo apt update
    sudo apt install -y rsyslog

    Ubuntu normally ships rsyslog already. This makes sure it is present and current.

  2. Create the folder for remote logs

    terminal
    sudo mkdir -p /var/log/remote
    sudo chown syslog:adm /var/log/remote
    sudo chmod 750 /var/log/remote

    rsyslog runs as user syslog and cannot create folders directly under /var/log, so create it first.

  3. Add the receiver configuration

    terminal
    sudo tee /etc/rsyslog.d/10-remote.conf > /dev/null <<'EOF'
    # Listen for syslog from the network
    module(load="imudp")
    input(type="imudp" port="514")
    module(load="imtcp")
    input(type="imtcp" port="514")
    
    # One folder per sender IP, one file per day
    template(name="PerHostDaily" type="string"
             string="/var/log/remote/%fromhost-ip%/%$year%-%$month%-%$day%.log")
    # Keep arrival time and sender IP, then the original line
    template(name="LineFmt" type="string"
             string="%timegenerated:::date-rfc3339% %fromhost-ip% %rawmsg%\n")
    
    # Everything not from this machine goes to the per-host files and stops there
    if $fromhost-ip != '127.0.0.1' then {
        action(type="omfile" dynaFile="PerHostDaily" template="LineFmt"
               dirCreateMode="0750" fileCreateMode="0640")
        stop
    }
    EOF

    Opens UDP and TCP 514, writes each sender to /var/log/remote/<sender-ip>/<date>.log, and keeps those messages out of the local /var/log/syslog. The file starts with 10- so it runs before Ubuntu's default rules.

    Check the heredoc kept its indentation. If you pasted with leading spaces, rsyslog still accepts them, but the EOF line must be at the start of the line.

  4. Check the configuration syntax

    terminal
    sudo rsyslogd -N1

    Validates the config without restarting. Fix any error before continuing.

    Expected output
    rsyslogd: version 8.x, config validation run (level 1), master config /etc/rsyslog.conf
    rsyslogd: End of config validation run. Bye.
  5. Restart and confirm it is listening

    terminal
    sudo systemctl restart rsyslog
    sudo ss -lunp | grep ':514'
    sudo ss -ltnp | grep ':514'

    -lun lists UDP listeners and -ltn TCP ones. You should see rsyslogd on port 514 in both.

  6. Open the firewall for your senders only

    terminal
    sudo ufw allow from 192.168.1.1 to any port 514 proto udp
    sudo ufw allow from 192.168.1.1 to any port 514 proto tcp
    sudo ufw status

    Replace 192.168.1.1 with each sender's IP. Skip this if ufw is not enabled, but then make sure another firewall limits access.

  7. Send a test message from another machine

    terminal
    logger -n 192.168.1.10 -P 514 -d "syslog test from $(hostname)"

    Run it on any Linux box (replace the server IP). -d uses UDP. Then read the file on the server in the next step.

  8. Read the received logs

    terminal
    ls /var/log/remote/
    sudo tail -f /var/log/remote/*/$(date +%Y-%m-%d).log

    You should see a folder per sender IP and new lines arriving. Press Ctrl+C to stop.

  9. Set up retention (compress after 1 day, delete after 90)

    terminal
    sudo tee /etc/cron.daily/remote-logs-clean > /dev/null <<'EOF'
    #!/bin/sh
    find /var/log/remote -type f -name '*.log' -mtime +1 -exec gzip -q {} \;
    find /var/log/remote -type f -name '*.gz' -mtime +90 -delete
    EOF
    sudo chmod +x /etc/cron.daily/remote-logs-clean

    Change 90 to your retention in days. Text logs typically compress very well, which saves disk.

How to verify it worked

  • ss shows rsyslogd on port 514 (UDP and TCP).
  • A test message creates /var/log/remote/<sender-ip>/<today>.log.
  • The real device (for example the FortiGate) shows up with its own folder.

How to undo it

Disable remote reception and restart:

terminal
sudo rm /etc/rsyslog.d/10-remote.conf
sudo systemctl restart rsyslog

Common errors and fixes

Nothing listening on 514 after restart Run sudo rsyslogd -N1 and journalctl -u rsyslog -n 50. Usually a typo in the config or another syslog service holds the port.
Test works locally but the device's logs never arrive Check the firewall (ufw status), the device's source IP, and watch the wire with sudo tcpdump -ni any port 514. See the 'no logs arriving' scenario.
Permission denied creating files Fix ownership: sudo chown -R syslog:adm /var/log/remote.
The disk fills up Retention is too long for the volume. Check du -sh /var/log/remote/*, reduce retention, or add disk. Consider a separate disk mounted on /var/log/remote.

Q&A

Why put the file in rsyslog.d/10-remote.conf?

Files in rsyslog.d are read in name order. A low number makes your rule run before the defaults, so remote messages are handled (and stopped) before they land in the local syslog file.

Why use the sender IP for the folder name?

Many network devices do not put a proper hostname in the syslog header, so %HOSTNAME% can be wrong or blank. The source IP is always known.

UDP or TCP?

The config accepts both. Let each device choose. Use TCP where losing messages matters.

Practice these as flashcards →

Terms in this scenario

Related scenarios

My notes

Saved in this browser only.