I’m in the process of re-configuring my home lab and would like to get some help figuring out log collection. My setup was a hodgepodge of systems/OSes using rsyslog to send syslogs to a syslog listener on my qnap but that’s not going to work anymore (partly because the qnap is gone).
My end-goal is going to be as homogeneous as I can manage. Mostly Debian 12 systems (phy and vm) and Docker containers. Does anyone know of a FOSS solution that can ingest journald, syslog, and if it’s even possible to send docker logs to a log collector?
Thanks
You can also take a look at open telemetry. It’s a huge open source project with lots of functionality. Handles logs just fine and also can provide metrics and traces too. Might be overkill for your needs but it’s an excellent tool.
I’ll check it out. Thanks.
I use a Graylog/Opensearch/Mongodb stack to log everything. I spent a good amount of time writing parsers for each source, but the benefit is that everything is normalized to make searching easier. I’m happy with it as a solution!
I also use graylog to aggregate logs from various devices (mostly from rsyslog over SSL/TLS). The only downsides for me are the license (not a big problem for a personal setup), and resource usage of the general graylog/elasticsearch stack. I still think it’s great.
I use this ansible role to install and manage it.
For simpler setups with resource constraints, I would simply use a rsyslog server as aggregator instead of graylog, and lnav for the analysis/filtering/parsing part
FWIW I use an elastic stack for that: filebeat, journalbeat to collect logs. Logstash to sort and parse them. Elasticsearch to store them. Not sure if it satisfies your FOSS requirement, as I don’t believe it’s entirely open source.
