mini-siem-elk

🛡️ Mini SIEM with ELK Stack

This project demonstrates how to set up a Mini Security Information and Event Management (SIEM) system using the ELK Stack, Filebeat, and Winlogbeat across multiple virtual machines.

📦 What is ELK Stack?

The ELK Stack is a powerful open-source log analysis platform composed of three main tools:

Tool	Description
Elasticsearch	A distributed, RESTful search and analytics engine to store and index logs
Logstash	A data processing pipeline for ingesting, transforming, and forwarding logs
Kibana	A UI for querying, visualizing, and exploring logs stored in Elasticsearch

🧩 Project Architecture

This Mini SIEM project is built using multiple virtual machines:

• Ubuntu Server 1 – Runs Elasticsearch, Logstash, and Kibana
• Ubuntu Server 2 – Sends logs using Filebeat (CSV, Apache)
• Windows 10 VM – Sends event logs using Winlogbeat
• Host Machine – Used to access Kibana dashboards via browser

🔄 Integration Overview

🔸 Kibana + Elasticsearch (Sequence)

• Enrollment Token: Auto-generated when Elasticsearch starts
• Username/Password: Provided during initial Elasticsearch launch
• Host Access: Configured via elasticsearch.yml and kibana.yml with VirtualBox Port Forwarding

👉 Setup Guide: 01-install-elasticsearch.md & 02-install-kibana.md

🔸 Filebeat Integration – CSV Data

• Filebeat reads from a .csv file and sends logs to Logstash.
• Logstash parses and forwards the data to Elasticsearch.
• Data is visualized in Kibana.

👉 Setup Guide: 04-integrate-filebeat.md

🔸 Filebeat Integration – Apache Logs

• Filebeat collects Apache access and error logs.
• Logstash filters logs using Grok and GeoIP plugins.
• Logs are indexed in Elasticsearch and explored via Kibana dashboards.

🔸 Filebeat → Logstash → Elasticsearch (Sequence Diagram)

🪟 Winlogbeat Integration (Windows Event Logs)

Winlogbeat is used to collect and forward Windows event logs to Logstash, which then indexes them in Elasticsearch. The data is visualized via Kibana.

👉 Setup Guide: 05-integrate-winlogbeat.md

⚠️ Problems Faced and Solutions

Problem Area	Description
Kibana Integration	Confusion about where to get enrollment token, username/password
Host Access Issues	Required editing network settings and config files (network.host, ports)
Logstash + Elasticsearch	Trouble establishing secure connection and formatting correct logstash.conf
Filebeat Configuration	Managing log input types, paths, and matching output settings in Logstash
Index Conflicts or Permissions	Ensuring correct index patterns and credentials in all services

⚙️ Component Setup Links

Component	Guide File
Elasticsearch	01-install-elasticsearch.md
Kibana	02-install-kibana.md
Logstash	03-install-logstash.md
Filebeat (Linux)	04-integrate-filebeat.md
Winlogbeat (Windows)	05-integrate-winlogbeat.md

🧰 Features Demonstrated

• ✅ Full setup of the ELK Stack (Elasticsearch, Logstash, Kibana)
• ✅ Log forwarding with Filebeat and Winlogbeat
• ✅ Data enrichment with Logstash filters (e.g., Grok, GeoIP)
• ✅ Visualization of custom log data (CSV, Apache, Windows logs)
• ✅ Port forwarding setup and secure VM-to-VM communication
• ✅ Basic filtering and dashboarding in Kibana

📁 Folder Structure

📁 mini-siem-elk/
├── images/
│   ├── 01-install-elasticsearch/
│   ├── 02-install-kibana/
│   ├── 03-install-logstash/
│   ├── 04-integrate-filebeat/
│   ├── 05-integrate-winlogbeat/
│   └── *.png (summary diagrams)
│
├── setup/
│   ├── 01-install-elasticsearch.md
│   ├── 02-install-kibana.md
│   ├── 03-install-logstash.md
│   ├── 04-integrate-filebeat.md
│   ├── 05-integrate-winlogbeat.md
│   └── README.md

🎯 Learning Outcome

This project helped reinforce several key cybersecurity and DevOps concepts:

• Building a full-featured log monitoring pipeline
• Cross-VM log shipping using Filebeat and Winlogbeat
• Writing custom Logstash pipelines
• Troubleshooting issues in distributed logging setups
• Real-time data visualization with filters and dashboards

🚀 Next Steps

You can extend this project by:

• Adding alerting via ElastAlert or Kibana rules
• Sending Syslog/Firewall logs
• Integrating with tools like Wazuh for host intrusion detection
• Monitoring metrics using Metricbeat

This site is open source. Improve this page.