This project implements a complete IoT monitoring system using MQTT, Elasticsearch, Logstash, and Kibana (ELK Stack), with ESP32 device support and remote access via ngrok.
- Docker and Docker Compose installed
- Ngrok account
- ESP32 device (optional for hardware testing)
- Arduino IDE (if using ESP32)
- MQTT Broker: Eclipse Mosquitto for message handling
- Elasticsearch: Database for data storage
- Logstash: Data processing pipeline
- Kibana: Data visualization and analysis
- Web Interface: Custom web interface
- Ngrok: Secure tunnel for remote access
-
Clone the repository
git clone [repository-url] cd [directory-name]
-
Start the system
- On Linux:
sh ./start.sh
- On Windows:
start.bat
- On Linux:
-
Verify services
- Kibana: http://localhost:5601
- Elasticsearch: http://localhost:9200
- Web Interface: http://localhost:3000
- Ngrok Interface: http://localhost:4040
- Access Kibana (http://localhost:5601)
- Navigate to Management > Stack Management > Index Patterns
- Create a new index pattern:
- Pattern name:
mqtt_logs*
- Timestamp field:
@timestamp
- Pattern name:
- Save the configuration
-
Send MQTT messages via curl
curl -X POST https://[ngrok-domain]/publish \ -H "Content-Type: application/json" \ -d '{"message": "Test message"}'
-
ESP32 Configuration
- Open the sketch file in Arduino IDE
- Update WiFi credentials
- Update MQTT broker address (from ngrok)
- Upload sketch to device
- Use Kibana to view real-time messages
- Check Docker container logs for debugging
- Monitor ngrok connections from web interface
-
If services fail to start:
docker-compose down docker-compose up --build
-
To reset volumes:
docker-compose down -v
The system uses Docker volumes for data persistence. All configurations are maintained in the respective configuration files in the project directory.