Implement ingest-service: MQTT -> ClickHouse (batched) + RabbitMQ event

Subscribes to devices/+/telemetry, validates payloads, buffers writes into
ClickHouse (flush on size or interval), and emits a per-reading event on
RabbitMQ for rule-engine-service to consume later. Adds the initial Postgres
and ClickHouse schema migrations, a Dockerfile, and wires the service into
docker-compose. Verified end-to-end via docker compose + mosquitto_pub: row
lands in ClickHouse and the event reaches the telemetry.new_reading queue.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-26 16:52:45 +05:00
co-authored by Claude Sonnet 5
parent 6ce487735f
commit 938700cf86
19 changed files with 990 additions and 2 deletions
+7
View File
@@ -32,6 +32,13 @@ MQTT_PORT=1883
# --- device-control-service (gRPC) ---
DEVICE_CONTROL_GRPC_PORT=50051
# --- ingest-service ---
INGEST_MQTT_CLIENT_ID=ingest-service
INGEST_MQTT_TOPIC=devices/+/telemetry
INGEST_BATCH_MAX_SIZE=500
INGEST_BATCH_FLUSH_INTERVAL=5s
INGEST_BATCH_FLUSH_TIMEOUT=10s
# --- Laravel app (stage 2) ---
APP_KEY=
APP_URL=http://localhost:8000