commit 6ce487735fb5be7ac486963651ed9f3d07f81bcf Author: Dmitry Gammel Date: Sun Jul 26 16:14:36 2026 +0500 Scaffold repo: infra docker-compose, service layout, README Sets up the microservice home-automation platform skeleton: docker-compose for mosquitto/postgres/clickhouse/redis/rabbitmq, per-service README stubs describing responsibilities, and top-level README with architecture/stack. Co-Authored-By: Claude Sonnet 5 diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..0a341c6 --- /dev/null +++ b/.env.example @@ -0,0 +1,37 @@ +# --- PostgreSQL (users, zones, devices, device_types, automation_rules) --- +POSTGRES_HOST=postgres +POSTGRES_PORT=5432 +POSTGRES_DB=home_automation +POSTGRES_USER=home_automation +POSTGRES_PASSWORD=change_me + +# --- ClickHouse (telemetry, time-series) --- +CLICKHOUSE_HOST=clickhouse +CLICKHOUSE_HTTP_PORT=8123 +CLICKHOUSE_NATIVE_PORT=9000 +CLICKHOUSE_DB=telemetry +CLICKHOUSE_USER=default +CLICKHOUSE_PASSWORD=change_me + +# --- Redis (device shadow: desired/reported state, status, last_seen) --- +REDIS_HOST=redis +REDIS_PORT=6379 +REDIS_PASSWORD= + +# --- RabbitMQ (async events between Go services) --- +RABBITMQ_HOST=rabbitmq +RABBITMQ_PORT=5672 +RABBITMQ_MANAGEMENT_PORT=15672 +RABBITMQ_USER=home_automation +RABBITMQ_PASSWORD=change_me + +# --- Mosquitto (MQTT broker, device <-> ingest/device-control) --- +MQTT_HOST=mosquitto +MQTT_PORT=1883 + +# --- device-control-service (gRPC) --- +DEVICE_CONTROL_GRPC_PORT=50051 + +# --- Laravel app (stage 2) --- +APP_KEY= +APP_URL=http://localhost:8000 diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..e3fea65 --- /dev/null +++ b/.gitignore @@ -0,0 +1,25 @@ +# Go +services/**/bin/ +services/**/*.exe +*.test +*.out + +# Laravel +laravel-app/vendor/ +laravel-app/node_modules/ +laravel-app/.env +laravel-app/storage/*.key +laravel-app/bootstrap/cache/* +!laravel-app/bootstrap/cache/.gitkeep + +# Env / secrets +.env +.env.local + +# Docker volumes data (if bind-mounted locally) +volumes/ + +# IDE / OS +.idea/ +.vscode/ +.DS_Store diff --git a/README.md b/README.md new file mode 100644 index 0000000..286cbde --- /dev/null +++ b/README.md @@ -0,0 +1,100 @@ +# Home Automation Platform + +A microservice-based home automation platform, built as a portfolio project +demonstrating Go (concurrency, message queues, MQTT), PHP/Laravel (API, +authorization), and polyglot persistence (PostgreSQL, ClickHouse, Redis). + +**This is not a "growbox project."** A growbox is the first implemented +**zone** with a first set of **device types** (light, pump, fan, temperature +sensor). The core system — device management, telemetry, automation rules, +users, notifications — operates purely on the abstractions *zone*, *device*, +*device type*, *sensor reading*, *command*, *rule*. Nothing growbox-specific +is hardcoded in the core: adding a "living room" zone with a "smart plug" +device type should require zero changes to service code, only new rows in +`device_types` / `zones` / `devices`. + +## Architecture + +``` +ESP32 (or emulator) + │ MQTT + ▼ +ingest-service (Go) ──► ClickHouse (raw telemetry) + │ + ▼ RabbitMQ ("new reading" event) +rule-engine-service (Go) ──► PostgreSQL (rules, cached) + │ gRPC (on rule match) + ▼ +device-control-service (Go) ──► MQTT (command) ──► device + │ + ▼ +Redis (device shadow: desired/reported state, status, last_seen) + +Laravel API (PHP) ──► PostgreSQL / ClickHouse / Redis + │ gRPC / HTTP + ▼ +device-control-service (manual commands from the UI) +``` + +## Stack + +| Concern | Technology | +|---|---| +| Device firmware / emulator | Go (`services/esp32-emulator`) | +| Device ↔ server protocol | MQTT (Mosquitto) | +| Telemetry / commands / rules backend | Go | +| Business logic / API / dashboard | PHP (Laravel) | +| Users, zones, devices, rules | PostgreSQL | +| Telemetry (time-series) | ClickHouse | +| Real-time device state | Redis (Device Shadow pattern) | +| Inter-service async events | RabbitMQ | +| Inter-service sync calls | gRPC / Protobuf | +| Metrics (stage 2.5) | Prometheus + Grafana | +| Containerization | Docker Compose | +| Web frontend (stage 3) | Vue.js + Inertia.js | +| Mobile app (stage 3) | Flutter | +| Notifications | Telegram Bot (MVP) / Firebase Cloud Messaging (later) | + +## Layout + +``` +services/ + ingest-service/ Go — MQTT → ClickHouse + RabbitMQ + device-control-service/ Go — gRPC + MQTT + Redis device shadow + rule-engine-service/ Go — RabbitMQ consumer + rule evaluation + health-check-service/ Go — offline detection ticker + esp32-emulator/ Go — fake device for local dev +laravel-app/ PHP/Laravel — API, auth, dashboard (stage 2) +proto/ Shared gRPC contracts (device_control.proto) +migrations/ + postgres/ users, zones, devices, device_types, automation_rules + clickhouse/ telemetry table +configs/mosquitto/ MQTT broker config +monitoring/ Prometheus + Grafana config (stage 2.5) +``` + +Each `services/*/README.md` describes that service's responsibility and +explicit non-responsibilities before any code exists there. + +## Build stages + +1. **Go services** — ingest, rule-engine, device-control, ESP32 emulator. +2. **Laravel** — auth (Sanctum, owner/viewer), CRUD for zones/devices/rules, + Blade dashboard, manual device control. MVP is considered done here. +3. *(optional, high value)* **Observability** — Prometheus metrics from the + Go services, a Grafana dashboard or two. +4. **Extensions** (no rush) — Telegram notifications, Vue.js/Inertia + frontend, Flutter app, real ESP32 hardware. + +## Running locally + +Infrastructure only for now (app services are added to `docker-compose.yml` +as they're implemented): + +```bash +cp .env.example .env +docker compose up -d +``` + +This brings up Mosquitto (`1883`), PostgreSQL (`5432`), ClickHouse (`8123`/`9000`), +Redis (`6379`), and RabbitMQ (`5672`, management UI on `15672`). diff --git a/configs/mosquitto/mosquitto.conf b/configs/mosquitto/mosquitto.conf new file mode 100644 index 0000000..6279187 --- /dev/null +++ b/configs/mosquitto/mosquitto.conf @@ -0,0 +1,12 @@ +listener 1883 +protocol mqtt + +# Local dev / MVP only: no auth on the broker. +# Before any real-device or public deployment, switch to per-device +# username/password or client certificates and set allow_anonymous false. +allow_anonymous true + +persistence true +persistence_location /mosquitto/data/ + +log_dest stdout diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..bc42a55 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,91 @@ +networks: + home-automation: + driver: bridge + +volumes: + postgres-data: + clickhouse-data: + redis-data: + rabbitmq-data: + mosquitto-data: + +services: + mosquitto: + image: eclipse-mosquitto:2 + ports: + - "1883:1883" + volumes: + - ./configs/mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro + - mosquitto-data:/mosquitto/data + networks: + - home-automation + restart: unless-stopped + + postgres: + image: postgres:16-alpine + environment: + POSTGRES_DB: ${POSTGRES_DB} + POSTGRES_USER: ${POSTGRES_USER} + POSTGRES_PASSWORD: ${POSTGRES_PASSWORD} + ports: + - "${POSTGRES_PORT}:5432" + volumes: + - postgres-data:/var/lib/postgresql/data + - ./migrations/postgres:/docker-entrypoint-initdb.d:ro + networks: + - home-automation + healthcheck: + test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"] + interval: 5s + timeout: 5s + retries: 10 + restart: unless-stopped + + clickhouse: + image: clickhouse/clickhouse-server:24-alpine + environment: + CLICKHOUSE_DB: ${CLICKHOUSE_DB} + CLICKHOUSE_USER: ${CLICKHOUSE_USER} + CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD} + ports: + - "${CLICKHOUSE_HTTP_PORT}:8123" + - "${CLICKHOUSE_NATIVE_PORT}:9000" + volumes: + - clickhouse-data:/var/lib/clickhouse + - ./migrations/clickhouse:/docker-entrypoint-initdb.d:ro + networks: + - home-automation + restart: unless-stopped + + redis: + image: redis:7-alpine + ports: + - "${REDIS_PORT}:6379" + volumes: + - redis-data:/data + networks: + - home-automation + restart: unless-stopped + + rabbitmq: + image: rabbitmq:3-management-alpine + environment: + RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER} + RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD} + ports: + - "${RABBITMQ_PORT}:5672" + - "${RABBITMQ_MANAGEMENT_PORT}:15672" + volumes: + - rabbitmq-data:/var/lib/rabbitmq + networks: + - home-automation + healthcheck: + test: ["CMD", "rabbitmq-diagnostics", "-q", "ping"] + interval: 10s + timeout: 5s + retries: 10 + restart: unless-stopped + + # App services (ingest-service, device-control-service, rule-engine-service, + # health-check-service, esp32-emulator, laravel-app) are added here as they + # get implemented — see services/*/README.md for the plan for each one. diff --git a/laravel-app/.gitkeep b/laravel-app/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/migrations/clickhouse/.gitkeep b/migrations/clickhouse/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/migrations/postgres/.gitkeep b/migrations/postgres/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/monitoring/grafana/.gitkeep b/monitoring/grafana/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/monitoring/prometheus/.gitkeep b/monitoring/prometheus/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/proto/.gitkeep b/proto/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/services/device-control-service/README.md b/services/device-control-service/README.md new file mode 100644 index 0000000..248801d --- /dev/null +++ b/services/device-control-service/README.md @@ -0,0 +1,20 @@ +# device-control-service (Go) + +Status: not implemented yet. + +Responsibility: +- Expose a gRPC API (see `proto/device_control.proto`) for issuing commands + to devices — called by Laravel (manual control) and rule-engine-service + (rule-triggered actions). +- Publish the command to the device's MQTT topic (`devices/{device_id}/commands`) + and listen for acknowledgements (`devices/{device_id}/ack`). +- Maintain the Device Shadow pattern in Redis: `desired_state` (what the user + wants) vs `reported_state` (what the device confirmed), plus `status` and + `last_seen`. +- Owns the health-check loop (or delegates to health-check-service): a ticker + goroutine that flips a device to `offline` when `last_seen` exceeds a + timeout, and emits an event for notification-service. + +Not this service's job: deciding *when* to send a command based on sensor +thresholds — that's rule-engine-service's logic. This service only executes +and tracks state for whatever command it's given. diff --git a/services/esp32-emulator/README.md b/services/esp32-emulator/README.md new file mode 100644 index 0000000..c736b1b --- /dev/null +++ b/services/esp32-emulator/README.md @@ -0,0 +1,11 @@ +# esp32-emulator (Go) + +Status: not implemented yet. + +Responsibility: +- Stand in for real ESP32 hardware during MVP development. +- Publish plausible telemetry (temperature/humidity/etc.) to + `devices/{device_id}/telemetry` over MQTT on an interval. +- Subscribe to `devices/{device_id}/commands` and reply on + `devices/{device_id}/ack`, so device-control-service has something real + to talk to. diff --git a/services/health-check-service/README.md b/services/health-check-service/README.md new file mode 100644 index 0000000..3b31cd1 --- /dev/null +++ b/services/health-check-service/README.md @@ -0,0 +1,11 @@ +# health-check-service (Go) + +Status: not implemented yet. May start as a goroutine inside +device-control-service and get split out into its own container later — +kept as a separate directory from day one so the split is a non-event. + +Responsibility: +- Periodically (ticker) check `last_seen` for every device in Redis. +- When a device exceeds the offline timeout, flip its `status` to `offline` + and publish an event for notification-service. +- Good place to demonstrate goroutines + graceful shutdown in Go. diff --git a/services/ingest-service/README.md b/services/ingest-service/README.md new file mode 100644 index 0000000..b2341da --- /dev/null +++ b/services/ingest-service/README.md @@ -0,0 +1,13 @@ +# ingest-service (Go) + +Status: not implemented yet. + +Responsibility: +- Subscribe to MQTT topics `devices/{device_id}/telemetry`. +- Validate/parse payload (device_id, sensor_type, value, timestamp). +- Batch-write raw readings into ClickHouse `telemetry` table. +- Publish a "new reading" event to RabbitMQ for rule-engine-service. + +Not this service's job: interpreting what a sensor reading *means* (thresholds, +actions) — that belongs to rule-engine-service. This service only ingests and +stores. diff --git a/services/rule-engine-service/README.md b/services/rule-engine-service/README.md new file mode 100644 index 0000000..c36078f --- /dev/null +++ b/services/rule-engine-service/README.md @@ -0,0 +1,18 @@ +# rule-engine-service (Go) + +Status: not implemented yet. + +Responsibility: +- Consume "new reading" events from RabbitMQ (async — delivery reliability + matters more than latency here). +- Load active `automation_rules` for the relevant zone/device from PostgreSQL, + caching in memory/Redis to avoid a DB round-trip per event. +- Evaluate rule conditions generically: `sensor_type X operator value` against + the reading — no hardcoded sensor/device names, everything comes from the + `automation_rules` row. +- On match: call device-control-service over gRPC (needs a fast pass/fail + result) and publish an event to notification-service over RabbitMQ (not + latency-critical). + +Not this service's job: talking to MQTT or Redis directly — device state +changes always go through device-control-service's gRPC API.