1 Commits
Author SHA1 Message Date
cactoandClaude Sonnet 5 8c5ebe2967 Translate all README files to Russian
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-26 17:06:30 +05:00
24 changed files with 118 additions and 1103 deletions
-7
View File
@@ -32,13 +32,6 @@ MQTT_PORT=1883
# --- device-control-service (gRPC) ---
DEVICE_CONTROL_GRPC_PORT=50051
# --- ingest-service ---
INGEST_MQTT_CLIENT_ID=ingest-service
INGEST_MQTT_TOPIC=devices/+/telemetry
INGEST_BATCH_MAX_SIZE=500
INGEST_BATCH_FLUSH_INTERVAL=5s
INGEST_BATCH_FLUSH_TIMEOUT=10s
# --- Laravel app (stage 2) ---
APP_KEY=
APP_URL=http://localhost:8000
+61 -59
View File
@@ -1,31 +1,32 @@
# Home Automation Platform
# Платформа домашней автоматизации
A microservice-based home automation platform, built as a portfolio project
demonstrating Go (concurrency, message queues, MQTT), PHP/Laravel (API,
authorization), and polyglot persistence (PostgreSQL, ClickHouse, Redis).
Pet-проект на микросервисной архитектуре для портфолио, демонстрирующий Go
(конкурентность, очереди сообщений, MQTT), PHP/Laravel (API, авторизация)
и работу с разными БД по назначению (PostgreSQL, ClickHouse, Redis).
**This is not a "growbox project."** A growbox is the first implemented
**zone** with a first set of **device types** (light, pump, fan, temperature
sensor). The core system — device management, telemetry, automation rules,
users, notifications — operates purely on the abstractions *zone*, *device*,
*device type*, *sensor reading*, *command*, *rule*. Nothing growbox-specific
is hardcoded in the core: adding a "living room" zone with a "smart plug"
device type should require zero changes to service code, only new rows in
**Это не «проект гроубокса».** Гроубокс — первая реализованная **зона**
с первым набором **типов устройств** (свет, полив, вентиляция, датчик
температуры). Ядро системы — управление устройствами, телеметрия, правила
автоматизации, пользователи, уведомления — оперирует исключительно
абстракциями *зона*, *устройство*, *тип устройства*, *показание датчика*,
*команда*, *правило*. Ничего специфичного для гроубокса не зашито в ядре:
добавление зоны «гостиная» с типом устройства «умная розетка» не должно
требовать изменений в коде сервисов — только новые строки в
`device_types` / `zones` / `devices`.
## Architecture
## Архитектура
```
ESP32 (or emulator)
ESP32 (или эмулятор)
│ MQTT
ingest-service (Go) ──► ClickHouse (raw telemetry)
ingest-service (Go) ──► ClickHouse (сырая телеметрия)
▼ RabbitMQ ("new reading" event)
rule-engine-service (Go) ──► PostgreSQL (rules, cached)
│ gRPC (on rule match)
▼ RabbitMQ (событие "новое показание")
rule-engine-service (Go) ──► PostgreSQL (правила, кэшируются)
│ gRPC (при срабатывании правила)
device-control-service (Go) ──► MQTT (command) ──► device
device-control-service (Go) ──► MQTT (команда) ──► устройство
Redis (device shadow: desired/reported state, status, last_seen)
@@ -33,68 +34,69 @@ Redis (device shadow: desired/reported state, status, last_seen)
Laravel API (PHP) ──► PostgreSQL / ClickHouse / Redis
│ gRPC / HTTP
device-control-service (manual commands from the UI)
device-control-service (ручные команды из UI)
```
## Stack
## Стек
| Concern | Technology |
| Назначение | Технология |
|---|---|
| Device firmware / emulator | Go (`services/esp32-emulator`) |
| Device ↔ server protocol | MQTT (Mosquitto) |
| Telemetry / commands / rules backend | Go |
| Business logic / API / dashboard | PHP (Laravel) |
| Users, zones, devices, rules | PostgreSQL |
| Telemetry (time-series) | ClickHouse |
| Real-time device state | Redis (Device Shadow pattern) |
| Inter-service async events | RabbitMQ |
| Inter-service sync calls | gRPC / Protobuf |
| Metrics (stage 2.5) | Prometheus + Grafana |
| Containerization | Docker Compose |
| Web frontend (stage 3) | Vue.js + Inertia.js |
| Mobile app (stage 3) | Flutter |
| Notifications | Telegram Bot (MVP) / Firebase Cloud Messaging (later) |
| Прошивка устройства / эмулятор | Go (`services/esp32-emulator`) |
| Протокол устройство ↔ сервер | MQTT (Mosquitto) |
| Backend телеметрии / команд / правил | Go |
| Бизнес-логика / API / дашборд | PHP (Laravel) |
| Пользователи, зоны, устройства, правила | PostgreSQL |
| Телеметрия (time-series) | ClickHouse |
| Состояние устройств в реальном времени | Redis (паттерн Device Shadow) |
| Асинхронные события между сервисами | RabbitMQ |
| Синхронные вызовы между сервисами | gRPC / Protobuf |
| Метрики (этап 2.5) | Prometheus + Grafana |
| Контейнеризация | Docker Compose |
| Веб-фронтенд (этап 3) | Vue.js + Inertia.js |
| Мобильное приложение (этап 3) | Flutter |
| Уведомления | Telegram-бот (MVP) / Firebase Cloud Messaging (позже) |
## Layout
## Структура репозитория
```
services/
ingest-service/ Go — MQTT → ClickHouse + RabbitMQ
device-control-service/ Go — gRPC + MQTT + Redis device shadow
rule-engine-service/ Go — RabbitMQ consumer + rule evaluation
health-check-service/ Go — offline detection ticker
esp32-emulator/ Go — fake device for local dev
laravel-app/ PHP/Laravel — API, auth, dashboard (stage 2)
proto/ Shared gRPC contracts (device_control.proto)
rule-engine-service/ Go — консьюмер RabbitMQ + оценка правил
health-check-service/ Go — тикер обнаружения офлайн-устройств
esp32-emulator/ Go — фейковое устройство для локальной разработки
laravel-app/ PHP/Laravel — API, авторизация, дашборд (этап 2)
proto/ Общие gRPC-контракты (device_control.proto)
migrations/
postgres/ users, zones, devices, device_types, automation_rules
clickhouse/ telemetry table
configs/mosquitto/ MQTT broker config
monitoring/ Prometheus + Grafana config (stage 2.5)
clickhouse/ таблица telemetry
configs/mosquitto/ Конфиг MQTT-брокера
monitoring/ Конфиги Prometheus + Grafana (этап 2.5)
```
Each `services/*/README.md` describes that service's responsibility and
explicit non-responsibilities before any code exists there.
Каждый `services/*/README.md` описывает зону ответственности сервиса и явно
то, что в неё не входит, — ещё до того, как там появится код.
## Build stages
## Этапы разработки
1. **Go services** — ingest, rule-engine, device-control, ESP32 emulator.
2. **Laravel**auth (Sanctum, owner/viewer), CRUD for zones/devices/rules,
Blade dashboard, manual device control. MVP is considered done here.
3. *(optional, high value)* **Observability** — Prometheus metrics from the
Go services, a Grafana dashboard or two.
4. **Extensions** (no rush) — Telegram notifications, Vue.js/Inertia
frontend, Flutter app, real ESP32 hardware.
1. **Go-сервисы** — ingest, rule-engine, device-control, эмулятор ESP32.
2. **Laravel**авторизация (Sanctum, owner/viewer), CRUD для зон/устройств/
правил, Blade-дашборд, ручное управление устройствами. MVP считается
готовым на этом этапе.
3. *(опционально, но сильно повышает ценность)* **Observability** — метрики
Prometheus с Go-сервисов, один-два дашборда в Grafana.
4. **Расширения** (не спеша) — Telegram-уведомления, фронтенд на
Vue.js/Inertia, приложение на Flutter, подключение реального ESP32.
## Running locally
## Запуск локально
Infrastructure only for now (app services are added to `docker-compose.yml`
as they're implemented):
Пока только инфраструктура (сервисы добавляются в `docker-compose.yml` по
мере реализации):
```bash
cp .env.example .env
docker compose up -d
```
This brings up Mosquitto (`1883`), PostgreSQL (`5432`), ClickHouse (`8123`/`9000`),
Redis (`6379`), and RabbitMQ (`5672`, management UI on `15672`).
Поднимутся Mosquitto (`1883`), PostgreSQL (`5432`), ClickHouse (`8123`/`9000`),
Redis (`6379`) и RabbitMQ (`5672`, веб-интерфейс управления на `15672`).
+1 -31
View File
@@ -55,11 +55,6 @@ services:
- ./migrations/clickhouse:/docker-entrypoint-initdb.d:ro
networks:
- home-automation
healthcheck:
test: ["CMD", "clickhouse-client", "--query", "SELECT 1"]
interval: 5s
timeout: 5s
retries: 10
restart: unless-stopped
redis:
@@ -91,31 +86,6 @@ services:
retries: 10
restart: unless-stopped
ingest-service:
build: ./services/ingest-service
environment:
MQTT_HOST: mosquitto
MQTT_PORT: 1883
CLICKHOUSE_HOST: clickhouse
CLICKHOUSE_NATIVE_PORT: 9000
CLICKHOUSE_DB: ${CLICKHOUSE_DB}
CLICKHOUSE_USER: ${CLICKHOUSE_USER}
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD}
RABBITMQ_HOST: rabbitmq
RABBITMQ_PORT: 5672
RABBITMQ_USER: ${RABBITMQ_USER}
RABBITMQ_PASSWORD: ${RABBITMQ_PASSWORD}
depends_on:
mosquitto:
condition: service_started
clickhouse:
condition: service_healthy
rabbitmq:
condition: service_healthy
networks:
- home-automation
restart: unless-stopped
# Remaining app services (device-control-service, rule-engine-service,
# App services (ingest-service, device-control-service, rule-engine-service,
# health-check-service, esp32-emulator, laravel-app) are added here as they
# get implemented — see services/*/README.md for the plan for each one.
View File
-9
View File
@@ -1,9 +0,0 @@
CREATE TABLE IF NOT EXISTS telemetry.telemetry (
device_id String,
zone_id String,
sensor_type String,
value Float64,
recorded_at DateTime,
received_at DateTime
) ENGINE = MergeTree()
ORDER BY (device_id, recorded_at);
View File
-59
View File
@@ -1,59 +0,0 @@
-- Core platform schema: users, zones, device types, devices, automation rules.
-- Deliberately generic — no growbox-specific tables or columns. A "growbox"
-- is just a row in zones with device_types like light/pump/fan attached to it.
CREATE TABLE users (
id BIGSERIAL PRIMARY KEY,
email TEXT NOT NULL UNIQUE,
password_hash TEXT NOT NULL,
role TEXT NOT NULL DEFAULT 'owner' CHECK (role IN ('owner', 'viewer')),
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE zones (
id BIGSERIAL PRIMARY KEY,
user_id BIGINT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name TEXT NOT NULL,
description TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE device_types (
id BIGSERIAL PRIMARY KEY,
code TEXT NOT NULL UNIQUE, -- light, pump, fan, sensor_temp_humidity, vacuum, ...
category TEXT NOT NULL CHECK (category IN ('actuator', 'sensor')),
capabilities JSONB NOT NULL DEFAULT '[]' -- e.g. ["turn_on","turn_off","set_level"]
);
CREATE TABLE devices (
id BIGSERIAL PRIMARY KEY,
user_id BIGINT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
zone_id BIGINT NOT NULL REFERENCES zones(id) ON DELETE CASCADE,
device_type_id BIGINT NOT NULL REFERENCES device_types(id),
name TEXT NOT NULL,
external_id TEXT NOT NULL UNIQUE, -- physical device id (ESP32 chip id, etc.)
protocol TEXT NOT NULL DEFAULT 'mqtt',
status TEXT NOT NULL DEFAULT 'offline' CHECK (status IN ('online', 'offline')),
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX devices_zone_id_idx ON devices(zone_id);
CREATE INDEX devices_external_id_idx ON devices(external_id);
CREATE TABLE automation_rules (
id BIGSERIAL PRIMARY KEY,
user_id BIGINT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
zone_id BIGINT NOT NULL REFERENCES zones(id) ON DELETE CASCADE,
target_device_id BIGINT NOT NULL REFERENCES devices(id) ON DELETE CASCADE,
condition_source_device_id BIGINT NOT NULL REFERENCES devices(id) ON DELETE CASCADE,
condition_sensor_type TEXT NOT NULL,
condition_operator TEXT NOT NULL CHECK (condition_operator IN ('>', '<', '>=', '<=', '=', '!=')),
condition_value DOUBLE PRECISION NOT NULL,
action_type TEXT NOT NULL, -- e.g. turn_on, turn_off, set_level
action_params JSONB NOT NULL DEFAULT '{}',
is_active BOOLEAN NOT NULL DEFAULT true,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX automation_rules_zone_id_idx ON automation_rules(zone_id);
CREATE INDEX automation_rules_active_idx ON automation_rules(is_active) WHERE is_active;
+16 -16
View File
@@ -1,20 +1,20 @@
# device-control-service (Go)
Status: not implemented yet.
Статус: пока не реализован.
Responsibility:
- Expose a gRPC API (see `proto/device_control.proto`) for issuing commands
to devices — called by Laravel (manual control) and rule-engine-service
(rule-triggered actions).
- Publish the command to the device's MQTT topic (`devices/{device_id}/commands`)
and listen for acknowledgements (`devices/{device_id}/ack`).
- Maintain the Device Shadow pattern in Redis: `desired_state` (what the user
wants) vs `reported_state` (what the device confirmed), plus `status` and
`last_seen`.
- Owns the health-check loop (or delegates to health-check-service): a ticker
goroutine that flips a device to `offline` when `last_seen` exceeds a
timeout, and emits an event for notification-service.
Зона ответственности:
- Предоставляет gRPC API (см. `proto/device_control.proto`) для отправки
команд устройствам — вызывается из Laravel (ручное управление) и из
rule-engine-service (действия по срабатыванию правил).
- Публикует команду в MQTT-топик устройства (`devices/{device_id}/commands`)
и слушает подтверждения (`devices/{device_id}/ack`).
- Поддерживает паттерн Device Shadow в Redis: `desired_state` (чего хочет
пользователь) против `reported_state` (что подтвердило устройство), плюс
`status` и `last_seen`.
- Владеет циклом health-check (либо делегирует его health-check-service):
горутина-тикер переводит устройство в `offline`, когда `last_seen`
превышает таймаут, и генерирует событие для notification-service.
Not this service's job: deciding *when* to send a command based on sensor
thresholds — that's rule-engine-service's logic. This service only executes
and tracks state for whatever command it's given.
Не входит в зону ответственности: решение о том, *когда* отправлять команду
на основе показаний датчиков — это логика rule-engine-service. Этот сервис
только исполняет команды и отслеживает состояние для той команды, что ему дали.
+8 -8
View File
@@ -1,11 +1,11 @@
# esp32-emulator (Go)
Status: not implemented yet.
Статус: пока не реализован.
Responsibility:
- Stand in for real ESP32 hardware during MVP development.
- Publish plausible telemetry (temperature/humidity/etc.) to
`devices/{device_id}/telemetry` over MQTT on an interval.
- Subscribe to `devices/{device_id}/commands` and reply on
`devices/{device_id}/ack`, so device-control-service has something real
to talk to.
Зона ответственности:
- Заменяет реальное железо ESP32 на время разработки MVP.
- Публикует правдоподобную телеметрию (температура/влажность/и т.д.) в
`devices/{device_id}/telemetry` по MQTT с заданным интервалом.
- Подписывается на `devices/{device_id}/commands` и отвечает в
`devices/{device_id}/ack`, чтобы device-control-service было с кем
по-настоящему взаимодействовать.
+9 -8
View File
@@ -1,11 +1,12 @@
# health-check-service (Go)
Status: not implemented yet. May start as a goroutine inside
device-control-service and get split out into its own container later —
kept as a separate directory from day one so the split is a non-event.
Статус: пока не реализован. Может начаться как горутина внутри
device-control-service и позже выделиться в отдельный контейнер — поэтому
с первого дня держим отдельную директорию, чтобы это разделение прошло
безболезненно.
Responsibility:
- Periodically (ticker) check `last_seen` for every device in Redis.
- When a device exceeds the offline timeout, flip its `status` to `offline`
and publish an event for notification-service.
- Good place to demonstrate goroutines + graceful shutdown in Go.
Зона ответственности:
- Периодически (по тикеру) проверяет `last_seen` каждого устройства в Redis.
- Когда устройство превышает таймаут офлайна, переводит его `status` в
`offline` и публикует событие для notification-service.
- Хороший повод продемонстрировать горутины + graceful shutdown в Go.
-12
View File
@@ -1,12 +0,0 @@
FROM golang:1.25-alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /out/ingest-service ./cmd/ingest-service
FROM alpine:3.20
RUN adduser -D -u 10001 app
COPY --from=build /out/ingest-service /usr/local/bin/ingest-service
USER app
ENTRYPOINT ["/usr/local/bin/ingest-service"]
+9 -30
View File
@@ -1,34 +1,13 @@
# ingest-service (Go)
Status: implemented (MVP).
Статус: пока не реализован.
Responsibility:
- Subscribe to MQTT topics `devices/{device_id}/telemetry`.
- Validate/parse payload (device_id, sensor_type, value, timestamp).
- Batch-write raw readings into ClickHouse `telemetry` table.
- Publish a "new reading" event to RabbitMQ for rule-engine-service.
Зона ответственности:
- Подписка на MQTT-топики `devices/{device_id}/telemetry`.
- Валидация/парсинг payload (device_id, sensor_type, value, timestamp).
- Батч-запись сырых показаний в таблицу ClickHouse `telemetry`.
- Публикация события «новое показание» в RabbitMQ для rule-engine-service.
Not this service's job: interpreting what a sensor reading *means* (thresholds,
actions) — that belongs to rule-engine-service. This service only ingests and
stores.
Note on `zone_id`: the telemetry payload carries `zone_id` directly (the
publishing device/emulator is configured with its own zone), rather than
ingest-service looking it up from PostgreSQL. This keeps ingest-service's
dependencies limited to MQTT/ClickHouse/RabbitMQ, matching the architecture
diagram in the root README. A real fleet where devices don't know their own
zone would need a device-registry lookup (cached, like rule-engine caches
rules) — that's a natural follow-up, not implemented yet.
## Running
```bash
cd services/ingest-service
go test ./...
go build ./cmd/ingest-service
```
Configuration is env-var driven (see the ingest-service section of the
repo-root `.env.example`). Via `docker compose up ingest-service` it talks to
the `mosquitto`/`clickhouse`/`rabbitmq` containers directly; connect from a
local `go run` by exporting `MQTT_HOST=localhost`, `CLICKHOUSE_HOST=localhost`, etc.
Не входит в зону ответственности: интерпретация того, что *означает*
показание датчика (пороги, действия) — это задача rule-engine-service. Этот
сервис только принимает и сохраняет данные.
@@ -1,118 +0,0 @@
// Command ingest-service subscribes to device telemetry over MQTT, batches
// it into ClickHouse, and publishes a "new reading" event per message for
// rule-engine-service to consume.
package main
import (
"context"
"log/slog"
"os"
"os/signal"
"sync"
"syscall"
"time"
"git.cactoz.su/cacto/home_automatization/services/ingest-service/internal/batch"
chstore "git.cactoz.su/cacto/home_automatization/services/ingest-service/internal/clickhouse"
"git.cactoz.su/cacto/home_automatization/services/ingest-service/internal/config"
"git.cactoz.su/cacto/home_automatization/services/ingest-service/internal/mqttclient"
"git.cactoz.su/cacto/home_automatization/services/ingest-service/internal/rabbitmq"
"git.cactoz.su/cacto/home_automatization/services/ingest-service/internal/telemetry"
)
// incomingBufferSize bounds how many parsed readings can wait for the worker
// goroutine before the MQTT callback starts dropping messages. It exists so
// a slow ClickHouse/RabbitMQ round-trip can't stall MQTT message delivery.
const incomingBufferSize = 1000
func main() {
logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
if err := run(logger); err != nil {
logger.Error("ingest-service exited with error", "error", err)
os.Exit(1)
}
}
func run(logger *slog.Logger) error {
cfg, err := config.Load()
if err != nil {
return err
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
store, err := chstore.Connect(ctx, chstore.Config{
Addr: cfg.ClickHouseAddr,
Database: cfg.ClickHouseDatabase,
Username: cfg.ClickHouseUsername,
Password: cfg.ClickHousePassword,
})
cancel()
if err != nil {
return err
}
defer store.Close()
publisher, err := rabbitmq.Connect(cfg.RabbitMQURL)
if err != nil {
return err
}
defer publisher.Close()
batcher := batch.New(cfg.BatchMaxSize, cfg.BatchFlushInterval, cfg.BatchFlushTimeout, store.InsertBatch, logger)
batcher.Start()
defer batcher.Stop()
incoming := make(chan telemetry.Reading, incomingBufferSize)
var workerWG sync.WaitGroup
workerWG.Add(1)
go func() {
defer workerWG.Done()
for reading := range incoming {
batcher.Add(reading)
pubCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
err := publisher.PublishReading(pubCtx, reading)
cancel()
if err != nil {
logger.Error("publish reading event failed", "device_id", reading.DeviceID, "error", err)
}
}
}()
subscriber, err := mqttclient.Connect(mqttclient.Config{
BrokerURL: cfg.MQTTBrokerURL,
ClientID: cfg.MQTTClientID,
Topic: cfg.MQTTTopic,
QoS: 1,
}, func(payload []byte, receivedAt time.Time) {
reading, err := telemetry.ParseReading(payload, receivedAt)
if err != nil {
logger.Warn("dropping invalid telemetry payload", "error", err)
return
}
select {
case incoming <- reading:
default:
logger.Error("dropping reading: worker backlog full", "device_id", reading.DeviceID)
}
}, logger)
if err != nil {
return err
}
logger.Info("ingest-service started", "mqtt_topic", cfg.MQTTTopic)
stop := make(chan os.Signal, 1)
signal.Notify(stop, syscall.SIGINT, syscall.SIGTERM)
<-stop
logger.Info("shutting down")
subscriber.Close()
close(incoming)
workerWG.Wait()
return nil
}
-30
View File
@@ -1,30 +0,0 @@
module git.cactoz.su/cacto/home_automatization/services/ingest-service
go 1.25.0
require (
github.com/ClickHouse/clickhouse-go/v2 v2.47.0
github.com/eclipse/paho.mqtt.golang v1.5.1
github.com/rabbitmq/amqp091-go v1.13.0
)
require (
github.com/ClickHouse/ch-go v0.73.0 // indirect
github.com/andybalholm/brotli v1.2.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/go-faster/city v1.0.1 // indirect
github.com/go-faster/errors v0.7.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/klauspost/compress v1.18.6 // indirect
github.com/paulmach/orb v0.13.0 // indirect
github.com/pierrec/lz4/v4 v4.1.27 // indirect
github.com/segmentio/asm v1.2.1 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
go.opentelemetry.io/otel v1.44.0 // indirect
go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/net v0.56.0 // indirect
golang.org/x/sync v0.20.0 // indirect
golang.org/x/sys v0.46.0 // indirect
)
-58
View File
@@ -1,58 +0,0 @@
github.com/ClickHouse/ch-go v0.73.0 h1:jsHiGRbQ3sz+gekvDFJF29LWDo5dzbJm5s1h8TWVP2M=
github.com/ClickHouse/ch-go v0.73.0/go.mod h1:wkFIxrqlXeRJ9cn3r5Fz5Qen9jl5aTMPuGZeuJpANNY=
github.com/ClickHouse/clickhouse-go/v2 v2.47.0 h1:ZDAzrnKSOPTIsm4tdUNfrii2yc8dk4SVRLC77BR7Z5Q=
github.com/ClickHouse/clickhouse-go/v2 v2.47.0/go.mod h1:sPj7C7UYQ2MWHcfX+4eGN6nwnCqwUKfgO6PcwKpd6K8=
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/eclipse/paho.mqtt.golang v1.5.1 h1:/VSOv3oDLlpqR2Epjn1Q7b2bSTplJIeV2ISgCl2W7nE=
github.com/eclipse/paho.mqtt.golang v1.5.1/go.mod h1:1/yJCneuyOoCOzKSsOTUc0AJfpsItBGWvYpBLimhArU=
github.com/go-faster/city v1.0.1 h1:4WAxSZ3V2Ws4QRDrscLEDcibJY8uf41H6AhXDrNDcGw=
github.com/go-faster/city v1.0.1/go.mod h1:jKcUJId49qdW3L1qKHH/3wPeUstCVpVSXTM6vO3VcTw=
github.com/go-faster/errors v0.7.1 h1:MkJTnDoEdi9pDabt1dpWf7AA8/BaSYZqibYyhZ20AYg=
github.com/go-faster/errors v0.7.1/go.mod h1:5ySTjWFiphBs07IKuiL69nxdfd5+fzh1u7FPGZP2quo=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/klauspost/compress v1.18.6 h1:2jupLlAwFm95+YDR+NwD2MEfFO9d4z4Prjl1XXDjuao=
github.com/klauspost/compress v1.18.6/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/paulmach/orb v0.13.0 h1:r7n7mQGGF+cj/CbcivEj9J3HGK+XR+yXnvzRdq9saIw=
github.com/paulmach/orb v0.13.0/go.mod h1:6scRWINywA2Jf05dcjOfLfxrUIMECvTSG2MVbRLxu/k=
github.com/pierrec/lz4/v4 v4.1.27 h1:+PhzhWDrjRj89TH2sw43nE3+4+W8lSxIuQadEHZyjUk=
github.com/pierrec/lz4/v4 v4.1.27/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rabbitmq/amqp091-go v1.13.0 h1:L8NA1WtF76C6KA3LAoufjfLgbist/If1UQYcsOjtxXA=
github.com/rabbitmq/amqp091-go v1.13.0/go.mod h1:Hy4jKW5kQART1u+JkDTF9YYOQUHXqMuhrgxOEeS7G4o=
github.com/segmentio/asm v1.2.1 h1:DTNbBqs57ioxAD4PrArqftgypG4/qNpXoJx8TVXxPR0=
github.com/segmentio/asm v1.2.1/go.mod h1:BqMnlJP91P8d+4ibuonYZw9mfnzI9HfxselHZr5aAcs=
github.com/shopspring/decimal v1.4.0 h1:bxl37RwXBklmTi0C79JfXCEBD1cqqHt0bbgBAGFp81k=
github.com/shopspring/decimal v1.4.0/go.mod h1:gawqmDU56v4yIKSwfBSFip1HdCCXN8/+DMd9qYNcwME=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
go.opentelemetry.io/otel v1.44.0 h1:JjwHmHpA4iZ3wBxluu2fbbE7j4kqlE8jXyAyPXH7HqU=
go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
@@ -1,105 +0,0 @@
// Package batch accumulates telemetry readings and flushes them in bulk,
// either once a size threshold is hit or on a fixed interval — so ClickHouse
// gets bulk inserts instead of one round-trip per reading.
package batch
import (
"context"
"log/slog"
"sync"
"time"
"git.cactoz.su/cacto/home_automatization/services/ingest-service/internal/telemetry"
)
// FlushFunc persists one batch. It is called with a context bounded by
// FlushTimeout, both for interval-triggered and shutdown-triggered flushes.
type FlushFunc func(ctx context.Context, readings []telemetry.Reading) error
// Batcher buffers readings in memory and flushes them via FlushFunc when
// MaxSize is reached or Interval elapses, whichever comes first.
type Batcher struct {
maxSize int
interval time.Duration
flushTimeout time.Duration
flush FlushFunc
logger *slog.Logger
mu sync.Mutex
buf []telemetry.Reading
stop chan struct{}
done chan struct{}
}
func New(maxSize int, interval, flushTimeout time.Duration, flush FlushFunc, logger *slog.Logger) *Batcher {
return &Batcher{
maxSize: maxSize,
interval: interval,
flushTimeout: flushTimeout,
flush: flush,
logger: logger,
}
}
// Start begins the interval-flush loop. Call once before the first Add.
func (b *Batcher) Start() {
b.stop = make(chan struct{})
b.done = make(chan struct{})
go b.loop()
}
func (b *Batcher) loop() {
defer close(b.done)
ticker := time.NewTicker(b.interval)
defer ticker.Stop()
for {
select {
case <-ticker.C:
b.flushNow("interval")
case <-b.stop:
b.flushNow("shutdown")
return
}
}
}
// Add appends a reading, flushing immediately if the buffer just reached
// maxSize rather than waiting for the next tick.
func (b *Batcher) Add(r telemetry.Reading) {
b.mu.Lock()
b.buf = append(b.buf, r)
full := len(b.buf) >= b.maxSize
b.mu.Unlock()
if full {
b.flushNow("size")
}
}
// Stop flushes any remaining buffered readings and waits for the flush loop
// to exit. Safe to call once, after Start.
func (b *Batcher) Stop() {
close(b.stop)
<-b.done
}
func (b *Batcher) flushNow(reason string) {
b.mu.Lock()
if len(b.buf) == 0 {
b.mu.Unlock()
return
}
readings := b.buf
b.buf = nil
b.mu.Unlock()
ctx, cancel := context.WithTimeout(context.Background(), b.flushTimeout)
defer cancel()
if err := b.flush(ctx, readings); err != nil {
b.logger.Error("batch flush failed", "reason", reason, "size", len(readings), "error", err)
return
}
b.logger.Debug("batch flushed", "reason", reason, "size", len(readings))
}
@@ -1,84 +0,0 @@
package batch
import (
"context"
"io"
"log/slog"
"testing"
"time"
"git.cactoz.su/cacto/home_automatization/services/ingest-service/internal/telemetry"
)
func discardLogger() *slog.Logger {
return slog.New(slog.NewTextHandler(io.Discard, nil))
}
func reading(deviceID string) telemetry.Reading {
return telemetry.Reading{DeviceID: deviceID, ZoneID: "z1", SensorType: "temperature", Value: 1}
}
func awaitBatch(t *testing.T, flushed chan []telemetry.Reading, timeout time.Duration) []telemetry.Reading {
t.Helper()
select {
case batch := <-flushed:
return batch
case <-time.After(timeout):
t.Fatal("timed out waiting for flush")
return nil
}
}
func TestBatcher_FlushesOnSize(t *testing.T) {
flushed := make(chan []telemetry.Reading, 1)
b := New(3, time.Hour, time.Second, func(_ context.Context, r []telemetry.Reading) error {
flushed <- r
return nil
}, discardLogger())
b.Start()
defer b.Stop()
b.Add(reading("a"))
b.Add(reading("b"))
b.Add(reading("c")) // hits maxSize, should trigger an immediate flush
batch := awaitBatch(t, flushed, time.Second)
if len(batch) != 3 {
t.Fatalf("got %d readings, want 3", len(batch))
}
}
func TestBatcher_FlushesOnInterval(t *testing.T) {
flushed := make(chan []telemetry.Reading, 1)
b := New(100, 20*time.Millisecond, time.Second, func(_ context.Context, r []telemetry.Reading) error {
flushed <- r
return nil
}, discardLogger())
b.Start()
defer b.Stop()
b.Add(reading("a"))
batch := awaitBatch(t, flushed, time.Second)
if len(batch) != 1 {
t.Fatalf("got %d readings, want 1", len(batch))
}
}
func TestBatcher_StopFlushesRemainder(t *testing.T) {
flushed := make(chan []telemetry.Reading, 1)
b := New(100, time.Hour, time.Second, func(_ context.Context, r []telemetry.Reading) error {
flushed <- r
return nil
}, discardLogger())
b.Start()
b.Add(reading("a"))
b.Add(reading("b"))
b.Stop()
batch := awaitBatch(t, flushed, time.Second)
if len(batch) != 2 {
t.Fatalf("got %d readings, want 2", len(batch))
}
}
@@ -1,76 +0,0 @@
// Package clickhouse writes batches of telemetry readings to ClickHouse.
package clickhouse
import (
"context"
"crypto/tls"
"fmt"
ch "github.com/ClickHouse/clickhouse-go/v2"
"github.com/ClickHouse/clickhouse-go/v2/lib/driver"
"git.cactoz.su/cacto/home_automatization/services/ingest-service/internal/telemetry"
)
type Config struct {
Addr string
Database string
Username string
Password string
UseTLS bool
}
type Store struct {
conn driver.Conn
}
func Connect(ctx context.Context, cfg Config) (*Store, error) {
opts := &ch.Options{
Addr: []string{cfg.Addr},
Auth: ch.Auth{
Database: cfg.Database,
Username: cfg.Username,
Password: cfg.Password,
},
}
if cfg.UseTLS {
opts.TLS = &tls.Config{}
}
conn, err := ch.Open(opts)
if err != nil {
return nil, fmt.Errorf("open clickhouse connection: %w", err)
}
if err := conn.Ping(ctx); err != nil {
return nil, fmt.Errorf("ping clickhouse: %w", err)
}
return &Store{conn: conn}, nil
}
func (s *Store) Close() error {
return s.conn.Close()
}
// InsertBatch writes readings to the telemetry table in a single INSERT.
func (s *Store) InsertBatch(ctx context.Context, readings []telemetry.Reading) error {
if len(readings) == 0 {
return nil
}
batch, err := s.conn.PrepareBatch(ctx, "INSERT INTO telemetry (device_id, zone_id, sensor_type, value, recorded_at, received_at)")
if err != nil {
return fmt.Errorf("prepare batch: %w", err)
}
defer batch.Close()
for _, r := range readings {
if err := batch.Append(r.DeviceID, r.ZoneID, r.SensorType, r.Value, r.RecordedAt, r.ReceivedAt); err != nil {
return fmt.Errorf("append reading for device %s: %w", r.DeviceID, err)
}
}
if err := batch.Send(); err != nil {
return fmt.Errorf("send batch: %w", err)
}
return nil
}
@@ -1,91 +0,0 @@
// Package config loads ingest-service settings from environment variables,
// matching the names used in the repo-root .env.example.
package config
import (
"fmt"
"os"
"strconv"
"time"
)
type Config struct {
MQTTBrokerURL string
MQTTClientID string
MQTTTopic string
ClickHouseAddr string
ClickHouseDatabase string
ClickHouseUsername string
ClickHousePassword string
RabbitMQURL string
BatchMaxSize int
BatchFlushInterval time.Duration
BatchFlushTimeout time.Duration
}
func Load() (Config, error) {
cfg := Config{
MQTTBrokerURL: fmt.Sprintf("tcp://%s:%s", getEnv("MQTT_HOST", "localhost"), getEnv("MQTT_PORT", "1883")),
MQTTClientID: getEnv("INGEST_MQTT_CLIENT_ID", "ingest-service"),
MQTTTopic: getEnv("INGEST_MQTT_TOPIC", "devices/+/telemetry"),
ClickHouseAddr: fmt.Sprintf("%s:%s", getEnv("CLICKHOUSE_HOST", "localhost"), getEnv("CLICKHOUSE_NATIVE_PORT", "9000")),
ClickHouseDatabase: getEnv("CLICKHOUSE_DB", "telemetry"),
ClickHouseUsername: getEnv("CLICKHOUSE_USER", "default"),
ClickHousePassword: getEnv("CLICKHOUSE_PASSWORD", ""),
RabbitMQURL: fmt.Sprintf("amqp://%s:%s@%s:%s/",
getEnv("RABBITMQ_USER", "guest"),
getEnv("RABBITMQ_PASSWORD", "guest"),
getEnv("RABBITMQ_HOST", "localhost"),
getEnv("RABBITMQ_PORT", "5672"),
),
}
var err error
if cfg.BatchMaxSize, err = getEnvInt("INGEST_BATCH_MAX_SIZE", 500); err != nil {
return Config{}, err
}
if cfg.BatchFlushInterval, err = getEnvDuration("INGEST_BATCH_FLUSH_INTERVAL", 5*time.Second); err != nil {
return Config{}, err
}
if cfg.BatchFlushTimeout, err = getEnvDuration("INGEST_BATCH_FLUSH_TIMEOUT", 10*time.Second); err != nil {
return Config{}, err
}
return cfg, nil
}
func getEnv(key, fallback string) string {
if v := os.Getenv(key); v != "" {
return v
}
return fallback
}
func getEnvInt(key string, fallback int) (int, error) {
v := os.Getenv(key)
if v == "" {
return fallback, nil
}
n, err := strconv.Atoi(v)
if err != nil {
return 0, fmt.Errorf("%s: %w", key, err)
}
return n, nil
}
func getEnvDuration(key string, fallback time.Duration) (time.Duration, error) {
v := os.Getenv(key)
if v == "" {
return fallback, nil
}
d, err := time.ParseDuration(v)
if err != nil {
return 0, fmt.Errorf("%s: %w", key, err)
}
return d, nil
}
@@ -1,60 +0,0 @@
// Package mqttclient subscribes to device telemetry topics over MQTT.
package mqttclient
import (
"fmt"
"log/slog"
"time"
mqtt "github.com/eclipse/paho.mqtt.golang"
)
// Handler receives one message's raw payload and the time it arrived.
type Handler func(payload []byte, receivedAt time.Time)
type Config struct {
BrokerURL string // e.g. tcp://mosquitto:1883
ClientID string
Topic string // e.g. devices/+/telemetry
QoS byte
}
type Subscriber struct {
client mqtt.Client
}
// Connect dials the broker and (re-)subscribes to cfg.Topic on every
// successful connection, so a dropped connection resubscribes automatically
// once auto-reconnect succeeds.
func Connect(cfg Config, handler Handler, logger *slog.Logger) (*Subscriber, error) {
opts := mqtt.NewClientOptions().
AddBroker(cfg.BrokerURL).
SetClientID(cfg.ClientID).
SetAutoReconnect(true).
SetConnectRetry(true).
SetOnConnectHandler(func(c mqtt.Client) {
token := c.Subscribe(cfg.Topic, cfg.QoS, func(_ mqtt.Client, msg mqtt.Message) {
handler(msg.Payload(), time.Now())
})
token.Wait()
if err := token.Error(); err != nil {
logger.Error("mqtt subscribe failed", "topic", cfg.Topic, "error", err)
}
}).
SetConnectionLostHandler(func(_ mqtt.Client, err error) {
logger.Warn("mqtt connection lost", "error", err)
})
client := mqtt.NewClient(opts)
token := client.Connect()
token.Wait()
if err := token.Error(); err != nil {
return nil, fmt.Errorf("connect to mqtt broker %s: %w", cfg.BrokerURL, err)
}
return &Subscriber{client: client}, nil
}
// Close disconnects, waiting up to quiesceMS for in-flight work to settle.
func (s *Subscriber) Close() {
s.client.Disconnect(250)
}
@@ -1,85 +0,0 @@
// Package rabbitmq publishes "new reading" events for rule-engine-service to
// consume. Delivery reliability matters more than latency here, so messages
// are persistent and the queue is durable.
package rabbitmq
import (
"context"
"encoding/json"
"fmt"
"time"
amqp "github.com/rabbitmq/amqp091-go"
"git.cactoz.su/cacto/home_automatization/services/ingest-service/internal/telemetry"
)
const NewReadingQueue = "telemetry.new_reading"
type Publisher struct {
conn *amqp.Connection
ch *amqp.Channel
}
func Connect(url string) (*Publisher, error) {
conn, err := amqp.Dial(url)
if err != nil {
return nil, fmt.Errorf("dial rabbitmq: %w", err)
}
ch, err := conn.Channel()
if err != nil {
conn.Close()
return nil, fmt.Errorf("open channel: %w", err)
}
if _, err := ch.QueueDeclare(NewReadingQueue, true, false, false, false, nil); err != nil {
ch.Close()
conn.Close()
return nil, fmt.Errorf("declare queue %s: %w", NewReadingQueue, err)
}
return &Publisher{conn: conn, ch: ch}, nil
}
func (p *Publisher) Close() error {
if err := p.ch.Close(); err != nil {
p.conn.Close()
return err
}
return p.conn.Close()
}
// newReadingEvent is the wire format published to rule-engine-service.
type newReadingEvent struct {
DeviceID string `json:"device_id"`
ZoneID string `json:"zone_id"`
SensorType string `json:"sensor_type"`
Value float64 `json:"value"`
RecordedAt time.Time `json:"recorded_at"`
}
// PublishReading emits one event per reading to NewReadingQueue.
func (p *Publisher) PublishReading(ctx context.Context, r telemetry.Reading) error {
body, err := json.Marshal(newReadingEvent{
DeviceID: r.DeviceID,
ZoneID: r.ZoneID,
SensorType: r.SensorType,
Value: r.Value,
RecordedAt: r.RecordedAt,
})
if err != nil {
return fmt.Errorf("marshal event: %w", err)
}
err = p.ch.PublishWithContext(ctx, "", NewReadingQueue, false, false, amqp.Publishing{
ContentType: "application/json",
DeliveryMode: amqp.Persistent,
Timestamp: time.Now(),
Body: body,
})
if err != nil {
return fmt.Errorf("publish reading for device %s: %w", r.DeviceID, err)
}
return nil
}
@@ -1,69 +0,0 @@
// Package telemetry parses and validates raw device telemetry payloads.
package telemetry
import (
"encoding/json"
"fmt"
"time"
)
// Reading is a single validated sensor reading, ready to be stored in
// ClickHouse and published as an event for rule-engine-service.
type Reading struct {
DeviceID string
ZoneID string
SensorType string
Value float64
RecordedAt time.Time
ReceivedAt time.Time
}
// payload mirrors the wire format devices publish to devices/{device_id}/telemetry.
//
// ZoneID travels with the payload (rather than being looked up from
// PostgreSQL) so ingest-service stays decoupled from the device registry,
// matching the ingest→ClickHouse/RabbitMQ-only data flow in the platform
// architecture — the publishing client is responsible for knowing its own zone.
type payload struct {
DeviceID string `json:"device_id"`
ZoneID string `json:"zone_id"`
SensorType string `json:"sensor_type"`
Value float64 `json:"value"`
Timestamp string `json:"timestamp"`
}
// ParseReading decodes and validates a raw MQTT message body. receivedAt is
// the time the server received the message, stamped independently of
// whatever the device claims, so ingest/processing delay can be measured later.
func ParseReading(raw []byte, receivedAt time.Time) (Reading, error) {
var p payload
if err := json.Unmarshal(raw, &p); err != nil {
return Reading{}, fmt.Errorf("invalid json: %w", err)
}
if p.DeviceID == "" {
return Reading{}, fmt.Errorf("device_id is required")
}
if p.ZoneID == "" {
return Reading{}, fmt.Errorf("zone_id is required")
}
if p.SensorType == "" {
return Reading{}, fmt.Errorf("sensor_type is required")
}
if p.Timestamp == "" {
return Reading{}, fmt.Errorf("timestamp is required")
}
recordedAt, err := time.Parse(time.RFC3339, p.Timestamp)
if err != nil {
return Reading{}, fmt.Errorf("invalid timestamp %q: %w", p.Timestamp, err)
}
return Reading{
DeviceID: p.DeviceID,
ZoneID: p.ZoneID,
SensorType: p.SensorType,
Value: p.Value,
RecordedAt: recordedAt,
ReceivedAt: receivedAt,
}, nil
}
@@ -1,74 +0,0 @@
package telemetry
import (
"testing"
"time"
)
func TestParseReading_Valid(t *testing.T) {
receivedAt := time.Date(2026, 7, 26, 12, 0, 5, 0, time.UTC)
raw := []byte(`{
"device_id": "esp32-abc123",
"zone_id": "growbox-1",
"sensor_type": "temperature",
"value": 24.5,
"timestamp": "2026-07-26T12:00:00Z"
}`)
got, err := ParseReading(raw, receivedAt)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
want := Reading{
DeviceID: "esp32-abc123",
ZoneID: "growbox-1",
SensorType: "temperature",
Value: 24.5,
RecordedAt: time.Date(2026, 7, 26, 12, 0, 0, 0, time.UTC),
ReceivedAt: receivedAt,
}
if got != want {
t.Fatalf("got %+v, want %+v", got, want)
}
}
func TestParseReading_Invalid(t *testing.T) {
receivedAt := time.Now()
cases := map[string]string{
"malformed json": `{not json`,
"missing device_id": `{
"zone_id": "growbox-1", "sensor_type": "temperature",
"value": 1, "timestamp": "2026-07-26T12:00:00Z"
}`,
"missing zone_id": `{
"device_id": "esp32-1", "sensor_type": "temperature",
"value": 1, "timestamp": "2026-07-26T12:00:00Z"
}`,
"missing sensor_type": `{
"device_id": "esp32-1", "zone_id": "growbox-1",
"value": 1, "timestamp": "2026-07-26T12:00:00Z"
}`,
"missing timestamp": `{
"device_id": "esp32-1", "zone_id": "growbox-1",
"sensor_type": "temperature", "value": 1
}`,
"invalid timestamp format": `{
"device_id": "esp32-1", "zone_id": "growbox-1",
"sensor_type": "temperature", "value": 1, "timestamp": "not-a-date"
}`,
"value wrong type": `{
"device_id": "esp32-1", "zone_id": "growbox-1",
"sensor_type": "temperature", "value": "not-a-number", "timestamp": "2026-07-26T12:00:00Z"
}`,
}
for name, raw := range cases {
t.Run(name, func(t *testing.T) {
if _, err := ParseReading([]byte(raw), receivedAt); err == nil {
t.Fatalf("expected error, got nil")
}
})
}
}
+14 -14
View File
@@ -1,18 +1,18 @@
# rule-engine-service (Go)
Status: not implemented yet.
Статус: пока не реализован.
Responsibility:
- Consume "new reading" events from RabbitMQ (async — delivery reliability
matters more than latency here).
- Load active `automation_rules` for the relevant zone/device from PostgreSQL,
caching in memory/Redis to avoid a DB round-trip per event.
- Evaluate rule conditions generically: `sensor_type X operator value` against
the reading — no hardcoded sensor/device names, everything comes from the
`automation_rules` row.
- On match: call device-control-service over gRPC (needs a fast pass/fail
result) and publish an event to notification-service over RabbitMQ (not
latency-critical).
Зона ответственности:
- Потребляет события «новое показание» из RabbitMQ (асинхронно — здесь
надёжность доставки важнее задержки).
- Загружает активные `automation_rules` для нужной зоны/устройства из
PostgreSQL, кэшируя в памяти/Redis, чтобы не ходить в БД на каждое событие.
- Оценивает условия правил обобщённо: `sensor_type X operator value` против
показания — никаких зашитых названий датчиков/устройств, всё берётся из
строки `automation_rules`.
- При срабатывании: вызывает device-control-service по gRPC (нужен быстрый
результат успех/неудача) и публикует событие в notification-service через
RabbitMQ (не критично к задержке).
Not this service's job: talking to MQTT or Redis directly — device state
changes always go through device-control-service's gRPC API.
Не входит в зону ответственности: прямое взаимодействие с MQTT или Redis
изменения состояния устройства всегда идут через gRPC API device-control-service.