cactoandClaude Sonnet 5
938700cf86
Subscribes to devices/+/telemetry, validates payloads, buffers writes into ClickHouse (flush on size or interval), and emits a per-reading event on RabbitMQ for rule-engine-service to consume later. Adds the initial Postgres and ClickHouse schema migrations, a Dockerfile, and wires the service into docker-compose. Verified end-to-end via docker compose + mosquitto_pub: row lands in ClickHouse and the event reaches the telemetry.new_reading queue. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
35 lines
1.4 KiB
Markdown
35 lines
1.4 KiB
Markdown
# ingest-service (Go)
|
|
|
|
Status: implemented (MVP).
|
|
|
|
Responsibility:
|
|
- Subscribe to MQTT topics `devices/{device_id}/telemetry`.
|
|
- Validate/parse payload (device_id, sensor_type, value, timestamp).
|
|
- Batch-write raw readings into ClickHouse `telemetry` table.
|
|
- Publish a "new reading" event to RabbitMQ for rule-engine-service.
|
|
|
|
Not this service's job: interpreting what a sensor reading *means* (thresholds,
|
|
actions) — that belongs to rule-engine-service. This service only ingests and
|
|
stores.
|
|
|
|
Note on `zone_id`: the telemetry payload carries `zone_id` directly (the
|
|
publishing device/emulator is configured with its own zone), rather than
|
|
ingest-service looking it up from PostgreSQL. This keeps ingest-service's
|
|
dependencies limited to MQTT/ClickHouse/RabbitMQ, matching the architecture
|
|
diagram in the root README. A real fleet where devices don't know their own
|
|
zone would need a device-registry lookup (cached, like rule-engine caches
|
|
rules) — that's a natural follow-up, not implemented yet.
|
|
|
|
## Running
|
|
|
|
```bash
|
|
cd services/ingest-service
|
|
go test ./...
|
|
go build ./cmd/ingest-service
|
|
```
|
|
|
|
Configuration is env-var driven (see the ingest-service section of the
|
|
repo-root `.env.example`). Via `docker compose up ingest-service` it talks to
|
|
the `mosquitto`/`clickhouse`/`rabbitmq` containers directly; connect from a
|
|
local `go run` by exporting `MQTT_HOST=localhost`, `CLICKHOUSE_HOST=localhost`, etc.
|