perf/refactor: pre-compute device_categories at write time to eliminate runtime aggregation overhead · airqo-platform/AirQo-api#6185

2026-03-08T22:31:23.000Z

Summary Currently, device_categories (including all_categories , primary_category , ownership_category , mobile_category , is_mobile , is_static ) is computed at read time via a MongoDB $addFields aggregation stage ( getDeviceCategoriesAddFieldsStage ) every time a device document is queried. This stage runs inline on every call to statics.list() and every pipeline that fetches device data. This issue tracks replacing that read-time computation with a write-time pre-computation approach: compute device_categories once when a device is created or updated, store it directly on the Device document, and replace the aggregation stage with a simple $project (or just read the stored field directly). Background device_categories is derived entirely from three stable device fields: Field | Source -- | -- category | Device document root deployment_type | Device document root ("static" \| "mobile") network | Device document root Affected Files src/device-registry/models/Device.js — add stored field + pre-save hook + recomputeCategories static src/device-registry/utils/device.util.js — deprecate / remove getDeviceCategoriesAddFieldsStage() src/device-registry/models/Event.js — remove promotionStageForEventPipeline + addFields stage from fetch() src/device-registry/models/Reading.js — remove $ifNull passthrough; read stored field directly src/device-registry/jobs/store-readings-job.js — replace passthrough with Device lookup read Migration script — one-time recomputeCategories({}) backfill Out of Scope Changing the shape of DeviceCategorySchema (no field additions/removals in this issue) Unit tests for computeDeviceCategories() — tracked separately in #5XXX The enrich-dev-cats PR currently in review — this issue is a follow-up optimisation after that PR lands Priority Low–Medium. The correctness fix (Event.js promotion stage + cron recompute) in the enrich-dev-cats PR resolves the immediate corruption bug. This issue is a performance and maintainability improvement for a subsequent PR. Labels: performance , refactor , device-registry Component: device-registry Related PRs: enrich-dev-cats branch (must land first)

(0 comments) (0 reactions) (0 assignees)JavaScript (24 forks)auto 404

device-registryenhancementgood first issue

Repository metrics

Stars: (26 stars)
PR merge metrics: (PR metrics pending)

Description

Field	Source
category	Device document root
deployment_type	Device document root ("static" \| "mobile")
network	Device document root

Contributor guide

Research direction: Study the existing getDeviceCategoriesAddFieldsStage function and the device model's pre save hook. Understand how device categories are derived from device fields. Then, implement the compute function and modify the model to store computed categories on write, replacing the aggregation stage with a simple projection.
Tech stack: javascriptnodejsmongodb
Domain: backend
Issue type: Performance
Difficulty: 3
Estimated time: Half day
Activity status: Fresh
Clarity: Clear
Prerequisites: Node.jsMongoDB
Newbie friendliness: 65

Repository metrics

Description

Contributor guide

Get fresh easy issues in your inbox.