From 3.5s to 500ms API Latency

Earther is a construction management platform. By 2020, as more enterprise accounts joined, the platform began slowing down significantly under normal daytime load. Client contracts were on the line, and the engineering team needed a clear plan to stabilize it.

Core endpoints had p95 response times around 3.5 seconds, with occasional full timeouts during peak hours. The codebase had accumulated multiple N+1 database queries, synchronous blocking tasks for jobs that could easily run in the background, and loose boundaries where modules queried across each other in tangled ways.

• →The platform was in active use by paying clients, so downtime had to be zero.
• →The team still had to ship customer feature commitments during the overhaul.
• →We had to work inside an existing Laravel and MySQL codebase with several years of legacy decisions.
• →The team was small, so the architecture had to be straightforward for everyone to maintain.

I investigated the bottlenecks, designed the refactoring plan, and wrote the core database and queue implementations while keeping our weekly team syncs aligned on the changes.

My first reaction was to throw Redis caching at every slow query, but I held back. Before changing any code, I spent two full days instrumenting the application and analyzing query logs. That revealed the true issue: several heavy endpoints were making 40 to 80 separate database queries per request. Caching would have just masked bad access patterns. The real fix was simplifying the data layer so each endpoint fetched only what it needed.

We broke the work into three clean stages: first, resolve N+1 queries with eager loading and selective field queries; second, move heavy PDF and report generation into RabbitMQ background workers; third, organize business logic into clear service objects. We explicitly skipped a full microservices rewrite because the monolith was fine once the data access was cleaned up.

• 01Profiled every slow route using Laravel Debugbar and MySQL slow logs, documenting query counts in a shared tracker.
• 02Replaced N+1 query loops with eager loading and batch queries across 12 primary endpoints.
• 03Offloaded 6 heavy tasks (PDF generation, spreadsheet exports, compliance checks) to asynchronous RabbitMQ workers.
• 04Created service classes so controllers stopped making direct cross-model queries.
• 05Added integration tests for each endpoint before refactoring to verify output stayed identical.
• 06Shipped changes incrementally, route by route, using feature flags while watching p95 latency graphs.

• —Background jobs introduced eventual consistency, requiring the frontend to display loading and pending states for exports.
• —Service classes added an extra file layer, which required a couple of pairing sessions so everyone felt comfortable navigating the code.
• —Running RabbitMQ introduced a separate infrastructure dependency that needed monitoring and alerting.

p95 API response times dropped from ~3.5s to under 500ms across all refactored routes. The platform remained fully online throughout the rollout, and load tests that previously timed out ran cleanly.

Taking time to measure before writing code saves weeks of wasted effort. Resisting the urge to apply quick caching patches let us fix the actual structural problems. Clear sequencing and good communication allowed us to refactor core systems without blocking product releases.