The first thing I do when someone reports a slow API is step back and measure.
Before changing a single line of code, I check the query logs and trace the full execution path. Guessing where the bottleneck lives almost always leads to fixing the wrong thing.
Here is how we reduced p95 response times on Earther from 3.5 seconds down to under 500 milliseconds without downtime or a rewrite.
The background
Earther is a construction management platform. By 2020, as the user base grew, several key endpoints began slowing down to 3.5 seconds, and some heavy requests timed out completely during peak hours.
The team suspected the database was struggling, which was partly true, but the real issue was how the application queried it.
Finding the real bottleneck
I spent two days running profilers and reviewing slow query logs.
What stood out was that several endpoints were running 40 to 80 separate database queries per request. The developers had not written broken logic; they were just navigating relationships inside loops without realizing the database cost.
This is the classic N+1 query problem. It works fine during local testing with 5 mock records, but quickly slows down when you query 50 records in production.
What an N+1 query looks like
# One query to fetch the user's orders
orders = Order.objects.filter(user=user)
# One additional query PER ORDER to fetch line items
for order in orders:
items = order.line_items.all()
# ...
If a user has 20 orders, that code runs 21 separate database queries. If an endpoint gets called thousands of times an hour, the database spends all its time handling connection and query overhead.
What we fixed
Once we mapped out where time was actually being spent, the plan became clear:
1. Fix the N+1 queries first
This had the biggest immediate impact. We replaced iterative queries with eager loading and batch fetches across 12 main routes. Query counts per request dropped from 40-80 down to 3-8.
# Before: N+1 queries
orders = Order.objects.filter(user=user)
for order in orders:
items = order.line_items.all()
# After: single batched query
orders = Order.objects.filter(user=user).prefetch_related('line_items')
2. Move heavy tasks to background workers
Certain tasks did not need to run synchronously inside the HTTP request. Generating PDF invoices, aggregating weekly spreadsheets, and running audit checks were blocking API responses unnecessarily.
We moved 6 of these compute-heavy tasks to asynchronous RabbitMQ workers. The API triggered the job and responded right away, while clients polled or listened for updates when the file was ready.
This completely eliminated request timeouts.
3. Set up clear service boundaries
To keep queries organized long-term, we created dedicated service classes. Instead of controllers making direct queries across unrelated database models, all business logic and data access went through dedicated service interfaces.
The tradeoffs
Every architecture choice comes with tradeoffs:
- Asynchronous tasks meant the frontend had to show loading and pending states for report downloads.
- Adding service layers meant developers had to follow a structured pattern rather than writing queries directly in controllers.
- Running RabbitMQ introduced a separate queue service that needed monitoring.
The result
p95 response times dropped from 3.5s to under 500ms across all refactored routes with zero platform downtime. We deployed one endpoint at a time and watched latency graphs at each step.
The takeaway
Good performance work is rarely about writing clever micro-optimizations. It is about understanding what data an endpoint actually needs and removing redundant work.
Taking two days to measure query logs before writing code felt slow at first, but it prevented us from wasting weeks applying temporary cache patches to broken queries.