MongoDB Lesson Learned: Remeber To Use Indexes

After launching my first node.js + MongoDB API in production, I was religiously monitoring it like parents watching over their first born child.

As traffic started to roll in, the API server began to quietly cry. The average response in milliseconds increased as more and more concurrent queries were happening in the DB. But this wasn’t happening on my local development environment when I was running three times more traffic in my stress tests. Of course localhost beats the production environment any day of the week right?

Long story short, I forgot to enable indexes on the production environment. After adding db.collection.ensureIndex({"items": 1}) to the most important key that I was querying, the beams of sun broke through the cloudy sky.

Lets look at the difference in our famous before and after example:

> GET /sites/?… 200 2691ms – 128b

> GET /sites/?… 200 91ms – 128b

And a chart for giggles

API response time

Thanks to the awesome folks that make these amazing tools

Update: 2/11/13

On a side note, this small instance was able to handle over 1.5k requests per minute without ever breaking a sweat. Big ups OpenShift.