After launching my first node.js + MongoDB API in production, I was religiously monitoring it like parents watching over their first born child.
As traffic started to roll in, the API server began to quietly cry. The average response in milliseconds increased as more and more concurrent queries were happening in the DB. But this wasn’t happening on my local development environment when I was running three times more traffic in my stress tests. Of course localhost beats the production environment any day of the week right?
Long story short, I forgot to enable indexes on the production environment. After adding db.collection.ensureIndex({"items": 1})
to the most important key that I was querying, the beams of sun broke through the cloudy sky.
Lets look at the difference in our famous before and after example:
Before
> GET /sites/?… 200 2691ms – 128b
After
> GET /sites/?… 200 91ms – 128b
And a chart for giggles
Thanks to the awesome folks that make these amazing tools
- New Relic – http://newrelic.com/
- MongoDB MMS – http://mms.mongodb.com/help/monitoring/
- JMeter – http://jmeter.apache.org/
Update: 2/11/13
On a side note, this small instance was able to handle over 1.5k requests per minute without ever breaking a sweat. Big ups OpenShift.
We created a small article about our learnings and findings with MongoDB and it’s setup in a large cluster.
That might be another interesting read for you: http://www.webpageanalyse.com/blog/findings-deploying-a-large-mongodb-sharded-cluster-with-replication-part-1
It’s going to be finish of mine day, except before ending I am reading this fantastic post to improve my know-how.