The idio platform was growing and they were in need of a monitoring solution that would tie in seamlessly with their auto-scaling infrastructure. As with all growing companies, pulling team members away from their duties in order to evaluate, install and customise a new monitoring platform is not a decision that is made lightly. idio were looking for a solution that’s would be easy to install, and would work to a minimum level straight out of the box, but with the ability to be flexible around custom metrics and monitoring of business numbers.
idio required simplicity, a clear UI, quick installation, and an automatic process for provisioning new servers and setting up monitoring & alerts on them. Sometimes scaling this quickly leads to bottlenecks and issues which don’t get picked up until it is too late. To prevent this, idio wanted a singular view onto all of their metrics so that they could see issues as they start to appear; and take measures to stop them from becoming bigger problems that bring their servers / websites down.
- The agent: The Server Density agent is lightweight, simple to install and provided idio with most of the metrics they needed straight out of the box. You can never look for a monitoring solution that will cover 100% of your needs, because there will always be an amount of customisation that you require, but the agent provides room for expansion with plugins and the API.
- The API: idio needed the ability to put custom data into their system, as well as gather standard metrics out the box. The Server Density API makes it simple to manage an entire account programmatically - an HTTP JSON API that is very simple and quick to get started with. This allowed idio to develop some custom agentless monitoring.
- Scalable monitoring: Server Density integrates with your deployment & scaling mechanisms, and syncs in real time. From starting new instances to upgrading or deleting old ones, the idio infrastructure is being monitored constantly including virtual, physical and cloud instances to create a view of their entire infrastructure; without any downtime.
- Hosted: It was important to idio that their team were not placed under the pressure of hosting their monitoring solution, however considering the rate of their growth they still needed an interface that could scale to cope with the strain of lots of servers and web checks. With features like regex searching and grouping, Server Density makes managing large infrastructure easy - and takes away the strain on teams of worrying about alert delivery and monitoring your monitoring.
- Plugins - The Server Density plugin architecture makes it easy to build scripts and gather the extra application-specific stats that you need. In the case of idio, they developed their own plugins, enabling them to track & alert on application-specific metrics, which they’ve found can be more useful than hardware data alone. Server Density also enables you to access custom plugins via the store, which reduces the need to build custom plugins and allows team to focus on growing out their platform and infrastructure.
- API - The Server Density API allows idio to hook everything into their deployment and scaling mechanisms. This ensures that the monitoring is always in sync with their existing infrastructure, so they never miss anything. Furthermore, the range of alerting systems that you’re able to tweak with the API has meant that the idio team can be notified of issues regardless of their platform or location.
What they've learnt
idio have learnt to watch how their queue sizes change over time, from this information they have been able to optimise the flow of data & allocation of resources by load. They have also been more aware of rising issues before they properly affect them so that preventative action can be taken.
Ultimately, Server Density has forced idio to think a lot about the metrics that are important to their business - having started with too many alerts which created more problems than they solved, they’ve now narrowed down the problems and are focusing their time on the areas and alerts that matter the most to their business.
- ~40 Linux Servers on AWS of various sizes, along with ~10 RDS instances.
- Primarily MySQL, but a mix of Cassandra & ElasticSearch as well.
- Github for code & Chef for deployment.
- Splunk Storm for log aggregation.
- 11 people using Server Density.