On: Sensu

tl;dr Yes, it's awesome, just have some patience during the setup.

What is Sensu?

Sensu is described as a monitoring router. It will handle the check scheduling and the storing of check results.

Getting Started

So, you're sold on the Sensu idea or you just want to try it out? Great! Instead of reiterating what the Sensu Devs have done I'll link to thier guide and provide details that helped me.

Official Guide: http://sensuapp.org/docs/0.17/installation-overview#introduction

Pro-tips:

  1. Install the omnibus package. DO NOT try to do it another way, it's a huge pain for little gain
  2. Use the embedded Ruby install. Change the value in /etc/default/sensu to True. You'll thank me later when everything 'just works'.
  3. If you've never dealt with RabbitMQ or Redis then only use one machine as the master node when you start out. You can always split out Redis and RabbitMQ in the future.
  4. As the guide does, start out with one client machine and one master. Add a couple of checks and go from there.
  5. While you don't NEED configuration management to setup Sensu, it makes your life significantly easy when you can deploy all the checks to each client with one command.
  6. Uchiwa doesn't offer much, it's nice to have to verify checks visually but it lacks a lot of features. (Read: Historical data, updating any value in configs, restarting sensu-server/api, ect.)
  7. Browse the sensu community plugin repo. Tons of plugins/handlers! If there isn't one there, make it and submit a pull request ;)
  8. Read through every check you are going to implement. Many require specific ruby gems to be installed on the client. (Using the embedded ruby, you can use the gem command located in /opt/sensu/embedded/bin/gem)

Got it setup? Now what?

Well cool! You did it! Now you're going to want to start getting more and more checks running. They are nearly dead simple to add, just make sure the file permissions are right and that all required gems are installed. I typically start out with global checks that I want running on every server. Then I'll go through and plan out which server specific checks should be added.

Handlers - The real magic

I really enjoy the Sensu handlers! They aren't terribly special or awe-inspiring but they work very well. Once you get your first Pushover notification or a PM on IRC I think you'll understand why too. Coming from years of poorly to not even monitored systems, this is incredible.

My Favorite Handlers:

  1. StatusPage.rb - Updates your statuspage.io automatically so you can just go and fix your stuff instead
  2. IRC.rb - Setup to private message you or a channel. It can auth with NickServ too making everything easier.
  3. Pushover.rb - Sends a pushover notification whenever stuff breaks. Great for smaller systems or where one person is managing a server.

Interfaces

So, you might be asking, where's my pretty UI? Well sadly this is still a work in progress. Uchiwa is the official UI but it really is limited in it's functionality. I've been researching Flapjack.io and hope to use that along side Uchiwa. There is no perfect solution yet, but maybe someone (you?) will write one.

Closing Notes

Sensu, while young, is a reliable and very much usable system that can easily monitor your system. It's excellent for a dynamic environment and makes any SysAdmins life easier.