Log Analysis with Graylog2 
by Lennart Koopmann (TORCH GmbH)
Friday, 09.05.2014, Stage 11, 14:00-14:45 Uhr
Track: dc5
Graylog2 is a free and open source log analysis tool that allows you to perform searches, analyse the data, build dashboards and set alarms using the streams system. Typical use cases range from debugging platform problems, monitoring exception counts to displaying average pizza delivery time per state on a dashboard. You will see a live demo of Graylog2 with the management tools, searches and analysis in action.
There are several ways to design a Graylog2 system to manage your machine data and logs: From small setups on a single box to highly available architectures with message brokers and load balancers that allow you to shut down parts of the system without losing any transmitted data. I will talk about best practice setups and what we learned about real life Graylog2 operations at our customers.
Another important part to consider is how to send in data. What transport? TCP? UDP? Maybe even using HTTP? What is GELF? There are a lot of things to consider regarding transport security, reducing load on the producing clients and most importantly structuring the data so you make sense out of it later. I will share our experiences of the last 4 years, how we failed, what broke and what we learned from it.
About the author Lennart Koopmann:
Lennart Koopmann has started the open source Graylog2 project in 2010 and founded TORCH as a company behind it in 2012. A team of developers at TORCH is working on Graylog2 in full time since 2013. Lennart worked at Jimdo and XING before where he collected years of experience in developing and maintaining huge distributed systems and architectures. When not working on Graylog2 he is training for an Ironman triathlon with differing success.