How to Define Elastic Search Templates for Apache Metron

When you onboard a new data source on Apache Metron and you use Elastic Search (ES) as your indexing + search engine you need to specify and submit an ES template before the indexing topology attempts the first write to the ES cluster. The template should contain the following items: Dynamic fields for possible geo enrichments of any ip address field, dynamic fields for other kinds … Continue reading How to Define Elastic Search Templates for Apache Metron

How to Create a New Parser for Apache Metron

This blog entry goes through the process of a Cyber Platform Operator creating a new parser for Apache Metron and everything you need to consider to make this process as smooth as possible. This can also be seen as a checklist or to-do list when you are creating a new parser. Assumption: You know what Metron is, the data source is fully onboarded on your … Continue reading How to Create a New Parser for Apache Metron

Apache Metron as an Example for a Real Time Data Processing Pipeline

In my previous blog post I was writing a little bit about what Apache Metron is and How to Onboard a New Data Source in Apache Metron. Now I want to shine some light on how the ingestion pipeline architecture looks like. Since I just got started with Apache Metron myself, I hope this helps to kickstart your cyber security efforts. Rather than going too … Continue reading Apache Metron as an Example for a Real Time Data Processing Pipeline

How to Onboard a New Data Source in Apache Metron

Introduction Apache Metron aims to be a tool for analysts in a cyber security team to help them defining intelligent alerts, detecting threats and work on them in real-time. This is the first blog post in a row to ease operations and share my experiences with Apache Metron. Thus, it serves as an introduction to Metron. Technical Introduction Apache Metron is a cyber security platform making heavy … Continue reading How to Onboard a New Data Source in Apache Metron

How to Troubleshoot an Apache Storm Topology

Apache Storm is a real-time, fault-tolerant, event-based streaming framework and platform that runs your code in a highly parallelized way on distributed nodes. It’s all about Spouts (processing units to read from data sources) and Bolts (general processing units). Storm is often used to read data from Apache Kafka and write the results back to Kafka or to a data store. Apache Storm and Apache Kafka are the … Continue reading How to Troubleshoot an Apache Storm Topology

Basics of Hadoop User Management

Hadoop is old, everyone has their own Hadoop cluster and everyone knows how to use it. It’s 2018, right? This article is just a collection of a few gotchas, dos and don’ts with respect to User Management that shouldn’t happen in 2018 anymore. Terminology Just a few terms and definitions so that everyone is on the same page for the rest of the article. Roll … Continue reading Basics of Hadoop User Management

4 Things Factorio Taught Me about DevOps

What is Factorio? Factorio is a computer game. You probably ask yourself, in which ways a computer game is related to this blog? Well, not at all – or is it? Let’s find out. Basically, in the game you take over the role of a character in 3rd person perspective, whose rocket ship crashed on a foreign planet. You don’t have anything, but a pick … Continue reading 4 Things Factorio Taught Me about DevOps