Field Notes

The Problem With a Field Called User - Security data normalization makes correlation possible. It can also change what an event appears to mean.

Security monitoring depends on our ability to combine information from systems that were never designed to speak the same language. Firewalls, operating systems, identity platforms, applications and cloud services all produce events describing different activities, using different formats and often different terminology.

Normalization attempts to solve this problem by giving the data a common structure. Instead of building a separate detection rule for every product, we map events into a consistent set of fields and use those fields for searches, correlation, dashboards and alerting.

It is an entirely reasonable approach and one of the foundations of modern SIEM engineering.

There is, however, an interesting problem hidden inside it. Two events can contain the same field name without describing the same thing. Even more concerning, two perfectly valid data models can assign different meanings to that field.

When that happens, the data can be technically correct while the conclusions drawn from it are wrong.

If Monitoring Fails With the Service, It Was Never Enough - Observability is only useful if enough of it survives the failure it is supposed to explain.

Categories: [Field Notes], [Observability]

Monitoring systems are usually designed around a simple question: what information do we need in order to understand the state of our infrastructure and applications? We collect metrics, logs, traces, application telemetry and synthetic measurements, and then use them to build a picture of how the e...