Analyze and filter server and application access log files.
When an application runs, it emits logs to help developers trace execution. Historically, these logs were unstructured plain-text strings: 2023-10-25 14:32:01 [ERROR] Failed to connect to DB at 192.168.1.5.
To analyze unstructured logs, systems use Regular Expressions to extract the timestamp, severity, and IP into separate columns. Writing a regex that perfectly handles every edge case in a giant block of text is notoriously fragile and slow.
Modern applications use Structured Logging. Instead of a text string, the application outputs a JSON object: { "time": "2023-10-25...", "level": "ERROR", "message": "Failed...", "ip": "192.168.1.5" }.
Because it is already structured, log ingestion engines (like ELK, Datadog, or Splunk) don't need expensive regex parsing. They can ingest the JSON directly, allowing you to instantly query "show me all ERRORs where IP = 192.168.1.5" across billions of rows in milliseconds.
Logs are a massive liability if not handled correctly. A common engineering failure is accidentally logging Personally Identifiable Information (PII) like passwords, credit card numbers (PANs), or API tokens in plain text.
If a sensitive token ends up in a central logging system, that entire logging cluster becomes a high-value target for hackers. Applications must implement strict middleware to redact or mask sensitive fields before the log is written to disk.
Why is 'Structured Logging' (e.g., logging as JSON objects) preferred over plain-text logging in modern large-scale applications?
