What is the core difference between ELK and Loki?

ELK indexes log content for full-text search — fast queries but high storage cost. Loki indexes only metadata (labels), not log content — low storage cost, but complex queries (full-text search) are slower than ELK. Recommendation: if you have high log volume (hundreds of GB daily) and primarily query by label filters, choose Loki. If you need full-text search and complex aggregation, choose ELK.

How long should logs be retained?

Depends on business needs. Recommended tiered strategy: hot storage (fast queries) — 7 days, warm storage (queryable but slower) — 30 days, cold storage (archived, restore on demand) — 1 year. By importance: security logs — 1+ year, business logs — 30-90 days, debug logs — 7 days.

What log collection tool should I use?

Filebeat (ELK ecosystem) and Promtail (Loki ecosystem) are the most common log collection agents. Both are lightweight, support multiple input sources (files, stdout, syslog), and auto-discover new log files. If your app runs on Docker or K8s, prefer collecting container stdout logs over writing log files.

Do logs affect application performance?

Yes. Excessive logging (especially DEBUG-level logs on every request) can bottleneck disk I/O. Recommendations: ① Production: log only WARN and ERROR by default, INFO for key operations, DEBUG only during troubleshooting; ② Use async logging (e.g., Winston async transports) to avoid blocking the main flow; ③ Configure log rotation to prevent oversized log files.

Centralized Log Management: ELK vs Loki Architecture and Selection

Logging is not about "writing to a file" — it is about "finding the root cause quickly when something breaks." This article covers log collection, storage, and querying, comparing ELK (Elasticsearch + Logstash + Kibana) and Loki + Grafana — for backend developers and ops engineers building or optimizing log systems.

The Bottom Line: The Core of a Log System Is Not “How Much You Store” — It Is “How Fast You Can Find”

Many teams focus on “can we collect all the logs” and overlook “can we find the issue quickly when something breaks.” A system storing terabytes of logs with multi-minute query times is worse than one storing 7 days with sub-second response.

1. Solution Comparison

ELK (Elasticsearch + Logstash + Kibana)

App → Filebeat → Logstash → Elasticsearch → Kibana

Full-text indexing on log content, flexible queries
High storage cost
Best for full-text search and complex aggregation

Loki + Grafana

App → Promtail → Loki → Grafana

Indexes only labels, not log content
Low storage cost
Best for label-based filtered queries

Selection Guide

Dimension	ELK	Loki
Query flexibility	Full-text, any field	Label-based, limited content search
Storage cost	High	Low
Query speed	Fast (full-text index)	Fast (label filter)
Ops complexity	High (3 components)	Low (1 component + Grafana)
Best for	Full-text search, complex analysis	Label filtering, K8s environments

2. Log Levels

Level Definitions

const logLevels = {
  ERROR: 'Functionality-breaking errors requiring immediate action',
  WARN:  'Potential issues requiring attention, not immediate action',
  INFO:  'Key operations, e.g., user login, order creation',
  DEBUG: 'Detailed debug info, enabled only during troubleshooting',
};

Production Configuration

const logger = winston.createLogger({
  level: 'warn',
  transports: [new winston.transports.Console()],
});

3. Log Content Standards

Structured Logging

// ❌ Unstructured
"User login failed"

// ✅ Structured
{
  "timestamp": "2026-07-21T10:00:00Z",
  "level": "WARN",
  "service": "auth-service",
  "requestId": "req-123",
  "userId": "user-456",
  "message": "User login failed",
  "reason": "Incorrect password",
  "duration_ms": 120
}

Required Fields

Field	Description	Example
timestamp	Event time	2026-07-21T10:00:00Z
level	Log level	ERROR, WARN, INFO, DEBUG
service	Service name	auth-service
requestId	Request trace ID	req-abc123
message	Description	User login failed
duration_ms	Duration in ms	120

4. Log Querying

Common Patterns

# Query all errors for a service
{service="auth-service"} |= "ERROR"

# Query complete request chain
{requestId="req-abc123"}

# Query errors with duration > 1s
{level="ERROR"} | logfmt | duration_ms > 1000

Summary

Layer	Key Principle	Common Mistake
Solution selection	Choose by query needs, not popularity	ELK for everything regardless of scale
Log levels	WARN+ in production	DEBUG in production
Log format	Structured JSON with required fields	String concatenation, unparseable
Querying	Trace by requestId	No trace ID, no context

The value of a log system is not “how much log you stored” — it is “how fast you can find the root cause when something breaks.” A well-designed log system turns troubleshooting from “finding a needle in a haystack” into “following a map.”

Need log system design or ops services? Contact us — tell us about your service scale and query needs, feasibility within 24 hours.