CloudWatch logging basics

What you'll learn

By the end of this you'll know how CloudWatch Logs works and how to use it effectively for a Node.js application running on AWS. You'll understand log groups, log streams, and retention policies. You'll know how to write structured JSON logs so they're queryable, how to use CloudWatch Insights to search logs with real queries, how to create metric filters that turn log patterns into numeric metrics, and how to set up alarms that notify you when error rates spike.

Logging on AWS is different from logging to a file or a third-party service. Getting the CloudWatch mental model right makes the difference between logs that are searchable and actionable versus logs that are technically there but practically useless.

Who this is for

• Developers who've deployed Lambda or ECS applications and want to understand the logs they're seeing in CloudWatch
• Anyone who wants to go beyond console.log to structured, queryable logging for production applications on AWS
• Backend developers setting up monitoring and alerting for the first time on AWS

You can skip this if you're already using a third-party logging service (Datadog, Grafana, Splunk) that pulls from CloudWatch and you're comfortable with their query interface. Come back if you want to understand what's happening at the CloudWatch layer, which those services ultimately pull from.

What is CloudWatch Logs?

CloudWatch Logs is AWS's managed log storage and query service. Lambda, API Gateway, ECS, EC2, and most other AWS services send their output to CloudWatch Logs automatically. Your application code can also write directly to CloudWatch Logs using the SDK, though most developers let their runtime (Lambda, ECS) do that automatically via stdout.

Plain English: CloudWatch Logs is where AWS puts the output from your running code. Every console.log in a Lambda function ends up in CloudWatch, organized by function name and execution time, queryable with a SQL-like language.

Simple idea: logs are grouped by source (a log group), broken into streams by instance or invocation, and each entry is a timestamped line of text. Structured JSON logging makes those lines queryable by field value rather than regex on raw text.

Prerequisites

• An AWS account with a Lambda function or ECS service already deployed (CloudWatch Logs is populated by running services)
• Basic understanding of Lambda or whichever AWS compute service you're using
• IAM permissions: logs:DescribeLogGroups, logs:GetLogEvents, logs:StartQuery, logs:GetQueryResults

Setup from zero

Step 1 — Find your log group

Lambda automatically creates a log group named /aws/lambda/your-function-name when the function first executes. Open the Lambda Console → your function → Monitor → View CloudWatch logs. This opens CloudWatch Logs filtered to your function's log group.

For ECS, log groups are configured in the task definition under the awslogs log driver:

"logConfiguration": {
  "logDriver": "awslogs",
  "options": {
    "awslogs-group": "/ecs/my-service",
    "awslogs-region": "us-east-1",
    "awslogs-stream-prefix": "ecs"
  }
}

Step 2 — Set a retention policy

By default, CloudWatch log groups retain logs indefinitely — and you pay for storage. Set a retention period to control costs:

aws logs put-retention-policy   --log-group-name "/aws/lambda/my-function"   --retention-in-days 30   --profile my-app-dev

Common retention periods: 7 days for debug logs, 30 days for application logs, 90 days for audit logs. Use Infrastructure as Code (CloudFormation or CDK) to set retention automatically when creating log groups.

Step 3 — Write structured logs

A plain console.log("something happened") in Lambda is stored as-is. To query by fields (level, requestId, error message), log JSON:

// src/lib/logger.ts
export function log(
  level: "info" | "warn" | "error",
  message: string,
  fields: Record<string, unknown> = {}
): void {
  console.log(
    JSON.stringify({
      timestamp: new Date().toISOString(),
      level,
      message,
      ...fields,
    })
  );
}

Use it in the Lambda handler:

import { log } from "./logger";

export const handler = async (event: APIGatewayProxyEventV2, context) => {
  log("info", "Request received", {
    requestId: context.awsRequestId,
    path: event.rawPath,
    method: event.requestContext.http.method,
  });

  try {
    // ... business logic
    log("info", "Request completed", { requestId: context.awsRequestId, statusCode: 200 });
    return { statusCode: 200, body: JSON.stringify({ ok: true }) };
  } catch (err) {
    log("error", "Request failed", {
      requestId: context.awsRequestId,
      error: err instanceof Error ? err.message : String(err),
    });
    return { statusCode: 500, body: JSON.stringify({ error: "Internal error" }) };
  }
};

> Little tip: Always include the context.awsRequestId in every log entry from a Lambda function. Each Lambda invocation gets a unique request ID. When you're debugging a specific failed request, the request ID lets you filter CloudWatch Insights to show only the log lines from that single invocation — even if thousands of invocations are interleaved in the same log stream.

The mental model

CloudWatch Logs has three levels of hierarchy: log groups, log streams, and log events.

Log group — the top-level container. One per application component: /aws/lambda/my-function, /ecs/my-service. This is the unit of retention policy, metric filter, and access control.

Log stream — a sequence of log events from one source. Lambda creates one log stream per function instance (2026/06/14/[$LATEST]abc123). ECS creates one per task. If 50 Lambda instances run simultaneously, there are 50 log streams writing in parallel to the same log group.

Log event — a single timestamped line of text. The text can be anything — plain text, JSON, or structured data — but JSON is the only format you can query by field value in Insights.

The most useful mental model for querying: think of CloudWatch Insights as SQL over your log lines. Each JSON field in your log entry is a column. stats count(*) by level counts log entries grouped by the level field. filter level = "error" shows only error entries. The Insights query language is small but powerful enough for the diagnostics you'll actually need.

Key terms

Log group — the container for log streams from one source. One log group per service or function. The unit for retention policy and metric filters.

Log stream — a time-ordered sequence of events from one source instance. Lambda creates one per instance; ECS creates one per task. Multiple streams can write to the same group simultaneously.

Metric filter — a rule that watches a log group for a text pattern and increments a CloudWatch metric each time the pattern matches. Used to turn log events (like error-level entries) into numeric metrics that can trigger alarms.

CloudWatch Insights — CloudWatch's query engine for log groups. Uses a SQL-like query language. Supports filtering, aggregation, and sorting across large volumes of logs across multiple log streams.

Alarm — a CloudWatch resource that watches a metric and transitions between OK, ALARM, and INSUFFICIENT_DATA states based on thresholds. Alarms can notify via SNS (email, SMS, Lambda invocation, PagerDuty).

Retention period — how long CloudWatch keeps log events before automatically deleting them. Set per log group. Default is indefinite (and costly at scale).

Step-by-step: querying logs with CloudWatch Insights

In the CloudWatch Console → Logs → Insights:

1. Select one or more log groups
2. Set the time range (last 1 hour, 24 hours, etc.)
3. Write a query

Count errors in the last 24 hours:

filter level = "error"
| stats count(*) as errorCount by bin(1h)

Find slow requests (over 2 seconds) — if you log a durationMs field:

filter durationMs > 2000
| sort durationMs desc
| limit 20

See all log events from one specific invocation:

filter requestId = "abc123-your-request-id-here"
| sort @timestamp asc

Show the most common error messages:

filter level = "error"
| stats count(*) as occurrences by error
| sort occurrences desc

> Little tip: CloudWatch Insights queries are not free — you're billed per GB of log data scanned. Narrow your time range and select only the relevant log group before running a query. A query across 30 days of high-volume logs can cost several dollars; the same query scoped to 1 hour costs cents. Always start narrow and expand only if the data you need isn't there.

Patterns

Metric filter for error rate — create a metric filter that counts JSON log entries where level = "error":

1. CloudWatch Console → Log groups → your log group → Metric filters → Create metric filter
2. Filter pattern: { $.level = "error" }
3. Metric name: ErrorCount, namespace: MyApp, value: 1

Now every error log entry increments the ErrorCount metric.

Alarm on error rate — attach an alarm to the metric:

1. CloudWatch Console → Alarms → Create alarm
2. Select the ErrorCount metric you just created
3. Threshold: greater than 10 errors in 5 minutes
4. Action: notify an SNS topic (email or PagerDuty)

Log Lambda initialization separately — code outside the handler runs once per cold start. Log it explicitly to measure cold start frequency:

// Module level — runs on cold start
const startTime = Date.now();
const s3 = new S3Client({ region: process.env.AWS_REGION! });
console.log(JSON.stringify({ level: "info", message: "Cold start", initMs: Date.now() - startTime }));

Common mistakes

Logging plain text instead of JSON — console.log("Error: " + err.message) is stored as a string. You can search for it with filter @message like "Error:", but you can't filter by error type, request ID, or any other field. JSON from the start costs no extra effort and pays dividends in every debugging session.

Not setting retention periods — log storage in CloudWatch is priced per GB. A high-volume Lambda logging 1 KB per invocation at 100 requests/second generates 8 GB/day. Without a retention policy, that accumulates indefinitely. Set retention policies on all log groups, even development ones.

Logging sensitive data — CloudWatch Logs is not encrypted by default (though you can enable KMS encryption per log group). Never log passwords, full JWT tokens, credit card numbers, or other regulated data. Redact or omit sensitive fields before logging the event object.

Troubleshooting

No logs appearing for a Lambda function — the Lambda execution role is missing logs:CreateLogGroup, logs:CreateLogStream, or logs:PutLogEvents permissions. Check the execution role's attached policies. Attaching the AWS managed policy AWSLambdaBasicExecutionRole grants the minimum CloudWatch permissions Lambda needs.

Logs appear but Insights queries find nothing — Insights only indexes logs received within its query window. Check that the log group and time range in Insights match when the logs were actually written. Also verify that the logs are JSON — Insights can only filter by named fields on valid JSON entries.

High CloudWatch costs from logs — enable log compression (PutLogEvents compresses automatically), reduce log verbosity for info-level entries, shorten retention periods, and filter out high-frequency debug logs before they're written. The AWS_LAMBDA_LOG_LEVEL environment variable controls Lambda's own internal logs.

Checklist

• [ ] Retention period set on every log group (7–90 days depending on use case)
• [ ] All application logs written as JSON with level, message, and requestId fields
• [ ] Lambda execution role includes logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents
• [ ] CloudWatch Insights used for querying, not manual log stream browsing
• [ ] Metric filter created for error-level log entries
• [ ] Alarm configured on the error metric with an SNS notification action
• [ ] No passwords, tokens, or PII in log entries
• [ ] Insights queries scoped to narrow time ranges and specific log groups

Practice task

Take a Lambda function with basic logging and upgrade it to structured JSON logging. Add fields: level, message, requestId, durationMs (measure the handler duration), and statusCode. Deploy and trigger the function several times with both successful and failing inputs. Open CloudWatch Insights and write three queries: one that lists all error entries with their messages, one that calculates the average durationMs per hour, and one that shows all log entries from a single specific invocation using its request ID. Then create a metric filter for error entries and an alarm that fires if more than 5 errors occur within 5 minutes.

FAQ

Should I use CloudWatch Logs or a third-party service like Datadog?

For simple applications with moderate log volume, CloudWatch Insights is sufficient and avoids a third-party dependency. For complex queries, dashboards across multiple services, long-term retention, or teams that prefer a dedicated observability platform, Datadog, Grafana, or similar tools add value — they subscribe to CloudWatch via a subscription filter and ingest logs in near-real-time. CloudWatch is the source either way.

How do I share logs between multiple AWS accounts?

Use CloudWatch Logs subscription filters to stream logs to a cross-account destination (Kinesis Data Firehose or a Lambda in another account). This is the standard pattern for centralizing logs from multiple AWS accounts in an organization into a single logging account.

Can I search logs in real time?

CloudWatch Insights operates on logs already stored, with a delay of a few seconds to a few minutes. For real-time log tailing during development, use aws logs tail /aws/lambda/my-function --follow from the AWS CLI — it streams new log events to your terminal as they arrive.

What to learn next

After CloudWatch logging basics: CloudWatch Embedded Metric Format (EMF) for publishing custom metrics directly from application code without metric filters, AWS X-Ray for distributed tracing across Lambda and API Gateway, and CloudWatch dashboards for building operational views across multiple metrics and alarms.

Related on Baseline

• AWS Lambda basics — the compute service that produces the logs covered here
• IAM least privilege — the CloudWatch Logs permissions a Lambda execution role needs
• AWS for Node apps — the credential and SDK foundation that CloudWatch SDK calls use

Takeaways

CloudWatch Logs organizes output into log groups (one per service), log streams (one per instance or task), and log events (individual timestamped entries). Structured JSON logging makes log entries queryable by field in CloudWatch Insights. Metric filters turn log patterns into numeric metrics; alarms notify you when those metrics cross thresholds. Set retention periods on all log groups from day one — log storage costs accumulate silently without them.

If you remember only one thing: log JSON from the start, not formatted strings. console.log(JSON.stringify({ level: "error", message: err.message, requestId })) takes the same effort as console.log("Error: " + err.message) but makes every future debugging session faster — because you can query by field, filter by level, and correlate by request ID across thousands of interleaved invocations instead of reading line by line.