Logging Best Practices for AWS Lambda and Microservices

The Benefits of Logging

Why should you even care about logging in the first place? As mentioned above, the “-ilities of logging” are essential to keeping things up, running, easy to use, performant, productive, and easier to maintain.

Visibility

Seeing into your system is not easy! Logging helps give you proper insight into your system.

Traceability

Microservices have increased the number of touch points for a transaction. Logging allows you to trace those transactions across your ecosystem.

Supportability

Combining the visibility and traceability, logging makes supporting your system more manageable for developers, support staff, and non-technical team members.

Telemetry

An honorary “-ility,” telemetry is an awesome benefit to the list. With the proper insight you’re your system, many organizations are able to collect enough data to get good telemetry on the KPIs within their system.

Logging Pain Points

There is no denying that logging pain points are amplified with serverless and can potentially worsen as your serverless footprint grows.

Logging Common Mistakes

Logging mistakes in AWS Lambda create situations that can be as confusing as no logging at all. The list of potential mistakes, unfortunately, is substantial:

  • No design for logging or thinking of logging as an afterthought instead of addressing it from the start
  • Lack of logging standards
  • Hard coding error messages
  • Lack of request identifiers to help you trace a request across microservice boundaries
  • Logging payloads or sensitive data (this happens frequently!
  • Logging too much information, which increases the price of your logging or storage solution
  • Logging too little, which doesn’t give you enough information to work from

Logging Best Practices in AWS Lambda

Standardize logs

Use whatever standards you would like, but ensure they apply across the ecosystem. We recommend and prefer JSON because so many logging targets today understand it.

Informative messages

Using informative logging messages increases both visibility and traceability by letting you see what exactly is occurring inside the system and at what time.

Transaction Identifiers

Using Transaction IDs and including them in every log message across Lambdas improves traceability and lets you trace your transaction from end to end.

Dynamic error messages

With informative, dynamic error messages, it’s easier to debug problems and pinpoint where an error occurred.

Persist logs into a centralized system

With serverless and volumes of microservices, you’ll need a means of combining and aggregating all of your logs. We recommend CloudWatch and CloudWatch Insights, ElasticSearch, ELK, Splunk, etc.

Log at appropriate levels

Determining the appropriate levels requires some thought. You can log at the INFO, ERROR, or DEBUG level for example.

Log the appropriate amount of data

This will optimize log storage costs while also maintaining the right level of visibility.

Use Lambda Layers

If you’re using Lambdas, consider using Lambda Layers to create a reusable logging framework across your Lambdas.

Conclusion

Conceptually, logging is straightforward. It’s also essential. But it becomes difficult with a growing collection of microservices. Big Compass has been through logging implementations with many clients, so we’re familiar with what can go wrong and how to mitigate those risks.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Aaron Lieberman

Aaron Lieberman

19 Followers

Aaron’s passion for technology drives him to find innovative ways to help advance organizations through technology.