{"id":763,"date":"2020-12-22T09:15:57","date_gmt":"2020-12-22T08:15:57","guid":{"rendered":"http:\/\/www.ingemark.com\/?p=763"},"modified":"2023-10-30T11:37:02","modified_gmt":"2023-10-30T10:37:02","slug":"improved-java-microservice-logging-with-mdc","status":"publish","type":"post","link":"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/","title":{"rendered":"Improved Java microservice logging with MDC"},"content":{"rendered":"\n<h3 class=\"wp-block-heading\">Challenge of analyzing logs in microservice architecture<\/h3>\n\n\n\n<p>Modern software systems are often developed using microservice architecture. In this architecture applications are designed as a collection of small components built around business needs. That means that one user request is most often handled by multiple components. In environments with many concurrent users this architecture can make it very difficult to track requests split between different log sources.<br><br>Example of logs from different components:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>16:53:21.139 INFO &#91;main] OrderController : Received new order request\n16:53:21.148 INFO &#91;main] OrderController : Creating new order\n16:53:21.222 INFO &#91;main] OrderController : Received new order request\n16:53:21.258 INFO &#91;main] OrderController : Creating new order\n16:53:21.299 INFO &#91;main] OrderController : Successfully created new order\n16:53:21.350 INFO &#91;main] OrderController : Successfully created new order\n16:53:21.152 INFO &#91;main] ProcessManager : Started process for new order\n16:53:21.260 INFO &#91;main] ProcessManager : Started process for new order\n16:53:21.288 INFO &#91;main] ProcessManager : Finished process for new order\n16:53:21.333 INFO &#91;main] ProcessManager : Finished process for new order<\/code><\/pre>\n\n\n\n<p>One improvement would be to add an user identifier in every log line. That would enable us to easily analyze all the requests in one component. Another improvement would be to introduce a request identifier that would be logged in all the components handling one request. That way we could correlate logs from all the components regarding one user request.<\/p>\n\n\n\n<p>If we were to add those details manually, it would look something like:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>logger.info(\"userId: {}, requestId: {}, Received new order request\", userId, requestId):<\/code><\/pre>\n\n\n\n<p>This method would be very error prone and especially hard to implement in existing components with a lot of code already written. It is very easy to forget to add additional information or make a mistake that will log wrong data.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Better logging with MDC<\/h3>\n\n\n\n<p>Alternative solution is to use Mapped Diagnostic Context (MDC). This enables us to easily add contextual information to each user request. MDC provides a key\/value structure that we can populate at the beginning of processing user requests. MDC will keep that information separately for each running thread and automatically include it in every log line that is produced.<br><\/p>\n\n\n\n<p>MDC is supported by all the major Java logging frameworks: log4j, log4j2 and logback. In a typical web application, we would populate MDC data in an interceptor, so that it is automatically done on each HTTP request.<br><\/p>\n\n\n\n<p>Example of interceptor:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">public class MdcInterceptor implements HandlerInterceptor {\n\n@Override\npublic boolean preHandle(HttpServletRequest request, HttpServletResponse response, Objectandler) throws Exception {\nMDC.put(\"userId\", request.getHeader(\"X-User-Id\"));\nMDC.put(\"requestId\", request.getHeader(\"X-Request-Id\"));\nreturn true;\n}\n@Override\n\npublic void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throws Exception {\nMDC.clear();\n}\n}<\/pre>\n\n\n\n<p>Now we can use MDC data in our logging configuration, by using %X pattern specifier, followed by our MDC key:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">\u2026 %date{ISO8601} %-5level [%thread] [%X{userId} %X{requestId}] %logger{100} : %msg%n ...<\/pre>\n\n\n\n<p>Example of logs with MDC data populated:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">16:53:21.139 INFO [main] [178 21961] OrderController : Received new order request\n16:53:21.148 INFO [main] [178 21961] OrderController : Creating new order\n16:53:21.222 INFO [main] [442 21962] OrderController : Received new order request\n16:53:21.258 INFO [main] [442 21962] OrderController : Creating new order\n16:53:21.299 INFO [main] [178 21961] OrderController : Successfully created new order\n16:53:21.350 INFO [main] [442 21962] OrderController : Successfully created new order\n16:53:21.152 INFO [main] [178 21961] ProcessManager : Started process for new order\n16:53:21.260 INFO [main] [442 21962] ProcessManager : Started process for new order\n16:53:21.288 INFO [main] [178 21961] ProcessManager : Finished process for new order\n16:53:21.333 INFO [main] [442 21962] ProcessManager : Finished process for new order<\/pre>\n\n\n\n<p>Now we can more easily analyze the logs and extract information from specific user or specific user request scattered throughout different logs.<br><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Spring Cloud Sleuth<\/h3>\n\n\n\n<p>One disadvantage of the proposed solution is that we have to populate HTTP headers manually each time a microservice endpoint is called. We have to make sure that on the first user request MDC data is properly generated and then propagated to each component handling user request<br>throughout the system.<br><\/p>\n\n\n\n<p>Alternative solution is to use Spring Cloud Sleuth &#8211; including this library will automatically add additional information like request IDs in your logs. But more importantly Sleuth also has support for propagating such information throughout different microservice components whether they are communicating synchronously using HTTP or asynchronously using Kafka.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Modern software systems are often developed using microservice architecture. In this architecture applications are designed as a collection of small components built around business needs. That means that one user request is most often handled by multiple components. In environments with many concurrent users this architecture can make it very difficult to track requests split between different log sources.<\/p>\n","protected":false},"author":17,"featured_media":859,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Improved Java microservice logging with MDC - Ingemark<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Improved Java microservice logging with MDC - Ingemark\" \/>\n<meta property=\"og:description\" content=\"Modern software systems are often developed using microservice architecture. In this architecture applications are designed as a collection of small components built around business needs. That means that one user request is most often handled by multiple components. In environments with many concurrent users this architecture can make it very difficult to track requests split between different log sources.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/\" \/>\n<meta property=\"og:site_name\" content=\"Ingemark\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/ingemark\" \/>\n<meta property=\"article:published_time\" content=\"2020-12-22T08:15:57+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-10-30T10:37:02+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.ingemark.com\/wp-content\/uploads\/2020\/12\/java-logging-with-mdc-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1960\" \/>\n\t<meta property=\"og:image:height\" content=\"1100\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Ingemark\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@ingemark_com\" \/>\n<meta name=\"twitter:site\" content=\"@ingemark_com\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Ingemark\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/\"},\"author\":{\"name\":\"Ingemark\",\"@id\":\"https:\/\/www.ingemark.com\/#\/schema\/person\/dc101ea6255e7b978f7da11f0d5d6f9d\"},\"headline\":\"Improved Java microservice logging with MDC\",\"datePublished\":\"2020-12-22T08:15:57+00:00\",\"dateModified\":\"2023-10-30T10:37:02+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/\"},\"wordCount\":447,\"publisher\":{\"@id\":\"https:\/\/www.ingemark.com\/#organization\"},\"articleSection\":[\"Technology\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/\",\"url\":\"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/\",\"name\":\"Improved Java microservice logging with MDC - Ingemark\",\"isPartOf\":{\"@id\":\"https:\/\/www.ingemark.com\/#website\"},\"datePublished\":\"2020-12-22T08:15:57+00:00\",\"dateModified\":\"2023-10-30T10:37:02+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.ingemark.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Improved Java microservice logging with MDC\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.ingemark.com\/#website\",\"url\":\"https:\/\/www.ingemark.com\/\",\"name\":\"Ingemark\",\"description\":\"Boutique software agency\",\"publisher\":{\"@id\":\"https:\/\/www.ingemark.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.ingemark.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.ingemark.com\/#organization\",\"name\":\"Ingemark\",\"url\":\"https:\/\/www.ingemark.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ingemark.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.ingemark.com\/wp-content\/uploads\/2023\/10\/ingemark-og-image.png\",\"contentUrl\":\"https:\/\/www.ingemark.com\/wp-content\/uploads\/2023\/10\/ingemark-og-image.png\",\"width\":1200,\"height\":628,\"caption\":\"Ingemark\"},\"image\":{\"@id\":\"https:\/\/www.ingemark.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/ingemark\",\"https:\/\/twitter.com\/ingemark_com\",\"https:\/\/linkedin.com\/company\/ingemark\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.ingemark.com\/#\/schema\/person\/dc101ea6255e7b978f7da11f0d5d6f9d\",\"name\":\"Ingemark\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ingemark.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e61d17bdebe506334a7c450c5fe02bf0?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e61d17bdebe506334a7c450c5fe02bf0?s=96&d=mm&r=g\",\"caption\":\"Ingemark\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Improved Java microservice logging with MDC - Ingemark","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/","og_locale":"en_US","og_type":"article","og_title":"Improved Java microservice logging with MDC - Ingemark","og_description":"Modern software systems are often developed using microservice architecture. In this architecture applications are designed as a collection of small components built around business needs. That means that one user request is most often handled by multiple components. In environments with many concurrent users this architecture can make it very difficult to track requests split between different log sources.","og_url":"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/","og_site_name":"Ingemark","article_publisher":"https:\/\/www.facebook.com\/ingemark","article_published_time":"2020-12-22T08:15:57+00:00","article_modified_time":"2023-10-30T10:37:02+00:00","og_image":[{"width":1960,"height":1100,"url":"https:\/\/www.ingemark.com\/wp-content\/uploads\/2020\/12\/java-logging-with-mdc-1.png","type":"image\/png"}],"author":"Ingemark","twitter_card":"summary_large_image","twitter_creator":"@ingemark_com","twitter_site":"@ingemark_com","twitter_misc":{"Written by":"Ingemark","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/#article","isPartOf":{"@id":"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/"},"author":{"name":"Ingemark","@id":"https:\/\/www.ingemark.com\/#\/schema\/person\/dc101ea6255e7b978f7da11f0d5d6f9d"},"headline":"Improved Java microservice logging with MDC","datePublished":"2020-12-22T08:15:57+00:00","dateModified":"2023-10-30T10:37:02+00:00","mainEntityOfPage":{"@id":"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/"},"wordCount":447,"publisher":{"@id":"https:\/\/www.ingemark.com\/#organization"},"articleSection":["Technology"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/","url":"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/","name":"Improved Java microservice logging with MDC - Ingemark","isPartOf":{"@id":"https:\/\/www.ingemark.com\/#website"},"datePublished":"2020-12-22T08:15:57+00:00","dateModified":"2023-10-30T10:37:02+00:00","breadcrumb":{"@id":"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.ingemark.com\/improved-java-microservice-logging-with-mdc\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.ingemark.com\/"},{"@type":"ListItem","position":2,"name":"Improved Java microservice logging with MDC"}]},{"@type":"WebSite","@id":"https:\/\/www.ingemark.com\/#website","url":"https:\/\/www.ingemark.com\/","name":"Ingemark","description":"Boutique software agency","publisher":{"@id":"https:\/\/www.ingemark.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ingemark.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.ingemark.com\/#organization","name":"Ingemark","url":"https:\/\/www.ingemark.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ingemark.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.ingemark.com\/wp-content\/uploads\/2023\/10\/ingemark-og-image.png","contentUrl":"https:\/\/www.ingemark.com\/wp-content\/uploads\/2023\/10\/ingemark-og-image.png","width":1200,"height":628,"caption":"Ingemark"},"image":{"@id":"https:\/\/www.ingemark.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/ingemark","https:\/\/twitter.com\/ingemark_com","https:\/\/linkedin.com\/company\/ingemark"]},{"@type":"Person","@id":"https:\/\/www.ingemark.com\/#\/schema\/person\/dc101ea6255e7b978f7da11f0d5d6f9d","name":"Ingemark","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ingemark.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e61d17bdebe506334a7c450c5fe02bf0?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e61d17bdebe506334a7c450c5fe02bf0?s=96&d=mm&r=g","caption":"Ingemark"}}]}},"_links":{"self":[{"href":"https:\/\/www.ingemark.com\/wp-json\/wp\/v2\/posts\/763"}],"collection":[{"href":"https:\/\/www.ingemark.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ingemark.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ingemark.com\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ingemark.com\/wp-json\/wp\/v2\/comments?post=763"}],"version-history":[{"count":1,"href":"https:\/\/www.ingemark.com\/wp-json\/wp\/v2\/posts\/763\/revisions"}],"predecessor-version":[{"id":4702,"href":"https:\/\/www.ingemark.com\/wp-json\/wp\/v2\/posts\/763\/revisions\/4702"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.ingemark.com\/wp-json\/wp\/v2\/media\/859"}],"wp:attachment":[{"href":"https:\/\/www.ingemark.com\/wp-json\/wp\/v2\/media?parent=763"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ingemark.com\/wp-json\/wp\/v2\/categories?post=763"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ingemark.com\/wp-json\/wp\/v2\/tags?post=763"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}