Feign Calls Pegged the CPU: Repeated HttpMessageConverter Initialization

Posted on Edited on Views: Word count in article: 3k Reading time ≈ 3 mins.
A production postmortem: Feign calls pegged the CPU because HttpMessageConverters were re-initialized on every call, re-running class loading and Jackson setup. Full diagnosis chain and fixes.

A record of one application incident: analysis, investigation, resolution.

A CPU alert came in and the business team reported that API responses were taking too long. This post records the entire process — from the alert, through diagnosis, to root cause analysis — ending at a CPU saturation problem caused by Feign calls.

🕰️ Version note: this post was written in 2024, on a stack of Feign 10.1.0, Spring 5.1.6, Java 8 (1.8.0_242). The ecosystem moves fast; read it against the versions you actually use.

Background

On the morning of the 9th, a CPU alert came in; at the same time, the business team reported that upstream services depending on this one were seeing very slow API responses.

1
2
3
4
5
6
7
8
Application alert - CPU usage change
[WARNING] project XXX, cluster qd-aliyun, partition bbbb-prod, app customer, instance customer-6fb6448688-m47jz: POD CPU request usage >= 90.000000%, current value 138.4971051199925%
Occurred: 2024/10/09 11:17:33


project XXX, cluster qd-aliyun, partition bbbb-prod, app customer, instance customer-6fb6448688-28pvs: POD CPU request usage >= 90.000000%, current value 157.7076205766934% — alert recovered
Occurred: 2024/10/09 11:06:33
Recovered: 2024/10/09 12:24:33

Traffic

Peak QPS per instance was around 100.

Why care about QPS? Because 100 QPS shouldn’t consume this much CPU — and the request and response bodies were small.

网络流入情况
网络流出情况

POD monitoring

POD quotas:

  • CPU request 2 cores, CPU limit 3 cores
  • Memory request 7 GiB, memory limit 9 GiB

POD CPU
POD TCP链接
POD 线程数

From the charts:

  • CPU load stayed high;
  • TCP connections and thread count climbed steeply starting at 11:40.

ARMS

Looking at the trace monitoring, we saw that the time was going into the customer service calling external APIs via Feign.

生产环境Arms监控 标识占用CPU过高接口

Interim mitigation

Scale out instances and increase the CPU allocation.

Root cause analysis

The investigation of the third-party APIs and the open-platform gateway is omitted here; the conclusion was that neither the third-party APIs nor the open-platform gateway had a problem.
They were checked first because the trace showed the time going into calls to third-party APIs.

getAddressWarehouse 放大

From the ARMS chart:

  • CPU time was concentrated in the Decoder and Encoder of the Feign calls;
  • The Decoder/Encoder time was concentrated in:
    • HttpMessageConverters#getDefaultConverters() =>
    • WebMvcConfigurationSupport#addDefaultHttpMessageConverters =>
    • ……(full call chain in the excerpt below)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
feign.ReflectiveFeign$BuildTemplateByResolvingArgs.create(Object[]) (14.37%, 1.43 minutes)
feign.ReflectiveFeign$BuildEncodedTemplateFromArgs.reesolve(Object[], RequestTemplate, Map) (14.37%, 1.43minutes)
org.springframework.cloud.openfeign.support.SpringEndcoder.encode(Object, Type, RequestTemplate) (14.28%,1.42 minutes)
com.jiankunking.common.core.feign.FeignClientsConfig$$ambda$938.56729293.get0bject() (13.98%, 1.39 minutes 
com.jiankunking.common.core.feign.FeignClientsConfig.lambda$feignEncoder$2() (13.98%, 1.39 minutees)
org.springframework.boot.autoconfigure.http.HttpmessaageConverters.<init>(HttpMessageConverter[]) (12.03%,1.19 minutes)
prg.springframework.boot.autoconfigure.http.Http.HttpMessageConverters.<init>(Collection) (12.03%, 119 minutes)
org.springframework.boot.autoconfigure.http.HttpmessaageConverters.<init>(boolean, Collection) (12.03%, 1.19 minutes)
prg.springframework.boot.autoconfigure.http.Http.HttpMessageConverters.getDefaultConverters()(12.02%, 1.19 minutes
org.springframework.boot.autoconfigure.http.HttpmessageConverters$1.defaultMessageConverters() (12.02%, 119 minutes)
org.springframework.web.servlet.config.annotation.WebMvcConfigurationSupport.getMessageConverters() (12.02%, 1.19 minutes)
org.springframework.web.servlet.config.annotation. WebMvcConfigurationSupport.addDefaultHttpMessageConverters(List) (12.02%, 1
org.springframework.http.converter.json.Jackson2ObjectMapperBuilder.build() (5.93%, 0.59 minutes)
org.springframework.http.converter.json.Jackson2ObjectMapperBuilder.configure(ObjectMapper)(5.91%, 0.59 minutes)
org.springframework.http.converter.json.Jackson2Objec:tMapperBuilder.registerWellKnownModulesIfAvailable(Map)(5.89%,0.58 min
org.springframework.util.ClassUtils.forName(String, CClassLoader)(5.84%, 0.58 minutes)
java.lang.Class.forName(String, boolean, Classloader) (5.83%, 0.58 minutes)
java.lang.Class.forName0(String, boolean, ClassLoader, Class) (5.83%, 0.58 minutes)
......

Custom Encoder and Decoder

Encoder

Look at the Encoder in com.jiankunking.common.core.feign.FeignClientsConfig:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
public Encoder feignEncoder() {
    ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(new RMappingJackson2HttpMessageConverter());
    return new SpringEncoder(objectFactory);
}

public class RMappingJackson2HttpMessageConverter extends MappingJackson2HttpMessageConverter {

    public RMappingJackson2HttpMessageConverter(ObjectMapper objectMapper) {
        super(objectMapper);
        List<MediaType> mediaTypes = new ArrayList<>();
        mediaTypes.add(MediaType.valueOf(MediaType.APPLICATION_JSON_UTF8_VALUE));
        mediaTypes.add(MediaType.valueOf(MediaType.TEXT_HTML_VALUE + ";charset=UTF-8"));
        setSupportedMediaTypes(mediaTypes);
    }

    RMappingJackson2HttpMessageConverter() {
        List<MediaType> mediaTypes = new ArrayList<>();
        mediaTypes.add(MediaType.valueOf(MediaType.APPLICATION_JSON_UTF8_VALUE));
        mediaTypes.add(MediaType.valueOf(MediaType.TEXT_HTML_VALUE + ";charset=UTF-8"));
        setSupportedMediaTypes(mediaTypes);
    }
}

Decoder

Look at the Decoder in com.jiankunking.common.core.feign.FeignClientsConfig:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
public Decoder feignDecoder() {
    HttpMessageConverter jacksonConverter = new MappingJackson2HttpMessageConverter(customObjectMapper());
    ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(jacksonConverter);
    return new ResponseEntityDecoder(new RSpringDecoder(objectFactory));
}

public ObjectMapper customObjectMapper() {
    ObjectMapper objectMapper = new ObjectMapper();

    objectMapper.registerModule(new StringToDateModule());
    objectMapper.configure(JsonParser.Feature.ALLOW_COMMENTS, true);
    objectMapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true);
    objectMapper.configure(JsonParser.Feature.ALLOW_SINGLE_QUOTES, true);
    objectMapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_CONTROL_CHARS, true);
    objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);

    return objectMapper;
}

A quick search for ‘spring feign encode jackson cpu usage high’
=> https://segmentfault.com/a/1190000043037032
=> https://mp.weixin.qq.com/s/RuqltkN9VdVQ1K3GKuJ-Gw
=> https://meantobe.github.io/2019/12/21/ClassLoader/

Source analysis

Look at the code around registerWellKnownModulesIfAvailable:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

@SuppressWarnings("unchecked")
private void registerWellKnownModulesIfAvailable(Map<Object, Module> modulesToRegister) {
	try {
		Class<? extends Module> jdk8ModuleClass = (Class<? extends Module>)
				ClassUtils.forName("com.fasterxml.jackson.datatype.jdk8.Jdk8Module", this.moduleClassLoader);
		Module jdk8Module = BeanUtils.instantiateClass(jdk8ModuleClass);
		modulesToRegister.put(jdk8Module.getTypeId(), jdk8Module);
	}
	catch (ClassNotFoundException ex) {
		// jackson-datatype-jdk8 not available
	}

	try {
		Class<? extends Module> javaTimeModuleClass = (Class<? extends Module>)
				ClassUtils.forName("com.fasterxml.jackson.datatype.jsr310.JavaTimeModule", this.moduleClassLoader);
		Module javaTimeModule = BeanUtils.instantiateClass(javaTimeModuleClass);
		modulesToRegister.put(javaTimeModule.getTypeId(), javaTimeModule);
	}
	catch (ClassNotFoundException ex) {
		// jackson-datatype-jsr310 not available
	}

	// Joda-Time present?
	if (ClassUtils.isPresent("org.joda.time.LocalDate", this.moduleClassLoader)) {
		try {
			Class<? extends Module> jodaModuleClass = (Class<? extends Module>)
					ClassUtils.forName("com.fasterxml.jackson.datatype.joda.JodaModule", this.moduleClassLoader);
			Module jodaModule = BeanUtils.instantiateClass(jodaModuleClass);
			modulesToRegister.put(jodaModule.getTypeId(), jodaModule);
		}
		catch (ClassNotFoundException ex) {
			// jackson-datatype-joda not available
		}
	}

	// Kotlin present?
	if (KotlinDetector.isKotlinPresent()) {
		try {
			Class<? extends Module> kotlinModuleClass = (Class<? extends Module>)
					ClassUtils.forName("com.fasterxml.jackson.module.kotlin.KotlinModule", this.moduleClassLoader);
			Module kotlinModule = BeanUtils.instantiateClass(kotlinModuleClass);
			modulesToRegister.put(kotlinModule.getTypeId(), kotlinModule);
		}
		catch (ClassNotFoundException ex) {
			if (!kotlinWarningLogged) {
				kotlinWarningLogged = true;
				logger.warn("For Jackson Kotlin classes support please add " +
						"\"com.fasterxml.jackson.module:jackson-module-kotlin\" to the classpath");
			}
		}
	}
}

The logic is: if Joda-Time’s LocalDate is on the classpath, load Jackson’s corresponding JodaModule (via the LaunchedURLClassLoader).

Why didn’t I suspect jdk8ModuleClass and javaTimeModuleClass? Because the common package already depends on these two artifacts:

1
2
compile "com.fasterxml.jackson.datatype:jackson-datatype-jdk8:${v.jacksonDatatype}"
compile "com.fasterxml.jackson.datatype:jackson-datatype-jsr310:${v.jacksonDatatype}"

That made the fix clear.

Solutions

Avoid repeated ClassLoader loading

Add this dependency to the project. After it loads once, later calls can retrieve it via findLoadedClass, reducing the resource cost of class loading.

1
2
3
4
5
<dependency>
    <groupId>com.fasterxml.jackson.datatype</groupId>
    <artifactId>jackson-datatype-joda</artifactId>
    <version>x.x.x</version>
</dependency>

Avoid re-initializing HttpMessageConverters

1
2
3
4
5
6
7
8
9
10
11
12
13
public Decoder feignDecoder() {
    HttpMessageConverter jacksonConverter = new MappingJackson2HttpMessageConverter(customObjectMapper());
    ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(false, Collections.singletonList(jacksonConverter));
    return new ResponseEntityDecoder(new RSpringDecoder(objectFactory));
}


public Encoder feignEncoder() {
    HttpMessageConverter jacksonConverter = new RMappingJackson2HttpMessageConverter(customObjectMapper());
    ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(false, Collections.singletonList(jacksonConverter));
    return new SpringEncoder(objectFactory);
}

Note that this approach breaks Feign calls whose return value is a simple type — for example, a String return throws:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
feign.codec.DecodeException: Error while extracting response for type [class java.lang.String] and content type [application/json]; nested exception is org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Cannot deserialize instance of `java.lang.String` out of START_OBJECT token; nested exception is com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of `java.lang.String` out of START_OBJECT token
 at [Source: (PushbackInputStream); line: 1, column: 1]
	at feign.SynchronousMethodHandler.decode(SynchronousMethodHandler.java:180) ~[feign-core-10.1.0.jar!/:na]
	at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:140) ~[feign-core-10.1.0.jar!/:na]
	at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:78) ~[feign-core-10.1.0.jar!/:na]
	at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:103) ~[feign-core-10.1.0.jar!/:na]
	at com.sun.proxy.$Proxy219.token(Unknown Source) ~[na:na]
	at com.jiankunking.gateway.controller.AuthController.getAccessTokenByCode(AuthController.java:399) [classes!/:na]
	at com.jiankunking.gateway.controller.AuthController.getCustomerToken(AuthController.java:86) [classes!/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_242]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_242]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_242]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_242]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:189) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:897) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) [tomcat-embed-websocket-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:96) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at com.jiankunking.common.core.filter.RequestIdFilter.doFilter(RequestIdFilter.java:73) [jsh-common-1.1.35-SNAPSHOT.jar!/:na]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:90) [spring-boot-actuator-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at com.jiankunking.common.security.JwtAuthenticationFilter.doFilterInternal(JwtAuthenticationFilter.java:71) [jsh-common-1.1.35-SNAPSHOT.jar!/:na]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:74) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.servlet.v3_1.OpenTelemetryHandlerMappingFilter.doFilter(OpenTelemetryHandlerMappingFilter.java:80) [otel-agent.jar:na]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:117) [spring-boot-actuator-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:106) [spring-boot-actuator-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:200) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(Ni

A further refinement:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
List<HttpMessageConverter<?>> decodeMessageConverters = new ArrayList<>();

List<HttpMessageConverter<?>> encodeMessageConverters = new ArrayList<>();

public FeignClientsConfig() {

    addDefaultHttpMessageConverters(decodeMessageConverters);
    addDefaultHttpMessageConverters(encodeMessageConverters);

    HttpMessageConverter mappingJackson2HttpMessageConverter = new MappingJackson2HttpMessageConverter(customObjectMapper());
    decodeMessageConverters.add(mappingJackson2HttpMessageConverter);

    HttpMessageConverter rMappingJackson2HttpMessageConverter = new RMappingJackson2HttpMessageConverter(customObjectMapper());
    encodeMessageConverters.add(rMappingJackson2HttpMessageConverter);

    log.info("==================Custom FeignClientsConfig=======================");
}

@Bean
@Override
public Decoder feignDecoder() {
    log.info("==================Override feignDecoder=======================");
    ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(false, getDecodeMessageConverters());
    return new ResponseEntityDecoder(new RSpringDecoder(objectFactory));
}

@Bean
@Override
public Encoder feignEncoder() {
    log.info("==================Override feignEncoder=======================");
    ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(false, getEncodeMessageConverters());
    return new SpringEncoder(objectFactory);
}

private List<HttpMessageConverter<?>> getDecodeMessageConverters() {
    return Collections.unmodifiableList(decodeMessageConverters);
}

private List<HttpMessageConverter<?>> getEncodeMessageConverters() {
    return Collections.unmodifiableList(encodeMessageConverters);
}


// copy from =>
// pkg:org.springframework.web.servlet.config.annotation
// WebMvcConfigurationSupport#addDefaultHttpMessageConverters
protected final void addDefaultHttpMessageConverters(List<HttpMessageConverter<?>> messageConverters) {
    StringHttpMessageConverter stringHttpMessageConverter = new StringHttpMessageConverter();
    stringHttpMessageConverter.setWriteAcceptCharset(false);  // see SPR-7316

    messageConverters.add(new ByteArrayHttpMessageConverter());
    messageConverters.add(stringHttpMessageConverter);
    messageConverters.add(new ResourceHttpMessageConverter());
    messageConverters.add(new ResourceRegionHttpMessageConverter());
    try {
        messageConverters.add(new SourceHttpMessageConverter<>());
    } catch (Throwable ex) {
        // Ignore when no TransformerFactory implementation is available...
    }
    messageConverters.add(new AllEncompassingFormHttpMessageConverter());

}

Results after the fix

CPU usage dropped by half

优化前后效果比对

Summary

When you customize Feign’s codecs with SpringEncoder / SpringDecoder, avoid re-initializing HttpMessageConverters on every call. If you don’t need the default HttpMessageConverters, pass false as the first argument when constructing HttpMessageConverters, so the defaults are never initialized.

This article is the English edition of a post originally published in Chinese.