衣舞晨风
0%

一次Fegin CPU占用过高导致的事故

发表于 更新于 分类于
本文字数： 28k 阅读时长 ≈ 25 分钟

记录一下
一次应用事故分析、排查、处理

背景介绍

9号上午收到CPU告警，同时业务反馈依赖该服务的上游服务接口响应耗时太长

1
2
3
4
5
6
7
8
应用告警-CPU使用率 告警变更
【WARNING】项目XXX,集群qd-aliyun,分区bbbb-prod,应用customer,实例customer-6fb6448688-m47jz, POD实例CPU请求使用率 >= 90.000000% 当前值138.4971051199925%
发生时间:2024/10/09 11:17:33


项目XXX,集群qd-aliyun,分区bbbb-prod,应用customer,实例customer-6fb6448688-28pvs, POD实例CPU请求使用率 >= 90.000000% 当前值157.7076205766934%告警已恢复
发生时间: 2024/10/09 11:06:33
恢复时间: 2024/10/09 12:24:33

服务访问量

单实例峰值QPS100左右

为啥要关注QPS，因为QPS100不应该消耗这么多CPU啊，而且请求、响应体都不大。


POD监控

POD配额

  • CPU请求 2 Core CPU上限 3 Core
  • 内存请求 7GiB 内存上限 9GiB



从图中可以看出

  • CPU负载一直很高
  • TCP链接及线程数从11点40开始陡峭上升

Arms

看下Trace监控发现，耗时主要是customer通过fegin调用外围接口导致的。

临时方案

临时处理方案：扩实例并增加CPU配置。

根因分析

此处略过排查三方接口跟开放平台网关的过程，此处的结论是：依赖的三方接口跟开放平台网关没有问题。
为啥会先排查三方接口跟开放平台网关是因为中Trace上来看是调用三方接口响应时间过长。


从Arms图看可以看出

  • CPU耗时集中在fegin调用的Decoder、Encoder
  • Decoder、Encoder耗时都集中在
    • HttpMessageConverters#getDefaultConverters()=>
    • WebMvcConfigurationSupport#addDefaultHttpMessageConverters=>
    • ……(具体调用链看下方摘要)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
feign.ReflectiveFeign$BuildTemplateByResolvingArgs.create(Object[]) (14.37%, 1.43 minutes)
feign.ReflectiveFeign$BuildEncodedTemplateFromArgs.reesolve(Object[], RequestTemplate, Map) (14.37%, 1.43minutes)
org.springframework.cloud.openfeign.support.SpringEndcoder.encode(Object, Type, RequestTemplate) (14.28%,1.42 minutes)
com.jiankunking.common.core.feign.FeignClientsConfig$$ambda$938.56729293.get0bject() (13.98%, 1.39 minutes 
com.jiankunking.common.core.feign.FeignClientsConfig.lambda$feignEncoder$2() (13.98%, 1.39 minutees)
org.springframework.boot.autoconfigure.http.HttpmessaageConverters.<init>(HttpMessageConverter[]) (12.03%,1.19 minutes)
prg.springframework.boot.autoconfigure.http.Http.HttpMessageConverters.<init>(Collection) (12.03%, 119 minutes)
org.springframework.boot.autoconfigure.http.HttpmessaageConverters.<init>(boolean, Collection) (12.03%, 1.19 minutes)
prg.springframework.boot.autoconfigure.http.Http.HttpMessageConverters.getDefaultConverters()(12.02%, 1.19 minutes
org.springframework.boot.autoconfigure.http.HttpmessageConverters$1.defaultMessageConverters() (12.02%, 119 minutes)
org.springframework.web.servlet.config.annotation.WebMvcConfigurationSupport.getMessageConverters() (12.02%, 1.19 minutes)
org.springframework.web.servlet.config.annotation. WebMvcConfigurationSupport.addDefaultHttpMessageConverters(List) (12.02%, 1
org.springframework.http.converter.json.Jackson2ObjectMapperBuilder.build() (5.93%, 0.59 minutes)
org.springframework.http.converter.json.Jackson2ObjectMapperBuilder.configure(ObjectMapper)(5.91%, 0.59 minutes)
org.springframework.http.converter.json.Jackson2Objec:tMapperBuilder.registerWellKnownModulesIfAvailable(Map)(5.89%,0.58 min
org.springframework.util.ClassUtils.forName(String, CClassLoader)(5.84%, 0.58 minutes)
java.lang.Class.forName(String, boolean, Classloader) (5.83%, 0.58 minutes)
java.lang.Class.forName0(String, boolean, ClassLoader, Class) (5.83%, 0.58 minutes)
......

自定义Encoder、Decoder

Encoder

看下jiankunking.common.core.feign.FeignClientsConfig中的Encoder

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
public Encoder feignEncoder() {
    ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(new RMappingJackson2HttpMessageConverter());
    return new SpringEncoder(objectFactory);
}

public class RMappingJackson2HttpMessageConverter extends MappingJackson2HttpMessageConverter {

    public RMappingJackson2HttpMessageConverter(ObjectMapper objectMapper) {
        super(objectMapper);
        List<MediaType> mediaTypes = new ArrayList<>();
        mediaTypes.add(MediaType.valueOf(MediaType.APPLICATION_JSON_UTF8_VALUE));
        mediaTypes.add(MediaType.valueOf(MediaType.TEXT_HTML_VALUE + ";charset=UTF-8"));
        setSupportedMediaTypes(mediaTypes);
    }

    RMappingJackson2HttpMessageConverter() {
        List<MediaType> mediaTypes = new ArrayList<>();
        mediaTypes.add(MediaType.valueOf(MediaType.APPLICATION_JSON_UTF8_VALUE));
        mediaTypes.add(MediaType.valueOf(MediaType.TEXT_HTML_VALUE + ";charset=UTF-8"));
        setSupportedMediaTypes(mediaTypes);
    }
}

Decoder

看下jiankunking.common.core.feign.FeignClientsConfig中的Decoder

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
public Decoder feignDecoder() {
    HttpMessageConverter jacksonConverter = new MappingJackson2HttpMessageConverter(customObjectMapper());
    ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(jacksonConverter);
    return new ResponseEntityDecoder(new RSpringDecoder(objectFactory));
}

public ObjectMapper customObjectMapper() {
    ObjectMapper objectMapper = new ObjectMapper();

    objectMapper.registerModule(new StringToDateModule());
    objectMapper.configure(JsonParser.Feature.ALLOW_COMMENTS, true);
    objectMapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true);
    objectMapper.configure(JsonParser.Feature.ALLOW_SINGLE_QUOTES, true);
    objectMapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_CONTROL_CHARS, true);
    objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);

    return objectMapper;
}

Google了一下:‘spring feign encode jackson cpu usage high’
=> https://segmentfault.com/a/1190000043037032
=> https://mp.weixin.qq.com/s/RuqltkN9VdVQ1K3GKuJ-Gw
=> https://meantobe.github.io/2019/12/21/ClassLoader/

源码分析

查看registerWellKnownModulesIfAvailable处的代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

@SuppressWarnings("unchecked")
private void registerWellKnownModulesIfAvailable(Map<Object, Module> modulesToRegister) {
	try {
		Class<? extends Module> jdk8ModuleClass = (Class<? extends Module>)
				ClassUtils.forName("com.fasterxml.jackson.datatype.jdk8.Jdk8Module", this.moduleClassLoader);
		Module jdk8Module = BeanUtils.instantiateClass(jdk8ModuleClass);
		modulesToRegister.put(jdk8Module.getTypeId(), jdk8Module);
	}
	catch (ClassNotFoundException ex) {
		// jackson-datatype-jdk8 not available
	}

	try {
		Class<? extends Module> javaTimeModuleClass = (Class<? extends Module>)
				ClassUtils.forName("com.fasterxml.jackson.datatype.jsr310.JavaTimeModule", this.moduleClassLoader);
		Module javaTimeModule = BeanUtils.instantiateClass(javaTimeModuleClass);
		modulesToRegister.put(javaTimeModule.getTypeId(), javaTimeModule);
	}
	catch (ClassNotFoundException ex) {
		// jackson-datatype-jsr310 not available
	}

	// Joda-Time present?
	if (ClassUtils.isPresent("org.joda.time.LocalDate", this.moduleClassLoader)) {
		try {
			Class<? extends Module> jodaModuleClass = (Class<? extends Module>)
					ClassUtils.forName("com.fasterxml.jackson.datatype.joda.JodaModule", this.moduleClassLoader);
			Module jodaModule = BeanUtils.instantiateClass(jodaModuleClass);
			modulesToRegister.put(jodaModule.getTypeId(), jodaModule);
		}
		catch (ClassNotFoundException ex) {
			// jackson-datatype-joda not available
		}
	}

	// Kotlin present?
	if (KotlinDetector.isKotlinPresent()) {
		try {
			Class<? extends Module> kotlinModuleClass = (Class<? extends Module>)
					ClassUtils.forName("com.fasterxml.jackson.module.kotlin.KotlinModule", this.moduleClassLoader);
			Module kotlinModule = BeanUtils.instantiateClass(kotlinModuleClass);
			modulesToRegister.put(kotlinModule.getTypeId(), kotlinModule);
		}
		catch (ClassNotFoundException ex) {
			if (!kotlinWarningLogged) {
				kotlinWarningLogged = true;
				logger.warn("For Jackson Kotlin classes support please add " +
						"\"com.fasterxml.jackson.module:jackson-module-kotlin\" to the classpath");
			}
		}
	}
}

可以看到其逻辑为若classpath中有JodaTime的LocalDate，则加载Jackson对应的JodaModule.LaunchedURLClassLoader.

为啥没有怀疑jdk8ModuleClass、javaTimeModuleClass这两个地方呢？因为common包中已经依赖了下面两个包

1
2
compile "com.fasterxml.jackson.datatype:jackson-datatype-jdk8:${v.jacksonDatatype}"
compile "com.fasterxml.jackson.datatype:jackson-datatype-jsr310:${v.jacksonDatatype}"

那么解决方案就很清晰了

解决方案

避免ClassLoader反复加载

将这个依赖添加到工程中。加载一次后，再次调用可以通过findLoadedClass获得，减少加载类导致的资源消耗。

1
2
3
4
5
<dependency>
    <groupId>com.fasterxml.jackson.datatype</groupId>
    <artifactId>jackson-datatype-joda</artifactId>
    <version>x.x.x</version>
</dependency>

避免HttpMessageConverters重复初始化

1
2
3
4
5
6
7
8
9
10
11
12
public Decoder feignDecoder() {
    HttpMessageConverter jacksonConverter = new MappingJackson2HttpMessageConverter(customObjectMapper());
    ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(false, Collections.singletonList(jacksonConverter));
    return new ResponseEntityDecoder(new RSpringDecoder(objectFactory));
}


public Encoder feignEncoder() {
    HttpMessageConverter jacksonConverter = new RMappingJackson2HttpMessageConverter(customObjectMapper());
    ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(false, Collections.singletonList(jacksonConverter));
    return new SpringEncoder(objectFactory);
}

注意这种处理方式,会导致feign返回值简单类型的调用异常,比如返回是String会抛出异常

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
feign.codec.DecodeException: Error while extracting response for type [class java.lang.String] and content type [application/json]; nested exception is org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Cannot deserialize instance of `java.lang.String` out of START_OBJECT token; nested exception is com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of `java.lang.String` out of START_OBJECT token
 at [Source: (PushbackInputStream); line: 1, column: 1]
	at feign.SynchronousMethodHandler.decode(SynchronousMethodHandler.java:180) ~[feign-core-10.1.0.jar!/:na]
	at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:140) ~[feign-core-10.1.0.jar!/:na]
	at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:78) ~[feign-core-10.1.0.jar!/:na]
	at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:103) ~[feign-core-10.1.0.jar!/:na]
	at com.sun.proxy.$Proxy219.token(Unknown Source) ~[na:na]
	at com.jiankunking.gateway.controller.AuthController.getAccessTokenByCode(AuthController.java:399) [classes!/:na]
	at com.jiankunking.gateway.controller.AuthController.getCustomerToken(AuthController.java:86) [classes!/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_242]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_242]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_242]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_242]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:189) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:897) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882) [spring-webmvc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) [tomcat-embed-websocket-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:96) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at com.jiankunking.common.core.filter.RequestIdFilter.doFilter(RequestIdFilter.java:73) [jsh-common-1.1.35-SNAPSHOT.jar!/:na]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:90) [spring-boot-actuator-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at com.jiankunking.common.security.JwtAuthenticationFilter.doFilterInternal(JwtAuthenticationFilter.java:71) [jsh-common-1.1.35-SNAPSHOT.jar!/:na]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:74) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178) [spring-security-web-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.servlet.v3_1.OpenTelemetryHandlerMappingFilter.doFilter(OpenTelemetryHandlerMappingFilter.java:80) [otel-agent.jar:na]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:117) [spring-boot-actuator-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:106) [spring-boot-actuator-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:200) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834) [tomcat-embed-core-9.0.17.jar!/:9.0.17]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(Ni

进一步处理方式

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
List<HttpMessageConverter<?>> decodeMessageConverters = new ArrayList<>();

List<HttpMessageConverter<?>> encodeMessageConverters = new ArrayList<>();

public FeignClientsConfig() {

    addDefaultHttpMessageConverters(decodeMessageConverters);
    addDefaultHttpMessageConverters(encodeMessageConverters);

    HttpMessageConverter mappingJackson2HttpMessageConverter = new MappingJackson2HttpMessageConverter(customObjectMapper());
    decodeMessageConverters.add(mappingJackson2HttpMessageConverter);

    HttpMessageConverter rMappingJackson2HttpMessageConverter = new RMappingJackson2HttpMessageConverter(customObjectMapper());
    encodeMessageConverters.add(rMappingJackson2HttpMessageConverter);

    log.info("==================Custom FeignClientsConfig=======================");
}

@Bean
@Override
public Decoder feignDecoder() {
    log.info("==================Override feignDecoder=======================");
    ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(false, getDecodeMessageConverters());
    return new ResponseEntityDecoder(new RSpringDecoder(objectFactory));
}

@Bean
@Override
public Encoder feignEncoder() {
    log.info("==================Override feignEncoder=======================");
    ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(false, getEncodeMessageConverters());
    return new SpringEncoder(objectFactory);
}

private List<HttpMessageConverter<?>> getDecodeMessageConverters() {
    return Collections.unmodifiableList(decodeMessageConverters);
}

private List<HttpMessageConverter<?>> getEncodeMessageConverters() {
    return Collections.unmodifiableList(encodeMessageConverters);
}


// copy from =>
// pkg:org.springframework.web.servlet.config.annotation
// WebMvcConfigurationSupport#addDefaultHttpMessageConverters
protected final void addDefaultHttpMessageConverters(List<HttpMessageConverter<?>> messageConverters) {
    StringHttpMessageConverter stringHttpMessageConverter = new StringHttpMessageConverter();
    stringHttpMessageConverter.setWriteAcceptCharset(false);  // see SPR-7316

    messageConverters.add(new ByteArrayHttpMessageConverter());
    messageConverters.add(stringHttpMessageConverter);
    messageConverters.add(new ResourceHttpMessageConverter());
    messageConverters.add(new ResourceRegionHttpMessageConverter());
    try {
        messageConverters.add(new SourceHttpMessageConverter<>());
    } catch (Throwable ex) {
        // Ignore when no TransformerFactory implementation is available...
    }
    messageConverters.add(new AllEncompassingFormHttpMessageConverter());

}

优化后效果

优化后CPU占用降低一半

总结

大家在自定义 Feign 的编解码器时，如果用到了 SpringEncoder / SpringDecoder，应避免 HttpMessageConverters 的重复初始化。如果不需要使用那些默认的 HttpMessageConverter，可以在初始化 HttpMessageConverters 时将第一个入参设置为 false，从而不初始化那些默认的 HttpMessageConverter。

相关文章

欢迎关注我的其它发布渠道