Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

集群重启 调度器出现问题 (使用的修改默认调度器) #34

Open
zcspk opened this issue Oct 18, 2022 · 2 comments
Open
Assignees

Comments

@zcspk
Copy link

zcspk commented Oct 18, 2022

E1018 09:42:10.621700 1 stats.go:128] [crane] failed to get node 's score: zcsnode2%!(EXTRA string=mem_usage_max_avg_1h, float64=33.699000000000005)
E1018 09:42:10.621708 1 stats.go:128] [crane] failed to get node 's score: zcsnode2%!(EXTRA string=mem_usage_max_avg_1d, float64=33.699000000000005)
I1018 09:42:10.621717 1 plugins.go:92] [crane] Node[zcsnode2]'s finalscore is 6, while score is 16 and hotvalue is 1.000000
E1018 09:48:25.615198 1 stats.go:128] [crane] failed to get node 's score: zcsmaster1%!(EXTRA string=cpu_usage_max_avg_1d, float64=45.77980000000001)
E1018 09:48:25.615301 1 stats.go:128] [crane] failed to get node 's score: zcsmaster1%!(EXTRA string=mem_usage_max_avg_1d, float64=71.2381)
I1018 09:48:25.615339 1 plugins.go:92] [crane] Node[zcsmaster1]'s finalscore is 35, while score is 35 and hotvalue is 0.000000
E1018 09:48:25.615397 1 stats.go:128] [crane] failed to get node 's score: zcsnode1%!(EXTRA string=cpu_usage_max_avg_1d, float64=47.9795)
E1018 09:48:25.615417 1 stats.go:128] [crane] failed to get node 's score: zcsnode1%!(EXTRA string=mem_usage_max_avg_1d, float64=75.73570000000001)
I1018 09:48:25.615424 1 plugins.go:92] [crane] Node[zcsnode1]'s finalscore is 37, while score is 37 and hotvalue is 0.000000
E1018 09:48:25.615447 1 stats.go:128] [crane] failed to get node 's score: zcsnode2%!(EXTRA string=cpu_usage_max_avg_1d, float64=47.5513)
E1018 09:48:25.615461 1 stats.go:128] [crane] failed to get node 's score: zcsnode2%!(EXTRA string=mem_usage_max_avg_1d, float64=83.1259)
I1018 09:48:25.615468 1 plugins.go:92] [crane] Node[zcsnode2]'s finalscore is 41, while score is 41 and hotvalue is 0.000000
E1018 09:48:56.352200 1 stats.go:128] [crane] failed to get node 's score: zcsmaster1%!(EXTRA string=cpu_usage_max_avg_1d, float64=45.77980000000001)
E1018 09:48:56.352275 1 stats.go:128] [crane] failed to get node 's score: zcsmaster1%!(EXTRA string=mem_usage_max_avg_1d, float64=71.2381)
I1018 09:48:56.352287 1 plugins.go:92] [crane] Node[zcsmaster1]'s finalscore is 35, while score is 35 and hotvalue is 0.000000
E1018 09:48:56.352346 1 stats.go:128] [crane] failed to get node 's score: zcsnode1%!(EXTRA string=cpu_usage_max_avg_1d, float64=47.9795)
E1018 09:48:56.352368 1 stats.go:128] [crane] failed to get node 's score: zcsnode1%!(EXTRA string=mem_usage_max_avg_1d, float64=75.73570000000001)
I1018 09:48:56.352379 1 plugins.go:92] [crane] Node[zcsnode1]'s finalscore is 37, while score is 37 and hotvalue is 0.000000
E1018 09:48:56.352415 1 stats.go:128] [crane] failed to get node 's score: zcsnode2%!(EXTRA string=cpu_usage_max_avg_1d, float64=47.5513)
E1018 09:48:56.352455 1 stats.go:128] [crane] failed to get node 's score: zcsnode2%!(EXTRA string=mem_usage_max_avg_1d, float64=83.1259)
I1018 09:48:56.352466 1 plugins.go:92] [crane] Node[zcsnode2]'s finalscore is 41, while score is 41 and hotvalue is 0.000000
E1018 09:51:48.506156 1 stats.go:128] [crane] failed to get node 's score: zcsmaster1%!(EXTRA string=cpu_usage_max_avg_1d, float64=45.854200000000006)
E1018 09:51:48.506282 1 stats.go:128] [crane] failed to get node 's score: zcsmaster1%!(EXTRA string=mem_usage_max_avg_1d, float64=71.34190000000001)
I1018 09:51:48.506296 1 plugins.go:92] [crane] Node[zcsmaster1]'s finalscore is 35, while score is 35 and hotvalue is 0.000000
E1018 09:51:48.506329 1 stats.go:128] [crane] failed to get node 's score: zcsnode1%!(EXTRA string=cpu_usage_max_avg_1d, float64=48.017900000000004)
E1018 09:51:48.506357 1 stats.go:128] [crane] failed to get node 's score: zcsnode1%!(EXTRA string=mem_usage_max_avg_1d, float64=75.80170000000001)
I1018 09:51:48.506364 1 plugins.go:92] [crane] Node[zcsnode1]'s finalscore is 37, while score is 37 and hotvalue is 0.000000
E1018 09:51:48.506390 1 stats.go:128] [crane] failed to get node 's score: zcsnode2%!(EXTRA string=cpu_usage_max_avg_1d, float64=47.545500000000004)
E1018 09:51:48.506408 1 stats.go:128] [crane] failed to get node 's score: zcsnode2%!(EXTRA string=mem_usage_max_avg_1d, float64=83.0675)
I1018 09:51:48.506416 1 plugins.go:92] [crane] Node[zcsnode2]'s finalscore is 41, while score is 41 and hotvalue is 0.000000

@xieydd xieydd self-assigned this Oct 18, 2022
@zcspk
Copy link
Author

zcspk commented Oct 19, 2022

  1. 问题一 重启集群后调度器无法获得评分 ,以及控制器无法搜集主机信息 。需要手动重启 控制器
  2. 问题二 crane-scheduler/deploy/controller/deployment.yaml 健康检查开启,pod无法启动。需要关闭健康检查(helm 部署文件也没有看到健康检查)

@xieydd
Copy link
Member

xieydd commented Dec 9, 2022

  1. Tha above log show you can not get the node metrics ?
  2. maybe you can increasse the initialDelaySeconds in https://github.com/gocrane/crane-scheduler/blob/main/deploy/controller/deployment.yaml#L48

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants