-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ejabberdctl restart sometimes fails #4137
Comments
Is this with Can you try this fix #4109 (comment) ? |
Yes, I had |
I am closing this as duplicate for now. If this issue still persists after the new commits, I will reopen it. |
I set ejabberd.ymlinclude_config_file:
/var/www/html/ipa.yml:
# 'SELF_IPV4A'
# 'SELF_IPV6A'
allow_only:
- "define_macro"
/opt/ejabberd/conf/ejabberd-secret-credentials.yml:
# 'SQL_PASSWORD'
allow_only:
- "define_macro"
hosts:
- zauber.dynv6.net
hide_sensitive_log_data: true
loglevel: debug
sql_type: mysql
sql_server: "localhost"
sql_database: "ejabberd"
sql_username: "ejabberd"
sql_password: 'SQL_PASSWORD'
sql_prepared_statements: true
auth_method: [sql]
default_db: sql
# update_sql_schema: true
certfiles:
- /etc/letsencrypt/live/zauber.dynv6.net/fullchain.pem
- /etc/letsencrypt/live/zauber.dynv6.net/privkey.pem
listen:
-
port: 5222
ip: "::"
module: ejabberd_c2s
max_stanza_size: 262144
shaper: c2s_shaper
access: c2s
starttls_required: true
-
port: 5223
ip: "::"
module: ejabberd_c2s
max_stanza_size: 262144
shaper: c2s_shaper
access: c2s
tls: true
-
port: 5269
ip: "::"
module: ejabberd_s2s_in
max_stanza_size: 524288
shaper: s2s_shaper
-
port: 5443
ip: "::"
module: ejabberd_http
tls: true
request_handlers:
/admin: ejabberd_web_admin
/api: mod_http_api
/bosh: mod_bosh
/captcha: ejabberd_captcha
/upload: mod_http_upload
/ws: ejabberd_http_ws
-
port: 5280
ip: "::"
module: ejabberd_http
request_handlers:
/admin: ejabberd_web_admin
# /.well-known/acme-challenge: ejabberd_acme
-
port: 3478
ip: "::"
transport: udp
module: ejabberd_stun
use_turn: true
## The server's public IPv4 address:
turn_ipv4_address: 'SELF_IPV4A'
## The server's public IPv6 address:
# turn_ipv6_address: "2001:db8::3"
-
port: 1883
ip: "::"
module: mod_mqtt
backlog: 1000
acme:
auto: false
s2s_use_starttls: optional
captcha_cmd: /usr/local/bin/captcha.sh
captcha_url: auto
acl:
admin:
user: admin1@zauber.dynv6.net
local:
user_regexp: ""
loopback:
ip:
- 127.0.0.0/8
- ::1/128
access_rules:
local:
allow: local
c2s:
deny: blocked
allow: all
announce:
allow: admin
configure:
allow: admin
muc_create:
allow: local
pubsub_createnode:
allow: local
trusted_network:
allow: loopback
api_permissions:
"console commands":
from:
- ejabberd_ctl
who: all
what: "*"
"admin access":
who:
access:
allow:
- acl: loopback
- acl: admin
oauth:
scope: "ejabberd:admin"
access:
allow:
- acl: loopback
- acl: admin
what:
- "*"
- "!stop"
- "!start"
"public commands":
who:
ip: 127.0.0.1/8
what:
- status
- connected_users_number
shaper:
normal:
rate: 3000
burst_size: 20000
fast: 100000
shaper_rules:
max_user_sessions: 10
max_user_offline_messages:
5000: admin
100: all
c2s_shaper:
none: admin
normal: all
s2s_shaper: fast
modules:
mod_adhoc: {}
mod_admin_extra: {}
mod_announce:
access: announce
mod_avatar: {}
mod_blocking: {}
mod_bosh: {}
mod_caps: {}
mod_carboncopy: {}
mod_client_state: {}
mod_configure: {}
mod_disco: {}
mod_fail2ban: {}
mod_http_api: {}
mod_http_upload:
put_url: https://@HOST@:5443/upload
custom_headers:
"Access-Control-Allow-Origin": "https://@HOST@"
"Access-Control-Allow-Methods": "GET,HEAD,PUT,OPTIONS"
"Access-Control-Allow-Headers": "Content-Type"
mod_last: {}
mod_mam:
db_type: sql
assume_mam_usage: true
default: always
mod_mqtt: {}
mod_muc:
access:
- allow
access_admin:
- allow: admin
access_create: muc_create
access_persistent: muc_create
access_mam:
- allow
default_room_options:
mam: true
mod_muc_admin: {}
mod_offline:
access_max_user_messages: max_user_offline_messages
mod_ping: {}
mod_privacy: {}
mod_private: {}
mod_proxy65:
access: local
max_connections: 5
mod_pubsub:
access_createnode: pubsub_createnode
plugins:
- flat
- pep
force_node_config:
## Avoid buggy clients to make their bookmarks public
storage:bookmarks:
access_model: whitelist
mod_push: {}
mod_push_keepalive: {}
mod_register:
captcha_protected: true
mod_roster:
versioning: true
mod_s2s_dialback: {}
mod_shared_roster: {}
mod_stream_mgmt:
resend_on_timeout: if_offline
mod_stun_disco: {}
mod_vcard: {}
mod_vcard_xupdate: {}
mod_version:
show_os: false ejabberd.log ends with
|
I noticed this only happens sometimes with |
I can reproduce the problem (or at least I think it's the same problem) one of every two or three restarts. The problem appears when using any binary installer from master branch, 23.10, 23.04, 22.05 When running
It is very easy to reproduce:
The problem does not appear when using the container, or ejabberd compiled from source code with Erlang 26.1 (the one used in the installers). In those cases, |
Is this seen with OpenSSL 3, and when no |
The problem is reproducible when using binary installers (23.10, older releases, also master release). Those installers are built by Line 73 in 66d701e
As you mentioned, fast_tls included in git some fixes. It's now tagged and used by ejabberd, but testing installers that use latest fast_tls throws the same problem. I guess it's worth trying to update binary installers to OpenSSL 3, but unfortunately I get compilation problems in that case. |
@badlop: OpenSSL 1.1.1 branch is now EOL, yes please look for OpenSSL 3.2.x: |
The newest ejabberd installers use OpenSSL 3.2.1 (since 1962fc8) and fast_tls 1.1.18, which includes the fix processone/fast_tls@da16622. Those installers can be downloaded from https://github.com/processone/ejabberd/actions/runs/7844499079 However, the problem mentioned in this issue is still present. |
Happy New Year!
Environment
Errors from error.log/crash.log
No errors
ejabberd.log
Bug description
After running
sudo ejabberdctl restart
ejabberd has written the log seen above and crashed. The ejabberd server was not available that day. There were no error messages and I could not find any information about what went wrong.Later, after a manual
sudo service ejabberd start
ejabberd started normally and has written the following lines to the log:Why could
sudo ejabberdctl restart
bring the server down? What could make ejabberd stuck/crash when "Building MQTT cache"?The text was updated successfully, but these errors were encountered: