[Google VM] An error occurred when initializing this installation - bitnami Lamp

[tacoma50]

Platform

Google Cloud Platform

bndiagnostic ID know more about bndiagnostic ID

725a6da6-56f5-9279-ed58-af34f8fd44db

bndiagnostic output

These same errors are also found on the original working VM where the machine image created from.
So it seems these errors are unrelated since they appear on both the old working VM and the new VM.

[Connectivity]

Server ports 22, 80 and/or 443 are not publicly accessible. Please check the
following guide to open server ports for remote access:

https://docs.bitnami.com/general/faq/administration/use-firewall/

[Apache]

Found recent error or warning messages in the Apache error log.
Press [Enter] to continue:

[Wed Oct 02 09:18:33.621632 2024] [core:error] [pid 1500:tid 140294860109504] 
[client **ip_address**:50418] AH10244: invalid URI path 
(/cgi-bin/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32
%65/%%32%65%%32%65/%%32%65%%32%65/bin/sh)

[Wed Oct 02 10:01:17.999079 2024] [authz_core:error] [pid 1500:tid 
140294007801536] [client **ip_address**:46810] AH01630: client denied by server 
configuration: /opt/bitnami/apache/cgi-bin

[Wed Oct 02 12:07:59.028652 2024] [authz_core:error] [pid 1500:tid 
140294902073024] [client **ip_address**:51878] AH01630: client denied by server 
configuration: /opt/bitnami/apache/cgi-bin





### bndiagnostic was not useful. Could you please tell us why?

These same errors are also found on the original working VM where the machine  image created from. So it seems these errors are unrelated since they appear on both  the old working VM and the new VM.

### Describe your issue as much as you can

I created a new Google VM from a machine image of a fully working bitnami VM. The new VM was created successfully with no errors. But when I log into the new VM I see this message:

  !!!!! IMPORTANT: An error occurred when initializing this installation !!!!! 
  │ Please check the initialization logs to get more information: 
  │ sudo cat /var/log/cloud-init-output.log
  │ Please try and re-create this installation. If that does not fix your issue, 
  │ open a Bitnami support ticket at https://github.com/bitnami/vms/issues

In /var/log/cloud-init-output.log I see this:

2024-10-01 23:30:51,087 - schema.py[WARNING]: Invalid cloud-config provided: Please run 'sudo cloud-init schema --system' to see the schema errors.
Cloud-init v. 22.4.2 running 'modules:config' at Tue, 01 Oct 2024 23:30:53 +0000. Up 10.81 seconds.

When I ran 'sudo cloud-init schema --system' I saw:
  # cloud-init schema --system
  Error:
  Cloud config schema errors: format-l1.c1: File None needs to begin with "#cloud-config"

[tacoma50]

I think I found the cause.

This is message I am seeing at login, it is part of the MOTD:
| !!!!! IMPORTANT: An error occurred when initializing this installation !!!!!
│ Please check the initialization logs to get more information:
│ sudo cat /var/log/cloud-init-output.log
│ Please try and re-create this installation. If that does not fix your issue,
│ open a Bitnami support ticket at https://github.com/bitnami/vms/issues

And I found this message in the /var/log/cloud-init-output.log:

Starting mariadb in background
^[[38;5;6mmariadb ^[[38;5;5m20:17:07.33 ^[[0m^[[38;5;1mERROR^[[0m ==> MySQL failed to start
xargs: bash: exited with status 255; aborting
^[[38;5;6m ^[[38;5;5m20:17:07.36 ^[[0m^[[38;5;2mINFO ^[[0m ==> Appending error message to motd banner
xargs: bash: exited with status 255; aborting
2024-10-01 20:17:07,382 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)

So why did MySQL fail to start? Is there anything I need to check?

It looks like mysql is working:

root@lampstack# mariadb --version
mariadb from 11.3.2-MariaDB, client 15.2 for Linux (x86_64) using readline 5.1

root@lampstack:~# mysqlcheck -c -u root -p --all-databases
Enter password:
mysql.column_stats OK
mysql.columns_priv OK

more of the /var/log/cloud-init-output.log here...
Cloud-init v. 22.4.2 running 'modules:config' at Tue, 01 Oct 2024 20:06:52 +0000. Up 14.30 seconds.
Cloud-init v. 22.4.2 running 'modules:final' at Tue, 01 Oct 2024 20:07:04 +0000. Up 26.10 seconds.
^[[38;5;6m ^[[38;5;5m20:07:04.87 ^[[0m^[[38;5;2mINFO ^[[0m ==> Enabling all services
Created symlink /etc/systemd/system/bitnami.service.wants/bitnami.varnish.service → /etc/systemd/system/bitnami.varnish.service.
^[[38;5;6m ^[[38;5;5m20:07:05.38 ^[[0m^[[38;5;2mINFO ^[[0m ==> Parsing environment variables specified within the user-data
^[[38;5;6mmariadb ^[[38;5;5m20:07:06.24 ^[[0m^[[38;5;2mINFO ^[[0m ==> Validating settings in MYSQL_/MARIADB_ env vars
^[[38;5;6mmariadb ^[[38;5;5m20:07:06.28 ^[[0m^[[38;5;2mINFO ^[[0m ==> Initializing mariadb database
^[[38;5;6mmariadb ^[[38;5;5m20:07:06.32 ^[[0m^[[38;5;2mINFO ^[[0m ==> Updating 'my.cnf' with custom configuration
^[[38;5;6mmariadb ^[[38;5;5m20:07:06.34 ^[[0m^[[38;5;2mINFO ^[[0m ==> Setting bind_address option
^[[38;5;6mmariadb ^[[38;5;5m20:07:06.38 ^[[0m^[[38;5;2mINFO ^[[0m ==> Setting slow_query_log option
^[[38;5;6mmariadb ^[[38;5;5m20:07:06.39 ^[[0m^[[38;5;2mINFO ^[[0m ==> Setting long_query_time option
^[[38;5;6mmariadb ^[[38;5;5m20:07:06.40 ^[[0m^[[38;5;2mINFO ^[[0m ==> Using persisted data
/opt/bitnami/mariadb/bin/mysql: Deprecated program name. It will be removed in a future release, use '/opt/bitnami/mariadb/bin/mariadb' instead
/opt/bitnami/mariadb/bin/mysql: Deprecated program name. It will be removed in a future release, use '/opt/bitnami/mariadb/bin/mariadb' instead
/opt/bitnami/mariadb/bin/mysql: Deprecated program name. It will be removed in a future release, use '/opt/bitnami/mariadb/bin/mariadb' instead
^[[38;5;6mmariadb ^[[38;5;5m20:07:06.49 ^[[0m^[[38;5;2mINFO ^[[0m ==> Running mysql_upgrade
^[[38;5;6mmariadb ^[[38;5;5m20:07:06.50 ^[[0m^[[38;5;2mINFO ^[[0m ==> Starting mariadb in background
^[[38;5;6mmariadb ^[[38;5;5m20:17:07.33 ^[[0m^[[38;5;1mERROR^[[0m ==> MySQL failed to start
xargs: bash: exited with status 255; aborting
^[[38;5;6m ^[[38;5;5m20:17:07.36 ^[[0m^[[38;5;2mINFO ^[[0m ==> Appending error message to motd banner
xargs: bash: exited with status 255; aborting
2024-10-01 20:17:07,382 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
2024-10-01 20:17:07,382 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py'>) failed
Cloud-init v. 22.4.2 finished at Tue, 01 Oct 2024 20:17:07 +0000. Datasource DataSourceGCELocal. Up 628.83 seconds
Cloud-init v. 22.4.2 running 'init-local' at Tue, 01 Oct 2024 20:31:08 +0000. Up 4.87 seconds.
Cloud-init v. 22.4.2 running 'init' at Tue, 01 Oct 2024 20:31:12 +0000. Up 8.80 seconds.

[tacoma50]

So what is causing this error when creating a new VM bitnami instance from a machine image and how would I avoid it:

^[[38;5;6mmariadb ^[[38;5;5m20:17:07.33 ^[[0m^[[38;5;1mERROR^[[0m ==> MySQL failed to start
xargs: bash: exited with status 255; aborting

I am sure anyone can duplicate this error by creating a new bitnami VM instance from a existing bitnami VM machine image.

In a nutshell:
It does not matter if a bitnami VM was created directly from the Google Marketplace or from a existing bitnami machine image...we always see the "WARNING]: Invalid cloud-config provided - schema error" in the /var/log/cloud-init-output.log after stop/start of the VM.
We only see the "MySQL failed to start" error if the instance was created from a bitnami machine image.

1. What is causing the "WARNING]: Invalid cloud-config provided - schema error" when a new bitnami VM is created from Google Marketplace?

2. More importantly: What is causing the "MySQL failed to start" error when a bitnami VM is created from a existing bitnami machine image?

I would feel more confident using a VM created from a machine image that did not have any errors during creation.

[jotamartos]

The warning messages shouldn't make the deployment fail. However, it seems that the database is not initialized correctly

^[[38;5;6mmariadb ^[[38;5;5m20:17:07.33 ^[[0m^[[38;5;1mERROR^[[0m ==> MySQL failed to start

I do not know which app you are deploying, but can you ensure that it has enough resources? Please note that some apps require high memory requirements to work.

[tacoma50]

Hi jotamartos 
I have broken this ticket into two separate issues which are:

1. What is causing the "[WARNING]: Invalid cloud-config provided - schema error" when a new bitnami-lamp VM is created from Google Marketplace?

2. More importantly: What is causing the "MySQL failed to start" error when a bitnami-lamp VM is created from an existing bitnami-lamp machine image?

Let's look at issue 1:
WARNING invalid cloud-config provided - schema error Found in /var/log/cloud-init-output.log log file.

As I stated this warning appears in the cloud-init  log with a brand new freshly created bitnami-lamp VM from the google Marketplace. I have not changed anything in the VM.
I would think that a new bitnami-lamp VM would not have any warnings from the get go. Can you explain what is causing this warning? Up until now we have been ignoring it.
You can easily recreate this warning by going to the Google Marketplace and creating a new BITNAMI PACKAGE FOR LAMP VM at: https://console.cloud.google.com/marketplace/product/bitnami-launchpad/lampstack

This is my VM info: 
 Location: us-central1-c   
 MachineType: n1-standard-1  (vCPU=1 Memory=3.75g)
 Architecture: x86/64
 Boot-Disk: 20g - SSD persistent disk

Issue 2: 
MySQL failed to start  Found in /var/log/cloud-init-output.log log file.

As I stated this warning occurs when a bitnami VM is created from an existing bitnami-lamp machine image.
You can recreate this warning by creating a brand new bitnami-lamp VM as described above, then create a machine image of this VM, then use that machine image to create a new bitnami-lmap VM. During the creation of this new VM from the image you will see the MySQL FAILED to start error in the log.

I would think that a new bitnami-lamp VM created from a bitnami-lamp machine image would not have any warnings from the get go. Can you explain this MySQL warning?

Also a basic check of MySQL appears that it's working: 
root@lampstack:~# mysqlcheck -c -u root -p --all-databases
  mysql.column_stats OK

You said: "I do not know which app you are deploying, but can you ensure that it has enough resources? Please note that some apps require high memory requirements to work."

We have not deployed any apps on the VM and you can see my VM size info above in issue1.

[jotamartos]

Hi @tacoma50,

I just tried to reproduce the issue and launched a Bitnami WordPress and LAMP solutions. I know you didn't launch WordPress, but another user mentioned this issue as well in #1690.

However, my deployments worked as expected, there is no error in the console and the different components were launched without problems

Please ensure you launch a VM with enough resources to run the deployment process. Could you please try to launch the VM again?

Thanks

[tacoma50]

Hi Jotamartos,

I am not running WORDPRESS so I cannot speak to that. Can you please test with my setup as follows:
Go the Google Marketplace and create a new BITNAMI PACKAGE FOR LAMP VM at:
https://console.cloud.google.com/marketplace/product/bitnami-launchpad/lampstack

With this exact VM:
Location: us-central1-c
MachineType: n1-standard-1 (vCPU=1 Memory=3.75g)
Architecture: x86/64
Boot-Disk: 20g - SSD persistent disk

Then stop and start this VM, then run: grep -i Invalid cloud-init-output.log

2024-06-04 14:37:07,323 - schema.py[WARNING]: Invalid cloud-config provided: Please run 'sudo cloud-init schema --system' to see the schema errors

then run: cloud-init schema --system
Error:
Cloud config schema errors: format-l1.c1: File None needs to begin with "#cloud-config"

thank you

[jotamartos]

Hi,

I tried to reproduce the issue, but it's just got a warning.

bitnami@jotalamp-vm:~$ grep -i Invalid /var/log/cloud-init-output.log 
2024-10-25 13:22:26,921 - schema.py[WARNING]: Invalid cloud-config provided: Please run 'sudo cloud-init schema --system' to see the schema errors.
2024-10-25 13:27:44,418 - schema.py[WARNING]: Invalid cloud-config provided: Please run 'sudo cloud-init schema --system' to see the schema errors.
bitnami@jotalamp-vm:~$ cat bitnami_credentials 
Welcome to Bitnami package for LAMP

******************************************************************************
The default password is 'XXXXXX'.
******************************************************************************

You can also use this password to access the databases and any other component the stack includes.

Please refer to https://docs.bitnami.com/google/infrastructure/lamp/ for more details.
bitnami@jotalamp-vm:~$ sudo /opt/bitnami/ctlscript.sh status
apache already running
mariadb already running
php-fpm already running

When deploying the VM the first time or when stopping/starting it, the output just shows a warning and all services are started properly

[tacoma50]

Hi Jotamartos,
I'm glad you were able to recreate this warning I see in issue 1:

stop and start this new VM, then run: grep -i Invalid /var/log/cloud-init-output.log
If you look further int ot warning you saw you will see:
2024-06-04 - schema.py[WARNING]: Invalid cloud-config provided: Please run 'sudo cloud-init schema --system' to see the schema errors
then run: cloud-init schema --system
Error: Cloud config schema errors: format-l1.c1: File None needs to begin with "#cloud-config"

So are you saying it is OK to ignore this Error:
error: Cloud config schema errors: format-l1.c1: File None needs to begin with "#cloud-config"

[jotamartos]

I said that I couldn't reproduce the issue, I only got warning messages. I used the latest version of the VM and didn't get the error message. I may be a transient issue with the version you launched (back in June). You can try launching the latest version and see if the issue is solved.

[tacoma50]

You said you were able recreate the warning and you posted this:

bitnami@jotalamp-vm:~$ grep -i Invalid /var/log/cloud-init-output.log
2024-10-25 13:22:26,921 - schema.py[WARNING]: Invalid cloud-config provided: Please run 'sudo cloud-init schema --
system' to see the schema errors.

When you see that message above, can you post the output of this command:
grep -i Invalid /var/log/cloud-init-output.log