Problems getting no-leaks working (to detect memory leaks)

[jamesspittal]

I have a PHP project and I run no-leaks as follows:

$ ./vendor/bin/roave-no-leaks --configuration test/phpunit.xml --coverage-html coverage

It runs, but produces no information about the memory leak:

$ ./vendor/bin/roave-no-leaks --configuration test/phpunit.xml --coverage-html coverage
PHPUnit 8.0.6 by Sebastian Bergmann and contributors.
.FF.FF...F......FF.FF
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /builds/organisation/organisation-app/vendor/phpunit/php-code-coverage/src/Driver/Xdebug.php on line 98

Other information:

• PHP is 7.3.33 (I have tested other versions also)
• PHP Unit is 8.0.6
• export XDEBUG_MODE=coverage
• Composer version 1.10.24 2021-12-09 20:06:33
• roave/no-leaks: 1.1.2

[asgrim]

PHP running out of memory can't be stopped. Increase your memory limit and try again perhaps. no-leaks can only work if the processing can finish first.

[jamesspittal]

Hi @asgrim

You wrote:

Increase your memory limit and try again perhaps.

In this environment, memory_limit (in php.ini) is set to 128M by default. I have increased this to 256M and run again.

I get the same output:

$ cat /usr/local/etc/php/php.ini-production | grep memory_limit
memory_limit = 256M
$ ./vendor/bin/roave-no-leaks --configuration test/phpunit.xml --coverage-html coverage
PHPUnit 8.0.6 by Sebastian Bergmann and contributors.
.FF.FF...F......FF.FF.
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 32768 bytes) in /builds/organisation/organisation-app/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php on line 40

Testing again with larger memory_limit values.

[Ocramius]

134217728 / 1024 / 1024 is still 128M

You really want to run it with 1Gb or such, though.

php -dmemory_limit=1G vendor/bin/roave-no-leaks, for example.

[jamesspittal]

Thanks @Ocramius

134217728 / 1024 / 1024 is still 128M

You are correct. I have fixed this issue (the memory_limit increase wasn't being applied).

Running with the -d flag (as you suggested) to set the memory limit to 1G. The output is as below:

$ php -dmemory_limit=1G vendor/bin/roave-no-leaks --configuration test/phpunit.xml --coverage-html coverage
PHPUnit 8.0.6 by Sebastian Bergmann and contributors.
.FF.FF...F......FF.FF.FF.FFFFFFF.FFFF.FFFFFF.F.FFFFFFFF.FFFFFFF  63 / 829 (  7%)
FFF.FFFFFFFFFFFFFFFFFFFFFFFFFFFFF.FFFFFFFFFFFFFFFFFFFFFFFFFFFFF 126 / 829 ( 15%)
F.FFFFFFFFFFFF.FFFFFFF.FFFFFFF.FFFFF.FFF.FFFF.FFFF............. 189 / 829 ( 22%)
..............F.E.............E.E.E.E.E.E.E.E.E.E.E.E.E.E.E.E.E 252 / 829 ( 30%)
.E.EE.E.EE.EE.EE.E.EEE.E.EEE.EE.E.EE.E.EE.EE.EE.EEEE.E.EE.E.E.E 315 / 829 ( 37%)
.EEEEEEE.EEEE.EEEEE.EE.E.E.E.EEEE.EEEE.EEEE.E.EEEEEE..FFF..EEEE 378 / 829 ( 45%)
..EEEEEE..EEEEEEEE..EEEEEEE..EEEE..EEEEEE..E..EEEEEEEPHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 20480 bytes) in /builds/organisation/organisation-app/vendor/phpunit/php-code-coverage/src/Driver/Xdebug.php on line 80

[Ocramius]

And PHPUnit also fails (without this tool)?

To me, it looks like you're just running a large test suite, which is indeed leaking somewhere, but to find the leaks, you must first be able to run it to completion.

You have two ways forward:

1. give it all the ram you can afford to, and hope for a full run
2. run only part of your test suite

The fact that it errors out is also potentially indicating that your test suite is highly stateful, which is a design flaw: does your suite run fine if you randomize execution order?

[asgrim]

If you're running a machine with lots of RAM, try memory_limit of just below what you have available, e.g. if you have 64GB RAM, try with 62GB for example (with caution, make sure you know what you're doing!). If you still cannot run it, there is something using a lot of RAM, at this point there is usually a clear explanation if you consider your codebase carefully.

[Ocramius]

Just to be clear: leaking tests should show, but after the full test suite has run (at which point this tool generates a report).

If tests cannot be repeated, that already is an indicator of unclean test shutdown.