MySQL total number of locks exceeded
The Problem
Occasionally on a very busy server, you may not be able to log in to Open-AudIT and will see an error as below.
A Database Error Occurred Error Number: 1206 The total number of locks exceeds the lock table size /* logon::check_defaults */ DELETE FROM `logs` WHERE (`severity` = 0 AND `timestamp` < (NOW() - INTERVAL 180 DAY)) OR (`severity` = 1 AND `timestamp` < (NOW() - INTERVAL 180 DAY)) OR (`severity` = 2 AND `timestamp` < (NOW() - INTERVAL 180 DAY)) OR (`severity` = 3 AND `timestamp` < (NOW() - INTERVAL 180 DAY)) OR (`severity` = 4 AND `timestamp` < (NOW() - INTERVAL 180 DAY)) OR (`severity` = 5 AND `timestamp` < (NOW() - INTERVAL 90 DAY)) OR (`severity` = 6 AND `timestamp` < (NOW() - INTERVAL 30 DAY)) OR (`severity` = 7 AND `timestamp` < (NOW() - INTERVAL 7 DAY)) Filename: controllers/logon.php Line Number: 351
Why?
Open-AudIT logs to a table in the database named (surprisingly) "logs". The issue is that we're running a query upon the logs table that affects too many rows at once.
The specific MySQL error code (1206) is listed here. MySQL has a configurable option for this called innodb_buffer_pool_size. This can be set in my.cnf (details below).
The Resolution
To resolve the issue you can:
- manually delete log entries while limiting the query and/or
- adjust the option and restart MySQL
- set the Open-AudIT config options for log purging
To manually delete the affected rows, run the below SQL.
Manually Delete Log Entries
NOTE - this command may need to be run several times.
NOTE #2 - If the command fails, try decreasing the LIMIT of 1000 to something smaller.
Windows
c:\xampplite\mysql\bin\mysql.exe -u openaudit -popenauditpassword -e "USE openaudit; DELETE FROM `logs` WHERE `severity` = 7 AND `timestamp` < (NOW() - INTERVAL 7 DAY) LIMIT 1000;"
Linux
mysql -u openaudit -popenauditpassword -e "USE openaudit; DELETE FROM `logs` WHERE `severity` = 7 AND `timestamp` < (NOW() - INTERVAL 7 DAY) LIMIT 1000;"
Adjust the configurable option in MySQL
Windows
Open the file c:\xampplite\mysql\bin\my.cnf in a text editor.
Find the value for innodb_buffer_pool_size and increase as required.
Restart the MySQL service.
Linux
There may be no actual option set. In that case, it will default to 16M. Add an entry (if required) or adjust the existing entry.
Restart the MySQL service.
Set the Open-AudIT config to purge log entries
As an Open-AudIT user who has the Admin role, go to menu -> Admin -> Configuration -> List Configuration.
You will see entries for log_retain_level_[0-7]. I would first adjust the "debug" level logs (ie - 7) to be purged more aggressively. Say 4 days. This is entirely up to you as the user. If you have increased your innodb_buffer_pool_size, this may not be required at all. You could even increase this retention period - assuming your innodb_buffer_pool_size (and server) can cope.