Ubuntu
fail2ban package

logfile fills incredibly fast

Asked by Dr. Default on 2018-06-23

After upgrading from 16.04 to 18.04 and reinstall of fail2ban, my logifles are growing incredibly fast.
In fact, made my VPS incommunicado (no spce on device left) so I changed to hourly logrotation and maxsize of logs.

However, it seems somewhere fail2ban thinks it needs to be in extreme heavy debug mode, even though I said normal level in the jail.local

Is the debuglevel hardcoded somewhere? Did I destroy some log or config file during upgrade?

part of LOG:
-------------------
2018-06-23 16:17:16,807 fail2ban.server 2018-06-23 16:17:16,808 fail2ban.server 2018-06-23 16:17:16,808 fail2ban.server 2018-06-23 16:17:16,809 fail2ban.server 2018-06-23 16:17:16,809 fail2ban 2018-06-23 16:17:16,828 fail2ban 2018-06-23 16:17:16,828 fail2ban 2018-06-23 16:17:16,829 fail2ban.transmitter 2018-06-23 16:17:16,830 fail2ban 2018-06-23 16:17:16,830 fail2ban.transmitter 2018-06-23 16:17:16,831 fail2ban 2018-06-23 16:17:16,831 fail2ban.transmitter 2018-06-23 16:17:16,831 fail2ban 2018-06-23 16:17:16,831 fail2ban.transmitter 2018-06-23 16:17:16,832 fail2ban 2018-06-23 16:17:16,832 fail2ban.transmitter 2018-06-23 16:17:16,851 fail2ban.database 2018-06-23 16:17:16,876 fail2ban 2018-06-23 16:17:16,877 fail2ban.transmitter 2018-06-23 16:17:16,877 fail2ban 2018-06-23 16:17:16,878 fail2ban.transmitter 2018-06-23 16:17:16,878 fail2ban.jail 2018-06-23 16:17:17,134 fail2ban.jail 2018-06-23 16:17:17,134 fail2ban.filter 2018-06-23 16:17:17,135 fail2ban.filter 2018-06-23 16:17:17,140 fail2ban.transmitter 2018-06-23 16:17:17,142 fail2ban.server 2018-06-23 16:17:17,146 fail2ban.server 2018-06-23 16:17:17,149 fail2ban.server 2018-06-23 16:17:17,153 fail2ban.server 2018-06-23 16:17:17,155 fail2ban.server 2018-06-23 16:17:17,158 fail2ban.server 2018-06-23 16:17:17,161 fail2ban.server 2018-06-23 16:17:17,163 fail2ban.server 2018-06-23 16:17:17,166 fail2ban.server 2018-06-23 16:17:17,169 fail2ban.server 2018-06-23 16:17:17,172 fail2ban.server 2018-06-23 16:17:17,174 fail2ban 2018-06-23 16:17:17,175 fail2ban.transmitter 2018-06-23 16:17:17,176 fail2ban.filter 2018-06-23 16:17:17,176 fail2ban.-------------------------------------------
[19008]: INFO Starting Fail2ban v0.10.2
[19008]: DEBUG Creating PID file /var/run/fail2ban/fail2ban.pid
[19008]: DEBUG Starting communication
[19008]: HEAVY server phase {'start': True, 'ready': True, 'start-ready': True}
[19008]: HEAVY client phase {'start': True, 'ready': True, 'start-ready': True, 'configure': True}
[19008]: HEAVY __waitOnServer: (True, 30)
[19008]: HEAVY Command: ['ping', '0.00625']
[19008]: HEAVY OK : 'pong'
[19008]: HEAVY Command: ['set', 'syslogsocket', 'auto']
[19008]: HEAVY OK : 'auto'
[19008]: HEAVY Command: ['set', 'loglevel', '1']
[19008]: HEAVY OK : '1'
[19008]: HEAVY Command: ['set', 'logtarget', '/var/log/fail2ban.log']
[19008]: HEAVY OK : '/var/log/fail2ban.log'
[19008]: HEAVY Command: ['set', 'dbfile', '/var/lib/fail2ban/fail2ban.sqlite3']
[19008]: INFO Connected to fail2ban persistent database '/var/lib/fail2ban/fail2ban.sqlite3'
[19008]: HEAVY OK : '/var/lib/fail2ban/fail2ban.sqlite3'
[19008]: HEAVY Command: ['set', 'dbpurgeage', '1d']
[19008]: HEAVY OK : 86400
[19008]: HEAVY Command: ['add', 'ssh', 'auto']
[19008]: INFO Creating new jail 'ssh'
[19008]: INFO Jail 'ssh' uses pyinotify {}
[19008]: DEBUG Setting usedns = warn for FilterPyinotify(Jail('ssh'))
[19008]: DEBUG Created FilterPyinotify(Jail('ssh'))
/>filterpyinotify[19008]: DEBUG Created FilterPyinotify
[19008]: INFO Initiated 'pyinotify' backend
[19008]: HEAVY OK : 'ssh'
[19008]: HEAVY Command: ['multi-set', 'ssh', 'addfailregex', ['^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*(?:error: PAM: )?[aA]uthentication (?:failure|error) for .* from <HOST>( via \\S+)?\\s*$', '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*(?:error: PAM: )?User not known to the underlying authentication module for .* from <HOST>\\s*$', '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*Failed \\S+ for .*? from <HOST>(?: port \\d*)?(?: ssh\\d*)?(: (ruser .*|(\\S+ ID \\S+ \$serial \\d+\$ CA )?\\S+ (?:[\\da-f]{2}:){15}[\\da-f]{2}(, client user ".*", client host ".*")?))?\\s*$', '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*ROOT LOGIN REFUSED.* FROM <HOST>\\s*$', '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*[iI](?:llegal|nvalid) user .* from <HOST>\\s*$', '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*User .+ from <HOST> not allowed because not listed in AllowUsers\\s*$', '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*User .+ from <HOST> not allowed because listed in DenyUsers\\s*$', '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*User .+ from <HOST> not allowed because not in any group\\s*$', '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*refused connect from \\S+ \$<HOST>\$\\s*$', '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*User .+ from <HOST> not allowed because a group is listed in DenyGroups\\s*$', "^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*User .+ from <HOST> not allowed because none of user's groups are listed in AllowGroups\\s*$"]]
[19008]: DEBUG failregex: '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*(?:error: PAM: )?[aA]uthentication (?:failure|error) for .* from <HOST>( via \\S+)?\\s*$'
[19008]: DEBUG failregex: '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*(?:error: PAM: )?User not known to the underlying authentication module for .* from <HOST>\\s*$'
[19008]: DEBUG failregex: '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*Failed \\S+ for .*? from <HOST>(?: port \\d*)?(?: ssh\\d*)?(: (ruser .*|(\\S+ ID \\S+ \$serial \\d+\$ CA )?\\S+ (?:[\\da-f]{2}:){15}[\\da-f]{2}(, client user ".*", client host ".*")?))?\\s*$'
[19008]: DEBUG failregex: '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*ROOT LOGIN REFUSED.* FROM <HOST>\\s*$'
[19008]: DEBUG failregex: '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*[iI](?:llegal|nvalid) user .* from <HOST>\\s*$'
[19008]: DEBUG failregex: '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*User .+ from <HOST> not allowed because not listed in AllowUsers\\s*$'
[19008]: DEBUG failregex: '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*User .+ from <HOST> not allowed because listed in DenyUsers\\s*$'
[19008]: DEBUG failregex: '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*User .+ from <HOST> not allowed because not in any group\\s*$'
[19008]: DEBUG failregex: '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*refused connect from \\S+ \$<HOST>\$\\s*$'
[19008]: DEBUG failregex: '^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*User .+ from <HOST> not allowed because a group is listed in DenyGroups\\s*$'
[19008]: DEBUG failregex: "^\\s*(<[^.]+\\.[^.]+>)?\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )?(?:(?:\\[\\d+\\])?:\\s+[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?|[\\[\$]?sshd(?:\\(\\S+\$)?[\\]\\)]?:?(?:\\[\\d+\\])?:?)?\\s(?:\\[ID \\d+ \\S+\\])?\\s*User .+ from <HOST> not allowed because none of user's groups are listed in AllowGroups\\s*$"
[19008]: HEAVY OK : True
[19008]: HEAVY Command: ['set', 'ssh', 'addlogpath', '/var/log/auth.log', 'head']
[19008]: INFO Added logfile: '/var/log/auth.log' (pos = 0, hash = f76af315de9397b2622d80ee90327609eabff76b)
/>filterpyinotify[19008]: DEBUG New <Watch wd=1 path=/var/log mask=1073745280 proc_fun=None auto_add=False exclude_filter=<function WatchManager.<lambda> at 0x7fc62383eb70> dir=True >
/>filterpyinotify[19008]: DEBUG Added monitor for the parent directory /var/log
/>filterpyinotify[19008]: DEBUG New <Watch wd=2 path=/var/log/auth.log mask=2 proc_fun=None auto_add=False exclude_filter=<function WatchManager.<lambda> at 0x7fc62383eb70> dir=False >
/>filterpyinotify[19008]: DEBUG Added file watcher for /var/log/auth.log
[19008]: DEBUG Seek to find time 1529762837.1761818 (2018-06-23 16:07:17), file size 207
/>datetemplate [19008]: DEBUG constructed regex (?:^|\b|\W)((?P<Y>(?:202|201)\d)(?P<_sep>[-/.])(?P<m>1[0-2]|0[1-9]|[1-9])(?P=_sep)(?P<d>3[0-1]|[1-2]\d|0[1-9]|[1-9]| [1-9])(?:T| ?)(?P<H>2[0-3]|[0-1]\d|\d):(?P<M>[0-5]\d|\d):(?P<S>6[0-1]|[0-5]\d|\d)(?:[.,](?P<f>[0-9]{1,6}))?(?:\s*(?P<z>Z|UTC|GMT|[+-][01]\d(?::?\d{2})?))?)(?=\b|\W|$)
/>datetemplate [19008]: DEBUG constructed regex ^(?:\W{0,2})?((?P<Y>(?:202|201)\d)(?P<_sep>[-/.])(?P<m>1[0-2]|0[1-9]|[1-9])(?P=_sep)(?P<d>3[0-1]|[1-2]\d|0[1-9]|[1-9]| [1-9])(?:T| ?)(?P<H>2[0-3]|[0-1]\d|\d):(?P<M>[0-5]\d|\d):(?P<S>6[0-1]|[0-5]\d|\d)(?:[.,](?P<f>[0-9]{1,6}))?(?:\s*(?P<z>Z|UTC|GMT|[+-][01]\d(?::?\d{2})?))?)(?=\b|\W|$)