I’m sure many of the companies are using Splunk as centralized log server, and are aware of the licensing method of it. As a Splunk customer you will pay for the amount of logs received by your Splunk environment which could be painful for many large companies, and still, Splunk is a very popular product.
I have to say, I understand that someone has to pay the bill, and usually the decision was made by the “business” instead of the IT professional, who works on the systems. I’ve met a couple employees who had the ongoing task to optimize the outgoing logs from the environment.
And they have no choice, somehow the outgoing logs must be filtered. It doesn’t matter if you don’t have the necessary information stored centrally, or you may have to log into the systems one by one which can be time consuming, when you have to analyse logs in a large environments. Aim is to decrease the outgoing amount.
This is why this article has been created. I’ll try to collect the options available in a basic vSphere environment to decrease the amount of outgoing logs.
In vCenter GUI the following log levels are available:
| Log Level | Description |
| None (Disable logging) | Turns off logging |
| Error (Errors only) | Displays only error log entries |
| Warning (Errors and warnings) | Displays warning and error log entries |
| Info (Normal logging) | Displays information, error, and warning log entries |
| Verbose (Verbose) | Displays information, error, warning, and verbose log entries |
| Trivia (Extended verbose) | Displays information, error, warning, verbose, and trivia log entries |
Caution: Trivia and Verbose mode should be used for troubleshooting and for a limited time only. After the troubleshooting is done, change back to the default!
vCenter
In vCenter server you can choose easily from the above options if you click on Hosts and Clusters tab › vCenter name › Configuration › Settings › General › Edit
Alternatively the same setting is available on the advanced settings tab:
If it’s not enough, and there are logs still to be filtered, rsyslog can be modified as well.
Check that the central syslog server is added on the VAMI interface. Go to the https://vCenterIP:5480 and log in. Select the syslog tab and configure your syslog server.
As next step open a cli to the vcenter ip and check the /etc/vmware-syslog/syslog.conf.
It should look like this:
*.* @192.168.1.1:514;RSYSLOG_SyslogProtocol23FormatThis needs to be edited according to the requirements:
*.warn*.err;*.crit;*.alert @192.168.1.1:514;RSYSLOG_SyslogProtocol23FormatOnce it’s done, restart rsyslog service
systemctl restart rsyslogYou can test the result with logger:
logger -p syslog.info "Info level / this will not appear"
logger -p syslog.warn "Warning level / this will be visible"
logger -p syslog.error "Error level / this will be visible"If you have external PSC(s) this should be done on that appliance(s) as well.
Note: If something changed on VAMI interface, this configuration may require to be added again!
Upgrade Manager
To change Update Manager’s log level we need the old flex gui. Click on Home > Administration > System Configuration > Services > Update manager > Manage.
That’s all. I hope this guide provides enough info to keep the unwanted logs away from your syslog servers. Having said all this: information is important and with log filtering only to error messages you may not have that visibility of your environment that you and your monitoring, reporting tool would need for proper analysis. If you implement log filtering you will be notified only if the issue happened and pro-active intervention becomes difficult.
Many other solutions exist and worth considering instead of Splunk. For example: VMware Log Insight using OSI (single log source) model or per CPU basis, where there is one flat fee for all log data sources from a single CPU, regardless of hypervisor or number of guest OSs.