Log Management & Log Monitoring For the Cloud Servers

Some server softwares perform only the general monitoring works. Log Monitoring is a separate topic. Servers, application, network can generate the log files. Errors, problems, and other information is logged and saved. Like we talked about Linux Dash or Cockpit or Tessera Log – they are for general server monitoring. Here is an Introduction to Log Management & Log Monitoring For the Cloud Servers. Using the Log Softwares Need Some Theoretical Knowledge. Actually we written about Centralized Log Management and commands to check the server logs before. For real hacking attempts like WordPress XML RPC attack, knowledge of the sysadmin is more important matter.

Basic Theoretical Part of Log Management & Log Monitoring

Log Management (LM) is an approach to deal with the large volumes of computer-generated log messages. Log Management (LM) involves :

1. Log collection
2. Centralized aggregation
3. Long-term retention
4. Log rotation
5. Log analysis in real-time
6. Log search and reporting.
7. Security, system operations, network operations and regulatory compliance

There are many challenges in effectively analyzing large volumes of diverse types logs :

1. Huge volume of log
2. Diversity of log format
3. Analysis preventive measures by the log formats
4. Security matters like intrusion-detection logs

There is no clearcut modern definition of logging. Log management is a complicated process and mistakes while approaching is not uncommon. It can be a golden chance for a hacker to get access to the logs. Logging can produce technical information usable for the maintenance of applications or websites :

• to define whether a reported bug is actually a bug
• to help analyze, reproduce and solve bugs
• to help test new features in a development stage

Log Monitors are the softwares or tools to monitor the log files. In order to detect problems automatically, sysadmins and set up tools t0 analyze the generated logs. These log monitors can scan the log files and search for known text patterns and rules. Once an event has been detected, the log monitoring system will send alert.

Log Management & Log Monitoring For the Cloud Servers

Sudden decision after reading some guide on some web hosts blog to try a new software on a production software for log management & log monitoring for the cloud servers is not great approach. It is not easy to give access to the /var/log directory. The monitoring softwares will also have some undetected bugs or backdoor. There are standards and best practices of log management defined by NIST for example to analyze the security logs.

As for the cloud servers, there are some cloud computing software like OpenStack or virtualization solution like OpenVZ is used to create the instances. Unlike a dedicated server, virtual servers has one monitoring always ON from the hosts – it is for network security. Because the customer can be a hacker. The customer can be a target of the hackers. Even in the so called unmanaged cloud servers, the network monitoring is always ON.

For just running a web server software for WordPress with low volume traffic, for one cloud server setup, installing the web host promoted Log Management & Log Monitoring tools not only wastes RAM and disk space, but also poses some risks. The web hosts will write the guides to increase their sell. Whether the website is up and running, monitoring it is one of the basic measure. Second step for these users is some tools like Linux Dash or Cockpit for generalized monitoring. Sometimes, the web hosts have such tools in their dashboard. When there will be unusual activities, unusual resource consumption will take place. Everyday checking log manually from SSH is a better approach for the unmanaged servers. It is more important to take regular working backup of the whole site and configurations.

When we talk about the Log Management & Log Monitoring Tools, we take it granted that 24 x7 some human is present for the servers. They are tools for the professionals monitoring hundreds of servers. No serious hacker will waste time to hack an ordinary small blog. Even if attempts done, they are quite basic. Penetration is, indeed illegal work and none roams on the street nude. It may be better for these users to have a professional service to check the potential weak points first and plan accordingly. When peoples are talking about the importance of log monitoring of breech of security, they are talking about the infrastructures with many servers.