Threat Detection - Logs, Log Sources and Analysis Make All the Difference

Threat detection has grown to a complex and messy activity in organizations. Many utilize Security Information and Event Management systems which can play a critical role in today's enterprise. In order to do their job, SIEMs depend on the logs generated by the enterprise's various systems. Sounds simple enough. However, in a typical Fortune 500 company scenario, an astounding amount of log data is generated. It's not at all unusual to see 10 Terabytes of plain text per month. Fact is, there can be hundreds, even thousands of sources of log data in the typical enterprise. Even small and medium sized businesses will be overwhelmed trying to collect, analyze, and store their log data. The questions are, then, “Can you collect AND analyze them all? Should you? Will the the infrastructure support storage and ongoing detection? Do you have the expertise in place to analyze logs and maintain the infrastructure to do so?”Logging and analyzing the data from every source system can be ponderous and quite possibly (or probably) a case of overkill. The challenge for IT is to determine what logs absolutely have to be collected, what logs should be collected, and what logs will do little more than bog down your operation. After all, at some point, the human element has to be considered. No company has the resources available to read and vet all of the information from all of the logs all of the time.The security needs must be weighed against the resource needs. An attack may seriously damage a large enterprise, but it can destroy an SME. Each company's data guardian has to determine which logs deserve which prioritization. Therefore, the issue of log monitoring has to be one of establishing priorities. And all logs need to be regularly monitored.Recommended LogsLogs can and should be used for both troubleshooting and intrusion detection. To accomplish these tasks, it’s recommended that the below should be collected and analyzed in order of priority:Log Sources

1. IDS/IPS Alert Detections (Blocked & Allowed), Access, & Configuration Changes
2. Advanced Endpoint Protection (such as Carbon Black Defense) logs
3. Firewall Logs/Connections, Access, Health, & Configuration Changes
4. Domain Controller Authentication, User Creation and Modification
5. Windows Event Application, Security, Powershell, & System
6. DNS Requests
7. Web Proxy Access/Errors
8. Remote Access/VPN Authentication & Connections
9. DHCP Lease Details
10. Two-Factor Authentication Access Attempts
11. Switching Logs & Netflow
12. SNMP Audit Where Relevant

Critical Logs

When deciding which logs to collect and feed into your SIEM for analysis, the main factor is this: what are the critical components of your network? What are the critical components of your business? These MUST be given top priority. There are some systems critical to any enterprise, for instance:

1. Your next-gen firewall and IPS. Collecting and analyzing your firewall & IPS logs are a proactive way to detect attempted invasions before they materialize, and take corrective action. In cases where the attack has been successful, you need to know about it as quickly as possible.
2. Your Endpoints/Advanced Endpoint Security Solutions
3. Your Domain Controller. This is important as it will allow you to view and analyze the actions of users’ network activity. Suspicious activity can then be detected and halted.
4. Your key application and database servers. If any unusual or malicious activity is occurring, it should be also detectable here.
5. Your Web Servers that are exposed to the internet. Web Server vulnerabilities has been the downfall of many otherwise-secure enterprises. Companies have been burned by depending solely on their firewalls for protection. Your database should employ use the vendor-recommended security measures, and its logs should be monitored regularly.

But beyond this list, companies must identify the key elements of their data infrastructure, the elements that provide life or death criticality. Most importantly, not every company will have the same answer. If you’re running an e-commerce company, for example, logs from your web server and your payment systems are critical. If you’re a financial services company, any attempted intrusion into any of your customer records is a major threat, and data from any system connected to your customer records must be collected and analyzed in as close to real time as possible. Third party payment processors must make sure their systems are meeting the latest PCI requirements. Healthcare providers must make sure they comply with HIPAA privacy mandates. The list goes on and on, but the bottom line is this: every company must identify their critical functions and protect them from intrusion. Your SIEM helps you accomplish this, but collection isn’t enough. If critical logs aren’t analyzed, you are exposing your business to the possibility of a catastrophe. For large enterprises, it can be a public relations black eye, a costly embarrassment; for small and medium-sized businesses, it could be the death penalty.

Conclusion

We’ve shown how logs are important, and given some general guidance as to which information is essential for analysis and security purposes. But no two businesses have the exact same needs. It's the responsibility of management to make sure that critical information is being collected and analyzed. Periodic self-evaluations and penetration testing is also highly recommended. Organizations must be disciplined enough to establish a regular schedule of self-assessments and tests, and to stick to that schedule. Gaps and weaknesses must be detected and corrected ASAP. Too often, businesses don’t do this until it’s too late.Logs should be gathered, aggregated, and stored when necessary. Your SIEM may do a great job at all of these. But, without the 24x7 monitoring and analysis, your SIEM simply cannot do the job it was meant to do.