Facing The Challenge Of Windows Logs Collection To Leverage Valuable

Facing the challenge(s) of Windows logs collection to leverage valuable IOCs Michel de Crevoisier Security Analyst, Radar Cyber Security 15.10.2019, Berne RadarServices // Classification: Public .

The five challenges RadarServices // Classification: Public

#1 High diversity of log sources Built-in Server roles Microsoft software 3rd party software Application ADFS Advanced Threat Analytics (ATA) Ivanti software Certification authority Exchange DHCP server Skype DNS server SQL Server IIS web server SYSMON NPS Radius Defender PowerShell Kaspersky Security System [ ] RadarServices // Classification: Public Veeam Backup [ ] 3

#2 Different log extensions EVTX ETL (standard Windows logs in XML format) (analytical logs, like DNS Server or PowerShell) RadarServices // Classification: Public TXT (IIS, NPS, DHCP, PowerShell Transcript, former DNS logs) 4

#3 Multiple architectural approaches Access method / Protocol (MS-EVEN6, RPC, WMI, ) Push vs Pull Agent vs Agentless Intermediate collector VS Direct sending to receiver Central file store vs Shared folder Managed agent VS Unmanaged agent RadarServices // Classification: Public 5

#4 Disabled and restrictive event logs Valuable event logs disabled Protected users (if configured, on DCs only) LSA (Local Security Authority) IIS web server DNS client Event logs with restrictive access SMB server SMB client IIS web server RadarServices // Classification: Public 6

#5 Operational constraints Security Data exchange Performance Avoid usage of high privileges Isolation between customer and security provider Data encryption Secured authentication method High availability Compression RadarServices // Classification: Public Configuration Easy deployment Minimize configuration changes Low impact on operating system Environment Cloud Domain VS Workgroup OT (Operational Technology) 7

Collecting standard Windows logs RadarServices // Classification: Public

WEF/WEC introduction Unified & built-in solution to collect standard Windows logs WEF (Windows Event Forwarding) WEC (Windows Event Collector) Authentication and encryption through Kerberos in a domain or TLS certificates in a Workgroup Collects and stores all requested events from WEF clients according XML subscriptions Data exchange over WinRM (push or pull) High availability capacities where clients send events to each WEC collector XML-based language to control event IDs to collect or to suppress noisy events Settings control over GPO EPS control rate Certain 3rd party software can also: Emulate a WEC server by spoofing a WinRM listener (e.g.: SYSLOG-NG Premium, NXLog Enterprise, AlienVault USM actually uses NXLog) Manage multiple WEC servers with a central management console (e.g.: SuperCharger from Logbinder) RadarServices // Classification: Public 9

Who is publishing about WEF/WEC? HP/ArcSight, Australian Cyber Security, 2017 2015 2013 2017 & 2019 RadarServices // Classification: Public 10

WEF/WEC performance Scaling out Technical characteristics Limitations Up to 4.000 source clients per collector (source: Microsoft) All collected events are saved in Forwarded Events log file Average logging is 5.000 EPS, can go up to 10.000 EPS (source: Microsoft) All events are mixed without any tagging possibilities Maximum recommended size per event log file: 4GB Only standard event logs (EVTX) can be forwarded Maximum recommended size for all Windows logs files: 16GB Compression possible with event log size reduction RadarServices // Classification: Public 11

WEF/WEC advanced approach The Palantir approach to the rescue Multiple event channels Different size and rotation strategy Channel can be tagged for SIEM ingestion Channel can be placed on different storage for better performance Preconfigured subscriptions XML query to specify the events to collect Specify the event channel destination RadarServices // Classification: Public 12

WEF/WEC advanced approach A look in production on a WEC server Deployment is not automatized Requires several manual actions Potential source of incorrect configuration Event channels RadarServices // Classification: Public Subscriptions 13

WEF/WEC deployment enhancement PowerShell at the rescue Automated WEC server role setup Automated Palantir toolset deployment Covers event channel and subscriptions Adjusts log file size and location Fixes SDDL permissions on WinRM service Available on GitHub https://github.com/rs-dev/windows-event-collector auto-deploy RadarServices // Classification: Public 14

WEF/WEC Injecting data with agent from the WEC server to your SIEM ArcSight agent NXLog agent Community RSYSLOG agent JSON Snare agent Source clients WEC collector SIEM CEF Splunk UF agent WinCollect agent Chosen agent software solution RadarServices // Classification: Public Winlogbeat agent Other/ External target / provider Other target External provider / Archiving solution 15

WEF/WEC Injecting data without agent from the WEC server to your SIEM NXLog agent Enterprise SYSLOG-NG Premium Source clients Certificates pushed on hosts Chosen software for WinRM server listener emulation RadarServices // Classification: Public SIEM Certificates are required on each source client ! 16

Collecting Windows DNS transaction logs RadarServices // Classification: Public

Collecting DNS transaction logs Technical possibilities overview DNS transactions logs Linux/Unix OS Windows OS DNS server logs 1 DNS debugging DNS client logs 2 ETW Firewall or 3rd party solution NIDS solution Mirrored traffic 3 ETL Server 2012 R2 RadarServices // Classification: Public Bind, Unbound, Dnsmasq, Passive DNS DNS Event log SYSMON (ID 22) Disabled 18

1 Collecting DNS transaction logs Old school approach with Debugging DNS logs Very simple access High impact on performance Only for debugging purpose Not supported by MS for production Does not include DNS answer Timestamp structure may change Delay before data is written ( 1min) No event ID RadarServices // Classification: Public 19

2 About ETW Event Tracing for Windows Efficient kernel-level tracing facility that allows to save kernel or application-defined events Allows to dynamically enable or disable logging in real time without any restart of the system Great open source projects available: KrabsETW (Microsoft) Performant C library to interact with ETW (https://github.com/Microsoft/krabsetw) PowerKrabsEtw TA-DNSETW PowerShell module built around the KrabsETW APIs (https://github.com/zacbrown/PowerKrabsEtw) Splunk plugin to collect DNS events from ETW using "KrabsETW" (https://github.com/secops4thewin/TA-DNSETW) SilkETW (FireEye) Flexible C# ETW wrapper running as a service - Blackhat 19 (https://github.com/fireeye/SilkETW) NXLog Community Windows agent provided with a native ETW module (im etw). Logs can be saved in a file and/or sent to a remote target RadarServices // Classification: Public 20

2 Collecting DNS transaction logs Advanced approach with native ETW Solutions for production System tools: Low impact on performance Event ID provided DNS answer is provided (but encoded) Not compatible with WEC Requires agent or script installation RadarServices // Classification: Public Built-in: Logman, Perfmon, Netsh Installable: Xperf, Tracelog, NetMon, Microsoft MMA, Tracelogging Splunk App “TA-DNSETW”: read ETW using the KrabsETW library from Microsoft NXLog Community No cache file Built-in module to read and forward ETW logs 21

3 About ETL Event Tracing Logs ETW trace session are saved into ETL log files ETL files can be placed on a shared folder on each DNS server to be read remotely Great open source tools available: ETL-to-EVTX PowerShell script that reads ETL logs and writes them into Windows Event Viewer (https://github.com/acalarch/ETL-to-EVTX) ETLParser (GCPartners) Executable which can decodes several types of ETL files (https://github.com/gcpartners/ETLParser) DNSplice Python script that parses DNS ETL files (https://github.com/nerdiosity/DNSplice) DNS Analytical App (Splunk) PowerShell script for Splunk UF that reads ETL logs (https://splunkbase.splunk.com/app/2937) NXLog Community Windows agent provided with a native ETL module. Logs can be saved in a file and/or sent to a remote target ETW2JSON (Microsoft) Read ETL file and convert it to JSON (https://github.com/microsoft/ETW2JSON) RadarServices // Classification: Public 22

3 Collecting DNS transaction logs Advanced approach with ETL Solutions for production Low impact on performance Event ID provided ETL file can be placed in a shared folder DNS answer is provided (but encoded) Not compatible with WEC per default (*) *ETL-to-EVTX script can convert ETL logs to EVTX log file RadarServices // Classification: Public System tools: Built-in: Tracerpt Installable: Microsoft Message Analyzer (MMA) Splunk App “DNS analytical”: PowerShell script that extracts ETL logs and send it to a remote listener NXLog Community Built-in module to read and forward ETL logs (**) **Currently in preview. Will be fully released in NXLog agent v5 according NXLog support 23

Steps and solutions overview RadarServices // Classification: Public

Overview of collecting methods 1: requires PowerShell script that extracts ETL content into EVTX log files 2: requires agent or plugin with ETL or ETW capacities 3: data in event log has no structure RadarServices // Classification: Public 4: not recommended, requires to query SCCM SQL Server database 5: requires SQL Server advanced configuration 6: pulling requires dealing with firewall, credentials and double NAT issues 7: only a limited set of logs are available. Per default, format and mapping are not maintained. SCOM is not a SIEM. 25

Steps for a proper log collection Download Palantir toolset ng Download and run the Radar deployment script https://github.com/rs-dev/windows-event-collector auto-deploy Configure advanced audit policies Enable PowerShell auditing Configure clients to target your WEC server(s) Install and configure your agent solution on your WEC server(s) to forward logs to your SIEM Enable auditing for permission changes (SACL) Start gathering data in your SIEM RadarServices // Classification: Public 26

. Thank You RadarServices // Classification: Public

ArcSight agent NXLog agent Community RSYSLOG agent Snare agent Splunk UF agent WinCollect agent Winlogbeat agent Injecting data with agent from the WEC server to your SIEM WEF/WEC 15 Chosen agent software solution Source clients WEC collector SIEM Other target / External provider JSON CEF Other target / External provider / Archiving solution