Skip to main content

Leveraging WEF and the HELK

In an effort to have some more content on this blog (wow life gets busy sometimes!) I thought I'd write up this post on how to configure Windows Event Forwarding and the awesome project, HELK.

On a bit of a side-note, this post coincides with an event we run up here in Toronto, Canada called the Canadian Collegiate Cyber Exercise (C3X). This year we are running the first real iteration (although we executed a pilot version last year which you can see here: https://www.youtube.com/watch?v=oycYKQzzHoU) and part of the design will be providing the students with HELK.

OK back on track. I'll admit that when I first learned about Windows Event Forwarding it seemed a little daunting. A few of the posts I first read seemed confusing and a lot of moving parts. To the newcomer, Windows Event Logs can also be fairly intimidating which is why projects such as HELK are so fantastic to be openly and freely distributed.

So why leverage WEF? Well, at a very basic level (and this whole post will be kept as basic as possible) it allows for a kind of centralized storage for you desktops and servers to push some or all the event logs you desire to and then from the storage location, called the collector, you can decide what to do with the logs you actually care about. Should you need more or different logs later down the road, you have them sitting and waiting for you in said collector which is nice for peace of mind.

Before digging in it's also worth mentioning that if this is something you plan on introducing into a production network, there are a lot of things to consider that this post will not be covering such as log size, clustering and load balance, fail over etc.

Enough of me ranting on. Let's kick this thing off!

To start, here's the virtual machine setup I'm going with:
  • pfSense - Virtual router
    • using this to park all the VM's behind.
    • NIC 1 - NAT - DHCP (our internet access)
    • NIC 2 - internal - 172.16.0.0/24
      • ip address = 172.16.0.1
  • Windows Server 2012R2 - Domain Controller
    • my domain for this post is weflab.local
    • NIC 1 - internal
      • ip address = 172.16.0.10
  • Windows Server 2012R2 - Windows Event Collector
    • where we will pull the logs into before shipping them off to HELK
    • NIC 1 - internal
      • ip address = 172.16.0.11
  • HELK
    • obviously :)
    • NIC 1 - internal
      • ip address = 172.16.0.12
I'm running this little setup on VirtualBox but the approach is little to no extra effort for Hyper-V or VMWare should your resources be different.

Another important note is that you will need a decent physical machine to run this on primarily due to HELKs CPU and RAM requirements. For HELK alone, you'll want 4 CPU cores and 16GB RAM (but for this simple demo I'll be reducing that down to 8GB RAM as you don't actually have to use the suggested amount of RAM).

Go ahead and start building the virtual machines. I will not be covering how to setup ActiveDirectory and the domain controller or pfSense since there's no shortage of blogs on how to do so. How you build users and groups and organizational units is entirely up to you. We will also be creating some GPOs which I will guide you through. 

One more note for the 2012R2 machines is I highly recommend you fully updating them. Start the Windows update process then go train for and run a marathon then they should be good when you get back. Reason being is you'll want to take advantage of WMF 5.1 (Windows Management Framework) and other updates that will provide you with better logging abilities primarily for PowerShell. We'll also need to install Sysmon and a configuration for it but I will be guiding you through that as well.

1 - Preparing the Domain Controller

For this whole post I'm only collecting logs from the DC for the sake of brevity but the process to ship off the logs can applied to any other machines you wish to work with.

I've created a super simple OU structure, a standard and local admin user, couple security groups, again, for brevity sake as you can see below.


Like I said above, how you wish to design your users and groups and whatnot is entirely up to you.

Next, I'm going to create a GPO that will configure certain logging actions I want. In an attempt to not make this post take 3 hours to read I just implemented the Malware Archaeology cheat sheets which I'm going to recommend to you. Go ahead and and configure those settings and apply the GPO to the OUs in your setup (I'm just applying it to the DC since it's my only machine being monitored). I've named by policy "GPO-Logging Policy". The PowerShell and logging/advanced logging will be enough for our needs but go as nuts as you like:

https://www.malwarearchaeology.com/cheat-sheets/

Below is a simple screenshot of this:

Next let's install and configure Sysmon and use @SwiftOnSecurity's configuration. Grab these both from:

https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon
https://github.com/SwiftOnSecurity/sysmon-config

In the image below, I'm installing Sysmon on the domain controller with a few arguments:

  • -i = we want to install Sysmon
  • -accepteula = agree to the EULA
  • -l = we want to log the loading of modules
  • -n = we want to log network connections
  • -h md5,sha256 = we want to hash images using MD5 and SHA256
After that I run Sysmon again but this time only specifying the "-c" flag and the path the @SwiftOnSeucrity sysmon-config:


Let's verify that it installed and is working properly by opening the Event Viewer and navigating to Applications and Services - > Microsoft -> Windows -> Sysmon -> Operational:


Awesome!

Next, we're going to create a new GPO for our event forwarding settings. I've called mine "GPO-Event Log Forwarding". What I've done for this little lab is allowed the NT AUTHORITY\NETWORK SERVICE account to read the event logs. There's other ways and permissions you could configure this but again, keeping it simple for now.

Open up the GPO and let's edit the settings. Go to Computer Configuration -> Policies -> Windows Settings -> Security Settings -> Restricted Groups and "Add Group". Type in full or start typing "Event Log Readers" and hit "Check Names" and it should automatically detect the "Event Log Readers" group (you'll know because there will be an underline beneath the text):

After that add the "NT AUTHORITY\NETWORK SERVICE" account as "Members of this group" (meaning we're adding the NETWORK SERVICE account as members of the Event Log Readers group). Close down the prompts and it should look like the following:


Don't close the Group Policy Management Editor just yet as there's more to do. We're also going to configure the WinRM service to start automatically. Browse to Computer Configuration -> Policies -> Windows Settings -> Security Settings -> System Services and select "Windows Remote Management (WS-Management)". Give it a double-click, define it and set to Automatic.



Finally, one more setting to configure. Still in the editor, lets configure Event Forwarding (makes sense right :). Browse to Computer Configuration -> Policies ->  Administrative Templates -> Windows Components -> Event Forwarding. In the right pane, select "Configure target Subscription Manager". Click the "Enabled" radio button and under "Options" click the "Show" button next to "SubscriptionManagers".

All you need to enter differently than mine is the FQDN of your collector (don't worry if you haven't configured it yet or even turned it on). So from the below image, replace "wec01.weflab.local" with your collectors address:


Now close down the dialogue boxes and the editor, drag the new forwarding GPO onto your OUs you want forwarding their logs (in my case just the Domain Controllers OU) and then either make some coffee or open a cmd/PowerShell prompt and run "gpupdate.exe /force" if you're impatient like me.


Awesome!

2 - Configuring the Collector

Now onto the collector. First things first, we need to enable WinRM on this machine as well (since that's going to be the transport the logs will be shipped and received over) as well as increase the log size for the "ForwardedEvents" logs and enable the collector service. Run the commands below from an elevated PowerShell prompt:


Next we need to do a few things. Let's break this down like so:
  • Install Sysmon and the config on the collector as well
    • Mainly because I/you may want to ship off the logs of your collector as well to HELK but also in a later step when we're selecting the specific logs to send to HELK, you will not see Sysmon in the list of available logs.
  • Install winlogbeat
  • Configure the collectors Subscription settings
    • Here we will be defining how the collector is going to tell the forwarding machines which logs it (the collector) is really interested in.
First, let's edit the winlogbeat.yml file. Open it up in you favourite text editor. In it's default state it will look like so:


We're going to edit this to just ship the "ForwardedEvents" log from the collector. Delete everything up the first row containing "-name: " but leave that there. Replace everything with "ForwardedEvents" like so:


Scroll down to the Kafka output section and set the IP address of your HELK instance (again if you haven't configured that yet or even turned on the VM that's perfectly fine, just think through what you WILL be assigning it to have for an ip address):


Awesome!

Close the config and let's install winlogbeat. You can download it from here:

Unzip it and you will see a bunch of files. Most importantly, it will already have a winlogbeat.yml file. Replace this with the one you grabbed and edited above from HELK. You'll also notice there's a winlogbeat.exe and install-service-winlogbeat.ps1 PowerShell script. On the Elastic website they recommend using the .exe to install it but I've had no issues using the PowerShell script. Go ahead and open the script if you'd like to see how it works.



After running it, verify the service installed and you will most likely need to start the service manually:


Awesome!

Next, and still on the collector, open up the Event Viewer. In the left hand pane, select "Subscriptions" then on the far right pane select "Create Subscription". Give your subscription a name, ensure the destination log is "Forwarded Events" then select the radio button for "Source computer initiated":


Click "Select Computer Groups".  There's a few ways to do this but here's what I did. Click "Add Domain Computer" then in the new box select "Advanced" then in the new new box click "Find Now". This will list everything in the domain for easy selection of which machines this configuration will affect. So for example, I could just select "DC01" (my domain controller) or "Domain Controllers" and it'll have the same affect. Select the machines or groups you configured forwarding for and click OK. 



Don't close the "Subscriptions Properties" box just yet as need to specify the "Events to Collect". Click "Select Events".

What I did, is use the list in HELKs winlogbeat.yml file before we edited it and selected all the event logs outlined in that config. If you need a reminder, these are them:


In the above image, some event logs are drilled down into specific event IDs and time frames but I'm going all out (lol) and grabbing those entire logs with a time frame of 1 hour. Again, this is all up to you:

Close it all down back to the Event Viewer main window. With the "Subscriptions" entry highlighted, you should see something that looks like this:


Don't freak out if under "Source Computers" you see a 0. Give it some time or a trusty reboot of everything and it'll show a 1 eventually. You can check this by clicking "Runtime Status" in the right pane.

When you do see a connected source computer, select the "Forwarded Events" log in the left pane and ensure the logs are correctly arriving in your collector:


Awesome!

3 - HELK

Phew we're almost done.

Go ahead and turn on your HELK VM. Mine is running Ubuntu Server 16. Check the HELK github page for install instructions (they're super simple and easy to get installed)


Once installed, you should be presented with a screen like so:


I'm accessing the web interface for Kabana from my collector so let's stroll over to it and make sure we didn't botch anything up:


Click "Discover" on the left side nav to make sure we are getting logs in (again, give this some time to populate):


And finally, let's see if we have some wonderful visualizations in one of the dashboards ( I picked Sysmon dashboard):


Awesome and voila! We have successfully forwarded our events to a collector which is shipping off our preferred logs to HELK.

I absolutely must thank @Cyb3rWard0g and @neu5ron for some late night assistance (sry for keeping you up Roberto lol)

Another big thanks to @SwiftOnSecurity for the awesome Sysmon config.

Happy hunting!!!

-ITG

Popular posts from this blog

We Break So We Can Build Better

Hello again friends!

Today I'm writing this post to shed a little preview light on how our team that builds C3X (Canadian Collegiate Cyber Exercise) will be approaching the design for the 2019 competition.

We are doing a lot of different and fun things this year which means all of us are very excited and at the same time have a beast of a project ahead of us.

I'm also going to discuss areas where you can help out and be a part of this wonderful event.

C3X Overview Entering its third year, the C3X is a competition that puts student teams from various Canadian colleges and universities who are enrolled in cyber security programs a chance to defend a controlled environment against a team of offensive security professionals. That's the short and sweet version but there is so much more.
Ben Wells (@1StealthMove) and I created this in 2016 and saw the first event happen in 2017. Why did we do this? Canada isn't exactly overflowing with combative challenges. Sure we have plen…

C3X2018 Genesis Review

What a year this has been. C3X2018 "Genesis" just wrapped up on October 24th and I couldn't be happier (and happy that it's over because I forgot what real sleep feels like).
For those of you who have no idea what I'm talking about, the Canadian Collegiate Cyber Exercise or C3X is an event that Ben Wells (@1StealthMove) and myself started working on in 2016, and brought  into reality in 2017 with a beta run of the event. You can watch the trailer from last year here:


So why did we create  C3X at all? Sadly, Canada doesn't have too many red vs. blue, cyber war game type events (at least to my knowledge) and as far as RedBlack is concerned, that's a shame. We have a lot of CTF's and the like, but there's no shortage of those already. We've also come to learn that many students do not get the required and needed exposure of  learning about defending Windows environments, ActiveDirectory and being able to put what they've learned in school to th…