Overview:

This Tines story will setup a Tines Webhook action as a receiver for JSON content/data coming from the Splunk Enterprise platform running on a Linux VPS. In this particular case, the data will be Splunk alerts coming from auth.log Sudo Session events. This is perfect for getting customized automated alerts/notifications from your Linux instance via Splunk forwarding. Although Splunk Enterprise natively has alert forwarding, the integration in tines allows you the ability to send the alerts to potentially more places than natively with Splunk, as well as automatically react to the alert via more complex integrations with tines. At the very least, tines will still provide the ability to customize the formatting and display of the alert so that you get the relevant information vs raw output.

This story uses the assistance of Splunk Enterprise, search queries, and alerts running on a Linux VPS. In this particular case, the alerts are being formatted and forwarded to a private Discord channel, but can easily be switched to Slack, etc. with minimal changes to the outgoing HTTP Request action payload. This information is also under the assumption you already have Splunk setup and the relevant logs you want configured/being forwarded into Splunk.

Some concepts you will need to be familiar with to create this story:

HTTP Requests
- Headers
- Payload content
- API Endpoints
  - POST
JSON formatting as well as interacting with JSON content
Webhooks
Minor string content formatting (optional for preference readability)
Splunk Enterprise Search, Alerts
Required beforehand: Splunk and relevant logs setup and already being forwarded into Splunk.

Guide:

[Intro]
Welcome back to Tines with Tyler. In this video, I’m going to show you how to integrate your existing Linux Splunk alerts into Tines, format the data, and send the alert out to a Discord channel via Discord webhook payload. This particular setup will involve Linux Sudo Session events coming from Splunk alerts. The final output can easily be swapped to a variety of other services (Slack, Email, text, other apps, APIs, etc.), but this example uses Discord channel.

[Main]

The first thing you want to do is open up your Tines story. Once opened, drag the Webhook action onto your storyboard.

If you click on the webhook action you will be able to see the URL and other options. The path and secret is built into the URL so you won’t usually need to use this information, but some services may require you specify in order to send data to the webook URL.

In this particular case, I am only concerned about receiving POST requests, so I am just going to select post as the only allowed verbs. Change the title to something relevant and descriptive to you/your project.

Now that is setup, we can copy the webhook URL and bring it over to our Splunk instance.

Login with your Splunk Admin account and go to Search tab to setup the search results you are wanting to create an alert for. In this particular case, I will be setting up an alert based off Sudo Sessions occurring. My Linux auth.log file is forwarded to Splunk under security_logs index. This search query is detailing I am looking for keywords “sudo*session opened”

Once you have defined the appropriate query, click to save it as an alert

On the alert box, customize how you want the alert triggered. In my case I have these settings… The important part to integrate with tines is to paste the tines Webhook action URL into the trigger action webhook in Splunk

Now, whenever the Splunk search query produces results and fits the alert trigger thresholds you set, Splunk will create a JSON alert to your tines Webhook receiver URL.

The next step is to decipher the raw data and format it into relevant information for your alert notification payload.

With that in mind, we can put an Event Transform Action to “Parse” the webhook data into convenient fields. This is how I setup my parsing for the Sudo Session payload.

After you have selected and formatted the information the way you want, you can then setup the Discord webhook payload. This will be done via another Event Transform action set to throttle (to prevent rate limits during high frequency alert periods), followed by an HTTP Request action setup to whatever specific Discord webhook URL you will want to deliver the alert to.

Unfortunately, there is no native test feature with Splunk to test alerts with dummy payloads, so the only way to make sure it is working properly is to actually create the event you are monitoring. In this case, initiate a Sudo session on the monitored server.

[End]
Now you have customized events being forwarded to your Discord if any Sudo sessions are opened on your Linux server! Check out other posts to see how to integrate other Linux events via Splunk, and if you’re interested in creating more projects or wanting to learn how to automate more cool things in tines, be sure to like, subscribe and check out the socials to stay up to date on the latest projects!