Configuring Splunk Instances Running K8s

The recommendations to configure Splunk instances that are running on K8S using the Splunk operator is by bundling .conf files in Splunk apps. Unfortunately some things can’t be configured with Splunk apps, e.g. adding a user. Then you have two options: Adding the user manually via CLI or UI and therefore not having the configuration in Git OR Deploying a second instance in K8S that is doing the configuration for the first instance Here’s the deployment YAML for this second option:...

October 27, 2022 · 1 min · Marcus Schiesser

Injecting passwords from Kubernetes secrets into Splunk config files

Currently, the Splunk K8S operator doesn’t support injecting passwords from Kubernetes secrets into the Splunk config. The workaround is to store a complete configuration file as a secret, see https://github.com/splunk/splunk-operator/issues/657 As configuration files can be quite large, I created a small bash script that is using template files for the configuration and filling in the secrets based on user input. You can create the K8S secret containing the configuration by calling:...

June 24, 2022 · 1 min · Marcus Schiesser

Storing Splunk Configuration for Kubernetes in Git

After having discussions about how to use Gitops with Splunk, I created this template to store the complete configuration (apps and system configuration) of a Splunk installation running on Kubernetes in Git: https://github.com/marcusschiesser/splunk-gitops. Having the complete configuration in a single repository is an important prerequisite to enjoy the benefits of GitOps. To ensure that this template is running with future versions of Splunk, it is solely extending Splunk Docker images according to the docker-splunk configuration....

May 9, 2022 · 1 min · Marcus Schiesser

Liveness And Readiness Checks For Splunk In K8s

Usually, you’ll use the Splunk Operator to run Splunk on K8S. There are some use cases where you might want to run Splunk without the operator though. As with any deployment, it’s good practice then to add liveness probes to restart Splunk if it’s not healthy anymore. Furthermore, as the Splunk container needs about one minute to startup, I’ll recommend adding a readiness probe. This ensures that no traffic is sent to a pod as long as Splunk hasn’t been fully started yet....

April 15, 2022 · 1 min · Marcus Schiesser

Running Mapped Subsearches Without Limits In Splunk

If you’re running saved searches in Splunk as subsearches inside of the map command, they are bound by the subsearch limitation. This is an alternative command that doesn’t have this limitation as it starts a new job for each subsearch. To use it, instead of calling: | makeresults | map test You’re using: | makeresults | mapsearch search=test Missing the full flexibility of map, the command also passes each event’s values as input parameters to each called saved search....

March 27, 2022 · 1 min · Marcus Schiesser