Splunk Interview Questions

If you're looking for Splunk Interview Questions & Answers for Experienced or Freshers, you are at the right place. There are a lot of opportunities from many reputed companies in the world. According to research, Splunk has a market share of about 36.2%. So, You still have the opportunity to move ahead in your career in Splunk. Mindmajix offers Advanced Splunk Interview Questions 2023 that helps you in cracking your interview & acquire your dream career as Splunk Developer.

We have categorized Splunk Interview Questions into 3 levels they are:

• Basic
• Splunk Architect
• Scenario-Based

Top 10 Frequently Asked Splunk Interview Questions

1. What is Splunk?
2. What are the common port numbers used by Splunk?
3. What are the components of Splunk/Splunk architecture?
4. What is Splunk indexer? What are the stages of Splunk indexing?
5. What are the types of Splunk licenses?
6. What is the Splunk app?
7. Where Splunk's default configuration does is stored?
8. what is the summary index in Splunk?
9. What is Splunk DB connect?
10. What are buckets? Explain Splunk bucket lifecycle?

Q) Splunk Vs ELK

Compare Splunk and Elastic Stack
Feature	Splunk	ELK
Log Management	Highly supported	Highly supported
Customizations	Available	Available
Advanced reporting	Available	Available
Robust Search	Capable of robust search	Capable of robust search
Deployment steps	Easy to deploy	Easy to deploy
Dashboard features	Offers more features than ELK	Relatively less when compared to Splunk
Community support	Highly professional community to support	Open Source has more benefits due to huge user base
Latest Version	Splunk 6.5	Elastic Stack 5.0
Pricing	Offers enterprise editions	Open Source
APIs supported/built on	RESTful API and 200 plus endpoints	RESTFul API and JSON
Third-party integrations	App portal provides 1000s of add-ons	Plethora of integrations and plugins

[ Related Article: Splunk vs ELK]

Basic Splunk Interview Questions and Answers

1) What is Splunk?

Splunk is Google for your machine data. It’s a software/Engine which can be used for searching, visualizing, Monitoring, reporting, etc of your enterprise data. Plunk takes valuable machine data and turns it into powerful operational intelligence by providing real-time insight into your data through charts, alerts, and, reports, etc.

If you want to enrich your career and become a professional in Splunk, then enroll in "Splunk Training" - This course will help you to achieve excellence in this domain.

2) Difference between Splunk and Hadoop

Splunk Cloud	Hadoop HDFS
Annual Subscription	Free
Cloud-based service	Distributed file system designed to run on commodity hardware
Splunk is a tool	Hadoop is a framework
Collect and Index Data	File System Namespace
Splunk is a log analysis platform	Hadoop is a BigData file system
Devices Supported: Windows, Android, iPhone/iPad, Mac	Devices Supported: Windows, Mac Web-based
Splunk DB Connect	Browser Interface
Splunk App for Stream	DFSAdmin
Splunk mainly monitors, log, and analyze machine-generated big data.	Hadoop is ideal for ETL and storage of data in HDFS and streamlining analysis
Splunk is an integrated solution for data analysis	Hadoop is an implementation of the Map-Reduce paradigm

3) What are the common port numbers used by Splunk?

Below are common port numbers used by Splunk, however, you can change them if required -

Service	Port number Used
Splunk Web Port	8000
Splunk Management Port	8089
Splunk Indexing Port	9997
Splunk Index Replication Port	8080
Splunk network port	514 (Used to get data in from network port i.e. UDP data)
KV store	8191

4) What are the components of Splunk/Splunk architecture?

Below are components of Splunk Architecture

• Search head: provides GUI for searching
• Indexer: indexes machine data
• Forwarder: Forwards logs to Indexer
• Deployment server: Mange’s Splunk components in a distributed environment.

5) Which is the latest Splunk version in use?

Latest Version Release - Splunk 9.0.

6) Compare Splunk VS Logstash Vs Sumo Logic

Splunk	Logstash	Sumo Logic
A big player in the log management tool space	An open-source log management tool	SaaS version of Splunk
On-Premises Model	SaaS	Used as part of the ELK stack along with ElasticSearch and Kibana
Installation: Setting it up locally	Installation: Setting up a communication out to the Sumo Logic cloud	Installation: Setting it all up on your own system
Need to plan for the hardware and space capacities that you will need.	Need to configure the sources that will gather and send the logs to Sumo Logic	Need to configure the inputs, filters, and outputs that you’ll want
An extensive amount of features are available	Has a good chunk of similar features to Splunk	Logstash gives you the most control over the tools between devloepment & Community
Can create and manage your own dashboards	Uses a panel-based dashboard system	On its own, Logstash doesn’t give you dashboards at all
Dashboards can be built in either XML code or through the Splunk dashboard editor	Most of the information is presented in a chart-based manner	As part of the ELK stack, Kibana is used as frontend reporting & visualization, metrics tools, such as Graphite, Librato, and DataDog.
600+ Integration Plugins available	Includes development automation tools, cloud platforms, OS platforms, and compliance and security tools	Logstash has a continuously growing plugin environment, over 160 plugins available
Pricing - $1800 – $60,000 per year	Pricing - Free lite version, Entry-level pricing is friendlier	Pricing - Free, avail paid subscriptions also

[ Related Article: Datadog vs Splunk]

7) What is a Splunk indexer? What are the stages of Splunk indexing?

The indexer is the Splunk Enterprise component that creates and manages indexes. The primary functions of an indexer are:

• Indexing incoming data.
• Searching the indexed data.

8) What is a Splunk forwarder and what are the types of Splunk forwarder?

There are two types of Splunk forwarder as below:

1. Universal forwarder (UF) - Splunk agent installed on the non-Splunk system to gather data locally, can’t parse or index data.
2. Heavyweight forwarder (HWF) - a full instance of Splunk with advanced functionality generally works as a remote collector, intermediate forwarder, and possible data filter because they parse data, they are not recommended for production systems.

9) what are the most important configuration files of Splunk OR can you tell the names of few important configuration files in Splunk?

• props.conf
• indexes.conf
• inputs.conf
• transforms.conf
• server.conf.

10) What are the types of Splunk licenses?

• Enterprise license
• free license
• Forwarder license
• Beta license
• Licenses for search heads (for distributed search)
• Licenses for cluster members (for index replication).

11) What is the Splunk app?

Splunk app is a container/directory of configurations, searches, dashboards, etc in Splunk.

[ Check out: Learn to Create Splunk Dashboards ]

12) Where Splunk default configuration does is stored?

$splunkhome/etc/system/default.

13) What features are not available in Splunk free?

Splunk free lacks these features:

• authentication and scheduled searches/alerting
• distributed search
• forwarding in TCP/HTTP (to non-Splunk)
• deployment management.

14) What happens if the license master is unreachable? 

The license slave will start a 24-hour timer, after which search will be blocked on the license slave (though indexing continues). Users will not be able to search data in that slave until it can reach the license master again.

15) what is a summary index in Splunk?

The Summary index is the default summary index (the index that Splunk Enterprise uses if you do not indicate another one). If you plan to run a variety of summary index reports you may need to create additional summary indexes.

Splunk Architect Interview Questions and Answers

16) What is Splunk DB connect?

Splunk DB Connect is a generic SQL database plugin for Splunk that allows you to easily integrate database information with Splunk queries and reports.

17) Can you write down a general regular expression for extracting IP addresses from logs?

There are multiple ways we can extract IP addresses from logs. Below are a few examples.
Regular Expression for extracting IP address:

18) What is the difference between stats vs transaction command?

The transaction command is most useful in two specific cases:

• Unique id (from one or more fields) alone is not sufficient to discriminate between two transactions. This is the case when the identifier is reused, for example, web sessions identified by cookie/client IP. In this case, time span or pauses are also used to segment the data into transactions. In other cases when an identifier is reused, say in DHCP logs, a particular message may identify the beginning or end of a transaction.
• When it is desirable to see the raw text of the events combined rather than analysis on the constituent fields of the events.
• In other cases, it’s usually better to use stats as the performance is higher, especially in a distributed search environment. Often there is a unique id and stats can be used.

19) How to troubleshoot Splunk performance issues?

The answer to this question would be very wide but basically, the interviewer would be looking for the following keywords in an interview:

• Check splunkd.log for any errors
• Check server performance issues i.e. CPU/memory usage, disk i/o, etc
• Install SOS (Splunk on Splunk) app and check for warnings and errors in the dashboard
• Check the number of saved searches currently running and their system resources consumption
• Install Firebug, which is a firefox extension. After it’s installed and enabled, log into Splunk (using firefox), open firebug’s panels, and switch to the ‘Net’ panel (you will have to enable it). The Net panel will show you the HTTP requests and responses along with the time spent on each. This will give you a lot of information quickly over which requests are hanging Splunk for a few seconds, and which are blameless. etc.

20) What are the buckets? Explain Splunk bucket lifecycle?

Splunk places indexed data in directories, called as “buckets”. It is physically a directory containing events of a certain period. A bucket moves through several stages as it ages:

• Hot: Contains newly indexed data. Open for writing. One or more hot buckets for each index.
• Warm: Data rolled from hot. There are many warm buckets.
• Colld: Data rolled from warm. There are many cold buckets.
• Frozen: Data rolled from cold. The indexer deletes frozen data by default, but you can also archive it. Archived data can later be thawed (Data in frozen buckets is not searchable).

By default, your buckets are located in $SPLUNK_HOME/var/lib/Splunk/default/db. You should see the hot-db there, and any warm buckets you have. By default, Splunk sets the bucket size to 10GB for 64bit systems and 750MB on 32bit systems.

21) What is the difference between stats and event stats commands?

Splunk Stats command generates summary statistics of all existing fields in your search results and saves them as values in new fields. Eventstats is similar to the stats command, except that aggregation results are added inline to each event and only if the aggregation is pertinent to that event.

Event Stats computes the requested statistics like stats but aggregates them to the original raw data.

22) Who are the biggest direct competitors to Splunk?

logstash, Loggly, LogLogic, sumo logic, etc.

23) Splunk licenses specify what?

How much data you can index per calendar day.

24) How does Splunk determine 1 day, from a licensing perspective?

Midnight to midnight on the clock of the license master

25) How are forwarder licenses purchased?

They are included with Splunk, no need to purchase them separately.

26) What is a command for restarting just the Splunk web server?

Splunk start Splunk web.

27) What is a command for restarting just the Splunk daemon?

Splunk start Splunk.

28) What is the command to check for running Splunk processes on Unix/Linux?

ps aux | grep Splunk.

29) What is Command to enable Splunk to boot start?

$SPLUNK_HOME/bin/Splunk enable boot-start.

30 How to disable Splunk boot start?

$SPLUNK_HOME/bin/Splunk disable boot-start.

[ Learn Splunk Eval Commands With Examples ]

Splunk Scenario Based Interview Questions

31) What is the source type in Splunk?

The source type is Splunk way of identifying data.

32) How to reset the Splunk admin password?

To reset your password log in to the server on which Splunk is installed and rename passwd file at the below location and then restart Splunk. After restart, you can log in using the default username: admin password:changeme.

33) How to disable Splunk launch message?

Set value OFFENSIVE=Less in splunk_launch.conf.

34) How to clear Splunk search history?

Delete the following file on Splunk server.

$splunk_home/var/log/splunk/searches.log

35) What is Splunk btool or how will you troubleshoot Splunk configuration files?

Splunk btool is a command-line tool that helps us to troubleshoot configuration file issues or just see what values are being used by your Splunk Enterprise installation in the existing environment.

[ Related Article: Splunk Tools ]

36) What is the difference between the Splunk app and Splunk add-on?

Basically, both contains preconfigured configuration and reports, etc, but the Splunk add-on does not have a visual app. Splunk apps have preconfigured visual apps.

37) What is .conf files precedence in Splunk?

File precedence is as follows:

• System local directory — highest priority
• App local directories
• App default directories
• System default directory — lowest priority.

38) What is a fish bucket or what is a fish bucket index?

It’s a directory or index at default location /opt/Splunk/var/lib/Splunk .It contains seek pointers and CRCs for the files you are indexing, so splunkd can tell if it has read them already. We can access it through GUI by searching for “index=_thefishbucket”.

39) How do I exclude some events from being indexed by Splunk?

This can be done by defining a regex to match the necessary event(s) and sending everything else to the null queue. Here is a basic example that will drop everything except events that contain the string login In props. conf:

——————————————————————–
[source::/var/log/foo]
# Transforms must be applied in this order
# to make sure events are dropped on the
# floor prior to making their way to the
# index processor

TRANSFORMS-set= setnull,setparsing

————————————————————————-
In transforms.conf
————————————————————————————–
[setnull] REGEX = . DEST_KEY = queue FORMAT = nullQueue

[setparsing]
REGEX = login
DEST_KEY = queue
FORMAT = indexQueue
—————————————————————————————

40) How can I tell when Splunk is finished indexing a log file?

By watching data from Splunk's metrics log in real-time.

index=”_internal” source=”*metrics.log” group=”per_sourcetype_thruput” series=”” | eval MB=kb/1024 | chart sum(MB)

or to watch everything happening split by source type….

index=”_internal” source=”*metrics.log” group=”per_sourcetype_thruput” | eval MB=kb/1024 | chart sum(MB) avg(eps) over series

And if you’re having trouble with data input and you want a way to troubleshoot it, particularly if your whitelist/blacklist rules aren't working the way you expect, go to this URL:

HTTPS://YOURSPLUNKHOST:8089/SERVICES/ADMIN/INPUTSTATUS

41) How to set the default search time in Splunk 6?

To do this in Splunk Enterprise 6.0, use ui-prefs.conf. If you set the value in $SPLUNK_HOME/etc/system/local, all your users should see it as the default setting. For example, if your $SPLUNK_HOME/etc/system/local/ui-prefs.conf file includes:

[search]
dispatch.earliest_time = @d
dispatch.latest_time = now

The default time range that all users will see in the search app will be today.

The configuration file reference for ui-prefs.conf is here: 

HTTP://DOCS.SPLUNK.COM/DOCUMENTATION/SPLUNK/LATEST/ADMIN/UI-PREFSCONF

42) What is the dispatch directory?

$SPLUNK_HOME/var/run/Splunk/dispatch contains a directory for each search that is running or has been completed. For example, a directory named 1434308943.358 will contain a CSV file of its search results, a search.log with details about the search execution, and other stuff. Using the defaults (which you can override in limits.conf), these directories will be deleted 10 minutes after the search completes – unless the user saves the search results, in which case the results will be deleted after 7 days.

43) What is the difference between search head pooling and search head clustering?

Both are features provided by Splunk for the high availability of Splunk search head in case anyone's search head goes down. Search head cluster is newly introduced and search head pooling will be removed in next upcoming versions. The search head cluster is managed by the captain and the captain controls its slaves. Search head cluster is more reliable and efficient than search head pooling.

44) If I want to add/onboard folder access logs from a windows machine to Splunk how can I add the same?

Below are steps to add folder access logs to Splunk:

1. Enable Object Access Audit through group policy on a windows machine on which folder is located
2. Enable auditing on the specific folder for which you want to monitor logs
3. Install Splunk universal forwarder on a Windows machine
4. Configure universal forwarder to send security logs to Splunk indexer.

45) How would you handle/troubleshoot Splunk license violation warning error?

License violation warning means Splunk has indexed more data than our purchased license quota. We have to identify which index/source type has received more data recently than the usual daily data volume. We can check on the Splunk license master pool-wise available quota and identify the pool for which violation is occurring. Once we know the pool for which we are receiving more data then we have to identify the top source type for which we are receiving more data than usual data. Once the source type is identified then we have to find outsource machine which is sending a huge number of logs and the root cause for the same and troubleshoot accordingly.

46) What is the MapReduce algorithm?

MapReduce algorithm is the secret behind Splunk's fast data searching speed. It’s an algorithm typically used for batch-based large-scale parallelization. It’s inspired by functional programming’s map() and reduce () functions.

47) How Splunk avoid duplicate indexing of logs?

At indexer, Splunk keeps track of indexed events in a directory called fish buckets (default location /opt/Splunk/var/lib/Splunk). It contains seek pointers and CRCs for the files you are indexing, so splunkd can tell if it has read them already. – See more at:

https://www.learnsplunk.com/splunk-indexer-configuration.html#sthash.t1ixi19P.dpuf.

48) What is the difference between Splunk SDK and Splunk framework?

Splunk SDKs are designed to allow you to develop applications from the ground up and not require Splunk Web or any components from the Splunk App Framework. These are separately licensed to you from Splunk Software and do not alter the Splunk Software.

Splunk App Framework resides within Splunk’s web server and permits you to customize the Splunk Web UI that comes with the product and develop Splunk apps using the Splunk web server. It is an important part of the features and functionalities of Splunk Software, which does not license users to modify anything in the Splunk Software.

Explore Splunk Sample Resumes! Download & Edit, Get Noticed by Top Employers!