transforms.conf basics

Question 1

What is transforms.conf?

Answer 1

It is a .conf file that allows us to make changes to the data that we onboarding, such as:

• Sending events to NULL queue
• Separating a single source type into multiple ones
• Host ans sources overrides based on regex
• Delimiter based field extractions
• Anonymizing data

Question 2

What is a Splunk’s pipeline?

Answer 2

Pipeline is the route that the data takes when it comes through Splunk, and depending on the phase of that process, different things happen to the data, such as line breaking, line merging, time extraction.

Question 3

What is a queue?

Answer 3

It is a memory space to store data between pipelines

Question 4

What kind of pipelines there is, and what do they do?

Answer 4

a) Parsing pipeline:
- parsing of utf8 decoding
- linebreaker
- header
- First place to split data stream into a single line event

b) Merging pipeline:
- aggregator
- line merging for multiline events
- time extraction

c) Typing pipeline:
- regex replacement
- annotator

d) Index pipeline:
- syslog out
- tcp out to another Splunk component
- indexer

Question 5

What kind of queues there is? (in order)

Answer 5

• parsing queue
• aggregator queue
• typing queue
• index queue

Question 6

What is the transforms.conf stanza for discarding specific events?

Answer 6

[]
REGEX=
DEST_KEY=queue
FORMAT=nullQueue

Question 7

What is the transforms.conf stanza for deleting all events BUT ones with xyz?

Answer 7

[]
REGEX=.
DEST_KEY=queue
FORMAT=nullQueue

[]
REGEX=xyz
DEST_key=queue
FORMAT=indexQueue

Question 8

What is the transforms.conf stanza for splitting a log into multiple sourcetypes?

Answer 8

[]
REGEX=
DEST_KEY=Metadata:Sourcetype
FORMAT=sourcetype::xyz

[]
REGEX=
DEST_KEY=Metadata:Sourcetype
FORMAT=sourcetype::xyz2

Question 9

What is the transforms.conf stanza for changing host name?

Answer 9

[]
REGEX=
DEST_KEY=Metadata:Host
FORMAT=

Question 10

What is the transforms.conf and props.conf for delimiter based field extraction?

Answer 10

props.conf:
[]
REPORT-=xyz

transforms.conf:
[xyz]
DELIMS=”,”
FIELDS=”saleTime”,”productName”,”somethingElse”,

Question 11

What is the transforms.conf for masking data?

Answer 11

[xyz]
REGEX=
DEST_KEY=_raw
FORMAT=

Question 12

How to anonymize data with props.conf?

Answer 12

[]

SEDCMD-= s///g