T-Forms(post E3 #1) Flashcards
what is transforms.conf
where we specify transformations and lookups that can then be applied to any event.
These transforms and lookups are referenced by name in props.
main difference between props and transforms
-props is responsible for how data breaks and parses
-transforms is responsible for how data looks
Props and Transfroms work together
Transforms will not work without props
-props says “do this, but go see transforms for instruction”
-transforms says, “this is how you do it”
Name more functions of Transforms.conf
-Manipulates data before it gets indexed
-Transforms and alters raw data
-Has the ability to change metadata
List the common uses of Transfroms
-Sending events to Null queue
-Separating a single sourcetype into multiple sourcetypes
-Host and source overrides based on Regex
-Delimiter-based field extractions
-Anonymizing data
what is a splunk pipeline
In Splunk, a pipeline refers to the sequence of data processing stages that data goes through as it is ingested, indexed, and made available for searching and analysis.
what is a splunk queue
queues are designed to help manage data flow, ensure data integrity, and handle system load efficiently.
What is Splunkd considered and what are its subprocesses?
*Considered a main process
*the core Splunk process that runs on all Splunk components
1-Parsing queue/parsing pipeline
(linebreaking occurs)
2-Aggregation queue/merging pipeline
(Line merging & Time extraction)
3-Typing queue/typing pipeline
(More Regex occurs here than in parsing pipeline)
4-Indexing queue/indexing pipeline
(syslog out,tcp out, indexer)Final stop is disk after this
What is the solution for when you do not want to index unwanted data?
Send events to null queue
Why do we want to prevent unwanted data from indexing ?
Do not want extra work, takes more time processing, and is a waste of storage
List all 4 Splunk pipelines in order
- Parsing pipeline
- Merging pipeline
- Typing pipeline
- Index pipeline
Explain process of Splunk pipeline and queues
see class notes
What should you do if backlog is caused because process is frozen?
Restart Instance
Facts about Regex in Splunk pipeline
-More Regex happening in Typing pipeline than parsing pipeline
-More Regex in transfroms.conf(mostly in typing pipeline)
What configurations do we control in Splunk pipeline?
props.conf and transforms.conf
