Data Onboarding part II

Question 1

What are the 7 + 2 attributes of props.conf?

Answer 1

• TIME_PREFIX
• MAX_TIMESTAMP_LOOKAHEAD
• TIME_FORMAT
• TZ
• SHOULD_LINEMERGE
• LINE_BREAKER
• TRUNCATE
• BREAK_ONLY_BEFORE
• BREAK_ONLY_BEFORE_DATE

Question 2

What does TIME_PREFIX do?

Answer 2

It is used to tell Splunk when to start to look for timestamp in your event. What regex precedes the timestamp. (kind off anchor)

Question 3

What does MAX_TIMESTAMP_LOOKAHEAD do?

Answer 3

enables splunk to run more efficently because it will not have to spend any extra time and resources to find the timestamp. You can tell Splunk your timestamp is 20 characters into your event so Splunk will not wast any time looking through the entire event.

Set a value for the MAX_TIMESTAMP_LOOKAHEAD setting to specify how far into an event past the TIME_PREFIX location to look for the timestamp.
By constraining the amount of tine to look ahead, you can improve both the accuracy and performance in determining and extracting the timestamp.

Question 4

What does TZ do?

Answer 4

it stand for time zone. It sets appropriate time zone for the host, so that the time then shows up correctly on your search head.

Question 5

What does SHOULD_LINEMERGE do?

Answer 5

It tells Splunk what to do with multiple lines. Is Splunk supposed to merge multiple lines into one event? Or should it treat each line as a new event? when you have multi-lined events, or some events that have line breaks that belong to the same evenet, this should be set as “true”. It is generally set to false, and used along with LINE_BREAKER to speed up processing.

Question 6

What does LINE_BREAKER do?

Answer 6

It tells Splunk where to break the data to start a new event. This is important, because if this is not set correctly, you can have data that is spread across multiple events. Line breaks uses regex to determine the pattern so when Splunk comes across this pattern Splunk knows that this signals the end of one event and the beginning of the next one. It should be used in conjuction with SHOULD_LINEMERGE=false

Question 7

What does TRUNCATE do?

Answer 7

It tells Splunk where to limit the size of event in bytes. By default Splunk limits the size of data to 10 000 bytes. Ten truncate to 999999 depending on your event size, so that the event does not truncate early. You can also use this in the case at extremely verbose data where you can only wish to capture part of the event and discard the rest.

Question 8

What does BREAK_ONLY_BEFORE do?

Answer 8

Detects a specific regex pattern, and will break event if the pattern is somewhere in the next line.

Question 9

What does BREAK_ONLY_BEFORE_DATE do?

Answer 9

If It detects a date in a line, it will break the event before that line.

Question 11

This is the “reversed” card. It first displays you the answer and you have to guess the question part of it.

It is used to tell Splunk when to start to look for timestamp in your event. What regex precedes the timestamp. (kind off anchor)

Answer 11

What does TIME_PREFIX do?

Question 12

This is the “reversed” card. It first displays you the answer and you have to guess the question part of it.

enables splunk to run more efficently because it will not have to spend any extra time and resources to find the timestamp. You can tell Splunk your timestamp is 20 characters into your event so Splunk will not wast any time looking through the entire event.

Set a value for the MAX_TIMESTAMP_LOOKAHEAD setting to specify how far into an event past the TIME_PREFIX location to look for the timestamp.
By constraining the amount of tine to look ahead, you can improve both the accuracy and performance in determining and extracting the timestamp.

Answer 12

What does MAX_TIMESTAMP_LOOKAHEAD do?

Question 13

This is the “reversed” card. It first displays you the answer and you have to guess the question part of it.

it stand for time zone. It sets appropriate time zone for the host, so that the time then shows up correctly on your search head.

Answer 13

What does TZ do?

Question 14

This is the “reversed” card. It first displays you the answer and you have to guess the question part of it.

It tells Splunk what to do with multiple lines. Is Splunk supposed to merge multiple lines into one event? Or should it treat each line as a new event? when you have multi-lined events, or some events that have line breaks that belong to the same evenet, this should be set as “true”. It is generally set to false, and used along with LINE_BREAKER to speed up processing.

Answer 14

What does SHOULD_LINEMERGE do?

Question 15

This is the “reversed” card. It first displays you the answer and you have to guess the question part of it.

It tells Splunk where to break the data to start a new event. This is important, because if this is not set correctly, you can have data that is spread across multiple events. Line breaks uses regex to determine the pattern so when Splunk comes across this pattern Splunk knows that this signals the end of one event and the beginning of the next one. It should be used in conjuction with SHOULD_LINEMERGE=false

Answer 15

What does LINE_BREAKER do?

Question 16

This is the “reversed” card. It first displays you the answer and you have to guess the question part of it.

It tells Splunk where to limit the size of event in bytes. By default Splunk limits the size of data to 10 000 bytes. Ten truncate to 999999 depending on your event size, so that the event does not truncate early. You can also use this in the case at extremely verbose data where you can only wish to capture part of the event and discard the rest.

Answer 16

What does TRUNCATE do?

Question 17

This is the “reversed” card. It first displays you the answer and you have to guess the question part of it.

Detects a specific regex pattern, and will break event if it finds that pattern somewhere in the line.

Answer 17

What does BREAK_ONLY_BEFORE do?

Question 18

This is the “reversed” card. It first displays you the answer and you have to guess the question part of it.

If It detects a date in a line, it will break the event before that line.

Answer 18

What does BREAK_ONLY_BEFORE_DATE do?

Question 19

How to test new logs with Splunk GUI?

Answer 19

a) Settings —> add data —> upload (max 500mb) —> select file
b) chose the source type (or use default or whatever)

c) Choose type of event breaks (either Auto, every line, or regex)
- If you select Regex, type the pattern that determines how the raw text stream is broken into events, before line merging takse place ( by default it is ([\r\n]+) The regex must containt a capture group

d) Configure timestamp if it has one
- Use “advanced” to help Splunk recognize the timestamp
- Configure timeprefix to help Splunk find the timestamp by recognising its prefix
*Use regex patter for this. It can also be a literal match*
- Tell Splunk how to interpret the timestamp by editing Timestamp format:
* Use splunk Docs to choose variables
* Eg. if its 13 digit epoch (10 seconds digits, 3 nanosec digits) it would look like this: %s%3N

• Configure lookahead (set up proper maximum timestamp length)
• Go to advanced:
  You can delete attrributes that you don’t need
  Use DATETIME_CONFIG = CURRENT if there is no timestamp in the event, so it assigns system time to each event as Splunk indexes them
  You can add more attributes
  Add truncate and tz if it is missing
  Choose proper timezone
• Copy to clipboard and save it somewhere to use it later when onboarding data

Question 20

What is epoch/system time?

Answer 20

It is the number of seconds after 1970-1-1.

Question 21

How to onboard data through UF?

Answer 21

a) If you don’t have test env it would be nice to create a test index
b) create custom TA’s (inputs.conf for UF, and props.conf for forwarders and indexers)
props. conf on UF need only the LINE_BREAKER attribute.

Question 22

How to configure serverclass.conf?

Answer 22