INTERVIEW PREP Flashcards
What’s minimum replication/search factor for idx cluster and sh cluster? What is replication and search factor?
sh default 2
Default ports?
8000 - web port 8191 - KV store port 8080 - Replication port (idx cluster) 8181 - search head cluster replication 9997 - receiving porg 514 - syslog port 8089 - mgmt port 8088 - hec
How to set up search head cluster, idx cluster, multisite cluster?
IDX cluster:
set up replication factor, search factor, security key, and cluster label in the server.conf of the cluster master
set up manager ip, mode, and pass4symmkey on the peer nodes and search head
SH cluster:
on the deployer set up pass4symmkey, and also label sh cluster;
initialize splunk init shcluster-config command, or use server.conf to set it up
select captain;
How to push configurations from deployers and indexer clusters?
IDX cluster: splunk validate cluster-bundle --check-restart splunk apply cluster-bundle SH clusteR: splunk apply shcluster-bundle
Where to place the configuration bundle on the deployer, DS, and CM?
$SPLUNK_HOME/etc/shcluster/apps
$SPLUNK_HOME/etc/deployment-apps
$SPLUNK_HOME/etc/master-apps/
What is the comapny name?
Symphtech
Splunk web not working and the KV store has errors how do you troubleshoot?
- see mongod.log and splunkd.log, remove mongod.lock file, see permissions for splunk.key
Architect the growth of the cluster while ensuring the
necessary storage requirements are met and retention
policies are set appropriately.
1
Schedule cronjobs using Cron Utility to automate deletion
of syslog data
1
Build syslog servers for network log management
1
Manage Logical Volume Manager (LVM)
1
Troubleshoot disk usage and memory issues on Splunk
indexers
2
Monitoring system health via Monitoring Console and
custom searches and visualizations.
2
Build custom TAs and Apps to ingest and visualize data
from various data sources
2
Review onboarded logs during last phases to ensure no PII
exist, using regex and transforms.conf
2
Manage ingestion on our license by reviewing verbose logs
for necessity of indexing; eliminating extraneous logs by
sending them to null queue or truncating logs
2
Developed 12 dashboards for Financial Board members
annual presentation
2
Built a series of Windows Security Dashboards that included
but are not limited to, Disabled AD Accounts Re-enabled,
AD password change attempts, Brute Force Attacks, Policy
Changes, Changes to Windows Firewalls, Privilege Use, etc.
2
Use advanced dashboard XML and CSS customizations to
achieve specific look on reports
2
Wrote SOPs and established runbooks to document
procedures during incidents
2
Manage the use and deployment of knowledge objects like
macros, tags, eventtypes, calculated fields, field aliases to fit
specific standards of use and avoid duplication
2
Ongoing tuning of the CIM for new data onboarding from
various branches of the organization and develop
preliminary data model searches to kick-off the utilization
of the data when handing off to Security Operations team
2
Actively working with end-users to aid in building efficient
search queries
2
Upgraded entire Splunk infrastructure in both dev and prod -
uction environments
2
Administrating Splunk servers for optimal performance
2
Leading log ingestion efforts; formatting back end configurations
2
Standardizing configurations across Splunk environment
2
Cleaning up custom apps on deployment server; implementing naming standards, app.conf, etc.
2
Leading code reviews to deploy new configurations on a weekly basis; granting approvals and
denials
2
Staggered crons to ensure that alerting is not backed up or skipped
2
Optimizing search head; rewriting SPL to improve run-time performances
2
Setting retention periods as a standard within company
2
Troubleshooting failed ingestion, missing logs and outage of Splunk servers
2
Installing forwarders on new servers as part of ingestion work
2
Designing and leading onboarding training for new hires
2
Reassign knowledge objects upon off-boarding Splunk users
2
Configure inputs and drilldowns on dashboards
2
User assistance on query development, troubleshooting, optimization
2
Upgraded entire Splunk infrastructure in both dev and production environments
2
Troubleshooted missing logs
2
Completed wide-scale project to ingest more than 300+ network devices through syslog filtration
ensuring each host is appropriately filtered into correct directories; utilized custom
configurations to bring the data into Splunk; made new ingested data CIM compliant, and
remedied existing non-compliant data
2
What is your experience with ES?
2
Part of Implementation Team building out several Splunk clusters
2
Ingested data to ensure monitoring of network devices, applications, including deployment of Nix
and Windows TAs to collect server logs
2
Performed the ingestion of several security tools, including Nessus Vulnerability scanner,
Acunetix, Malwarebytes and Bluvector; including modifying out of the box apps for use in
destination environment
2
Worked on props, transforms, inputs, authentication, authorize and severclass configurations on a daily basis
2
Performed field extractions, look ups and macros for various data sources
2
Troubleshooted data transference issues within Splunk; troubleshoot outages
2
Addressed misuse of platform to prevent search head failures, rewrite multi-paneled dashboards
to use base searches, remedy resource intensive dashboards, and shorten long running searches
2
Addressed misuse of platform to prevent search head failures, rewrite multi-paneled dashboards
to use base searches, remedy resource intensive dashboards, and shorten long running searches
2
Principal engineer responsible for designing and maintaining production-quality dashboards,customizing XML via Source Code, including development of a collection of dashboards to
support field tech teams on monitoring the 9,000+ cameras, sensors and alert systems for
building access
2
Create savedsearches to aid Splunk non-technical users in simplifying complex queries
2
Create scheduled reports for summary indexing for 90 day trends of alert failures.
2
Created and monitored Logical Volume Manager (LVM) and Swap Space
2
Utilized Vagrant to provision servers for Developers in Testing Environment
2
Configured and administered LDAP, DNS, DHCP, TLS/SSL on Linux servers
2
Administered SSH, NFS and FTP on Linux servers
2
Administered and configured Apache
2
Perform server updates, patching, upgrades and package installations using rpm, yum and wget
2
Disk Partitioning
2
Deployed and managed virtualization technologies; KVM and VMWare
2
Deployed virtual servers using templates
2
Performed RPM and YUM package installation.
2
Checked System Logs to diagnose errors and resolve them
2
Performed system backup and compression using tar, gzip and bzip
2
Provisioned accounts: added new user accounts, removed users, changed ownerships of groups
using chown, chgrp commands. Modified file permissions using chmod, and set special
permissions using ACL.
2
Automated and scheduled jobs using Cron utility
2
Monitored and resolved service desk tickets using Jira Ticketing system
2
Created and updated Process Documentation for future reference
2
Do you have any experience migrating Splunk?
2
What are macros, and how have you used them?
2
Regex: How to match a digit? How to match a word character?
2
Where do Splunk buckets and databases reside?
2
How would you perform index time field extracion?
2
Tell us about yourself
2
What is splunk’s smart store?
2
Could you tell me something about the default configuration files?
2
When heavy forwarder is down but you have access to SSH what do you do? How would you troubleshoot this?
2
Why could a search be inefficient?
2
