Sunday, May 21, 2023

Some useful Splunk queries for Datastax/Cassandra

1) Compaction error:

index="index_name" sourcetype="cassandra_cluster:system"  "Not enough space for compaction" 


2) DSE Process manual shutdown

index="index_name" "dse service shutdown" | table _time, host, _raw | rename _time as Time _raw as Event | convert ctime(Time) as Time 


3) Hints corrupted:

index="index_name" level=ERROR "*file is corrupted*" HintsDispatchExecutor | table _time host _raw 

4) Some Cassandra Exceptions:

index="index_name" cassandra_exception="InvalidRequestException" OR cassandra_exception="AlreadyExistsException" OR cassandra_exception="ConfigurationException" OR cassandra_exception="FunctionFailureException" OR cassandra_exception="WriteFailureException" OR cassandra_exception="ReadFailureException" OR cassandra_exception="CoordinationFailureException" OR cassandra_exception="RequestExecutionException" OR cassandra_exception="QueryValidationException" OR cassandra_exception="NoHostAvailableException" OR cassandra_exception="DriverException" OR cassandra_exception="UnavailableException" OR cassandra_exception="ReadTimeoutException" OR cassandra_exception="OperationTimedOutException" | eval suppress_field = host."-".cassandra_exception | table _time, host,suppress_field, _raw | rename _time as Time _raw as Event | convert ctime(Time) As Time 

5) OS Out of Memory killer invoked:

index="index_name" source="/var/log/messages" "oom-killer" OR "Out of memory: Kill process" OR "Killed process" | rex field=_raw "^(?:[^:\n]*:){3}\s+(?P<invoked_oom>\w+) invoked oom-killer" | rex field=_raw "^(?:[^ \n]* ){10}(?P<killed_pid>[^\(]+)\((?P<killed_process>\w+)" | transaction _time | table host invoked_oom killed_process _raw 


***

No comments:

Post a Comment