Splunk rex Command: Field Extraction with Regular Expressions

What Is the rex Command?

The rex command extracts fields from event data using PCRE (Perl-Compatible Regular Expressions) at search time. It’s your go-to tool when data arrives in unstructured or semi-structured formats that Splunk’s automatic field extraction doesn’t cover.

Core Syntax

| rex [field=<source_field>] "<regex_with_named_groups>"

By default, rex operates on _raw. Use field= to target a specific field.

Named Capture Groups

Use (?P<field_name>pattern) to name extracted fields:

| rex "user=(?P<extracted_user>[^\s]+)"

Security Operations Examples

1. Extracting IPs from Unstructured Logs

index=syslog
| rex "(?P<attacker_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+failed"
| stats count BY attacker_ip
| sort - count

2. Parsing Email Addresses from Alert Text

index=email sourcetype=exchange
| rex "From:\s*(?P<sender>[\w\.\-]+@[\w\.\-]+\.\w+)"
| rex "To:\s*(?P<recipient>[\w\.\-]+@[\w\.\-]+\.\w+)"
| stats count BY sender recipient

3. Extracting URLs from Proxy Logs

index=proxy
| rex "(?P<full_url>https?://[^\s"]+)"
| rex field=full_url "https?://(?P<domain>[^/]+)"
| stats count BY domain | sort - count

4. Custom Error Code Parsing

index=application sourcetype=app_log
| rex "ERROR\[(?P<error_code>\d+)\]:\s*(?P<error_message>[^\]]+)"
| stats count BY error_code error_message

rex in sed Mode — Find and Replace

Use mode=sed to transform field values inline:

| rex mode=sed field=email "s/@.*//g"

This strips everything after the @ symbol, leaving just the username portion.

Masking Sensitive Data (NIST SC-28)

| rex mode=sed field=ssn "s/\d{3}-\d{2}-/***-**-/g"
| rex mode=sed field=credit_card "s/\d{12}/************/g"

Best Practices

• Test regex on regex101.com first — Validate your patterns outside Splunk before embedding them in searches.
• Use field= to target specific fields — Avoid running regex against _raw when you know which field contains your data.
• Be specific with capture groups — Greedy patterns like (.+) cause unexpected matches. Use [^\s]+ or [^,]+ to bound captures.
• Use rex for search-time extraction, props.conf for persistent — If you need the same extraction in every search, define it in transforms.conf.
• Chain multiple rex commands — Extract different fields from the same event with consecutive rex pipes.

Next in the series: The lookup command — enriching events with external reference data.

---