crowdstrike: improvements and fixes to ingest pipelines for all data streams by navnit-elastic · Pull Request #16730 · elastic/integrations

navnit-elastic · 2025-12-30T11:24:13Z

Proposed commit message

crowdstrike: improvements and fixes to ingest pipelines for all data streams

fixes all the issues outlined in issue #15973.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

[ ]

How to test this PR locally

Pipeline Tests for all data streams:

--- Test results for package: crowdstrike - START ---
╭─────────────┬───────────────┬───────────┬──────────────────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE     │ DATA STREAM   │ TEST TYPE │ TEST NAME                                                                    │ RESULT │ TIME ELAPSED │
├─────────────┼───────────────┼───────────┼──────────────────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ crowdstrike │ alert         │ pipeline  │ (ingest pipeline warnings test-alert.log)                                    │ PASS   │ 415.419166ms │
│ crowdstrike │ alert         │ pipeline  │ test-alert.log                                                               │ PASS   │ 265.621417ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-event-stream.log)                             │ PASS   │ 399.122209ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-audit-events.log)                      │ PASS   │ 446.721667ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-auth-activity.log)                     │ PASS   │   453.8645ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-cspmioa-streaming.log)                 │ PASS   │  491.57225ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-cspmsearch-streaming.log)              │ PASS   │ 474.186875ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-data-protection-detection-summary.log) │ PASS   │ 434.634875ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-detection-summary.log)                 │ PASS   │ 433.706167ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-epp-detection-summary.log)             │ PASS   │ 471.211625ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-events.log)                            │ PASS   │ 457.642625ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-firewall.log)                          │ PASS   │ 404.285209ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-identity-protection-incident.log)      │ PASS   │ 394.308375ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-incident-summary.log)                  │ PASS   │ 406.211708ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-ipd-summary.log)                       │ PASS   │ 408.486666ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-mobile-detection-summary.log)          │ PASS   │ 431.840333ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-recon-notification.log)                │ PASS   │ 404.128291ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-remote-response.log)                   │ PASS   │ 408.239333ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-sample.log)                            │ PASS   │   407.6305ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-tags-list.log)                         │ PASS   │ 433.912208ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-tags.log)                              │ PASS   │ 408.622958ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-user-activity.log)                     │ PASS   │ 420.845833ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-xdr-detection-summary.log)             │ PASS   │ 448.744916ms │
│ crowdstrike │ falcon        │ pipeline  │ test-event-stream.log                                                        │ PASS   │ 145.416708ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-audit-events.log                                                 │ PASS   │ 111.738417ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-auth-activity.log                                                │ PASS   │  78.738125ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-cspmioa-streaming.log                                            │ PASS   │  78.328625ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-cspmsearch-streaming.log                                         │ PASS   │  77.802042ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-data-protection-detection-summary.log                            │ PASS   │  93.265666ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-detection-summary.log                                            │ PASS   │  90.241458ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-epp-detection-summary.log                                        │ PASS   │ 259.862917ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-events.log                                                       │ PASS   │  83.804334ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-firewall.log                                                     │ PASS   │  83.687667ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-identity-protection-incident.log                                 │ PASS   │   80.10225ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-incident-summary.log                                             │ PASS   │  68.054958ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-ipd-summary.log                                                  │ PASS   │  77.081084ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-mobile-detection-summary.log                                     │ PASS   │  85.974041ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-recon-notification.log                                           │ PASS   │  66.996458ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-remote-response.log                                              │ PASS   │  75.048334ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-sample.log                                                       │ PASS   │ 104.718625ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-tags-list.log                                                    │ PASS   │  67.588667ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-tags.log                                                         │ PASS   │  68.034917ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-user-activity.log                                                │ PASS   │  70.480083ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-xdr-detection-summary.log                                        │ PASS   │     77.884ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-data.log)                                     │ PASS   │ 422.628334ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-cspm-ioa.log)                             │ PASS   │ 507.362333ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-cspm-iom-evaluation.log)                  │ PASS   │    444.716ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-cspm-iom.log)                             │ PASS   │ 415.004458ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-data-protection-detection-summary.log)    │ PASS   │ 458.365542ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-epp-detection-summary.log)                │ PASS   │ 466.797125ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-lengthy-field-delete.log)                 │ PASS   │ 465.865333ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-lengthy-field-index.log)                  │ PASS   │ 437.518416ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr.log)                                      │ PASS   │ 472.625291ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdrv2-notmanaged.log)                         │ PASS   │ 507.205334ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-linux.log)                                    │ PASS   │ 458.793875ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-macos.log)                                    │ PASS   │   473.1535ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-tags-formats.log)                             │ PASS   │ 424.751959ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-windows.log)                                  │ PASS   │    460.481ms │
│ crowdstrike │ fdr           │ pipeline  │ test-data.log                                                                │ PASS   │ 139.903166ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-cspm-ioa.log                                                        │ PASS   │ 112.283542ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-cspm-iom-evaluation.log                                             │ PASS   │  99.427125ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-cspm-iom.log                                                        │ PASS   │ 100.011291ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-data-protection-detection-summary.log                               │ PASS   │  99.198041ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-epp-detection-summary.log                                           │ PASS   │  254.41975ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-lengthy-field-delete.log                                            │ PASS   │  84.978917ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-lengthy-field-index.log                                             │ PASS   │  82.725333ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr.log                                                                 │ PASS   │ 1.420559792s │
│ crowdstrike │ fdr           │ pipeline  │ test-fdrv2-notmanaged.log                                                    │ PASS   │  78.446291ms │
│ crowdstrike │ fdr           │ pipeline  │ test-linux.log                                                               │ PASS   │ 175.896458ms │
│ crowdstrike │ fdr           │ pipeline  │ test-macos.log                                                               │ PASS   │ 256.396833ms │
│ crowdstrike │ fdr           │ pipeline  │ test-tags-formats.log                                                        │ PASS   │  77.859875ms │
│ crowdstrike │ fdr           │ pipeline  │ test-windows.log                                                             │ PASS   │ 1.279709542s │
│ crowdstrike │ host          │ pipeline  │ (ingest pipeline warnings test-host.log)                                     │ PASS   │ 397.009542ms │
│ crowdstrike │ host          │ pipeline  │ test-host.log                                                                │ PASS   │   65.42125ms │
│ crowdstrike │ vulnerability │ pipeline  │ (ingest pipeline warnings test-vulnerability.log)                            │ PASS   │ 417.808917ms │
│ crowdstrike │ vulnerability │ pipeline  │ test-vulnerability.log                                                       │ PASS   │  99.117083ms │
╰─────────────┴───────────────┴───────────┴──────────────────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: crowdstrike - END   ---
Done

Related issues

Relates [CrowdStrike] Pipeline improvements and fixes #15973

Screenshots

elasticmachine · 2026-01-05T05:49:08Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

kcreddy · 2026-01-06T10:23:00Z

...rowdstrike/data_stream/falcon/elasticsearch/ingest_pipeline/identity_protection_incident.yml

      formats:
        - UNIX
-      tag: date_event_start_time_epoch
+      tag: date_event_end_time_epoch


Can we have a different tag for this one than above?

Thanks! Fixed both issues and also cleaned up duplicate tags across other pipeline files.

Did you use any script for updating the tags?
I think its easier to review the script instead of reviewing all changes in latest commit.

No script was used. I made the changes manually. Mostly the changes followed a pattern:
added _ms suffix for UNIX_MS date format tags, added descriptive suffix to foreach processors, fixed other duplicate tags due to copy paste error.

packages/crowdstrike/data_stream/fdr/elasticsearch/ingest_pipeline/default.yml

navnit-elastic · 2026-01-06T11:48:34Z

packages/crowdstrike/data_stream/alert/elasticsearch/ingest_pipeline/default.yml

      tag: rename_child_process_ids
      target_field: crowdstrike.alert.child_process_ids
      ignore_missing: true
-  - rename:


This processor is redundant.

navnit-elastic · 2026-01-06T11:51:20Z

...rowdstrike/data_stream/falcon/elasticsearch/ingest_pipeline/identity_protection_incident.yml

      formats:
        - UNIX
-      tag: date_event_start_time_epoch
+      tag: date_event_end_time_epoch


Thanks! Fixed both issues and also cleaned up duplicate tags across other pipeline files.

elastic-vault-github-plugin-prod · 2026-01-06T12:20:02Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

elasticmachine · 2026-01-06T12:20:05Z

💚 Build Succeeded

Buildkite Build
Commit: 5166263

History

💚 Build #36016 succeeded c507690
💚 Build #35982 succeeded c98679a

cc @navnit-elastic

kcreddy

Thanks, LGTM.

elastic-vault-github-plugin-prod · 2026-01-06T13:53:17Z

Package crowdstrike - 3.2.0 containing this change is available at https://epr.elastic.co/package/crowdstrike/3.2.0/

navnit-elastic added 4 commits December 29, 2025 16:19

crowdstrike: improve and fix fdr pipelines

51469f3

crowdstrike: improve and fix falcon pipelines

e0653e1

crowdstrike: improve and fix alert,host,vulnerability pipelines

cdc507d

Merge branch 'main' into 15973-crowdstrike

ba7934f

navnit-elastic self-assigned this Dec 30, 2025

navnit-elastic force-pushed the 15973-crowdstrike branch 2 times, most recently from ba57063 to c98679a Compare December 30, 2025 11:32

update changelog

c507690

navnit-elastic force-pushed the 15973-crowdstrike branch from c98679a to c507690 Compare December 31, 2025 11:58

navnit-elastic marked this pull request as ready for review January 5, 2026 05:49

navnit-elastic requested a review from a team as a code owner January 5, 2026 05:49

kcreddy reviewed Jan 6, 2026

View reviewed changes

Fix duplicate processor tags

5166263

navnit-elastic commented Jan 6, 2026

View reviewed changes

kcreddy approved these changes Jan 6, 2026

View reviewed changes

navnit-elastic merged commit ee1a104 into elastic:main Jan 6, 2026
8 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

crowdstrike: improvements and fixes to ingest pipelines for all data streams#16730

crowdstrike: improvements and fixes to ingest pipelines for all data streams#16730
navnit-elastic merged 6 commits intoelastic:mainfrom
navnit-elastic:15973-crowdstrike

navnit-elastic commented Dec 30, 2025 •

edited

Loading

Uh oh!

elasticmachine commented Jan 5, 2026

Uh oh!

kcreddy Jan 6, 2026

Uh oh!

navnit-elastic Jan 6, 2026

Uh oh!

kcreddy Jan 6, 2026

Uh oh!

navnit-elastic Jan 6, 2026

Uh oh!

Uh oh!

navnit-elastic Jan 6, 2026

Uh oh!

navnit-elastic Jan 6, 2026

Uh oh!

elastic-vault-github-plugin-prod bot commented Jan 6, 2026

Uh oh!

elasticmachine commented Jan 6, 2026

Uh oh!

kcreddy left a comment

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod bot commented Jan 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

navnit-elastic commented Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

Uh oh!

elasticmachine commented Jan 5, 2026

Uh oh!

kcreddy Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

navnit-elastic Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

kcreddy Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

navnit-elastic Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

navnit-elastic Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

navnit-elastic Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

elastic-vault-github-plugin-prod bot commented Jan 6, 2026

🚀 Benchmarks report

Uh oh!

elasticmachine commented Jan 6, 2026

💚 Build Succeeded

History

Uh oh!

kcreddy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod bot commented Jan 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

navnit-elastic commented Dec 30, 2025 •

edited

Loading