Collect Palo Alto Networks firewall logs

Palo Alto Networks Firewall

Overview

This document describes how you can configure syslog and a Google SecOps forwarder to collect Palo Alto Networks firewall logs. This document also explains how Palo Alto Networks firewall log fields map to Google SecOps Unified Data Model (UDM) fields. For an overview about Google SecOps data ingestion, see Data ingestion to Google SecOps. An ingestion label identifies the parser which normalizes raw log data to structured UDM format. The information in this document applies to the parser with the PAN_FIREWALL ingestion label.

Before you begin

• Ensure that the Palo Alto Networks firewall product is properly deployed and configured. For detailed setup instructions, refer to the PAN-OS Documentation.

• To understand the components deployed to collect Palo Alto Networks firewall logs, review the deployment architecture. Each customer deployment might differ from this representation and might be more complex. The following diagram shows how you can configure syslog on a Palo Alto Networks firewall and install a Google SecOps forwarder on a Linux server to forward log data to Google SecOps. The parser supports logs written in the following data formats: Comma Separated Values (CSV), Common Event Format (CEF), and Log Event Extended Format (LEEF).

  

• Verify the log formats and PAN-OS versions that the Google SecOps parser supports. The following table lists the log formats and the corresponding PAN-OS versions that the Google SecOps parser supports:

  Log format	PAN-OS version
  CSV	10.1.3
  CEF	10.0.0
  LEEF	9.1.0

• Verify the Palo Alto Networks firewall log types that the Google SecOps parser supports. The Google SecOps parser supports the following Palo Alto Networks firewall log types:

  • Traffic
  • Threat
  • WildFire submissions
  • Tunnel inspection
  • Config
  • System
  • HIP match
  • IP-Tag
  • User-ID
  • Decryption
  • Authentication
  • URL filtering
  • Data filtering
  • GlobalProtect
  • Correlation
  • GTP
  • SCTP
  • Audit

  For more information about the Palo Alto Networks firewall log types, see PAN-OS log types.

• Ensure that all systems in the deployment architecture are configured in the UTC time zone.

• Before you use the Palo Alto Networks firewall parser, review the changes in field mappings between the previous parser and the current Palo Alto Networks firewall parser. As part of the migration, ensure that the rules, searches, dashboards, or other processes that depend on the original fields use the updated fields.

  For example, in the previous parser version, the category log field is mapped to the security_result.description UDM field. In the current Palo Alto Networks firewall parser, the category log field is mapped to the security_result.category_details UDM field. If you migrate to the current Palo Alto Networks firewall parser and use the category field in your rules, you need to modify the rules to use the security_result.category_details UDM field of the current parser.

Configure syslog and the Google Security Operations forwarder

To configure syslog and the Google SecOps forwarder, complete the following steps:

1. To monitor CSV logs, configure the syslog server profile. For more information, see Configure the syslog server profile. When you configure the syslog server profile, specify "Default" as the custom log format.
2. To monitor CEF logs, configure the Palo Alto Networks firewall to forward CEF logs. For more information, download the PAN-OS CEF Integration guide PDF and see the "Configuration of Palo Alto Networks NGFW to output CEF events" section.
3. To monitor LEEF logs, configure the syslog server profile. For more information, see Custom log forwarding in LEEF format.

4. Configure the Google SecOps forwarder to send logs to Google Security Operations. For more information, see Installing and configuring the forwarder on Linux. The following is an example of a Google SecOps forwarder configuration:

     - syslog:
         common:
           enabled: true
           data_type: PAN_FIREWALL
           batch_n_seconds: 10
           batch_n_bytes: 1048576
         tcp_address: 0.0.0.0:10518
         connection_timeout_sec: 60
   

Supported Palo Alto Networks firewall log formats

The Palo Alto Networks firewall parser supports logs in LEEF,CEF and CSV format.

Supported Palo Alto Networks firewall sample logs

• LEEF

  <14>Jan 22 02:20:19 device_host LEEF:1.0|Palo Alto Networks|PAN-OS Syslog Integration|10.2.12-h4|Microsoft MSOFFICE(52033)|ReceiveTime=2025/01/22 02:20:18|SerialNumber=01250100xxxx|cat=THREAT|Subtype=wildfire|devTime=Jan 22 2025 08:20:18 GMT|src=198.50.100.1|dst=198.50.100.2|srcPostNAT=198.50.100.3|dstPostNAT=198.50.100.4|RuleName=AZURE-US-NEW-CNF_Inbound_To_Azure-ALLOW|usrName=|SourceUser=|DestinationUser=|Application=smtp-base|VirtualSystem=vsys1|SourceZone=McD-Global-Zone|DestinationZone=Azure-Zone|IngressInterface=ae1.111|EgressInterface=ae2.409|LogForwardingProfile=Default-Traffic-Logging|SessionID=35331795|RepeatCount=1|srcPort=21578|dstPort=25|srcPostNATPort=0|dstPostNATPort=0|Flags=0x2000|proto=tcp|action=allow|Miscellaneous=\"......3...................xls\"|ThreatID=Microsoft MSOFFICE(52033)|URLCategory=malicious|sev=4|Severity=high|Direction=client-to-server|sequence=7462614601465681755|ActionFlags=0x8000000000000000|SourceLocation=198.50.100.1-198.50.100.255|DestinationLocation=United States|ContentType=|PCAP_ID=0|FileDigest=0ea04c99bf188c2e4207f60f92ca7c6f5088c7943ee63f45c50032bbd2bf7ea9|Cloud=demo.com|URLIndex=1|RequestMethod=|FileType=ms-office|Sender=sender@ab.myownpersonaldomain.com|Subject=\"............:.................................................................................-.........(Name)-2025-01-22...............:Y107202501220005, ............:........................, ...............:.........\"|Recipient=abc@demo.myownpersonaldomain.com|ReportID=117022282776|DeviceGroupHierarchyL1=143|DeviceGroupHierarchyL2=144|DeviceGroupHierarchyL3=39|DeviceGroupHierarchyL4=0|vSrcName=|DeviceName=device_host|SrcUUID=|DstUUID=|TunnelID=0|MonitorTag=|ParentSessionID=0|ParentStartTime=|TunnelType=N/A|ThreatCategory=N/A|ContentVer=WildFire-0
  

• CEF

  14>1 2024-04-04T16:21:56+02:00 FW-PERIMETRAL-AVG-01 - - - - CEF:0|Palo Alto Networks|PAN-OS|10.1.10-h2|end|TRAFFIC|1|src=198.51.100.1 dst=198.51.100.2 srcTranslatedAddress=198.51.100.3 dstTranslatedAddress=198.51.100.4 rule=FW_USER_NATS2_APP suser= duser= app=bittorrent vs=vsys1 sz=INSIDE dz=EXTERNAL InboundInterface=ae2.2266 OutboundInterface=ae1 lp=log_forwarding sid=2935823 cnt=1 spt=6881 dpt=51413 srcTranslatedPort=0 dstTranslatedPort=0 flags=0x7a proto=udp act=allow tbytes=475 in=150 out=325 pkt=2 pktReceived=1 pktSent=1 start=Apr 04 2024 14:21:56 GMT stime=1206 urlcat=any externalId=externalId reason=aged-out DGl1=11 DGl2=161 DGl3=0 DGl4=0 VsysName=STONESOFT dvchost=FW-PERIMETRAL-AVG-01 cat=from-policy ActionFlags=0x8000000000000000 srcUUID= dstUUID= TunnelID=0 MonitorTag= ParentSessionID=0 ParentStartTime= TunnelType=N/A SCTPAssocID=0 SCTPChunks=0 SCTPChunkSent=0 SCTPChunksRcv=0 RuleUUID=746c3eb6-3d51-4679-8438-bd0e00e170a8 HTTP2Con=0 LinkChange=0 PolicyID= LinkDetail= SDWANCluster= SDWANDevice= SDWANClustype= SDWANSite= DynamicUsrgrp= XFFIP= srcDevCat= srcDevProf= srcDevModel= srcDevVendor= srcDevOS= srcDevOSv= srcHostname= srcMac= dstDevCat= dstDevProf= dstDevModel= dstDevVendor= dstDevOS= dstDevOSv= dstHostname= dstMac= ContainerName= PODNamespace= PODName= srcEDL= dstEDL= GPHostID= EPSerial= srcDAG= dstDAG= HASessionOwner= TimeHighRes=2024-04-04T16:21:56.250+02:00 ASServiceType= ASServiceDiff="
  

• CSV

  1,2021/10/24 15:30:07,,CONFIG,0,2561,2021/10/24 15:30:07,198.51.100.0,,set,admin,Web,Succeeded, network virtual-router  VR1,,VR1  { ecmp { algorithm { ip-modulo ; } } protocol { bgp { routing-options { graceful-restart { enable yes; } } enable no; } rip { enable no; } ospf { enable no; } ospfv3 { enable no; } } routing-table { ip { static-route { vr1-log  { path-monitor { enable no; failure-condition any; hold-time 2; } nexthop { ip-address 198.51.100.0; } bfd { profile None; } interface ethernet1/1; metric 10; destination 0.0.0.0/0; route-table { unicast ; } } } } } interface [ ethernet1/1 ethernet1/2 ]; } ,7022390503849066572,0x0,0,0,0,0,,PA-VM,0,
  

Field mapping reference: PAN firewall logs fields to UDM fields

This section explains how the parser maps Palo Alto Networks firewall log fields to Google SecOps UDM event fields for each log type. The Google SecOps label key refers to the name of the key mapped to Labels.key UDM field.

For example, in the case of the "Virtual System" field, the field name is "cs3" in CEF format and is "VirtualSystem" in LEEF format. The UDM field "about.labels.key" contains the value "vsys" and the UDM field "about.labels.value" contains the value of that field. Some of the CEF or LEEF field names do not have a name corresponding to the CSV field names. In such cases, if you add your own variable name in custom log format in the syslog profile, the parser does not map it to the UDM field.

Refer to the following sections for mapping reference of each log type:

• System
• Config
• Threat/wildfire
• Traffic
• User ID
• HIP match
• IP tag
• Decryption
• Tunnel
• Authentication
• URL
• Data
• GlobalProtect
• Correlation
• GTP
• SCTP
• Audit

System

The following table lists the log fields of the system log type and their corresponding UDM fields.

Config

The following table lists the log fields of the config log type and their corresponding UDM fields.

Threat/WildFire

The following table lists the log fields of the Threat/WildFire log type and their corresponding UDM fields.

Traffic

The following table lists the log fields of the traffic log type and their corresponding UDM fields.

User-ID

The following table lists the log fields of the user-id log type and their corresponding UDM fields.

HIP match

The following table lists the log fields of the HIP match log type and their corresponding UDM fields.

IP tag

The following table lists the log fields of the IP tag log type and their corresponding UDM fields.

Decryption

The following table lists the log fields of the decryption log type and their corresponding UDM fields.

Tunnel

The following table lists the log fields of the tunnel log type and their corresponding UDM fields.

Authentication

The following table lists the log fields of the authentication log type and their corresponding UDM fields.

URL

The following table lists the log fields of the URL log type and their corresponding UDM fields.

Data

The following table lists the log fields of the data log type and their corresponding UDM fields.

GlobalProtect

The following table lists the log fields of the GlobalProtect log type and their corresponding UDM fields.

Correlation

The following table lists the log fields of the Correlation log type and their corresponding UDM fields.

GTP

The following table lists the log fields of the gtp log type and their corresponding UDM fields.

SCTP

Audit

Field mapping reference: Log types to UDM event type

The following table lists the Palo Alto Networks firewall log types and their corresponding UDM event types.

Log type	UDM event type
Traffic	NETWORK_CONNECTION
Threat	NETWORK_CONNECTION
URL Filtering	NETWORK_CONNECTION
WildFire	NETWORK_CONNECTION WildFire submissions logs are a subtype of Threat log type and use the same syslog format.
Data Filtering	NETWORK_CONNECTION
Tunnel	NETWORK_CONNECTION
GTP	NETWORK_CONNECTION
Config	SETTING_MODIFICATION/SETTING_CREATION/SETTING_DELETION/SETTING_UNCATEGORIZED The value of the "Command (cmd)" field determines the UDM event type mapping. If the cmd field value is add or clone, SETTING_CREATION is set. If the cmd field value is delete, SETTING_DELETION is set. If the cmd field value is edit, move, rename, set, or commit, SETTING_MODIFICATION is set. If the cmd field value does not contain any values, then SETTING_UNCATEGORIZED is set.
System	If the subtype value is "dhcp", then NETWORK_DHCP is set. If the subtype value is "auth", then USER_LOGIN is set. If the description value is "logged in", then USER_LOGIN is set. If the description value is "logged out", then USER_LOGOUT is set. For other values of the subtype, GENERIC_EVENT is set.
HIP Match	NETWORK_CONNECTION
IP Tag	GENERIC_EVENT
User-ID	USER_LOGIN/USER_LOGOUT/USER_UNCATEGORIZED If subtype value is "login", then USER_LOGIN is set. If subtype value is "logout", then USER_LOGOUT is set. If subtype does not contain any value, then USER_UNCATEGORIZED is set.
Decryption	NETWORK_CONNECTION
Authentication	GENERIC_EVENT
SCTP	NETWORK_CONNECTION
Audit	GENERIC_EVENT

UDM Mapping Delta

UDM Mapping Delta reference: Palo Alto Networks Firewall

The following table lists delta between Old UDM Mapping of Palo Alto Networks Firewall and New UDM Mapping of Palo Alto Networks Firewall.

Palo Alto Networks Firewall Strata Logging Service

Overview

Palo Alto Networks® Strata Logging Service provides cloud-based, centralized log storage and aggregation for your on-premise, virtual (private cloud and public cloud) firewalls, for Prisma Access, and for cloud-delivered services such as Cortex XDR.Strata Logging Service is secure, resilient, and fault-tolerant, and it ensures your logging data is up-to-date and available when you need it. It provides a scalable logging infrastructure that alleviates the need for you to plan and deploy Log Collectors to meet your log retention needs. If you already have on-premise Log Collectors, the new Strata Logging Service can complement your existing setup. You can augment your existing log collection infrastructure with the cloud-based Strata Logging Service to expand operational capacity as your business grows, or to meet the capacity needs for new locations.With this service, Palo Alto Networks takes care of the ongoing maintenance and monitoring of the logging infrastructure so that you can focus on your business.

• Verify the log formats and PAN-OS versions that the Strata Logging Service parser supports. The following table lists the log formats and the corresponding PAN-OS versions that the Strata Logging Service parser supports:

  Log format	PAN-OS version
  JSON	12.1

• Verify the Palo Alto Networks firewall log types that the Google SecOps parser supports. The Google SecOps parser supports the following Palo Alto Networks firewall log types:

  • Traffic
  • Threat
  • Tunnel inspection
  • System
  • HIP match
  • IP-Tag
  • User-ID
  • Decryption
  • Authentication
  • URL filtering
  • GlobalProtect

Strata Logging Service Deployment

• Ensure that the Palo Alto Networks firewall product is properly deployed and configured. For detailed setup instructions, refer to the PAN-OS Documentation and then follow this document of deployment before sending logs to strata logging service Strata Logging Service Deployment Prerequisites

Start Sending Logs to Strata Logging Service:

To Start Sending Logs to Strata Logging Service, following the steps:

1. Install a supported PAN-OS® version
2. Activate Strata Logging Service- Activating Strata Logging Service includes provisioning the certificate that the firewalls need to securely connect to Strata Logging Service.
3. Onboard firewalls to Strata Logging Service with or without Panorama

For detailed onboarding steps, refer to the Documentation.

Forward Logs from Strata Logging Service

To meet your long-term storage, reporting and monitoring, or legal and compliance needs, you can configure Strata Logging Service to forward logs to an HTTPS server or to the following SIEMs:

1. Exabeam
2. Google Chronicle
3. Microsoft Sentinel
4. Splunk HTTP Event Collector (HEC)

Use the HTTPS forward method to forward the logs using the Strata Logging Service for detailed information follow this Documentation.

Supported Palo Alto Networks Strata Logging Service log firewall formats

The Palo Alto Networks Strata Logging Service firewall parser supports logs in JSON format.

Supported Palo Alto Networks Strata Logging Service sample logs

• JSON

  {"source": "Palo Alto Networks FLS LF", "host": "dummy-loghost", "time": "1730265996460", "event": {"TimeReceived": "2024-10-30T05:25:50.000000Z", "DeviceSN": "no-serial", "LogType": "TRAFFIC", "Subtype": "end", "ConfigVersion": "10.2", "TimeGenerated": "2024-10-30T05:25:40.000000Z", "SourceAddress": "198.51.100.6", "DestinationAddress": "198.51.100.6", "NATSource": "", "NATDestination": "", "Rule": "egress-dns-ping-traceroute", "SourceUser": null, "DestinationUser": null, "Application": "dns-base", "VirtualLocation": "vsys1", "FromZone": "VA8280-RN", "ToZone": "inter-fw", "InboundInterface": "tunnel.101", "OutboundInterface": "tunnel.4005", "LogSetting": "Cortex Data Lake", "SessionID": 754194, "RepeatCount": 1, "SourcePort": 53578, "DestinationPort": 53, "NATSourcePort": 0, "NATDestinationPort": 0, "Protocol": "udp", "Action": "allow", "Bytes": 214, "BytesSent": 72, "BytesReceived": 142, "PacketsTotal": 2, "SessionStartTime": "2024-10-30T05:25:10.000000Z", "SessionDuration": 0, "URLCategory": "any", "SequenceNo": 7382192512716388639, "SourceLocation": "198.51.100.6-198.51.255.255", "DestinationLocation": "198.51.100.6-198.51.255.255", "PacketsSent": 1, "PacketsReceived": 1, "SessionEndReason": "aged-out", "DGHierarchyLevel1": 65537, "DGHierarchyLevel2": 65538, "DGHierarchyLevel3": 65541, "DGHierarchyLevel4": 0, "VirtualSystemName": "", "DeviceName": "VA8280-RN", "ActionSource": "from-policy", "SourceUUID": null, "DestinationUUID": null, "IMSI": 0, "IMEI": null, "ParentSessionID": 0, "ParentStarttime": "1970-01-01T00:00:00.000000Z", "Tunnel": "N/A", "EndpointAssociationID": 72057594037927936, "ChunksTotal": 0, "ChunksSent": 0, "ChunksReceived": 0, "RuleUUID": "95cfc3cc-cb00-4758-af1d-de9ab5f07f97", "HTTP2Connection": 0, "LinkChangeCount": 0, "SDWANPolicyName": null, "LinkSwitches": null, "SDWANCluster": null, "SDWANDeviceType": null, "SDWANClusterType": null, "SDWANSite": null, "DynamicUserGroupName": null, "X-Forwarded-ForIP": null, "SourceDeviceCategory": null, "SourceDeviceProfile": null, "SourceDeviceModel": null, "SourceDeviceVendor": null, "SourceDeviceOSFamily": null, "SourceDeviceOSVersion": null, "SourceDeviceHost": null, "SourceDeviceMac": null, "DestinationDeviceCategory": null, "DestinationDeviceProfile": null, "DestinationDeviceModel": null, "DestinationDeviceVendor": null, "DestinationDeviceOSFamily": null, "DestinationDeviceOSVersion": null, "DestinationDeviceHost": null, "DestinationDeviceMac": null, "ContainerID": null, "ContainerNameSpace": null, "ContainerName": null, "SourceEDL": null, "DestinationEDL": null, "GPHostID": null, "EndpointSerialNumber": null, "SourceDynamicAddressGroup": null, "DestinationDynamicAddressGroup": null, "HASessionOwner": null, "TimeGeneratedHighResolution": "2024-10-30T05:25:41.009000Z", "NSSAINetworkSliceType": null, "NSSAINetworkSliceDifferentiator": null}}"
  

Field mapping reference: PAN firewall logs fields to UDM fields

This section explains how the parser maps Palo Alto Networks Strata Logging Service firewall log fields to Google UDM event fields for each log type.

Refer to the following sections for mapping reference of each log type:

• System
• Threat
• Traffic
• User ID
• HIP match
• IP tag
• Decryption
• Tunnel
• Authentication
• URL
• GlobalProtect

System

The following table lists the log fields of the System log type and their corresponding UDM fields.

Threat

The following table lists the log fields of the Threat log type and their corresponding UDM fields.

Traffic

The following table lists the log fields of the Traffic log type and their corresponding UDM fields.

User-ID

The following table lists the log fields of the User-Id log type and their corresponding UDM fields.

HIP match

The following table lists the log fields of the HIP match log type and their corresponding UDM fields.

IP tag

The following table lists the log fields of the IP tag log type and their corresponding UDM fields.

Decryption

The following table lists the log fields of the Decryption log type and their corresponding UDM fields.

Tunnel

The following table lists the log fields of the Tunnel log type and their corresponding UDM fields.

Authentication

The following table lists the log fields of the Authentication log type and their corresponding UDM fields.

URL

The following table lists the log fields of the URL log type and their corresponding UDM fields.

GlobalProtect

The following table lists the log fields of the GlobalProtect log type and their corresponding UDM fields.

Field mapping reference: Log types to UDM event type

The following table lists the Palo Alto Networks Strata Logging Service firewall log types and their corresponding UDM event types.

Log type	UDM event type
Traffic	NETWORK_CONNECTION
Threat	NETWORK_CONNECTION
URL Filtering	NETWORK_CONNECTION
Tunnel	NETWORK_CONNECTION
System	If the subtype value is "dhcp", then NETWORK_DHCP is set. If the subtype value is "auth", then USER_LOGIN is set. If the description value is "logged in", then USER_LOGIN is set. If the description value is "logged out", then USER_LOGOUT is set. For other values of the subtype, GENERIC_EVENT is set.
HIP Match	NETWORK_CONNECTION
IP Tag	GENERIC_EVENT
User-ID	USER_LOGIN/USER_LOGOUT/USER_UNCATEGORIZED If subtype value is "login", then USER_LOGIN is set. If subtype value is "logout", then USER_LOGOUT is set. If subtype does not contain any value, then USER_UNCATEGORIZED is set.
Decryption	NETWORK_CONNECTION
Authentication	STATUS_UNCATEGORIZED
Globalprotect	USER_LOGIN/USER_LOGOUT/USER_RESOURCE_ACCESS If subtype value is "auth", then USER_LOGIN is set. If subtype value is "logout", then USER_LOGOUT is set. If subtype does not contain any value, then USER_RESOURCE_ACCESS is set.

What's next

• Data ingestion to Google SecOps

Need more help? Get answers from Community members and Google SecOps professionals.