Introducing the all-new TIBCO Community site!

For current users, please click "Sign In" to reset your password and access the enhanced features. If you're a first-time visitor, we extend a warm welcome—click "Sign Up" to become a part of the TIBCO Community!

If you're seeking alternative community sites, explore ibi, Jaspersoft, and Spotfire.

Jump to content
  • Insight into how TIBCO LogLogic® Log Management Intelligence relays events to syslog destinations to retain original source identification


    Manoj Chaurasia

    Table of Contents


    Back to LogLogic Overview page


    TIBCO LogLogic® provides the industry's first enterprise-class, end-to-end log management solution. Using LogLogic® log management solutions, IT organizations can analyze and archive log and machine data for the purpose of compliance and legal protection, decision support for security remediation, and increased system performance and improved availability of overall infrastructure.


    Introduction

    This article explains how LogLogic® LMI relays syslog data to downstream hosts and shows what the events will look like when received by those downstream hosts.

    LogLogic® LMI supports both TCP and UDP syslog for relaying events to downstream hosts. The downstream hosts can be other LogLogic® LMI appliances or 3rd party products. The destination type does not impact how the transmitting LMI appliance sends the events. In fact, only the message routing protocol and the original event format plays a part in how the events are transmitted downstream.

    Events that comply with the syslog RFCs must contain the syslog priority value. This is displayed as the <N> value at the beginning of every syslog message. Events that do not contain this can still be transmitted but the transmitting LMI's behavior differs depending on whether a given event already contains this priority value. File-based data that is forwarded downstream using syslog is a popular example of events that do not comply with the syslog RFCs, and therefore will not contain the <N> value, but must still be forwarded correctly to downstream syslog-based hosts. So the first question is how does LogLogic® LMI transmit events that do comply with the syslog RFCs as well as those which do not? Because this behavior differs depending on the selected message routing protocol this article will explain how those scenarios differ so you can be better prepared when using 3rd party products to receive the forwarded events.


    Scenario List

    This article identifies six different scenarios that a user may want to use when forwarding events. These scenarios are:

    1. UDP syslog with syslog priority
    2. UDP syslog without syslog priority and without encryption
    3. UDP syslog without syslog priority and with encryption (not applicable)
    4. TCP syslog with syslog priority
    5. TCP syslog without syslog priority and without encryption
    6. TCP syslog without syslog priority and with encryption

    Note that syslog priority refers to the <N> value that exists at the beginning of every properly formatted syslog event. This value represents the facility and severity of the event as defined by the sender.

    A summary of these scenarios in the form of a table is available at the end of this article if you wish to skip to that. Otherwise, we'll start with analyzing the simplest scenarios first, which involves the use of UDP syslog. When using LMI to forward events the source IP in the network packet header, not the syslog header, is altered to be that of the original source that generated the event. In this manner, the destination host will receive the packets in such a way that the events look like they are being sent directly from the originating source rather than the LogLogic® LMI appliance. This behavior is described as spoofing whereby LMI uses another IP different from its own IP in the packer headers. This behavior also has the consequence of requiring any and all firewalls between LMI and the destination host to be opened to allow all expected source IPs even though the packets will be transmitted by LMI.


    Scenario 1 UDP syslog with leading <<N>>

    Here is an event excerpt of an example with <N> to show what it looks like inside a network packet, as seen in a packet capture when transmitted with UDP syslog:

     <13>1 2019-05-22T12:34:23.498-05:00 192.168.1.200 MSWinEventLog    0    Security    0    Tue May 22 17:34:23 2019    5156    Microsoft-Windows-Security-Auditing

     

    The above representation is also how applications on destination hosts will receive the events.


    Scenario 2 UDP syslog without syslog priority <> and without encryption

    Here is an event excerpt of an example without <N> to show what it looks like inside a network packet when transmitted with UDP syslog. The only difference between this scenario and the previous one is the lack of <N> in this message however the behavior remains unaltered because the original source IP is being specified in the packet header itself rather than in the message. Note: These Windows events were collected by TIBCO LogLogic® Universal Collector which is responsible for adding the syslog header to each event so testing this scenario with an event without the <N> required manually editing the event before manually re-sending it to LMI.

     1 2019-05-22T12:34:23.498-05:00 192.168.1.200 MSWinEventLog    0    Security    0    Tue May 22 17:34:23 2019    5156    Microsoft-Windows-Security-Auditing 

     


    Scenario 3 UDP syslog without syslog priority and with encryption (not applicable)

    This scenario is not applicable because LogLogic LMI does not support using encryption with UDP syslog.

    The next three scenarios function quite differently from the UDP syslog scenarios because these involve TCP syslog. When using LMI to forward events the source IP in the network packet header will always be that of the transmitting LMI appliance. This aspect greatly assists with making firewall configurations much more secure. So then how do destination hosts know what the original source is for each event? LogLogic® LMI inserts a new syslog header in front of the original syslog payload. This header conforms to the syslog RFCs by utilizing the hostname field to specify the event's originating host IP. Prior to LogLogic® LMI 5.6.1 this IP was in IPv4 notation but starting in 5.6.1 LMI supports IPv6 therefore all addresses are specified using IPv6 notation.


    Scenario 4 TCP syslog with syslog priority

    Here is an event excerpt of an example with <N> to show what it looks like inside a network packet, as seen in a packet capture when transmitted with TCP syslog:

     <13>Jun 3 13:22:30 ::ffff:10.1.2.3: 1 2019-05-22T12:34:23.498-05:00 192.168.1.200 MSWinEventLog    0    Security    0    Tue May 22 17:34:23 2019    5156    Microsoft-Windows-Security-Auditing

     

    The next item for this scenario to be explained here may differ among products. LogLogic LMI reads the IP inserted into the hostname field (i.e. 10.1.2.3) of the new header to use as the original event source but then it strips the new header from the event before storing the event. So in the LMI storage, we see this:

     <13>1 2019-05-22T12:34:23.498-05:00 192.168.1.200 MSWinEventLog    0    Security    0    Tue May 22 17:34:23 2019    5156    Microsoft-Windows-Security-Auditing

     

    Note 1: Third-party products may or may not strip the newly added header and may or may not be able to read the IPv6 formatted address. Refer to your product's documentation for how to build parsing rules that can read IPv6 addresses.

    Note 2: This scenario's results do not differ based on whether encryption is enabled therefore this scenario wasn't analyzed with and without encryption.


    Scenario 5 TCP syslog without syslog priority and without encryption

    Here is an event excerpt of an example without <N> to show what it looks like inside a network packet, as seen in a packet capture when transmitted with TCP syslog:

     1 2019-05-22T12:34:23.498-05:00 192.168.1.200 MSWinEventLog    0    Security    0    Tue May 22 17:34:23 2019    5156    Microsoft-Windows-Security-Auditing

     

    Normally, with a properly formatted syslog event, LMI would insert a new syslog header with the originating source IP address, as described in scenario 4. But the criteria by which LMI determines whether the event is a properly formatted syslog event is simply the presence of the syslog priority (i.e. the <N> value). Since that value is missing in this scenario through manual manipulation of the event content the behavior will differ. Specifically, LMI will not add any new syslog header. This means the only piece of information destination hosts have for identifying the event source is the source IP in the packet header, which is LMI's IP in this scenario. So destination hosts will therefore identify all non-syslog formatted events as originating from the upstream LMI IP when transmitted downstream using TCP syslog. This behavior is documented in the LogLogic® LMI Admin Guide. Here is what the event looks like as stored by downstream hosts like LMI and, as you can see, it's the same content as seen in the packet capture during transmission:

     1 2019-05-22T12:34:23.498-05:00 192.168.1.200 MSWinEventLog    0    Security    0    Tue May 22 17:34:23 2019    5156    Microsoft-Windows-Security-Auditing

     


    Scenario 6 TCP syslog without syslog priority and with encryption

    This scenario is similar to the previous scenario but with the modification of encryption added to the configuration. Because LogLogic® LMI uses Secure Shell (SSH) encryption rather than certificate-based encryption, the events are first forwarded to a local port in order to be inserted into the SSH tunnel thus utilizing SSH's built-in port forwarding capability. The events are transmitted to the destination appliance where they exit the tunnel and are then forwarded to the internal tcp/514 port for processing along with unencrypted TCP syslog data. The use of SSH's port forwarding results in a source IP of 127.0.0.1 in the packet header as seen by the destination host but that IP is set by the transmitting appliance. This behavior is documented in the LogLogic® LMI Admin Guide.

    Note: LogLogic® LMI only supports this scenario when routing to other LMI appliances.

    Here is an event excerpt of an example without <N> to show what it looks like inside a network packet, as seen in a packet capture when transmitted with TCP syslog prior to the data entering the tunnel:

     1 2019-05-22T12:34:23.498-05:00 192.168.1.200 MSWinEventLog 0 Security 0 Tue May 22 17:34:23 2019 5156 Microsoft-Windows-Security-Auditing

     

    It is stored looking the same as above.


    Summary of the Scenarios

    Here is a summary of the scenarios to compare and contrast them:

     

    Screenshot2022-11-07at1_54_45PM.thumb.png.d538d71452f5e47779570a89459a8974.png


    Other Resources

    • If you have issues implementing the above procedure let us know by posting the issues on the LogLogic Discussion page in the TIBCO community.

    • To learn more refer to our documentation.

    • For issues that require additional assistance open a case.


    User Feedback

    Recommended Comments

    There are no comments to display.


×
×
  • Create New...