ArcSight Forwarder Connector. Send wherever we want
Good day, habrasoobschestvo!
In this article, I will share my practical experience on uploading events from ArcSight ESM. I will review the functionality in detail and provide step by step instructions on how to configure the ArcSight Forwarder Connector, as well as describe the interesting life hacks.
To begin with, let's see why events are generally sent from Arcsight ESM (after all, they live well in the database).
')
We can implement all these points using the ArcSight Forwarder Connector, but the initial configuration starts at the ArcSight ESM itself:
After creating the account and filtering on Arcsight ESM, we can proceed to the installation and configuration of the Arcsight Forwarder Connector
Configure destination types:
Tips and tricks for working with Arcsight Forwarder Connector
As a result, we can observe that ArcSight has a lot of integration capabilities, both with its systems and with external sources. But for the sake of truth, it is worth saying that the flow of events is gradually increasing, the same events must be sent to several sources, there must be a constant interaction of various systems ... and here the capabilities of a regular connector end.
Therefore, Micro Focus engineers developed a new architecture called the ArcSight Data Platform. A distinctive feature of this architecture is the ArcSight Event Broker product, which serves to route a huge stream of events to various systems (ESM, Logger, UEBA, Investigate, Hadoop, etc.) and is capable of processing more than 500,000 EPS !!!
In this article, I will share my practical experience on uploading events from ArcSight ESM. I will review the functionality in detail and provide step by step instructions on how to configure the ArcSight Forwarder Connector, as well as describe the interesting life hacks.
To begin with, let's see why events are generally sent from Arcsight ESM (after all, they live well in the database).
')
- Your company has several ESM servers and the task is to send events to the central ArcSight ESM for global correlation.
- You have a third-party system (for example, ELK) to which you need to send events from ArcSight
- Sending syslog events in CEF format
- Uploading events to a CSV file for further work
We can implement all these points using the ArcSight Forwarder Connector, but the initial configuration starts at the ArcSight ESM itself:
Initial Setup on Arcsight ESM
All work is done on the latest version of Arcsight ESM 6.11 (but this also applies to previous versions).
To begin, create an account that will be used to upload events from ESM.
In the navigation pane, go to the “Users” section and in the “Customer User Groups” directory, create your own “Forwarder Events” directory
Click on the newly created group “Forwarder Event” and create a user, for example “fwd”
For a new user, you must specify the type of account and password.
User Type = Forwarding Connector
Now you need to create a filter for the events that we want to unload from Arcsight ESM. To do this, select the section “Filters” in the navigation panel and create a filter with the conditions we need.
For example, I plan to unload all correlation events from ESM, so my filter will look like this
After the filter is created, it must be imposed on the group “Forwarder Events”, which includes the user “fwd”.
Go to the navigation panel in the section "Users" and for the group "Forwarder Events" select "Edit Access Control"
Then in the "ACL Editor" go to the tab "Events" and click "Add" to add our previously created filter
This completes the settings from the side of Arcsight ESM.
To begin, create an account that will be used to upload events from ESM.
In the navigation pane, go to the “Users” section and in the “Customer User Groups” directory, create your own “Forwarder Events” directory
Click on the newly created group “Forwarder Event” and create a user, for example “fwd”
For a new user, you must specify the type of account and password.
User Type = Forwarding Connector
Now you need to create a filter for the events that we want to unload from Arcsight ESM. To do this, select the section “Filters” in the navigation panel and create a filter with the conditions we need.
For example, I plan to unload all correlation events from ESM, so my filter will look like this
After the filter is created, it must be imposed on the group “Forwarder Events”, which includes the user “fwd”.
Go to the navigation panel in the section "Users" and for the group "Forwarder Events" select "Edit Access Control"
Then in the "ACL Editor" go to the tab "Events" and click "Add" to add our previously created filter
This completes the settings from the side of Arcsight ESM.
After creating the account and filtering on Arcsight ESM, we can proceed to the installation and configuration of the Arcsight Forwarder Connector
Install and configure ArcSight Forwarder Connector
To install the Arcsight Forwarder Connector, we need any server with Linux and the latest version (ArcSight-7.5.0.7986.0-SuperConnector-Linux64.bin) connector.
First of all, we need to make our file executable:
Now let's proceed to the installation of the connector itself:
Let's get acquainted with the information, press "Enter" and specify the installation directory:
Next, we refuse to create links by selecting “4” and confirm the installation
In the end, we will receive information about the successful installation and further instructions on launching the connector tuner.
Now we’ll make a preliminary setup.
Run ./runagentsetup.sh
Select the item “Add a Connector” and type “ArcSight Forwarding Connector (Enhanced)”
Next, the connector will prompt you to hide the input of parameters (login / password)
Now we set the parameters of the ESM server from which we will collect events and indicate the parameters of the previously created account “fwd”
Now you need to import the ArcSight ESM certificate to our connector.
When integration with ESM is successful, the connector will offer you several options for sending events.
Next, I will describe the setting for each of the options.
First of all, we need to make our file executable:
chmod +x ArcSight-7.5.0.7986.0-SuperConnector-Linux64.binNow let's proceed to the installation of the connector itself:
./ArcSight-7.5.0.7986.0-SuperConnector-Linux64.binLet's get acquainted with the information, press "Enter" and specify the installation directory:
/opt/arcsight/forwarderNext, we refuse to create links by selecting “4” and confirm the installation
In the end, we will receive information about the successful installation and further instructions on launching the connector tuner.
Now we’ll make a preliminary setup.
Run ./runagentsetup.sh
/opt/arcsight/forwarder/current/bin/runagentsetup.shSelect the item “Add a Connector” and type “ArcSight Forwarding Connector (Enhanced)”
Next, the connector will prompt you to hide the input of parameters (login / password)
Now we set the parameters of the ESM server from which we will collect events and indicate the parameters of the previously created account “fwd”
Now you need to import the ArcSight ESM certificate to our connector.
When integration with ESM is successful, the connector will offer you several options for sending events.
Next, I will describe the setting for each of the options.
Configure destination types:
Sending events to ArcSight ESM
Enter the data of the ESM server to which we will send events. Here you will need to specify the login and password of the standard account.
Now we specify the name of the connector that will appear on the destination ESM server
Import certificate to connector
At this setting is over. It remains to determine only the launch parameter of the connector. We can install the connector as a service with automatic start, or as an application that needs to be started manually
I usually choose to create an auto-start service.
On the destination server, check the registration of the connector and the arrival of events on it
Now we specify the name of the connector that will appear on the destination ESM server
Import certificate to connector
At this setting is over. It remains to determine only the launch parameter of the connector. We can install the connector as a service with automatic start, or as an application that needs to be started manually
/opt/arcsight/forwarder/current/bin/arcsight agentsI usually choose to create an auto-start service.
On the destination server, check the registration of the connector and the arrival of events on it
Sending events to ArcSight Logger
The first step is to create a “Receiver” at the Arcsight Logger itself. To do this, select the section “Configuration” in the Logger, then “Receivers” and click Add.
Give the name to our recipient and choose the type of events to be received.
Now go to the setting on the connector
Set the parameters for connecting to the Logger and specify our created Receiver - FWD_ESM
Import certificate for connector
Checking the arrival of events on ArcSight Logger
Give the name to our recipient and choose the type of events to be received.
Now go to the setting on the connector
Set the parameters for connecting to the Logger and specify our created Receiver - FWD_ESM
Import certificate for connector
Checking the arrival of events on ArcSight Logger
Sending Syslog Events in CEF Format
It's all elementary. Enter only the destination address, on which port we will send and specify the data transfer protocol
We check the arrival of events, for example in ELK
We check the arrival of events, for example in ELK
Sending events by uploading to a CSV file
In this case, we need to specify only the directory where the csv file will be created, which fields to unload and the file rotation time
Sending events to HPE Operations Manager
The transmission of events occurs via the SNMP protocol with the further display of events in IT Operations Management
Tips and tricks for working with Arcsight Forwarder Connector
Uploading correlation events along with baseline
By default, the Arcsight Forwarder Connector unloads only correlation events. But what to do if basic events are necessary, for example, for a detailed investigation of the incident.
To do this, we need to specify the ID-connector and user-ID as a parameter, as well as register all this in the ESM configuration file.
ID-connector can be found with the command
ID-user "fwd" can be viewed in his profile on ESM
Next, we need on the Arcsight ESM server itself to add an additional parameter to the server.properties file
Stop the server
Enter the parameters
We start the ESM server
Now the correlation events will be unloaded along with the base
To do this, we need to specify the ID-connector and user-ID as a parameter, as well as register all this in the ESM configuration file.
ID-connector can be found with the command
cat /opt/arcsight/forwarder/current/user/agent/agent.properties | grep entityidID-user "fwd" can be viewed in his profile on ESM
Next, we need on the Arcsight ESM server itself to add an additional parameter to the server.properties file
Stop the server
/etc/init.d/arcsight_services stop allEnter the parameters
vi /opt/arcsight/manager/config/server.propertieseventstream.cfc=(connectro ID).(forwarder user ID)We start the ESM server
/etc/init.d/arcsight_services start allNow the correlation events will be unloaded along with the base
Additional event filtering on Forwarder Connector
After installing the Arcsight Forwarder Connector and connecting all the necessary destination nodes, run
Select “Modify Connector”
Next “Add, modify, or remove destinations”
Next, select the source of the destination for filtering.
Select "Modify destination settings"
In this menu all settings of the connector are set. In our case, point 10 is required - Filters
Assigning filtering: in my case, discard all events that are NOT EQUAL to the value in the deviceVendor field.
Thus, we can send a diverse stream of events to the Forwarder Connector and scatter events to the destination source we need.
/opt/arcsight/forwarder/current/bin/runagentsetup.shSelect “Modify Connector”
Next “Add, modify, or remove destinations”
Next, select the source of the destination for filtering.
Select "Modify destination settings"
In this menu all settings of the connector are set. In our case, point 10 is required - Filters
Assigning filtering: in my case, discard all events that are NOT EQUAL to the value in the deviceVendor field.
Thus, we can send a diverse stream of events to the Forwarder Connector and scatter events to the destination source we need.
Setting coding of paged events
To correctly display events containing Russian characters, you need to set additional parameters in the agent.wrapper.conf file on the Forwarder Connector.
We enter the following lines (make no mistake with the ordinal numbering of wrapper.java.additional)
vi /opt/arcsight/forwarder/current/user/agent/agent.wrapper.confWe enter the following lines (make no mistake with the ordinal numbering of wrapper.java.additional)
wrapper.java.additional.10=-Dfile.encoding=UTF8
wrapper.java.additional.11=-Duser.language=ru
wrapper.java.additional.12=-Duser.region=RUAs a result, we can observe that ArcSight has a lot of integration capabilities, both with its systems and with external sources. But for the sake of truth, it is worth saying that the flow of events is gradually increasing, the same events must be sent to several sources, there must be a constant interaction of various systems ... and here the capabilities of a regular connector end.
Therefore, Micro Focus engineers developed a new architecture called the ArcSight Data Platform. A distinctive feature of this architecture is the ArcSight Event Broker product, which serves to route a huge stream of events to various systems (ESM, Logger, UEBA, Investigate, Hadoop, etc.) and is capable of processing more than 500,000 EPS !!!
Source: https://habr.com/ru/post/347642/
All Articles