Monitor Tomcat with Splunk in RWS Development

Projects: c2platform/rws/ansible-gis, c2platform.core, c2platform.mw

Overview

This guide details setting up a Tomcat node on Ubuntu 22 with Jolokia for Java monitoring using Splunk. The setup is akin to the gsd-tomcat1 and gsd-tomcat2 nodes, as described in Tomcat SSL/TLS and Java Keystore and TrustStore Configuration for Linux and Windows Hosts , with the key distinction that this node is dedicated to monitoring.

Prerequisites

• Setup the RWS Development Environment on Ubuntu 22: Install Ansible, Vagrant, LXD, Virtualbox and clone the project directory.
• Setting up Splunk Enterprise in RWS Development Environment: This guide aids the GIS Platform team in integrating and experimenting with Splunk Enterprise monitoring within the RWS development environment.

Ensure both the reverse proxy node and the Splunk node are operating in the RWS development environment with:

vagrant up gsd-rproxy1 gsd-splunk

Setup

To start the Tomcat node gsd-tomcat3, run the command:

vagrant up gsd-tomcat3

Verify

Open Firefox with your profile (see Configure a FireFox Profile ) and visit the URLs to check that the “Hello World” application runs without SSL/TLS certificate errors:

1. Hello World
2. Jolokia

The second link should display the Jolokia output, which should match the jolokia.json file.

Setup Monitoring

Once the gsd-tomcat3 node is running, configure it to send logs to Splunk. The Tomcat logs are found at /opt/tomcat/tomcat/logs/.

After the Splunk node gsd-splunk is ready, access the Splunk web interface at Splunk Web Interface.

Configure Tomcat Logging**

• Ensure the Tomcat logs are generating properly in the /opt/tomcat/tomcat/logs/ directory.
• Edit the logging configuration in Tomcat to ensure logs are in the desired format.

Install Forwarder**

On the gsd-tomcat3 node, install the Splunk Universal Forwarder.

ansible-playbook plays/mgmt/splunk_uf.yml -i hosts.ini --limit gsd-tomcat3

ansible-playbook plays/mgmt/splunk_uf.yml -i hosts.ini --limit gsd-tomcat2

The node gsd-tomcat3 is part of the Ansible group splunk_universalforwarder

Configure Forwarder**

• Configure the forwarder to watch the Tomcat log directory.
• Ensure the forwarder is pointing to your Splunk server address and port.

Set up the Splunk Add-on for Tomcat - Splunk Documentation https://docs.splunk.com/Documentation/AddOns/released/Tomcat/Setup

monitorRole secret

Install Tomcat https://splunk.c2platform.org/en-US/app/Splunk_TA_tomcat/configuration

Verify Data Reception**

• Once configured, verify if logs are being sent to the Splunk index.
• Check the splunkd.log for any errors or issues.

Create Dashboards/Alerts**

• Use the Splunk Web Interface to create dashboards to visualize the log data.
• Set up alerts for any critical metrics you wish to monitor.

Additional Information

For further details and guidance, refer to the provided resources and documentation.