Don’t treat Camunda as a system of record

Using Camunda as a system of record can lead to several issues, such as a bloated data store that causes performance problems. There’s also the risk of storing personally identifiable information (PII) or other sensitive data in systems that don’t require it. To avoid these challenges, keep variable data to a minimum, only storing what’s necessary for the process.

For critical information, use a separate data store and reference it in the process using an ID, rather than storing it directly in Camunda. Ultimately, the data you store in Camunda should be strictly relevant to the process flow itself, helping to maintain both efficiency and security.

One way to keep variable data small is to make use of the Result Expression when using a connector. Most connectors will offer an Output Mapping to store the result of the connector. I commonly see users storing the entire result instead of making use of the Result Expression to store only the data they need.

For example, if you use the REST connector, you may get a result containing the status code of the response, some headers, and then some additional data from the endpoint. By using the Result Expression, you can map just the data you need to variables.

Prevent running into backpressure

Backpressure is a crucial mechanism in Zeebe that helps maintain system stability when processing slows down. It kicks in when the broker experiences high latency, preventing new events from being accepted until the system can catch up and the processing speeds return to normal. This ensures that the broker doesn’t become overwhelmed and continues to function efficiently.

To avoid hitting backpressure, there are several proactive steps you can take.

First, conduct thorough load testing to simulate real-world traffic and identify potential bottlenecks. The Camunda community provides two GitHub projects to help you perform load tests: the benchmarking toolset and the process automator.

Check out this in-depth blog post about benchmarking in Camunda.

Next, review your hardware specs to ensure they meet the performance requirements for your use case—insufficient resources can cause delays in processing. There are two common pitfalls when choosing your hardware, both relating to the hard drives attached to Zeebe brokers: be sure that your drives have a consistent minimum of 1000 IOPS and are not NFS. Slower drives, and the latency incurred by NFS, will cause your Zeebe brokers to perform inefficiently.

Make your system observable

Observability refers to the ability to track and analyze your system’s performance in real time. This gives you insight into how well your application is functioning and allows you to catch issues before they become significant problems.

Despite its importance, I often see people delay setting up observability until it’s too late. Being proactive about monitoring your application’s health can save a lot of time and effort in the long run by helping you identify and resolve issues early.

To help you get started monitoring your Camunda platform, Camunda comes out of the box with support for Prometheus and OpenTelemetry. You can review the metrics we provide in our documentation. We also provide dashboards for Grafana to make visualizing these metrics quick and easy.

In addition to monitoring the metrics provided by our application, be sure that you can collect and monitor application logs. These are critical for determining the root cause of issues that may arise. If accessing raw logs is an issue in your team, consider leveraging cloud vendor solutions like AWS CloudWatch or Azure Monitor.

Set up data retention early

Databases have limits, and as they grow, they can start to slow down. Storing unnecessary or outdated data causes your system to become bloated, which will negatively impact query performance and overall responsiveness. That’s where data retention policies come in.

By defining what data should be kept and what should be discarded, you ensure that your database stays lean and efficient, preventing performance issues as your system scales.

Don’t wait until your database becomes overwhelming to start cleaning it up. Make data retention a key part of your system planning from the beginning, so you can keep things running smoothly as your data grows.

Check out our documentation for more information on setting data retention policies for all the Camunda components:

• Zeebe Exporter
• Operate
• Tasklist
• Optimize

While there is always more to learn as you get deeper into Camunda 8, avoiding these common pitfalls will help you get off to a strong start. And if you don’t want to deal with the hassle of hosting, sizing, and monitoring your platform, we also offer a SaaS option.

For anyone looking for a hand, of course, be sure to check out the docs, our forum, or contact your Camunda support representative directly, and we’ll be happy to help!

The post How to Succeed When Getting Started with Camunda 8 appeared first on Camunda.