> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nx1cloud.com/llms.txt
> Use this file to discover all available pages before exploring further.

# NiFi best practices

> Best practices for using NiFi efficiently.

This NiFi best practices page describes several efficient ways to use NiFi.

## Flow design

The following best practices focus on structuring flows so they're easy to maintain
as complexity grows:

* **Minimize redundant processing**: Avoid duplicating tasks by routing data efficiently
  and using shared resources.
* **Modularize with process groups**: Break complex flows into smaller, logical groups to
  improve readability and maintainability.
* **Plan for scalability**: Consider potential growth in data volume and design flows that
  can scale horizontally if needed.
* **Use descriptive names**: Name processors, connections, and process groups clearly to
  reflect their purpose.

## Data handling

The following best practices address how FlowFiles move through a flow:

* **Avoid blocking queues**: Monitor and configure queue thresholds to prevent back-pressure
  from slowing flows unnecessarily.
* **Manage large files carefully**: Use streaming or chunked processing for very large
  FlowFiles to avoid memory issues.
* **Use attributes wisely**: Store metadata in FlowFile attributes to simplify routing
  and transformation.
* **Validate data**: Use schema validation or record processors to catch errors early.

## Processor configuration

The following best practices focus on configuring processors and shared services to
ensure predictable behavior and efficient resource usage:

* **Check required properties**: Ensure all processor properties are properly configured
  before starting.
* **Enable only what's needed**: Disable unused processors or services to reduce
  resource consumption.
* **Monitor dependencies**: Before disabling or modifying a service, check which processors
  rely on it to avoid flow disruptions.
* **Test incrementally**: Build and test flows in small increments to catch issues early.
* **Use controller services**: Centralize shared connections and configuration for
  consistency and easier maintenance.

## Monitoring and troubleshooting

The following best practices help detect, diagnose, and resolve operational
issues during flow execution:

* **Document common issues**: Maintain a knowledge base of known problems and
  solutions for your team.
* **Keep an eye on queue sizes**: Large queues indicate potential bottlenecks or
  resource constraints.
* **Monitor bulletins and logs**: Regularly review warnings, errors, and system messages
  to catch issues early.
* **Use data provenance**: Trace FlowFiles to understand data paths and transformations.

## Performance optimization

The following best practices focus on tuning scheduling, concurrency, and
logging to maintain throughput under load:

* **Adjust scheduling and concurrency**: Tune processor scheduling intervals and
  concurrent tasks based on data volume and system capacity.
* **Avoid excessive logging**: Logging at the `DEBUG` level in production can impact
  performance; use it primarily for troubleshooting.
* **Balance flows**: Distribute processing tasks evenly across processors to
  prevent hotspots.

## Security and governance

The following best practices address access control, credential handling,
and secure communication within and across NiFi flows:

* **Manage access control**: Use NiFi's policies and, if applicable, OpenID Connect (OIDC)
  authentication to control who can view or modify flows.
* **Protect sensitive information**: Use NiFi's sensitive property handling to secure
  passwords, keys, and tokens.
* **Scope appropriately**: Use process group-level services for local, isolated configurations.
  Use global services for common resources.
* **Use secure connections**: Enable TLS/SSL for HTTP and data transfers.

## Additional resources

* To get an overview of NiFi, refer to the [NiFi in NexusOne](./nifi-in-nx1) page.
* To learn practical ways to use NiFi in the NexusOne environment, refer to the
  [NiFi hands-on examples](./nifi-hands-on-examples) page.
* For more details about NiFi, refer to the
  [NiFi official documentation](https://nifi.apache.org/docs/nifi-docs/).