Skip to main content
The Debezium server reads Change Data Captures (CDC) that happen within a database and streams the events to a destination. This best practices page contains guidelines for building reliable and performant CDC pipelines with appropriate table filtering, snapshot strategies, schema management, and monitoring.

​
Table filtering

This is a recommended approach used to minimize overhead and improve performance by capturing only the tables you need. Ensure you apply the following best practices:
  • Use schema and table inclusion patterns during portal configuration.
  • Exclude audit, temporary, or staging tables. For example: table.include.list=schema1.orders,schema1.customers.
  • Avoid overly broad wildcards that capture unnecessary tables.
  • Review and refine table filters as your data model evolves.

​
Source database optimization

Optimize your source database configuration for CDC performance by applying the following best practices:
  • Connection management: Ensure adequate connection pool sizes on the database side.
  • Transaction log retention: Configure appropriate retention periods for CDC logs. Write-Ahead Logging (WAL) for PostgreSQL, and binary logs for MySQL.
  • Database performance: Monitor source database load during initial snapshot and adjust maintenance windows accordingly.
  • Network bandwidth: Ensure sufficient bandwidth between the source database and NexusOne environment.

​
Schema change management

Though NexusOne manages schema evolution, you can prepare for schema changes by applying the following best practices:
  • Test schema changes: Always test schema modifications in a non-production environment first.
  • Coordinate changes: Plan schema changes with awareness of the current allow-field-addition=false default
  • Monitor for incompatible changes: Be aware that certain data type changes may require pipeline reconfiguration. For example, from a STRING to an INTEGER.

​
Initial snapshot considerations

NexusOne uses the initial snapshot mode by default. To ensure smooth initial loads:
  • Schedule during off-peak hours: Run initial deployments when the source database load is low.
  • Large table handling: For very large tables, such as a multi-terabyte, discuss incremental snapshot options with the NexusOne support team.
  • Monitor progress: Use the NexusOne portal to track snapshot completion status.
  • Network stability: Ensure stable network connectivity during the initial snapshot phase.

​
Additional resources

  • To get an overview of the Debezium server, refer to the Debezium in NexusOne page.
  • To learn how to troubleshoot common Debezium server issues in the NexusOne environment, refer to the Troubleshoot common Debezium server issues page.
  • If you are using the NexusOne portal and want to learn how to use the Debezium server, refer to the Ingest page.