Guides

Tracking Data Changes with Pennylane API

Suggest Edits

Changelog Endpoints

Changelog endpoints allow you to track changes (creation, updates, deletion) on specific resources in near real-time. They are particularly useful for maintaining data synchronization between your system and Pennylane.

Available Changelog Endpoints

Changes are available for the following resources:

  • Customer Invoices (/api/external/v2/changelogs/customer_invoices)
  • Supplier Invoices (/api/external/v2/changelogs/supplier_invoices)
  • Products (/api/external/v2/changelogs/products)
  • Customers (/api/external/v2/changelogs/customers)
  • Suppliers (/api/external/v2/changelogs/suppliers)

When to Use Changelog Endpoints

Use changelog endpoints when you need to:

  • Keep your local copy of Pennylane data up to date
  • React to specific changes in resources (e.g., when an invoice is created)
  • Build an audit trail of changes
  • Implement incremental synchronization instead of fetching all resources repeatedly

How It Works

  1. Initial Sync:

    • Without a start_date parameter, the endpoint returns changes starting from 4 weeks ago
    • You can specify a more recent start_date if you only need recent changes
    • Note that you cannot request changes older than 4 weeks

  2. Processing Changes:

    • Changes are returned in chronological order (oldest first)
    • Each change contains the resource ID and the type of operation
    • Use the resource ID to fetch complete data from the corresponding endpoint

  3. Pagination:

    • Results are paginated with a default limit of 20 items (max 1000)
    • Use the next_cursor to fetch subsequent pages when has_more is true
    • The cursor is only valid for the current request session
    • You can read more about how to use cursor-based pagination

Important Notes

  • Changes are retained for 4 weeks
  • Results are paginated with a default limit of 20 items (max 1000)
  • The operation field indicates the type of change:
    • insert: Resource created
    • update: Resource modified
    • delete: Resource deleted
  • You cannot use both start_date and cursor in the same request

Best Practices

  1. Regular Polling:

    • Poll the changelog endpoint at regular intervals (e.g., every few hours)
    • Store the processed_at of the last change you've processed
    • On next poll, use this timestamp as start_date to get only new changes

  2. Processing Changes:

    • Process changes in chronological order
    • Use pagination (via cursor) only to navigate through a large result set
    • Once all changes are processed for a request, start a new request with an updated start_date

  3. Resource Fetching:
    When you receive changes from the changelog endpoints, you'll need to fetch the complete resource data. Here's how to do it efficiently:

    1. Using Resource IDs:

      • Each change event includes a resource id
      • Use these IDs to fetch the current state of resources from their respective endpoints
      • For deleted resources (operation: "delete"), the resource will no longer be available

    2. Batch Fetching:

      • Instead of fetching each resource individually, collect multiple IDs
      • Use the filter parameter with the in operator on resource endpoints
      • Example filter: [{"field": "id", "operator": "in", "value": [123, 456, 789]}]
      • This reduces the number of API calls and improves performance

    3. Available Resource Endpoints:

      • Customer Invoices: /api/external/v2/customer_invoices
      • Supplier Invoices: /api/external/v2/supplier_invoices
      • Products: /api/external/v2/products
      • Customers: /api/external/v2/customers
      • Suppliers: /api/external/v2/suppliers

    4. Handling Missing Resources:

      • Some resources might have been deleted since the change was recorded
      • Implement proper error handling for these cases

Updated 1 day ago