How to Dismiss Empty Records In a Collection By Key Field

When working with a buffer-based connector, you may receive a collection of elements where some records are fully populated, while others contain empty, null, or zero values. With our new feature, you can now define when to exclude unwanted records from your collection automatically.

How It Works

Simply choose a key field and specify which values should trigger record dismissal. You can select one or more of the following options:

• dismiss-record-on-null – Removes records where the key field is null.
• dismiss-record-on-emptystring – Removes records where the key field is an empty string ("").
• dismiss-record-on-spaces – Removes records where the key field contains only spaces.
• dismiss-record-on-zeros – Removes records where the key field contains only zeros.

You can apply multiple dismissal rules at once. For example, if you choose to dismiss records where a string field is empty, contains spaces, or is null, any record matching these criteria will be removed.

Example

Input Data

If you set "name" as the key field and apply the options empty string, spaces, and null, the following dataset:

{
    "people": [
        { "name": "Danny", "age": 25 },
        { "name": "", "age": 0 },
        { "name": " ", "age": 0 },
        { "name": null, "age": 0 }
    ]
}

The result is:

{
    "people": [
        { "name": "Danny", "age": 25 }
    ]
}

With this feature, you have full control over your data, ensuring cleaner and more relevant collections without unnecessary manual filtering.

How to configure this in the Hub

To enable this feature in the HUB, follow these steps:

1. Navigate to the module scope and go to relevant Asset.
2. Go to the Asset Output.
3. On the list’s object level, set the following additional attributes.
   • You must specify the key field name and one or more of the dismiss-record-on properties.

Required Properties

Example

In the attached images, we set the key field as itemName, and enabled the dismiss-record-on-null option.
This means any object where itemName is null will be automatically dismissed from the collection.

With this configuration, you can efficiently clean up your data while ensuring only relevant records are retained. 🚀