GroupingPartitionRouter
A decorator on top of a partition router that groups partitions into batches of a specified size. This is useful for APIs that support filtering by multiple partition keys in a single request. Note that per-partition incremental syncs may not work as expected because the grouping of partitions might change between syncs, potentially leading to inconsistent state tracking.
- type:
object - required:
["type","group_size","underlying_partition_router"]
type
- Required: Yes
- type:
string - enum:
["GroupingPartitionRouter"]
group_size
The number of partitions to include in each group. This determines how many partition values are batched together in a single slice.
- Required: Yes
- type:
integer
Examples:
10
50
underlying_partition_router
The partition router whose output will be grouped. This can be any valid partition router component.
-
Required: Yes
-
anyOf:
- Allowed type:
ListPartitionRouter
- Allowed type:
-
anyOf:
- Allowed type:
SubstreamPartitionRouter
- Allowed type:
-
anyOf:
- Allowed type:
UnionPartitionRouter
- Allowed type:
-
anyOf:
- Allowed type:
CustomPartitionRouter
- Allowed type:
deduplicate
If true, ensures that partitions are unique within each group by removing duplicates based on the partition key.
- Required: No
- type:
boolean - default:
true
$parameters
- Required: No
- type:
object