flyte.extras

Flyte extras package. This package provides various utilities that make it possible to build highly customized workflows.

ContainerTask: Execute arbitrary pre-containerized applications, without needing the flyte-sdk to be installed. This extra uses flyte copilot system to inject inputs and slurp outputs from the container run.
DynamicBatcher / TokenBatcher: Maximize resource utilization by batching work from many concurrent producers through a single async processing function. DynamicBatcher is the general-purpose base; TokenBatcher is a convenience subclass for token-budgeted LLM inference with reusable containers.

Directory

Class	Description
`BatchStats`	Monitoring statistics exposed by :attr:`DynamicBatcher.
`ContainerTask`	This is an intermediate class that represents Flyte Tasks that run a container at execution time.
`DynamicBatcher`	Batches records from many concurrent producers and runs them through.
`Prompt`	Simple prompt record with built-in token estimation.
`TokenBatcher`	Token-aware batcher for LLM inference workloads.

Protocol	Description
`CostEstimator`	Protocol for records that can estimate their own processing cost.
`TokenEstimator`	Protocol for records that can estimate their own token count.