Categories: Image

Flux.2-Klein pipeline for real-time webcam stream processing in 30 FPS

I have built a pipeline based on the Flux.2-Klein-4B model that allows processing of a video stream with low latency (about 0.2 seconds) on a single RTX5090 GPU.
It is free and open-source, you can try it locally:
https://github.com/tensorforger/FluxRT

Under the hood, it uses a custom spatial-aware KV-cache, so it only recomputes a small number of image tokens per frame, specifically where something is moving or changing.
It also uses frame interpolation with the RIFE model, which can multiply FPS by a factor of 2, 4, 8, etc. I have found that 4 is the most appropriate for my setup.

Depending on scene dynamics, the output stream achieves up to 50 FPS in mostly static scenes and around 20 FPS when the entire input image is changing rapidly. Benchmark results are in the repo.

There is also a Gradio demo, several minimal cv2 examples, and a simple paint-style app with real-time canvas updates.

submitted by /u/TensorForger
[link] [comments]

AI Generated Robotic Content

Share
Published by
AI Generated Robotic Content
Tags: ai images

Recent Posts

The Roadmap to Mastering AI Agent Evaluation

Let's not waste any more time.

11 hours ago

SpaceX wants to build AI data centers in space. Will it work?

The race to build data centers in space is gaining momentum as AI drives unprecedented…

11 hours ago

Monitor and debug generative AI inference with SageMaker detailed metrics and Insights dashboard on CloudWatch

Monitoring and troubleshooting generative AI inference endpoints operating at scale is challenging. When your large…

1 day ago

Amazon Bedrock AgentCore harness is now generally available: Go from idea to production-grade agent in minutes

A year ago, Simon Willison wrote one of the cleanest definitions of an agent that…

1 day ago

How growing UK midsize businesses are building in the AI era

The UK’s 5-million-plus small and midsize businesses and enterprises (SMBs) are the backbone of our…

2 days ago

Amazon SageMaker AI Async Inference now supports inline request payloads

Today, we’re announcing inline payload support for Amazon SageMaker AI Async Inference. Customers can now…

2 days ago