This post is an attempt at sharing what I learnt the hard way while writing my firsts high throughput Numaflow pipelines. It took me some time to convince myself that I had a resource efficient and safe pipeline, and I had to troubleshoot a few misconfiguration errors I made.
5 tips for high throughput workloads in Numaflow
· 10 min read
A software and devops engineer, with a passion for data and audio.