[{"data":1,"prerenderedAt":749},["ShallowReactive",2],{"blog-post-\u002Fblog\u002Fflink-vs-kafka-streams":3},{"id":4,"title":5,"body":6,"date":735,"description":736,"extension":737,"meta":738,"navigation":739,"ogImage":740,"ogImageHeight":740,"ogImageWidth":740,"path":741,"pinned":742,"seo":743,"sitemap":744,"stem":745,"tags":746,"__hash__":748},"blog\u002Fblog\u002Fflink-vs-kafka-streams.md","Flink vs Kafka Streams 2026: Which Should You Use?",{"type":7,"value":8,"toc":709},"minimark",[9,27,37,50,55,145,149,155,296,305,310,317,320,323,327,335,345,348,358,362,446,449,456,463,466,470,473,484,491,505,533,537,540,550,559,563,576,580,593,601,607,616,620,623,635,643,650,654,658,661,665,668,672,675,679,682,686,689,693,696,700,703,706],[10,11,12,13,20,21,26],"p",{},"Kafka stores and moves events; ",[14,15,19],"a",{"href":16,"rel":17},"https:\u002F\u002Fflink.apache.org\u002F",[18],"nofollow","Apache Flink"," and ",[14,22,25],{"href":23,"rel":24},"https:\u002F\u002Fkafka.apache.org\u002F43\u002Fstreams\u002Fcore-concepts\u002F",[18],"Kafka Streams"," process them. Both support stateful aggregations, event-time windows, joins, and exactly-once output to Kafka. The main architectural differences are where processing runs and which systems it connects to.",[10,28,29,32,33,36],{},[30,31,25],"strong",{}," is a Java library that runs inside your application and reads and writes topics in a single Kafka cluster. ",[30,34,35],{},"Flink"," is a separate runtime. Its jobs run on your own cluster or a managed service and can connect to Kafka and many other systems.",[10,38,39,40,44,45,49],{},"Throughout, the examples use five-minute order totals per customer from an ",[41,42,43],"code",{},"orders"," topic to show how the two handle state recovery, late-arriving events, and exactly-once output. For Kafka Streams setup in Spring Boot or Quarkus, see ",[14,46,48],{"href":47},"\u002Fblog\u002Fkafka-streams-in-java","Kafka Streams in Java",".",[51,52,54],"h2",{"id":53},"quick-decision","Quick decision",[56,57,58,74],"table",{},[59,60,61],"thead",{},[62,63,64,68,71],"tr",{},[65,66,67],"th",{},"Situation",[65,69,70],{},"Best starting point",[65,72,73],{},"Main caveat",[75,76,77,90,102,115,132],"tbody",{},[62,78,79,85,87],{},[80,81,82],"td",{},[30,83,84],{},"JVM service whose inputs and outputs are all Kafka topics",[80,86,25],{},[80,88,89],{},"Parallelism depends on topic partition counts; all topics must be in one cluster.",[62,91,92,97,99],{},[80,93,94],{},[30,95,96],{},"Databases, files, or CDC streams as sources or sinks",[80,98,35],{},[80,100,101],{},"You run a Flink cluster or pay for a managed one.",[62,103,104,109,112],{},[80,105,106],{},[30,107,108],{},"Team that wants to write pipelines in SQL",[80,110,111],{},"Flink SQL",[80,113,114],{},"Joins and aggregations without a time bound can grow state indefinitely. A state TTL caps it, but expiring old state can change results.",[62,116,117,122,125],{},[80,118,119],{},[30,120,121],{},"Python team",[80,123,124],{},"PyFlink",[80,126,127,128,49],{},"Still runs on the JVM Flink runtime. For a library without a cluster, see ",[14,129,131],{"href":130},"\u002Fblog\u002Fkafka-streams-in-python","Kafka Streams in Python",[62,133,134,139,142],{},[80,135,136],{},[30,137,138],{},"Already on Confluent Cloud, wanting managed SQL",[80,140,141],{},"Confluent Cloud for Apache Flink",[80,143,144],{},"ksqlDB stays supported for existing queries, but Confluent recommends Flink for new ones.",[51,146,148],{"id":147},"flink-vs-kafka-streams-compared","Flink vs Kafka Streams compared",[10,150,151,154],{},[30,152,153],{},"Capabilities verified against each release's documentation and source:"," 20 Oct 2026",[56,156,157,184],{},[59,158,159],{},[62,160,161,164,171],{},[65,162,163],{},"Area",[65,165,166],{},[14,167,170],{"href":168,"rel":169},"https:\u002F\u002Fgithub.com\u002Fapache\u002Fkafka\u002Ftree\u002F4.3.1",[18],"Kafka Streams 4.3.1",[65,172,173,178,179],{},[14,174,177],{"href":175,"rel":176},"https:\u002F\u002Fgithub.com\u002Fapache\u002Fflink\u002Ftree\u002Frelease-2.2.1",[18],"Flink 2.2.1"," with ",[14,180,183],{"href":181,"rel":182},"https:\u002F\u002Fgithub.com\u002Fapache\u002Fflink-connector-kafka\u002Ftree\u002Fv5.0.0",[18],"Kafka connector 5.0.0",[75,185,186,199,212,225,238,251,264,283],{},[62,187,188,193,196],{},[80,189,190],{},[30,191,192],{},"Runs as",[80,194,195],{},"Library inside your JVM application",[80,197,198],{},"Jobs on a Flink cluster, self-managed or managed",[62,200,201,206,209],{},[80,202,203],{},[30,204,205],{},"Sources and sinks",[80,207,208],{},"Kafka topics in one cluster",[80,210,211],{},"Kafka in any number of clusters, files, JDBC, CDC, and other connectors",[62,213,214,219,222],{},[80,215,216],{},[30,217,218],{},"APIs",[80,220,221],{},"DSL and Processor API in Java",[80,223,224],{},"DataStream API in Java, Table API, and SQL; Python through PyFlink",[62,226,227,232,235],{},[80,228,229],{},[30,230,231],{},"State",[80,233,234],{},"Local RocksDB or in-memory stores, backed by changelog topics",[80,236,237],{},"Heap or RocksDB state backends, snapshotted to checkpoint storage such as S3",[62,239,240,245,248],{},[80,241,242],{},[30,243,244],{},"Recovery",[80,246,247],{},"Replays changelog topics into the stores",[80,249,250],{},"Restores state from the latest checkpoint and rewinds Kafka to the checkpointed offsets",[62,252,253,258,261],{},[80,254,255],{},[30,256,257],{},"Scaling",[80,259,260],{},"More instances, limited by partition counts",[80,262,263],{},"Parallelism set per operator, rescaled from a checkpoint or savepoint",[62,265,266,271,277],{},[80,267,268],{},[30,269,270],{},"Exactly-once",[80,272,273,274],{},"Kafka to Kafka in one cluster, with ",[41,275,276],{},"exactly_once_v2",[80,278,279,280],{},"Checkpoint-based; end to end with transactional sinks such as ",[41,281,282],{},"KafkaSink",[62,284,285,290,293],{},[80,286,287],{},[30,288,289],{},"Managed options",[80,291,292],{},"None; you deploy the application",[80,294,295],{},"Confluent Cloud for Apache Flink, Amazon Managed Service for Apache Flink, and others",[10,297,298,299,304],{},"Flink 2.3.0 shipped on 25 Jun 2026, but this comparison uses 2.2.1 because Kafka connector 5.0.0 ",[14,300,303],{"href":301,"rel":302},"https:\u002F\u002Fflink.apache.org\u002Fdownloads\u002F",[18],"officially supports"," Flink 2.1.x and 2.2.x, not 2.3.x.",[306,307,309],"h3",{"id":308},"deployment-and-operations","Deployment and operations",[10,311,312,313,316],{},"Kafka Streams runs inside your own Java service. To scale, you start more copies with the same ",[41,314,315],{},"application.id",", and they split the partitions between them. State is backed up to Kafka topics, so you don't need any infrastructure outside Kafka.",[10,318,319],{},"Flink runs as a separate cluster. A JobManager coordinates jobs and TaskManagers run them, often on Kubernetes through the Flink Kubernetes Operator. The cluster also needs storage such as S3 for checkpoints, and upgrades usually go through a savepoint.",[10,321,322],{},"Managed services such as Confluent Cloud for Apache Flink and Amazon Managed Service for Apache Flink run the cluster for you. Check which Flink version and connectors a service supports before you rely on them.",[306,324,326],{"id":325},"state-recovery-and-scaling","State, recovery, and scaling",[10,328,329,330,334],{},"Kafka Streams keeps each customer's running total in a local RocksDB store, backed up to a changelog topic. When an instance stops, other instances replay that topic to rebuild its state. ",[14,331,333],{"href":332},"\u002Fblog\u002Fkafka-streams-in-java#restoration-standby-replicas-and-scaling","Standby replicas"," keep ready copies on other instances to speed this up.",[10,336,337,338,340,341,344],{},"Without repartitioning, 12 ",[41,339,43],{}," partitions allow at most 12 aggregation tasks. Adding partitions later can separate new orders from stored totals, so scale further with ",[41,342,343],{},"repartition()"," into a topic with more partitions, which adds a hop through Kafka.",[10,346,347],{},"Flink keeps state in memory or in RocksDB, and regularly saves all of it, with the Kafka offsets, as a checkpoint. After a failure, the job restarts from the last checkpoint. A savepoint is a snapshot you trigger yourself, used for upgrades and rescaling.",[10,349,350,351,354,355,357],{},"After ",[41,352,353],{},"keyBy",", Flink operators can run more parallel tasks than ",[41,356,43],{}," has partitions, up to their max parallelism. Choose that value before going live, because changing it later makes existing checkpoints and savepoints incompatible.",[306,359,361],{"id":360},"event-time-windows-and-joins","Event time, windows, and joins",[56,363,364,378],{},[59,365,366],{},[62,367,368,371,374,376],{},[65,369,370],{},"Window type",[65,372,373],{},"Example",[65,375,25],{},[65,377,35],{},[75,379,380,395,415,430],{},[62,381,382,387,390,393],{},[80,383,384],{},[30,385,386],{},"Fixed, no overlap",[80,388,389],{},"10:00–10:05, 10:05–10:10",[80,391,392],{},"Tumbling",[80,394,392],{},[62,396,397,402,405,408],{},[80,398,399],{},[30,400,401],{},"Fixed size, overlapping",[80,403,404],{},"The last 5 minutes, updated every minute",[80,406,407],{},"Hopping",[80,409,410,411,414],{},"Sliding (",[41,412,413],{},"HOP"," in SQL)",[62,416,417,422,425,428],{},[80,418,419],{},[30,420,421],{},"Ends after inactivity",[80,423,424],{},"A customer's orders until 30 minutes pass with none",[80,426,427],{},"Session",[80,429,427],{},[62,431,432,437,440,443],{},[80,433,434],{},[30,435,436],{},"Based on time between records",[80,438,439],{},"Orders within 5 minutes of each other",[80,441,442],{},"Sliding",[80,444,445],{},"No direct equivalent",[10,447,448],{},"To keep the comparison simple, the order totals here use fixed windows. Orders are placed by event time, the time on the order, not the wall clock. Kafka Streams closes a window once event time passes its end plus a grace period. Later orders for it are dropped.",[10,450,451,452,455],{},"Flink uses a watermark, roughly the newest order time minus an allowance, and emits a window's total when the watermark passes its end. Kafka Streams instead emits updates while the window is open. ",[41,453,454],{},"allowedLateness"," lets late orders still update a Flink window.",[10,457,458,459,462],{},"Both need new orders to move event time, so if orders stop, the last window stays open. In Flink, an idle partition, or an idle source reader when parallelism exceeds the partition count, stalls all windows. ",[41,460,461],{},"withIdleness"," lets active partitions advance event time.",[10,464,465],{},"Kafka Streams joins usually need both topics partitioned the same way, with the same partition count and partitioner. GlobalKTable and foreign-key joins are the exceptions. Flink reshuffles records itself, so it can join any two streams.",[306,467,469],{"id":468},"exactly-once-and-output-latency","Exactly-once and output latency",[10,471,472],{},"Exactly-once means each order is counted once in the output, even if the job fails and reprocesses some input. Both frameworks do this by committing their output in Kafka transactions.",[10,474,475,476,479,480,483],{},"Consumers that read with ",[41,477,478],{},"read_committed"," see results only after the commit, which adds delay on top of any wait for the window. Consumers on the default ",[41,481,482],{},"read_uncommitted"," see results sooner but can also see output from failed attempts.",[10,485,486,487,490],{},"In Kafka Streams, set ",[41,488,489],{},"processing.guarantee=exactly_once_v2",". It then commits every 100 ms by default instead of every 30 seconds, and covers only topics in the same Kafka cluster.",[10,492,493,494,500,501,504],{},"In Flink, enable checkpointing and set ",[14,495,498],{"href":496,"rel":497},"https:\u002F\u002Fnightlies.apache.org\u002Fflink\u002Fflink-docs-release-2.2\u002Fdocs\u002Fconnectors\u002Fdatastream\u002Fkafka\u002F#fault-tolerance",[18],[41,499,282],{}," to ",[41,502,503],{},"EXACTLY_ONCE",". The sink commits once per checkpoint, so latency depends on how often checkpoints run and how long they take. Two sink settings also matter:",[506,507,508,518],"ul",{},[509,510,511,517],"li",{},[30,512,513,516],{},[41,514,515],{},"transactionalIdPrefix",":"," Give each Kafka sink its own prefix and keep it the same across restarts. A changed prefix can leave old transactions open, which holds back new output until they time out.",[509,519,520,525,526,528,529,532],{},[30,521,522,516],{},[41,523,524],{},"transaction.timeout.ms"," ",[41,527,282],{}," defaults to one hour, above the broker's default 15-minute limit (",[41,530,531],{},"transaction.max.timeout.ms","), so the job fails at startup. Raise the broker limit or lower the timeout, keeping it above your longest checkpoint plus recovery.",[51,534,536],{"id":535},"flink-sql-vs-ksqldb","Flink SQL vs ksqlDB",[10,538,539],{},"ksqlDB is a SQL server built on Kafka Streams, under the source-available Confluent Community License. Its persistent queries read and write Kafka topics; external sources and sinks connect through Kafka Connect.",[10,541,542,543,20,546,549],{},"Flink SQL is part of Apache Flink, under the Apache License 2.0. It reads and writes Kafka with the ",[41,544,545],{},"kafka",[41,547,548],{},"upsert-kafka"," connectors. Other connectors cover databases, files, and CDC streams, so one query can join an order stream with a database table.",[10,551,552,553,558],{},"ksqlDB now ships with Confluent Platform, and its standalone Docker image repository is archived. On Confluent Cloud, Confluent ",[14,554,557],{"href":555,"rel":556},"https:\u002F\u002Fdocs.confluent.io\u002Fcloud\u002Fcurrent\u002Fksqldb\u002Foverview.html",[18],"recommends"," Flink for new SQL pipelines and keeps existing ksqlDB applications fully supported.",[51,560,562],{"id":561},"inspect-flink-and-kafka-streams-topics-with-kafma","Inspect Flink and Kafka Streams topics with Kafma",[10,564,565,566,570,571,575],{},"Both frameworks read and write ordinary Kafka topics, so the same checks apply to either. Kafma is a desktop ",[14,567,569],{"href":568},"\u002F","Kafka IDE",". Use its ",[14,572,574],{"href":573},"\u002Ffeatures\u002Fkafka-console","Kafka console"," to send test input, read output topics, and watch how far a job has read.",[306,577,579],{"id":578},"check-the-output-totals","Check the output totals",[10,581,582,583,587,588,592],{},"Send a few orders for one customer with the ",[14,584,586],{"href":585},"\u002Fdocs\u002Fconsole\u002Fproducing-messages","producer panel",", then open the output topic and ",[14,589,591],{"href":590},"\u002Fdocs\u002Fconsole\u002Fconsuming-messages#search-and-inspect-kafka-messages","search"," for that customer ID. Each five-minute total should equal the sum of the orders you sent in that window.",[10,594,595,596,600],{},"Windows close only as newer orders arrive, so use ",[14,597,599],{"href":598},"\u002Fdocs\u002Fconsole\u002Fproducing-messages#generate-kafka-test-messages-on-a-loop","Loop"," to keep orders coming until the final total appears.",[10,602,603,604,606],{},"With exactly-once, Kafma shows totals only after they are committed, as a ",[41,605,478],{}," consumer would. If Flink totals never show up, check that its checkpoints complete.",[10,608,609],{},[610,611],"img",{"alt":612,"height":613,"src":614,"width":615},"Kafma showing the customer-order-totals record with a total amount of 36",822,"https:\u002F\u002Fmedia.kafma.app\u002Fblog\u002Fkafka-streams-in-python\u002Fkafma-customer-order-total.png",1564,[306,617,619],{"id":618},"watch-a-flink-jobs-committed-offsets","Watch a Flink job's committed offsets",[10,621,622],{},"Flink reads Kafka without joining a consumer group the usual way, so its group shows no members. With checkpointing on, it still commits the offsets from each completed checkpoint to that group by default.",[10,624,625,626,630,631,634],{},"After the first commit, open ",[14,627,629],{"href":628},"\u002Fdocs\u002Fconsole\u002Fconsuming-messages#live-and-watch-group-modes","Watch Group"," on the job's ",[41,632,633],{},"group.id"," to follow its progress as of the last checkpoint, which can trail what it has actually read. These offsets are for monitoring only. After a failure, Flink resumes from its checkpoint, not from Kafka.",[10,636,637],{},[610,638],{"alt":639,"height":640,"src":641,"width":642},"Kafma watching pending and consumed records around a consumer group's committed offsets",1368,"https:\u002F\u002Fmedia.kafma.app\u002Fdocs\u002Fconsole\u002Fconsuming-messages\u002Fwatch-group.png",1562,[10,644,645,649],{},[14,646,648],{"href":647},"\u002Fdownload","Download Kafma"," and connect to the cluster your jobs use.",[51,651,653],{"id":652},"frequently-asked-questions","Frequently asked questions",[306,655,657],{"id":656},"what-is-the-difference-between-kafka-and-flink","What is the difference between Kafka and Flink?",[10,659,660],{},"Kafka stores events durably in topics and delivers them to consumers. Flink is a processing engine that reads those events, computes results such as windowed totals, and writes them to Kafka or other systems.",[306,662,664],{"id":663},"can-flink-replace-kafka-streams","Can Flink replace Kafka Streams?",[10,666,667],{},"For most workloads, yes: Flink covers Kafka Streams' windows, joins, and exactly-once Kafka output. You move the processing logic out of your service into a Flink job. Kafka Streams state does not carry over, so plan to rebuild it by reprocessing the input.",[306,669,671],{"id":670},"is-ksqldb-deprecated","Is ksqlDB deprecated?",[10,673,674],{},"There is no official deprecation. On Confluent Cloud, Confluent recommends Flink for new stream processing workloads and keeps ksqlDB fully supported for existing applications. On Confluent Platform, ksqlDB still ships as a component.",[306,676,678],{"id":677},"can-kafka-streams-read-from-non-kafka-sources","Can Kafka Streams read from non-Kafka sources?",[10,680,681],{},"No. Kafka Streams reads and writes topics in one Kafka cluster. To include a database or files, bring the data into Kafka first, for example with a Kafka Connect source connector, and write results out with a sink connector.",[306,683,685],{"id":684},"does-flink-need-kafka","Does Flink need Kafka?",[10,687,688],{},"No. Flink reads files, databases, CDC streams, and other message systems through its connectors.",[306,690,692],{"id":691},"which-is-faster-flink-or-kafka-streams","Which is faster, Flink or Kafka Streams?",[10,694,695],{},"Neither is always faster. Compare throughput and end-to-end latency on your workload with the same processing guarantees. For exactly-once Kafka output, commit and checkpoint settings also affect when results become visible.",[51,697,699],{"id":698},"conclusion","Conclusion",[10,701,702],{},"Choose Kafka Streams when processing belongs in a JVM service and all its topics are in one cluster. Choose Flink for other sources and sinks, SQL, Python, or scaling processing after the source without adding partitions, if you can run or pay for a Flink cluster.",[10,704,705],{},"Either way, decide on the settings that are hard to change after launch: input partition counts for Kafka Streams and max parallelism for Flink. Then test with known input and check output totals and committed offsets before deploying.",[10,707,708],{},"This guide is maintained by the team behind Kafma.",{"title":710,"searchDepth":711,"depth":711,"links":712},"",3,[713,715,721,722,726,734],{"id":53,"depth":714,"text":54},2,{"id":147,"depth":714,"text":148,"children":716},[717,718,719,720],{"id":308,"depth":711,"text":309},{"id":325,"depth":711,"text":326},{"id":360,"depth":711,"text":361},{"id":468,"depth":711,"text":469},{"id":535,"depth":714,"text":536},{"id":561,"depth":714,"text":562,"children":723},[724,725],{"id":578,"depth":711,"text":579},{"id":618,"depth":711,"text":619},{"id":652,"depth":714,"text":653,"children":727},[728,729,730,731,732,733],{"id":656,"depth":711,"text":657},{"id":663,"depth":711,"text":664},{"id":670,"depth":711,"text":671},{"id":677,"depth":711,"text":678},{"id":684,"depth":711,"text":685},{"id":691,"depth":711,"text":692},{"id":698,"depth":714,"text":699},"2026-10-20","Flink vs Kafka Streams: a library inside your app or a separate runtime. Compare deployment, state, scaling, event time, exactly-once, and ksqlDB.","md",{},true,null,"\u002Fblog\u002Fflink-vs-kafka-streams",false,{"title":5,"description":736},{"loc":741},"blog\u002Fflink-vs-kafka-streams",[747,19,25],"Kafka Stream Processing","0gOENMLBxwxh3Ebe9Td_WwppnFiI-awwMAtrhOUKPDs",1792479165728]