[{"data":1,"prerenderedAt":1628},["ShallowReactive",2],{"blog-post-\u002Fblog\u002Fkafka-streams-in-java":3},{"id":4,"title":5,"body":6,"date":1613,"description":1614,"extension":1615,"meta":1616,"navigation":1617,"ogImage":1618,"ogImageHeight":1618,"ogImageWidth":1618,"path":1619,"pinned":1620,"seo":1621,"sitemap":1622,"stem":1623,"tags":1624,"__hash__":1627},"blog\u002Fblog\u002Fkafka-streams-in-java.md","Kafka Streams in Java: Spring Boot and Spring Cloud Stream",{"type":7,"value":8,"toc":1576},"minimark",[9,20,39,47,52,192,196,202,455,458,461,467,472,479,491,496,543,549,553,573,583,587,644,649,653,662,677,693,698,702,763,768,771,791,795,843,848,852,855,974,983,992,1014,1018,1024,1028,1046,1049,1065,1079,1086,1090,1098,1109,1119,1126,1130,1133,1139,1142,1151,1163,1166,1172,1180,1184,1187,1191,1212,1219,1222,1225,1229,1232,1239,1250,1253,1258,1262,1265,1278,1282,1294,1297,1312,1321,1325,1336,1339,1346,1354,1358,1369,1377,1385,1392,1396,1400,1412,1429,1433,1441,1445,1458,1462,1476,1481,1485,1494,1498,1504,1508,1527,1537,1551,1554,1557,1561,1567,1573],[10,11,12,19],"p",{},[13,14,18],"a",{"href":15,"rel":16},"https:\u002F\u002Fkafka.apache.org\u002F43\u002Fstreams\u002Fcore-concepts\u002F",[17],"nofollow","Kafka Streams"," is a Java library that processes Kafka records inside your application, with no separate processing cluster. It handles aggregations, windows, and joins, and keeps their state in local stores that can be recovered from Kafka.",[10,21,22,23,26,27,30,31,34,35,38],{},"Use ",[24,25,18],"strong",{}," directly, or through ",[24,28,29],{},"Spring Boot"," with Spring Kafka, the ",[24,32,33],{},"Spring Cloud Stream"," Kafka Streams binder, or the ",[24,36,37],{},"Quarkus"," extension. The three integrations wrap the same library but differ in configuration, lifecycle, serialization defaults, and error handling.",[10,40,41,42,46],{},"This guide compares these options, then covers Maven dependencies and version alignment, state store recovery, and testing with and without a broker. For plain producers and consumers, see ",[13,43,45],{"href":44},"\u002Fblog\u002Fkafka-java-clients","Kafka Java clients",".",[48,49,51],"h2",{"id":50},"quick-decision","Quick decision",[53,54,55,71],"table",{},[56,57,58],"thead",{},[59,60,61,65,68],"tr",{},[62,63,64],"th",{},"Situation",[62,66,67],{},"Best starting point",[62,69,70],{},"Main caveat",[72,73,74,96,119,143,164,179],"tbody",{},[59,75,76,82,88],{},[77,78,79],"td",{},[24,80,81],{},"Java service without Spring or Quarkus",[77,83,84],{},[13,85,87],{"href":86},"#apache-kafka-streams-in-plain-java","Apache Kafka Streams",[77,89,90,91,95],{},"Your code configures, starts, and closes ",[92,93,94],"code",{},"KafkaStreams"," and decides what happens when a stream thread fails.",[59,97,98,103,112],{},[77,99,100],{},[24,101,102],{},"Spring Boot application",[77,104,105],{},[13,106,108,109],{"href":107},"#kafka-streams-with-spring-boot-and-spring-kafka","Spring Kafka with ",[92,110,111],{},"@EnableKafkaStreams",[77,113,114,115,46],{},"Kafka Streams features follow the version Boot manages; see the ",[13,116,118],{"href":117},"#kafka-streams-maven-dependencies-and-version-alignment","version table",[59,120,121,126,132],{},[77,122,123],{},[24,124,125],{},"Team already using Spring Cloud Stream functions",[77,127,128],{},[13,129,131],{"href":130},"#spring-cloud-stream-kafka-streams-binder","Kafka Streams binder",[77,133,134,135,138,139,142],{},"Adds function bindings and Serde inference, not more Kafka Streams features. ",[92,136,137],{},"sendToDlq"," covers deserialization errors; wrap your own logic in ",[92,140,141],{},"DltAwareProcessor"," to send its failures to a DLT.",[59,144,145,150,158],{},[77,146,147],{},[24,148,149],{},"Quarkus application",[77,151,152],{},[13,153,155],{"href":154},"#quarkus-kafka-streams",[92,156,157],{},"quarkus-kafka-streams",[77,159,160,161,46],{},"Unlike the binder, it does not infer Serdes; health checks need ",[92,162,163],{},"quarkus-smallrye-health",[59,165,166,171,176],{},[77,167,168],{},[24,169,170],{},"Only producing and consuming records",[77,172,173],{},[13,174,175],{"href":44},"A Kafka Java client",[77,177,178],{},"You implement any per-key state, windows, or joins yourself.",[59,180,181,186,189],{},[77,182,183],{},[24,184,185],{},"Separate processing cluster, or many non-Kafka sources and sinks",[77,187,188],{},"Flink or Spark Structured Streaming",[77,190,191],{},"Both add a runtime to deploy and operate.",[48,193,195],{"id":194},"kafka-streams-integrations-compared","Kafka Streams integrations compared",[10,197,198,201],{},[24,199,200],{},"Capabilities verified against each release's source and documentation:"," 19 Oct 2026",[53,203,204,239],{},[56,205,206],{},[59,207,208,211,218,225,232],{},[62,209,210],{},"Area",[62,212,213],{},[13,214,217],{"href":215,"rel":216},"https:\u002F\u002Fgithub.com\u002Fapache\u002Fkafka\u002Ftree\u002F4.3.1",[17],"Kafka Streams 4.3.1",[62,219,220],{},[13,221,224],{"href":222,"rel":223},"https:\u002F\u002Fgithub.com\u002Fspring-projects\u002Fspring-boot\u002Freleases\u002Ftag\u002Fv4.1.1",[17],"Spring Boot 4.1.1",[62,226,227],{},[13,228,231],{"href":229,"rel":230},"https:\u002F\u002Fgithub.com\u002Fspring-cloud\u002Fspring-cloud-stream\u002Freleases\u002Ftag\u002Fv5.0.3",[17],"Spring Cloud Stream 5.0.3",[62,233,234],{},[13,235,238],{"href":236,"rel":237},"https:\u002F\u002Fgithub.com\u002Fquarkusio\u002Fquarkus\u002Freleases\u002Ftag\u002F3.40.1",[17],"Quarkus 3.40.1",[72,240,241,270,309,330,364,392,409,433],{},[59,242,243,248,253,261,266],{},[77,244,245],{},[24,246,247],{},"Maven artifact",[77,249,250],{},[92,251,252],{},"kafka-streams",[77,254,255,258,259],{},[92,256,257],{},"spring-kafka"," and ",[92,260,252],{},[77,262,263],{},[92,264,265],{},"spring-cloud-stream-binder-kafka-streams",[77,267,268],{},[92,269,157],{},[59,271,272,277,283,288,304],{},[77,273,274],{},[24,275,276],{},"Topology",[77,278,279,282],{},[92,280,281],{},"StreamsBuilder"," or Processor API",[77,284,285,286],{},"Stream beans on an injected ",[92,287,281],{},[77,289,290,293,294,297,298,258,301],{},[92,291,292],{},"Function"," or ",[92,295,296],{},"Consumer"," beans over ",[92,299,300],{},"KStream",[92,302,303],{},"KTable",[77,305,306,307],{},"CDI method returning a ",[92,308,276],{},[59,310,311,316,319,324,327],{},[77,312,313],{},[24,314,315],{},"Lifecycle",[77,317,318],{},"Your code",[77,320,321],{},[92,322,323],{},"StreamsBuilderFactoryBean",[77,325,326],{},"The binder",[77,328,329],{},"The extension, after listed topics exist",[59,331,332,337,342,347,356],{},[77,333,334],{},[24,335,336],{},"Configuration",[77,338,339],{},[92,340,341],{},"Properties",[77,343,344],{},[92,345,346],{},"spring.kafka.streams.*",[77,348,349,352,353],{},[92,350,351],{},"spring.cloud.stream.bindings.*",", ",[92,354,355],{},"spring.cloud.stream.kafka.streams.*",[77,357,358,352,361],{},[92,359,360],{},"quarkus.kafka-streams.*",[92,362,363],{},"kafka-streams.*",[59,365,366,374,377,382,387],{},[77,367,368],{},[24,369,370,371],{},"Default ",[92,372,373],{},"application.id",[77,375,376],{},"None; required",[77,378,379],{},[92,380,381],{},"spring.application.name",[77,383,384,386],{},[92,385,381],{}," if set (single processor only), else generated from the function name",[77,388,389],{},[92,390,391],{},"quarkus.application.name",[59,393,394,399,402,404,407],{},[77,395,396],{},[24,397,398],{},"Serdes",[77,400,401],{},"Explicit or configured defaults",[77,403,401],{},[77,405,406],{},"Inferred; explicitly configurable",[77,408,401],{},[59,410,411,416,419,422,428],{},[77,412,413],{},[24,414,415],{},"Failed records to a DLQ",[77,417,418],{},"Native DLQ from 4.2",[77,420,421],{},"Recovering exception handlers",[77,423,424,352,426],{},[92,425,137],{},[92,427,141],{},[77,429,430,431],{},"Native DLQ through ",[92,432,363],{},[59,434,435,440,443,448,452],{},[77,436,437],{},[24,438,439],{},"Broker test option",[77,441,442],{},"Testcontainers",[77,444,445],{},[92,446,447],{},"@EmbeddedKafka",[77,449,450],{},[92,451,447],{},[77,453,454],{},"Dev Services",[10,456,457],{},"A Serde pairs a serializer with a deserializer for a key or value type.",[10,459,460],{},"Every column gets Kafka Streams' state stores, windows, joins, and exactly-once processing. Which features are available, and how they behave, depends on the Kafka Streams version each one resolves and on its configuration.",[10,462,463,464,46],{},"The sections below use a running example: count orders per customer ID and keep the counts in a state store named ",[92,465,466],{},"order-counts",[468,469,471],"h3",{"id":470},"apache-kafka-streams-in-plain-java","Apache Kafka Streams in plain Java",[10,473,474,475,478],{},"The ",[92,476,477],{},"org.apache.kafka:kafka-streams"," library has two APIs. The DSL covers filtering, grouping, counting, windows, and joins; the Processor API lets you write custom processors that use state stores directly. One topology can mix both.",[10,480,481,482,484,485,487,488,490],{},"For the order count, you group the orders stream by key, count it into ",[92,483,466],{},", and write the changes to an output topic. Your code then creates ",[92,486,94],{}," from the topology and its ",[92,489,341],{},", starts it, and closes it on shutdown.",[10,492,493],{},[24,494,495],{},"Capabilities and limitations",[497,498,499,509,527,537],"ul",{},[500,501,502,505,506,508],"li",{},[24,503,504],{},"Lifecycle:"," ",[92,507,94],{}," runs its own stream threads. Register an uncaught exception handler to choose whether a failed thread is replaced or the client shuts down, and close the instance on shutdown so local state is flushed.",[500,510,511,514,515,517,518,505,521,526],{},[24,512,513],{},"Default failures:"," Deserialization and processing errors shut down the local ",[92,516,94],{}," instance by default. Use the ",[92,519,520],{},"LogAndContinue",[13,522,525],{"href":523,"rel":524},"https:\u002F\u002Fkafka.apache.org\u002F43\u002Fstreams\u002Fdeveloper-guide\u002Fconfig-streams\u002F",[17],"handlers"," to skip failed records instead.",[500,528,529,532,533,536],{},[24,530,531],{},"Native DLQ:"," From Kafka 4.2, ",[92,534,535],{},"errors.dead.letter.queue.topic.name"," makes the built-in handlers also send failed records to a dead-letter topic. Custom handlers must implement DLQ support explicitly.",[500,538,539,542],{},[24,540,541],{},"Versions:"," You pick the Kafka Streams version directly, with no framework dependency management in between.",[10,544,545,548],{},[24,546,547],{},"When to choose:"," Use plain Kafka Streams for services without Spring or Quarkus, or when you want to manage configuration and lifecycle yourself.",[468,550,552],{"id":551},"kafka-streams-with-spring-boot-and-spring-kafka","Kafka Streams with Spring Boot and Spring Kafka",[10,554,555,556,558,559,561,562,564,565,569,570,572],{},"Spring Kafka manages Kafka Streams through a ",[92,557,323],{},". With ",[92,560,252],{}," on the classpath and ",[92,563,111],{}," on a configuration class, ",[13,566,29],{"href":567,"rel":568},"https:\u002F\u002Fdocs.spring.io\u002Fspring-boot\u002Freference\u002Fmessaging\u002Fkafka.html#messaging.kafka.streams",[17]," builds its configuration from ",[92,571,346],{}," properties.",[10,574,575,576,579,580,582],{},"For the order count, a ",[92,577,578],{},"@Bean"," method receives the shared ",[92,581,281],{},", defines the stream, and returns it. The factory bean starts the topology with the application context and closes it on shutdown.",[10,584,585],{},[24,586,495],{},[497,588,589,607,613,628],{},[500,590,591,505,594,597,598,600,601,603,604,46],{},[24,592,593],{},"Configuration:",[92,595,596],{},"spring.kafka.streams.application-id"," defaults to ",[92,599,381],{},". Boot-supported settings use ",[92,602,346],{},"; other Kafka Streams properties go under ",[92,605,606],{},"spring.kafka.streams.properties.*",[500,608,609,612],{},[24,610,611],{},"Several topologies:"," Each topology with its own lifecycle needs its own factory bean and application ID.",[500,614,615,505,618,621,622,627],{},[24,616,617],{},"Failed records:",[92,619,620],{},"RecoveringDeserializationExceptionHandler"," can send records that fail deserialization to a dead-letter topic. Spring Kafka 4.1 adds ",[13,623,626],{"href":624,"rel":625},"https:\u002F\u002Fdocs.spring.io\u002Fspring-kafka\u002Freference\u002Fstreams.html",[17],"matching handlers"," for processing and production errors, which can use Kafka's native DLQ.",[500,629,630,505,633,636,637,640,641,46],{},[24,631,632],{},"Helpers:",[92,634,635],{},"KafkaStreamsInteractiveQueryService"," reads state stores, ",[92,638,639],{},"KafkaStreamsMicrometerListener"," registers metrics, and Spring Kafka 4.1 can switch to the streams rebalance protocol with ",[92,642,643],{},"setGroupProtocol()",[10,645,646,648],{},[24,647,547],{}," Use Spring Kafka for Kafka Streams in Spring Boot services. You do not need Spring Cloud Stream to run a topology in Spring Boot.",[468,650,652],{"id":651},"spring-cloud-stream-kafka-streams-binder","Spring Cloud Stream Kafka Streams binder",[10,654,474,655,657,658,661],{},[92,656,265],{}," artifact binds functions to Kafka topics. For the order count, a ",[92,659,660],{},"Function\u003CKStream\u003CString, Order>, KStream\u003CString, Long>>"," bean receives the input stream and returns the counts.",[10,663,664,665,258,668,671,672,46],{},"Binding properties map the function's ",[92,666,667],{},"-in-0",[92,669,670],{},"-out-0"," bindings to topics; see the ",[13,673,676],{"href":674,"rel":675},"https:\u002F\u002Fdocs.spring.io\u002Fspring-cloud-stream\u002Freference\u002Fkafka\u002Fkafka-streams-binder\u002Fprogramming-model.html",[17],"programming model",[10,678,679,680,683,684,689,690,692],{},"Each bound processor runs as its own Kafka Streams instance; composed functions such as ",[92,681,682],{},"foo|bar"," form one processor. ",[13,685,688],{"href":686,"rel":687},"https:\u002F\u002Fdocs.spring.io\u002Fspring-cloud-stream\u002Freference\u002Fkafka\u002Fkafka-streams-binder\u002Fancillaries-to-the-programming-model.html",[17],"Without an explicit ID",", the binder uses ",[92,691,381],{},", or generates one from the function bean name if that is not set.",[10,694,695,697],{},[92,696,381],{}," suits a single processor only; with several, they would all share that ID. Set IDs explicitly in production, one per function, and give every replica of a processor the same ID.",[10,699,700],{},[24,701,495],{},[497,703,704,714,728,746,757],{},[500,705,706,709,710,713],{},[24,707,708],{},"Serdes:"," The binder infers Serdes from the function's generic types. When no Kafka Serde matches, it falls back to Spring Kafka's ",[92,711,712],{},"JacksonJsonSerde",", so configure Serdes explicitly for Avro or Protobuf topics.",[500,715,716,719,720,723,724,727],{},[24,717,718],{},"Deserialization errors:"," With ",[92,721,722],{},"deserializationExceptionHandler: sendToDlq",", records that fail deserialization go to ",[92,725,726],{},"error.\u003Cinput-topic>.\u003Capplication-id>"," by default.",[500,729,730,733,734,736,737,740,741,46],{},[24,731,732],{},"Errors in your code:"," Wrap the logic in ",[92,735,141],{}," to publish failures to a DLT, or in ",[92,738,739],{},"RecordRecoverableProcessor"," to handle them yourself; see ",[13,742,745],{"href":743,"rel":744},"https:\u002F\u002Fdocs.spring.io\u002Fspring-cloud-stream\u002Freference\u002Fkafka\u002Fkafka-streams-binder\u002Ferror-handling.html",[17],"error handling",[500,747,748,505,751,753,754,46],{},[24,749,750],{},"DLT keys:",[92,752,141],{}," publishes through the regular Kafka binder, so configure the DLT binding's key serializer unless keys are ",[92,755,756],{},"byte[]",[500,758,759,762],{},[24,760,761],{},"Status:"," The Kafka Streams binder is maintained; version 5.0.3 shipped on 20 Aug 2026. The deprecation in Spring Cloud Stream 4.3.0 applies to the reactive Kafka binder.",[10,764,765,767],{},[24,766,547],{}," Use the binder when your services already use Spring Cloud Stream function bindings. It adds configuration on top of Kafka Streams, not more stream-processing features.",[468,769,770],{"id":157},"Quarkus Kafka Streams",[10,772,773,774,776,777,779,780,782,783,785,786,46],{},"With ",[92,775,157],{},", a CDI producer method returns the ",[92,778,276],{}," and Quarkus starts Kafka Streams for it. The application ID defaults to ",[92,781,391],{},", and ",[92,784,363],{}," properties pass straight through; see the ",[13,787,790],{"href":788,"rel":789},"https:\u002F\u002Fquarkus.io\u002Fguides\u002Fkafka-streams",[17],"Quarkus guide",[10,792,793],{},[24,794,495],{},[497,796,797,810,825,831],{},[500,798,799,802,803,806,807,809],{},[24,800,801],{},"Startup:"," The extension waits until every topic in ",[92,804,805],{},"quarkus.kafka-streams.topics"," exists before starting the topology. With ",[92,808,163],{},", it adds a readiness check for those topics and a liveness check for the Kafka Streams state.",[500,811,812,815,816,820,821,824],{},[24,813,814],{},"Development:"," Dev mode reloads the topology after code changes. ",[13,817,454],{"href":818,"rel":819},"https:\u002F\u002Fquarkus.io\u002Fguides\u002Fkafka-dev-services",[17]," starts a Kafka broker in dev mode and tests when Docker is available and ",[92,822,823],{},"kafka.bootstrap.servers"," is not set.",[500,826,827,830],{},[24,828,829],{},"Native executables:"," Kafka Streams applications can build as native executables.",[500,832,833,835,836,838,839,842],{},[24,834,708],{}," As in plain Kafka Streams, set Serdes in the topology code or as defaults through ",[92,837,363],{},"; ",[92,840,841],{},"ObjectMapperSerde"," covers JSON values.",[10,844,845,847],{},[24,846,547],{}," Use the extension for Quarkus services, especially when you want built-in health checks and a broker for local tests.",[48,849,851],{"id":850},"kafka-streams-maven-dependencies-and-version-alignment","Kafka Streams Maven dependencies and version alignment",[10,853,854],{},"In Spring Boot and Quarkus, the framework's dependency management picks the Kafka Streams version. Check the resolved version before you rely on a feature from a newer Kafka release, such as the native DLQ or the streams rebalance protocol from 4.2.",[53,856,857,870],{},[56,858,859],{},[59,860,861,864,867],{},[62,862,863],{},"Setup",[62,865,866],{},"Artifact to add",[62,868,869],{},"Kafka Streams version",[72,871,872,892,909,922,942,956],{},[59,873,874,879,883],{},[77,875,876],{},[24,877,878],{},"Plain Java",[77,880,881],{},[92,882,477],{},[77,884,885,886,891],{},"Your choice; the ",[13,887,890],{"href":888,"rel":889},"https:\u002F\u002Fkafka.apache.org\u002Fcommunity\u002Fdownloads\u002F",[17],"latest release"," is 4.3.1 (25 Jun 2026)",[59,893,894,898,906],{},[77,895,896],{},[24,897,224],{},[77,899,900,258,903,905],{},[92,901,902],{},"org.springframework.kafka:spring-kafka",[92,904,252],{},", versions from Boot",[77,907,908],{},"4.2.1, with Spring Kafka 4.1.1",[59,910,911,916,919],{},[77,912,913],{},[24,914,915],{},"Spring Boot 4.0.8",[77,917,918],{},"Same as Spring Boot 4.1",[77,920,921],{},"4.1.2, with Spring Kafka 4.0.7",[59,923,924,929,939],{},[77,925,926],{},[24,927,928],{},"Spring Cloud Stream binder",[77,930,931,932,935,936,938],{},"Import ",[92,933,934],{},"spring-cloud-dependencies"," 2025.1.3, then add ",[92,937,265],{}," without a version (5.0.3)",[77,940,941],{},"Managed by your Spring Boot version",[59,943,944,948,953],{},[77,945,946],{},[24,947,238],{},[77,949,950],{},[92,951,952],{},"io.quarkus:quarkus-kafka-streams",[77,954,955],{},"4.2.1",[59,957,958,963,969],{},[77,959,960],{},[24,961,962],{},"Tests",[77,964,965,968],{},[92,966,967],{},"org.apache.kafka:kafka-streams-test-utils"," in test scope",[77,970,971,972],{},"Same as ",[92,973,252],{},[10,975,976,977,293,980,46],{},"To see the versions your build resolves, run ",[92,978,979],{},"mvn dependency:tree -Dincludes=org.apache.kafka",[92,981,982],{},"gradle dependencyInsight --dependency kafka-streams",[10,984,985,986,991],{},"Spring Cloud 2025.1 supports Spring Boot 4.0 and, from 2025.1.2, Spring Boot 4.1; see the ",[13,987,990],{"href":988,"rel":989},"https:\u002F\u002Fspring.io\u002Fprojects\u002Fspring-cloud",[17],"compatibility table",". The binder is built against Kafka Streams 4.1.2, but Spring Boot's dependency management decides the version your application uses.",[10,993,773,994,997,998,1001,1002,1005,1006,1009,1010,1013],{},[92,995,996],{},"spring-boot-starter-parent",", override ",[92,999,1000],{},"kafka.version"," to keep the Kafka artifacts aligned, including ",[92,1003,1004],{},"kafka-streams-test-utils",". If you only import ",[92,1007,1008],{},"spring-boot-dependencies",", override the relevant ",[92,1011,1012],{},"dependencyManagement"," entries instead.",[48,1015,1017],{"id":1016},"kafka-streams-state-stores-and-recovery","Kafka Streams state stores and recovery",[10,1019,1020,1021,1023],{},"A state store holds data a stateful operation needs between records, such as each customer's count in ",[92,1022,466],{},". Each instance keeps the stores for the partitions it owns; by default, each store is backed by a changelog topic.",[468,1025,1027],{"id":1026},"rocksdb-and-in-memory-stores","RocksDB and in-memory stores",[10,1029,1030,1031,258,1034,1037,1038,1041,1042,1045],{},"DSL operations such as ",[92,1032,1033],{},"count()",[92,1035,1036],{},"aggregate()"," use persistent RocksDB stores by default, kept under ",[92,1039,1040],{},"state.dir"," (default ",[92,1043,1044],{},"${java.io.tmpdir}\u002Fkafka-streams",").",[10,1047,1048],{},"In-memory stores avoid disk I\u002FO, but after a restart they are rebuilt from the changelog, or start empty if logging is disabled.",[10,1050,1051,1052,1055,1056,1059,1060,46],{},"Set the store type for all DSL operations with ",[92,1053,1054],{},"dsl.store.suppliers.class",", or for one operation with ",[92,1057,1058],{},"Materialized",". RocksDB uses off-heap memory for its block cache and write buffers, so leave room beyond the JVM heap; see ",[13,1061,1064],{"href":1062,"rel":1063},"https:\u002F\u002Fkafka.apache.org\u002F43\u002Fstreams\u002Fdeveloper-guide\u002Fmemory-mgmt\u002F",[17],"memory management",[10,1066,1067,1068,258,1071,1074,1075,1078],{},"Internal topics are named ",[92,1069,1070],{},"\u003Capplication.id>-\u003Cname>-changelog",[92,1072,1073],{},"-repartition",". Name stores explicitly, for example with ",[92,1076,1077],{},"Materialized.as(\"order-counts\")",", because generated names can change when the topology changes.",[10,1080,1081,1082,1085],{},"Since Kafka 4.1, setting ",[92,1083,1084],{},"ensure.explicit.internal.resource.naming=true"," stops the application from starting while any internal name is generated.",[468,1087,1089],{"id":1088},"querying-state-with-interactive-queries","Querying state with Interactive Queries",[10,1091,1092,1097],{},[13,1093,1096],{"href":1094,"rel":1095},"https:\u002F\u002Fkafka.apache.org\u002F43\u002Fstreams\u002Fdeveloper-guide\u002Finteractive-queries\u002F",[17],"Interactive Queries"," read a running instance's local stores, for example to return one customer's count from a REST endpoint. Each instance holds only the partitions it owns, so a key may live on another instance.",[10,1099,1100,1101,1104,1105,1108],{},"To reach that instance, set a unique ",[92,1102,1103],{},"application.server"," on each instance and look up the key's host with ",[92,1106,1107],{},"queryMetadataForKey()",". The call to the other instance goes through an HTTP or RPC endpoint you provide.",[10,1110,1111,1112,1114,1115,1118],{},"Spring Kafka's ",[92,1113,635],{}," and the binder's ",[92,1116,1117],{},"InteractiveQueryService"," wrap the store and host lookups. The Quarkus guide shows the same pattern with REST endpoints.",[10,1120,1121,1122,1125],{},"While a store is migrating or restoring, lookups throw ",[92,1123,1124],{},"InvalidStateStoreException","; both Spring helpers can retry them.",[468,1127,1129],{"id":1128},"restoration-standby-replicas-and-scaling","Restoration, standby replicas, and scaling",[10,1131,1132],{},"When a task moves to another instance, Kafka Streams replays the store's changelog before processing resumes, which takes time for large stores.",[10,1134,1135,1138],{},[92,1136,1137],{},"num.standby.replicas"," (default 0) keeps copies of each stateful task's stores on other instances, so a failover only replays the changelog records a standby has not caught up on.",[10,1140,1141],{},"With one standby per task, run at least two instances and plan for roughly twice the total state-store capacity.",[10,1143,1144,1145,258,1148,46],{},"When a new instance joins, the default classic protocol keeps tasks on caught-up instances and warms up state on the new one first, controlled by ",[92,1146,1147],{},"acceptable.recovery.lag",[92,1149,1150],{},"max.warmup.replicas",[10,1152,1153,1154,1157,1158,46],{},"Caught-up warmups take over at the next probing rebalance, every ",[92,1155,1156],{},"probing.rebalance.interval.ms"," (10 minutes by default). See ",[13,1159,1162],{"href":1160,"rel":1161},"https:\u002F\u002Fkafka.apache.org\u002F43\u002Fstreams\u002Farchitecture\u002F",[17],"Streams architecture",[10,1164,1165],{},"In this single-input topology, five input partitions allow at most five active tasks; a sixth instance gets no active task but can host standby or warmup tasks.",[10,1167,773,1168,1171],{},[92,1169,1170],{},"exactly_once_v2",", an unclean shutdown can make restores longer: Kafka Streams 4.3 still wipes the affected stores and rebuilds them from the changelog.",[10,1173,1174,1175,1179],{},"Kafka 4.2 also added a broker-side rebalance protocol for Kafka Streams; see ",[13,1176,1178],{"href":1177},"#can-kafka-streams-use-the-new-rebalance-protocol","the FAQ"," for what it supports.",[48,1181,1183],{"id":1182},"kafka-streams-testing","Kafka Streams testing",[10,1185,1186],{},"Test the topology without a broker first, then test what depends on Kafka itself against a broker.",[468,1188,1190],{"id":1189},"test-a-topology-with-topologytestdriver","Test a topology with TopologyTestDriver",[10,1192,1193,1200,1201,1203,1204,1207,1208,1211],{},[13,1194,1197],{"href":1195,"rel":1196},"https:\u002F\u002Fkafka.apache.org\u002F43\u002Fstreams\u002Fdeveloper-guide\u002Ftesting\u002F",[17],[92,1198,1199],{},"TopologyTestDriver"," from ",[92,1202,1004],{}," runs a topology in one process without a broker. You pipe records into ",[92,1205,1206],{},"TestInputTopic",", read ",[92,1209,1210],{},"TestOutputTopic",", and read state stores directly, for example to check that three orders leave a count of 3.",[10,1213,1214,1215,1218],{},"It also controls time: event-time punctuations follow record timestamps, and ",[92,1216,1217],{},"advanceWallClockTime()"," fires wall-clock punctuations, so time-based logic runs without waiting.",[10,1220,1221],{},"The driver processes records synchronously and commits after each input, so it does not reproduce production caching or commit timing.",[10,1223,1224],{},"In Spring or Quarkus, keep the stream definition in a method you can call from a plain unit test.",[468,1226,1228],{"id":1227},"integration-tests-with-a-kafka-broker","Integration tests with a Kafka broker",[10,1230,1231],{},"The test driver does not create internal topics, rebalance between instances, restore state from changelogs, or run Kafka transactions. Test those behaviors against a broker.",[10,1233,1234,1235,1238],{},"Schema Registry Serdes can run inside the driver, for example against Confluent's in-memory ",[92,1236,1237],{},"mock:\u002F\u002F"," registry. Test the connection to your real registry and schema compatibility separately.",[10,1240,1111,1241,1243,1244,1249],{},[92,1242,447],{}," starts an in-process KRaft broker for tests (",[13,1245,1248],{"href":1246,"rel":1247},"https:\u002F\u002Fdocs.spring.io\u002Fspring-kafka\u002Freference\u002Ftesting.html",[17],"testing guide","). Quarkus Dev Services starts a Kafka container in dev mode and tests. For plain Java, Testcontainers' Kafka module does the same.",[10,1251,1252],{},"A useful broker test runs two instances with the same application ID and records one customer's count, N. Stop the instance that owns that customer's partition, wait for the other to take over, send one more order, and check that the count becomes N+1.",[10,1254,1255,1256,46],{},"When running both instances on the same host, give each a separate ",[92,1257,1040],{},[48,1259,1261],{"id":1260},"inspect-kafka-streams-topics-with-kafma","Inspect Kafka Streams topics with Kafma",[10,1263,1264],{},"Unit tests check the topology logic. A deployed application can still use the wrong Serde, restore slowly, or fill a dead-letter topic, and those problems show up in topics and group positions.",[10,1266,1267,1268,1272,1273,1277],{},"Kafma is a desktop ",[13,1269,1271],{"href":1270},"\u002F","Kafka IDE"," that runs beside your Java application. Use its ",[13,1274,1276],{"href":1275},"\u002Ffeatures\u002Fkafka-console","Kafka console"," to send input, read output and dead-letter topics, and watch the application's group.",[468,1279,1281],{"id":1280},"send-keyed-input-and-check-the-counts","Send keyed input and check the counts",[10,1283,1284,1285,1289,1290,1293],{},"Use the ",[13,1286,1288],{"href":1287},"\u002Fdocs\u002Fconsole\u002Fproducing-messages","producer panel"," to send orders with the customer ID as a String key, then open the output topic and watch that customer's count increase. ",[24,1291,1292],{},"Loop"," keeps orders arriving at a fixed interval, which the restart test below needs.",[10,1295,1296],{},"By default, counts can take up to 30 seconds to appear, and several orders may show up as one update, because Kafka Streams caches results until the next commit.",[10,1298,1299,1300,1303,1304,505,1307,1311],{},"Counts written with ",[92,1301,1302],{},"Serdes.Long()"," are 8-byte values; select the ",[24,1305,1306],{},"Long",[13,1308,1310],{"href":1309},"\u002Fdocs\u002Fconsole\u002Fdecoding-messages","decoder"," to read them. With Schema Registry configured, Kafma decodes Avro, Protobuf, and JSON Schema records automatically.",[10,1313,1314],{},[1315,1316],"img",{"alt":1317,"height":1318,"src":1319,"width":1320},"Kafma displaying a JSON order record with its Spring TypeId header, key, and metadata",1520,"https:\u002F\u002Fmedia.kafma.app\u002Fblog\u002Fkafka-java-clients\u002Fspring-json-type-header.png",3024,[468,1322,1324],{"id":1323},"watch-the-application-during-restarts","Watch the application during restarts",[10,1326,1327,1328,1330,1331,1335],{},"With the default classic protocol, a Kafka Streams application appears as a consumer group named after its ",[92,1329,373],{},". Open ",[13,1332,1334],{"href":1333},"\u002Fdocs\u002Fconsole\u002Fconsuming-messages#live-and-watch-group-modes","Watch Group"," on it, then stop an instance or start a second one.",[10,1337,1338],{},"While a moved task restores its store, its committed positions stop advancing, and lag grows if orders keep arriving. Kafma reads the group's positions without joining it, so watching does not trigger a rebalance.",[10,1340,474,1341,1345],{},[13,1342,1344],{"href":1343},"\u002Ffeatures\u002Fkafka-consumer-groups","consumer group UI"," shows each member's partitions as ownership moves.",[10,1347,1348],{},[1315,1349],{"alt":1350,"height":1351,"src":1352,"width":1353},"Kafma watching pending and consumed records around a consumer group's committed offsets",1368,"https:\u002F\u002Fmedia.kafma.app\u002Fdocs\u002Fconsole\u002Fconsuming-messages\u002Fwatch-group.png",1562,[468,1355,1357],{"id":1356},"inspect-and-replay-dead-letter-records","Inspect and replay dead-letter records",[10,1359,1360,1361,1364,1365,1368],{},"Kafka Streams' native DLQ records add ",[92,1362,1363],{},"__streams.errors.*"," headers with the exception, stack trace, and source topic, partition, and offset. Spring Kafka's handlers add ",[92,1366,1367],{},"kafka_dlt-*"," headers instead. Expand a record in Kafma to read them.",[10,1370,1371,1372,1376],{},"For records that failed deserialization on the input topic, fix the cause, then use ",[13,1373,1375],{"href":1374},"\u002Fdocs\u002Fconsole\u002Freplay-forward-and-export#replay-kafka-messages-to-a-topic","Replay to topic"," to load the record into the producer for that topic. Remove the error headers, check the key and value encodings, and produce it.",[10,1378,1379],{},[1315,1380],{"alt":1381,"height":1382,"src":1383,"width":1384},"Choosing a topic to replay a Kafka message to in Kafma",846,"https:\u002F\u002Fmedia.kafma.app\u002Fdocs\u002Fconsole\u002Freplay-forward-and-export\u002Freplay.png",1568,[10,1386,1387,1391],{},[13,1388,1390],{"href":1389},"\u002Fdownload","Download Kafma"," and connect to the same test cluster as your application.",[48,1393,1395],{"id":1394},"frequently-asked-questions","Frequently asked questions",[468,1397,1399],{"id":1398},"what-is-the-difference-between-the-kafka-binder-and-the-kafka-streams-binder","What is the difference between the Kafka binder and the Kafka Streams binder?",[10,1401,1402,1403,1406,1407,258,1409,1411],{},"Both are Spring Cloud Stream binders for Kafka. ",[92,1404,1405],{},"spring-cloud-stream-binder-kafka"," passes individual messages to ",[92,1408,296],{},[92,1410,292],{}," beans and runs on Spring Kafka listener containers.",[10,1413,1414,1416,1417,352,1419,1421,1422,1425,1426,46],{},[92,1415,265],{}," passes ",[92,1418,300],{},[92,1420,303],{},", or ",[92,1423,1424],{},"GlobalKTable"," objects to your functions and runs a Kafka Streams topology. For the message binder, see ",[13,1427,45],{"href":1428},"\u002Fblog\u002Fkafka-java-clients#spring-cloud-stream-kafka-binder",[468,1430,1432],{"id":1431},"do-you-need-spring-cloud-stream-to-use-kafka-streams-with-spring-boot","Do you need Spring Cloud Stream to use Kafka Streams with Spring Boot?",[10,1434,1435,1436,258,1438,1440],{},"No. Spring Kafka runs Kafka Streams in Spring Boot with ",[92,1437,111],{},[92,1439,346],{}," properties. Spring Cloud Stream adds function bindings on top; add it when your services already use them.",[468,1442,1444],{"id":1443},"can-spring-kafka-listeners-and-kafka-streams-run-in-the-same-application","Can Spring Kafka listeners and Kafka Streams run in the same application?",[10,1446,1447,1448,352,1451,1454,1455,1457],{},"Yes. One Spring Boot application can use ",[92,1449,1450],{},"KafkaTemplate",[92,1452,1453],{},"@KafkaListener",", and a Kafka Streams topology together. Give the topology an ",[92,1456,373],{}," that differs from your listener group IDs, because Kafka Streams uses it as its group ID.",[468,1459,1461],{"id":1460},"what-is-the-difference-between-kstream-and-ktable","What is the difference between KStream and KTable?",[10,1463,1464,1465,1467,1468,1470,1471,1473,1474,46],{},"A ",[92,1466,300],{}," treats each record as a separate event, such as an order. A ",[92,1469,303],{}," treats each record as an update to the latest value for its key, such as a customer's current count; counting a ",[92,1472,300],{}," by key produces a ",[92,1475,303],{},[10,1477,1464,1478,1480],{},[92,1479,1424],{}," gives every instance a full copy of a table.",[468,1482,1484],{"id":1483},"is-the-spring-cloud-stream-kafka-streams-binder-deprecated","Is the Spring Cloud Stream Kafka Streams binder deprecated?",[10,1486,1487,1488,1493],{},"No. Spring Cloud Stream 5.0.3 shipped the Kafka Streams binder on 20 Aug 2026. The ",[13,1489,1492],{"href":1490,"rel":1491},"https:\u002F\u002Fdocs.spring.io\u002Fspring-cloud-stream\u002Freference\u002Fkafka\u002Fkafka-reactive-binder\u002Foverview.html",[17],"deprecation"," announced in Spring Cloud Stream 4.3.0 covers the reactive Kafka binder, after Reactor Kafka was discontinued.",[468,1495,1497],{"id":1496},"does-kafka-streams-require-a-separate-cluster","Does Kafka Streams require a separate cluster?",[10,1499,1500,1501,1503],{},"It needs a Kafka cluster but no separate processing cluster. The library runs inside your application, and instances with the same ",[92,1502,373],{}," share the work by dividing the input partitions among them.",[468,1505,1507],{"id":1506},"can-kafka-streams-use-the-new-rebalance-protocol","Can Kafka Streams use the new rebalance protocol?",[10,1509,1510,1511,1513,1514,1519,1520,1523,1524,46],{},"Yes, when brokers and clients run Kafka 4.2 or later (see the ",[13,1512,118],{"href":117},"). New 4.2 clusters support the ",[13,1515,1518],{"href":1516,"rel":1517},"https:\u002F\u002Fkafka.apache.org\u002F43\u002Fstreams\u002Fdeveloper-guide\u002Fstreams-rebalance-protocol\u002F",[17],"streams rebalance protocol"," (KIP-1071) by default; upgraded clusters first need ",[92,1521,1522],{},"streams.version=1"," enabled with ",[92,1525,1526],{},"kafka-features.sh",[10,1528,1529,1530,1533,1534,46],{},"Applications still default to ",[92,1531,1532],{},"classic"," and opt in with ",[92,1535,1536],{},"group.protocol=streams",[10,1538,1539,1540,1543,1544,1547,1548,1550],{},"The application then registers as a streams group instead of a consumer group. Standby replicas become a group setting, ",[92,1541,1542],{},"streams.num.standby.replicas",", set with ",[92,1545,1546],{},"kafka-configs.sh","; the client's ",[92,1549,1137],{}," is ignored.",[10,1552,1553],{},"Static membership, warmup tasks, regex subscriptions, and online migration are not supported yet.",[10,1555,1556],{},"To migrate an existing application offline, stop every instance, wait until the group is empty, and restart with the new setting; only committed offsets carry over. Use 4.2.1 or later brokers for this, since 4.2.0 has a migration bug.",[48,1558,1560],{"id":1559},"conclusion","Conclusion",[10,1562,1563,1564,1566],{},"In Spring Boot, start with Spring Kafka and ",[92,1565,111],{},"; in Quarkus, use the extension; elsewhere, use Kafka Streams directly. Add the Spring Cloud Stream binder when your services already use its function bindings.",[10,1568,1569,1570,1572],{},"Whichever you choose, check the Kafka Streams version it resolves, name your state stores, and decide where failed records go. Test the topology with ",[92,1571,1199],{}," first, then test restores and rebalances against a broker.",[10,1574,1575],{},"This guide is maintained by the team behind Kafma.",{"title":1577,"searchDepth":1578,"depth":1578,"links":1579},"",3,[1580,1582,1588,1589,1594,1598,1603,1612],{"id":50,"depth":1581,"text":51},2,{"id":194,"depth":1581,"text":195,"children":1583},[1584,1585,1586,1587],{"id":470,"depth":1578,"text":471},{"id":551,"depth":1578,"text":552},{"id":651,"depth":1578,"text":652},{"id":157,"depth":1578,"text":770},{"id":850,"depth":1581,"text":851},{"id":1016,"depth":1581,"text":1017,"children":1590},[1591,1592,1593],{"id":1026,"depth":1578,"text":1027},{"id":1088,"depth":1578,"text":1089},{"id":1128,"depth":1578,"text":1129},{"id":1182,"depth":1581,"text":1183,"children":1595},[1596,1597],{"id":1189,"depth":1578,"text":1190},{"id":1227,"depth":1578,"text":1228},{"id":1260,"depth":1581,"text":1261,"children":1599},[1600,1601,1602],{"id":1280,"depth":1578,"text":1281},{"id":1323,"depth":1578,"text":1324},{"id":1356,"depth":1578,"text":1357},{"id":1394,"depth":1581,"text":1395,"children":1604},[1605,1606,1607,1608,1609,1610,1611],{"id":1398,"depth":1578,"text":1399},{"id":1431,"depth":1578,"text":1432},{"id":1443,"depth":1578,"text":1444},{"id":1460,"depth":1578,"text":1461},{"id":1483,"depth":1578,"text":1484},{"id":1496,"depth":1578,"text":1497},{"id":1506,"depth":1578,"text":1507},{"id":1559,"depth":1581,"text":1560},"2026-10-19","Run Kafka Streams in plain Java, Spring Boot, Spring Cloud Stream, or Quarkus. Compare Maven dependencies, state stores, and testing.","md",{},true,null,"\u002Fblog\u002Fkafka-streams-in-java",false,{"title":5,"description":1614},{"loc":1619},"blog\u002Fkafka-streams-in-java",[1625,18,1626],"Kafka Stream Processing","Java","V4ho9mMpWRzUNbMyDL68wXWdaa4nuT1r4y0domdf8wo",1792397975353]