[{"data":1,"prerenderedAt":1677},["ShallowReactive",2],{"blog-post-\u002Fblog\u002Fkafka-streams-in-golang":3},{"id":4,"title":5,"body":6,"date":1662,"description":1663,"extension":1664,"meta":1665,"navigation":1666,"ogImage":1667,"path":1668,"pinned":1669,"seo":1670,"sitemap":1671,"stem":1672,"tags":1673,"__hash__":1676},"blog\u002Fblog\u002Fkafka-streams-in-golang.md","Kafka Streams in Golang: Goka vs Watermill vs go-streams",{"type":7,"value":8,"toc":1624},"minimark",[9,37,45,50,134,138,141,144,148,151,399,406,410,425,430,522,528,531,552,556,614,619,621,646,650,686,691,694,707,711,795,800,804,812,822,826,908,912,1050,1053,1057,1222,1229,1233,1318,1321,1325,1347,1350,1354,1357,1368,1372,1394,1404,1408,1417,1420,1424,1437,1447,1453,1462,1466,1478,1486,1493,1497,1501,1504,1508,1511,1515,1518,1522,1534,1538,1541,1545,1565,1569,1578,1582,1590,1594,1605,1609,1612,1618,1621],[10,11,12,19,20,24,25,28,29,32,33,36],"p",{},[13,14,18],"a",{"href":15,"rel":16},"https:\u002F\u002Fkafka.apache.org\u002Fdocumentation\u002Fstreams\u002F",[17],"nofollow","Apache Kafka Streams"," has no Go API. Go projects commonly discussed as Kafka Streams alternatives solve different problems. ",[21,22,23],"strong",{},"Goka"," maintains per-key state in Kafka-backed tables that can recover after a restart. ",[21,26,27],{},"Watermill"," organizes event-driven handlers with Kafka as one of several transports. ",[21,30,31],{},"go-streams"," builds in-process pipelines from sources, transformations, and sinks. ",[21,34,35],{},"Redpanda Connect",", formerly Benthos, runs configured data pipelines, usually as a separate service.",[10,38,39,40,44],{},"This guide compares where each tool runs, how it uses Kafka, and whether its state survives a restart. Capabilities are checked against specific releases, and a dated GitHub snapshot compares maintenance activity. If you only need to produce and consume records, see ",[13,41,43],{"href":42},"\u002Fblog\u002Fkafka-golang-clients","Kafka Go Clients: franz-go vs kafka-go vs Sarama",".",[46,47,49],"h2",{"id":48},"quick-decision","Quick decision",[51,52,53,69],"table",{},[54,55,56],"thead",{},[57,58,59,63,66],"tr",{},[60,61,62],"th",{},"Situation",[60,64,65],{},"Best starting point",[60,67,68],{},"Main caveat",[70,71,72,85,97,109,121],"tbody",{},[57,73,74,80,82],{},[75,76,77],"td",{},[21,78,79],{},"Aggregates or joins with recoverable per-key state",[75,81,23],{},[75,83,84],{},"Input pauses while local state restores from compacted table topics; no built-in window operator.",[57,86,87,92,94],{},[75,88,89],{},[21,90,91],{},"CQRS, sagas, or the same handlers on Kafka and another broker",[75,93,27],{},[75,95,96],{},"Kafka is a transport; Watermill does not provide Kafka Streams state stores.",[57,98,99,104,106],{},[75,100,101],{},[21,102,103],{},"In-process map\u002Ffilter\u002Fwindow pipeline with Kafka as a source or sink",[75,105,31],{},[75,107,108],{},"Windows live in memory; the built-in Kafka connectors can lose records on a crash or sink error. No commits or merged PRs in the 90-day snapshot.",[57,110,111,116,118],{},[75,112,113],{},[21,114,115],{},"Configured pipelines between Kafka and other systems",[75,117,35],{},[75,119,120],{},"Usually runs as a separate service; enterprise features require an Enterprise license.",[57,122,123,128,131],{},[75,124,125],{},[21,126,127],{},"Porting a Java Kafka Streams topology to Go",[75,129,130],{},"Keep it on the JVM if API compatibility matters; otherwise redesign for Go",[75,132,133],{},"None of these four is API-compatible with Kafka Streams; stateful windows and joins need a new design.",[46,135,137],{"id":136},"where-state-lives","Where state lives",[10,139,140],{},"A running total or join needs state from earlier records. If another instance must resume that work after a crash or rebalance, the state needs a recovery path. Of the four tools here, Goka is built around Kafka-backed per-key tables: processor instances keep local copies of the table state for their assigned partitions and restore that state from Kafka when ownership changes. Goka does not provide a built-in window operator.",[10,142,143],{},"Watermill organizes message handlers while the application owns their state. go-streams keeps its window data in process memory rather than a Kafka changelog. Redpanda Connect holds window data in memory. After a restart, unacknowledged records are consumed again rather than restoring the previous window; processing-time window boundaries may change. Its Kafka-backed cache is not a co-partitioned processor state store. Choose these tools for their messaging or integration model, rather than expecting Goka-style state recovery.",[46,145,147],{"id":146},"goka-watermill-go-streams-and-redpanda-connect-compared","Goka, Watermill, go-streams, and Redpanda Connect compared",[10,149,150],{},"Capabilities checked against project documentation and source on 27 Sep 2026.",[51,152,153,188],{},[54,154,155],{},[57,156,157,160,167,174,181],{},[60,158,159],{},"Area",[60,161,162],{},[13,163,166],{"href":164,"rel":165},"https:\u002F\u002Fgithub.com\u002Flovoo\u002Fgoka\u002Freleases\u002Ftag\u002Fv1.1.17",[17],"Goka v1.1.17",[60,168,169],{},[13,170,173],{"href":171,"rel":172},"https:\u002F\u002Fgithub.com\u002FThreeDotsLabs\u002Fwatermill\u002Freleases\u002Ftag\u002Fv1.5.3",[17],"Watermill v1.5.3",[60,175,176],{},[13,177,180],{"href":178,"rel":179},"https:\u002F\u002Fgithub.com\u002Freugn\u002Fgo-streams\u002Freleases\u002Ftag\u002Fv0.13.0",[17],"go-streams v0.13.0",[60,182,183],{},[13,184,187],{"href":185,"rel":186},"https:\u002F\u002Fgithub.com\u002Fredpanda-data\u002Fconnect\u002Freleases\u002Ftag\u002Fv4.111.0",[17],"Redpanda Connect v4.111.0",[70,189,190,215,250,283,305,328,351,380],{},[57,191,192,197,200,202,204],{},[75,193,194],{},[21,195,196],{},"Runs as",[75,198,199],{},"Library in your service",[75,201,199],{},[75,203,199],{},[75,205,206,207],{},"Standalone binary or container; embeddable through ",[13,208,211],{"href":209,"rel":210},"https:\u002F\u002Fpkg.go.dev\u002Fgithub.com\u002Fredpanda-data\u002Fbenthos\u002Fv4\u002Fpublic\u002Fservice",[17],[212,213,214],"code",{},"public\u002Fservice",[57,216,217,222,225,235,247],{},[75,218,219],{},[21,220,221],{},"Programming model",[75,223,224],{},"Processor callbacks, group tables, views, emitters",[75,226,227,230,231,234],{},[212,228,229],{},"Publisher"," \u002F ",[212,232,233],{},"Subscriber",", router, CQRS components",[75,236,237,240,241,240,244],{},[212,238,239],{},"Source",", ",[212,242,243],{},"Flow",[212,245,246],{},"Sink",[75,248,249],{},"YAML inputs, processors, and outputs; Bloblang mappings",[57,251,252,257,260,267,273],{},[75,253,254],{},[21,255,256],{},"Kafka client underneath",[75,258,259],{},"Sarama",[75,261,262,263,266],{},"Sarama (",[212,264,265],{},"watermill-kafka"," v3.1.4)",[75,268,262,269,272],{},[212,270,271],{},"kafka"," module)",[75,274,275,276,279,280,282],{},"franz-go (",[212,277,278],{},"redpanda"," input, output, and cache); deprecated ",[212,281,271],{}," components use Sarama",[57,284,285,290,293,296,299],{},[75,286,287],{},[21,288,289],{},"Default key partitioning",[75,291,292],{},"FNV-1a",[75,294,295],{},"No key by default; FNV-1a when a key is set",[75,297,298],{},"Depends on sink input; keyed records use the configured Sarama partitioner (FNV-1a by default)",[75,300,301,302,304],{},"Murmur2 for keyed records; key is optional (",[212,303,278],{}," output)",[57,306,307,312,315,318,321],{},[75,308,309],{},[21,310,311],{},"Delivery on the Kafka path",[75,313,314],{},"At least once",[75,316,317],{},"At least once; no exactly-once",[75,319,320],{},"Built-in connectors mark offsets before processing; records can be lost",[75,322,323,324,327],{},"At least once; ",[212,325,326],{},"system_window"," can drop expired windows under back pressure",[57,329,330,335,342,345,348],{},[75,331,332],{},[21,333,334],{},"Kafka-backed processor state",[75,336,337,338,341],{},"Group tables in compacted ",[212,339,340],{},"\u003Cgroup>-table"," topics, restored to local storage",[75,343,344],{},"No built-in state store; application-owned",[75,346,347],{},"None; windows live in memory",[75,349,350],{},"No co-partitioned processor state; optional Kafka-backed cache",[57,352,353,358,368,371,374],{},[75,354,355],{},[21,356,357],{},"Windows \u002F joins",[75,359,360,363,364,367],{},[212,361,362],{},"Join"," and ",[212,365,366],{},"Lookup"," tables; no built-in windows",[75,369,370],{},"You implement them",[75,372,373],{},"In-memory tumbling, sliding, and session windows",[75,375,376,377,379],{},"In-memory tumbling or sliding windows (",[212,378,326],{},"); cache lookups for enrichment",[57,381,382,387,390,393,396],{},[75,383,384],{},[21,385,386],{},"Other systems",[75,388,389],{},"Kafka-focused",[75,391,392],{},"Many pub\u002Fsub backends",[75,394,395],{},"Many connectors",[75,397,398],{},"Many inputs, outputs, and caches",[10,400,401,402,405],{},"All four use a Kafka client underneath. The client and its configuration affect broker compatibility, consumer-group behavior, and partitioning. Check those settings and the record format when combining services. See the ",[13,403,404],{"href":42},"Kafka Go client comparison"," for franz-go vs kafka-go vs Sarama vs confluent-kafka-go.",[407,408,23],"h3",{"id":409},"goka",[10,411,412,416,417,420,421,424],{},[13,413,23],{"href":414,"rel":415},"https:\u002F\u002Fgithub.com\u002Flovoo\u002Fgoka",[17]," attaches a key-value table to a Kafka consumer group. A processor consumes input topics, updates local table state, and can emit further Kafka records. The table is persisted in Kafka so another instance can restore it after a rebalance or crash. A ",[212,418,419],{},"View"," reads a table; an ",[212,422,423],{},"Emitter"," writes into a stream.",[10,426,427],{},[21,428,429],{},"Capabilities and limitations",[431,432,433,447,459,479,490,511],"ul",{},[434,435,436,439,440,443,444,446],"li",{},[21,437,438],{},"State and recovery:"," Group tables are the reason to use Goka. ",[212,441,442],{},"Persist"," stores the group table in a compacted topic named ",[212,445,340],{},", and each instance keeps its partitions' table state in local storage, LevelDB by default. Input processing is blocked until the tables are recovered, so plan disk and restore time for reassignment.",[434,448,449,452,453,458],{},[21,450,451],{},"Delivery:"," Goka delivers messages ",[13,454,457],{"href":455,"rel":456},"https:\u002F\u002Fgithub.com\u002Flovoo\u002Fgoka#readme",[17],"at least once",". After a failure, a message can be processed again, so a callback that increments a counter can apply the same message twice. Make non-idempotent state updates and external side effects safe to retry, using durable deduplication state when needed.",[434,460,461,464,465,467,468,470,471,474,475,478],{},[21,462,463],{},"Joins and windows:"," ",[212,466,362],{}," reads a co-partitioned table, and ",[212,469,366],{}," reads a table that need not be co-partitioned. There is no built-in windowing; model windows as keyed table state and emitted events. You write callbacks over decoded messages rather than a Java-style ",[212,472,473],{},"KStream","\u002F",[212,476,477],{},"KTable"," DSL.",[434,480,481,484,485,489],{},[21,482,483],{},"Client:"," Goka uses ",[13,486,259],{"href":487,"rel":488},"https:\u002F\u002Fgithub.com\u002FIBM\u002Fsarama",[17],". Kafka version and consumer-group behavior follow the Sarama release you pin.",[434,491,492,495,496,240,499,502,503,506,507,44],{},[21,493,494],{},"Partitioning:"," Goka defaults to FNV-1a for producing records and locating keys in views. Producers writing its co-partitioned input topics must place the same key on the same partition. When integrating a producer with a different default, such as franz-go or the Java client, align the partitioners and Goka's ",[212,497,498],{},"WithHasher",[212,500,501],{},"WithViewHasher",", and, if used, ",[212,504,505],{},"WithEmitterHasher"," settings. Changing the hasher does not move records already in a topic. See ",[13,508,510],{"href":509},"\u002Fblog\u002Fkafka-golang-clients#keep-keyed-records-on-the-same-partitions","Keep keyed records on the same partitions",[434,512,513,516,517,521],{},[21,514,515],{},"Maintenance:"," The ",[13,518,520],{"href":519},"#github-repository-metrics","repository snapshot"," shows 7 commits and 2 merged PRs in 90 days, with PR merges spread across three people. Its 30 open issues have a median age of 3.4 years.",[10,523,524,527],{},[21,525,526],{},"When to choose:"," Use Goka when the service must keep per-key state that survives process failure and is partitioned with a consumer group. Use a Go Kafka client when each record is independent. Use Watermill when the hard part is application choreography, not Kafka tables.",[407,529,27],{"id":530},"watermill",[10,532,533,537,538,541,542,546,547,240,549,551],{},[13,534,27],{"href":535,"rel":536},"https:\u002F\u002Fwatermill.io\u002F",[17]," builds event-driven Go services. You publish and subscribe to ",[212,539,540],{},"message.Message"," values, then use routers and CQRS components to build workflows such as sagas. ",[13,543,265],{"href":544,"rel":545},"https:\u002F\u002Fgithub.com\u002FThreeDotsLabs\u002Fwatermill-kafka",[17]," is the Kafka adapter; it uses Sarama (",[212,548,229],{},[212,550,233],{},", consumer groups). The same handlers can sit on NATS, RabbitMQ, Redis streams, or SQL.",[10,553,554],{},[21,555,429],{},[431,557,558,564,573,598,604],{},[434,559,560,563],{},[21,561,562],{},"Kafka is a transport:"," watermill-kafka publishes and consumes through Sarama. Watermill does not create changelog topics or restore aggregates from Kafka. The Kafka adapter does not support exactly-once delivery, so handlers must tolerate redelivery.",[434,565,566,568,569,572],{},[21,567,494],{}," The default marshaler publishes records without a Kafka key, so Sarama assigns them to random partitions and per-key ordering is lost. Use ",[212,570,571],{},"NewWithPartitioningMarshaler"," with a key function, or a custom marshaler, when consumers rely on per-key order or the topic must be co-partitioned with Goka tables. With a key, Sarama's default FNV-1a partitioner matches Goka's default.",[434,574,575,464,578,581,582,585,586,589,590,593,594,597],{},[21,576,577],{},"Acks and retries:",[212,579,580],{},"Ack"," marks the offset for commit. ",[212,583,584],{},"Nack"," redelivers the message after ",[212,587,588],{},"NackResendSleep"," (100 ms by default), holding back later messages in that partition. Use bounded retries with the ",[212,591,592],{},"Retry"," middleware and route unprocessable records to a poison queue with ",[212,595,596],{},"PoisonQueue"," when the partition should continue.",[434,599,600,603],{},[21,601,602],{},"Portability:"," Handlers can move to another Pub\u002FSub, but keys, consumer groups, and Schema Registry framing still need Kafka-specific configuration.",[434,605,606,608,609,611,612,44],{},[21,607,515],{}," Watermill has the broadest visible usage of the four in the ",[13,610,520],{"href":519},", but the core repository had 2 commits and 2 merged PRs in 90 days, and one account handled 95% of recorded merges. The snapshot covers the core repository, not ",[212,613,265],{},[10,615,616,618],{},[21,617,526],{}," Use Watermill when you are building sagas, CQRS, or a service that may use Kafka today and another broker later. Do not choose it to replace Kafka Streams tables.",[407,620,31],{"id":31},[10,622,623,627,628,630,631,633,634,636,637,642,643,645],{},[13,624,31],{"href":625,"rel":626},"https:\u002F\u002Fgithub.com\u002Freugn\u002Fgo-streams",[17]," is a pipeline DSL: a ",[212,629,239],{}," emits, ",[212,632,243],{}," stages transform (map, filter, throttle, and tumbling, sliding, or session windows), and a ",[212,635,246],{}," consumes. The ",[13,638,641],{"href":639,"rel":640},"https:\u002F\u002Fgithub.com\u002Freugn\u002Fgo-streams\u002Fblob\u002Fv0.13.0\u002Fexamples\u002Fkafka\u002Fmain.go",[17],"Kafka example"," uses the Sarama-based ",[212,644,271],{}," module as source and sink. Other connectors cover Pulsar, NATS, Redis, Aerospike, cloud storage, WebSocket, and files.",[10,647,648],{},[21,649,429],{},[431,651,652,658,673,679],{},[434,653,654,657],{},[21,655,656],{},"Windows are in-process:"," A five-second tumbling window lives in that Go process. If the process stops, the window is gone unless you persist it yourself. That is not Goka's Kafka-backed table and not Kafka Streams changelog restore.",[434,659,660,516,663,668,669,672],{},[21,661,662],{},"Offsets and delivery:",[13,664,667],{"href":665,"rel":666},"https:\u002F\u002Fgithub.com\u002Freugn\u002Fgo-streams\u002Fblob\u002Fv0.13.0\u002Fkafka\u002Fkafka_sarama.go",[17],"Sarama source"," marks each message as consumed before passing it downstream, so Sarama's auto-commit can record the offset before the flow or sink finishes. If the process stops, records in flight or buffered in a window can be lost rather than redelivered. The Sarama sink also logs a failed ",[212,670,671],{},"SendMessage"," and moves on, so an output failure can lose a record even while the process keeps running.",[434,674,675,678],{},[21,676,677],{},"Kafka's role:"," Connector I\u002FO through Sarama. When the sink forwards a consumed Kafka message, it copies the key and value but not the headers. go-streams does not add a Streams topology or group table.",[434,680,681,516,683,685],{},[21,682,515],{},[13,684,520],{"href":519}," shows no commits or merged PRs in 90 days, and the latest GitHub Release, v0.13.0, dates from May 2025. One account handled all recorded merges.",[10,687,688,690],{},[21,689,526],{}," Use go-streams for an in-process pipeline with Kafka at the edges when the built-in connectors' delivery behavior is acceptable, or when you provide your own source and sink with the offset and error handling you need. Use Goka when state must survive restarts. Use Watermill when you need messaging patterns beyond a linear flow.",[407,692,35],{"id":693},"redpanda-connect",[10,695,696,700,701,706],{},[13,697,35],{"href":698,"rel":699},"https:\u002F\u002Fgithub.com\u002Fredpanda-data\u002Fconnect",[17]," is a stream processor configured in YAML: inputs read from systems such as Kafka, processors transform messages with the Bloblang mapping language, and outputs write the results. It started as Benthos; Redpanda acquired the project in 2024 and renamed it Redpanda Connect. It usually runs as its own binary or container, and Go programs can embed it or add plugins through the ",[13,702,704],{"href":209,"rel":703},[17],[212,705,214],{}," API.",[10,708,709],{},[21,710,429],{},[431,712,713,731,739,750,770,788],{},[434,714,715,718,719,721,722,724,725,728,729,44],{},[21,716,717],{},"Kafka clients:"," Use the ",[212,720,278],{}," input and output, which run on franz-go. Since v4.68.0, the ",[212,723,271],{}," (Sarama) and ",[212,726,727],{},"kafka_franz"," inputs and outputs are deprecated and will be removed in the next major version; their functionality moved into ",[212,730,278],{},[434,732,733,735,736,738],{},[21,734,451],{}," With at-least-once inputs and outputs, the normal pipeline acknowledges messages after output delivery without persisting them to disk in transit. A buffer such as ",[212,737,326],{}," can weaken that guarantee by intentionally dropping expired windows under back pressure.",[434,740,741,516,743,745,746,749],{},[21,742,494],{},[212,744,278],{}," output uses Murmur2 for keyed records by default and has no FNV-1a option. Its ",[212,747,748],{},"key"," field is optional, so set it when Goka needs per-key routing. Before writing co-partitioned input topics, check that the output's partition placement matches Goka's configured hasher.",[434,751,752,516,755,761,762,769],{},[21,753,754],{},"Windows and state:",[13,756,759],{"href":757,"rel":758},"https:\u002F\u002Fdocs.redpanda.com\u002Fredpanda-connect\u002Fcomponents\u002Fbuffers\u002Fsystem_window\u002F",[17],[212,760,326],{}," buffer groups messages into tumbling or sliding windows in memory, by processing time by default. Messages in an unfinished window are not acknowledged, so after a restart they are consumed again and window boundaries may differ. Under sustained back pressure, the buffer can intentionally drop expired windows, so not every input record is guaranteed to reach an output. Caches hold key-value state; the ",[13,763,766,768],{"href":764,"rel":765},"https:\u002F\u002Fdocs.redpanda.com\u002Fredpanda-connect\u002Fcomponents\u002Fcaches\u002Fredpanda\u002F",[17],[212,767,278],{}," cache"," stores it in a Kafka topic but rescans the partition on reads, so its documentation recommends a compacted topic and an in-memory cache in front.",[434,771,772,775,776,781,782,787],{},[21,773,774],{},"Licensing:"," Most components are Apache-2.0; enterprise features use the ",[13,777,780],{"href":778,"rel":779},"https:\u002F\u002Fgithub.com\u002Fredpanda-data\u002Fconnect\u002Fblob\u002Fv4.111.0\u002Flicenses\u002FREADME.md",[17],"Redpanda Community License",". The ",[13,783,786],{"href":784,"rel":785},"https:\u002F\u002Fgithub.com\u002Fwarpstreamlabs\u002Fbento",[17],"Bento"," fork continues from the codebase before the license change.",[434,789,790,516,792,794],{},[21,791,515],{},[13,793,520],{"href":519}," shows the most activity of the four: 156 commits and 153 merged PRs in 90 days, 83 GitHub Releases in 12 months, and merges spread across 15 people. Its open issue and PR backlogs are also the largest.",[10,796,797,799],{},[21,798,526],{}," Use Redpanda Connect when the job is moving and transforming data between Kafka and other systems, and you prefer configuring a pipeline service over writing processing code. Use Goka when per-key state must be co-partitioned with the input and restored from Kafka.",[46,801,803],{"id":802},"github-repository-metrics","GitHub repository metrics",[10,805,806,807,811],{},"The tables below compare the four projects' GitHub activity, maintenance, and public usage using the same measurement windows and assessment rules as the ",[13,808,810],{"href":809},"\u002Fblog\u002Fkafka-golang-clients#github-repository-metrics","Go client comparison",". Snapshot: 27 Sep 2026.",[10,813,814,815,818,819,821],{},"Watermill figures cover the core ",[212,816,817],{},"ThreeDotsLabs\u002Fwatermill"," repository, not the separate ",[212,820,265],{}," adapter.",[407,823,825],{"id":824},"project-overview","Project overview",[51,827,828,856],{},[54,829,830],{},[57,831,832,835,840,846,851],{},[60,833,834],{},"Attribute",[60,836,837],{},[13,838,23],{"href":414,"rel":839},[17],[60,841,842],{},[13,843,27],{"href":844,"rel":845},"https:\u002F\u002Fgithub.com\u002FThreeDotsLabs\u002Fwatermill",[17],[60,847,848],{},[13,849,31],{"href":625,"rel":850},[17],[60,852,853],{},[13,854,35],{"href":698,"rel":855},[17],[70,857,858,875,894],{},[57,859,860,863,866,869,872],{},[75,861,862],{},"Repository created",[75,864,865],{},"28 Mar 2017",[75,867,868],{},"8 Nov 2018",[75,870,871],{},"30 Apr 2019",[75,873,874],{},"22 Mar 2016",[57,876,877,880,883,886,888],{},[75,878,879],{},"License",[75,881,882],{},"BSD-3-Clause",[75,884,885],{},"MIT",[75,887,885],{},[75,889,890,891],{},"Apache-2.0; enterprise features under the ",[13,892,780],{"href":778,"rel":893},[17],[57,895,896,899,902,904,906],{},[75,897,898],{},"Repository status",[75,900,901],{},"Active · original",[75,903,901],{},[75,905,901],{},[75,907,901],{},[407,909,911],{"id":910},"activity","Activity",[51,913,914,929],{},[54,915,916],{},[57,917,918,921,923,925,927],{},[60,919,920],{},"Metric",[60,922,23],{},[60,924,27],{},[60,926,31],{},[60,928,35],{},[70,930,931,948,965,982,999,1016,1033],{},[57,932,933,936,939,942,945],{},[75,934,935],{},"Latest default-branch commit",[75,937,938],{},"24 Sep 2026",[75,940,941],{},"25 Aug 2026",[75,943,944],{},"14 Jan 2026",[75,946,947],{},"25 Sep 2026",[57,949,950,953,956,959,962],{},[75,951,952],{},"Latest GitHub Release",[75,954,955],{},"v1.1.17 (6 Aug 2026)",[75,957,958],{},"v1.5.3 (25 Aug 2026)",[75,960,961],{},"v0.13.0 (11 May 2025)",[75,963,964],{},"v4.111.0 (25 Sep 2026)",[57,966,967,970,973,976,979],{},[75,968,969],{},"GitHub Releases (12 mo)",[75,971,972],{},"3",[75,974,975],{},"2",[75,977,978],{},"0",[75,980,981],{},"83",[57,983,984,987,990,993,996],{},[75,985,986],{},"Commits",[75,988,989],{},"7 (90 d) · 12 (12 mo)",[75,991,992],{},"2 (90 d) · 20 (12 mo)",[75,994,995],{},"0 (90 d) · 2 (12 mo)",[75,997,998],{},"156 (90 d) · 1,059 (12 mo)",[57,1000,1001,1004,1007,1010,1013],{},[75,1002,1003],{},"Issue flow (90 d)",[75,1005,1006],{},"0 opened · 0 closed",[75,1008,1009],{},"3 opened · 1 closed",[75,1011,1012],{},"1 opened · 0 closed",[75,1014,1015],{},"18 opened · 5 closed",[57,1017,1018,1021,1024,1027,1030],{},[75,1019,1020],{},"PR flow (90 d)",[75,1022,1023],{},"11 opened · 2 merged",[75,1025,1026],{},"5 opened · 2 merged",[75,1028,1029],{},"0 opened · 0 merged",[75,1031,1032],{},"282 opened · 153 merged",[57,1034,1035,1038,1041,1044,1047],{},[75,1036,1037],{},"Activity assessment",[75,1039,1040],{},"🟡 7 commits and 2 merged PRs in the last 90 days.",[75,1042,1043],{},"🟡 2 commits and 2 merged PRs in the last 90 days.",[75,1045,1046],{},"🔴 No commits and no merged PRs in the last 90 days.",[75,1048,1049],{},"🟢 156 commits and 153 merged PRs in the last 90 days.",[10,1051,1052],{},"Release figures count GitHub Releases, not Go module tags.",[407,1054,1056],{"id":1055},"maintenance","Maintenance",[51,1058,1059,1073],{},[54,1060,1061],{},[57,1062,1063,1065,1067,1069,1071],{},[60,1064,920],{},[60,1066,23],{},[60,1068,27],{},[60,1070,31],{},[60,1072,35],{},[70,1074,1075,1091,1108,1125,1141,1158,1175,1188,1205],{},[57,1076,1077,1080,1083,1086,1088],{},[75,1078,1079],{},"Active commit authors (12 mo)",[75,1081,1082],{},"4",[75,1084,1085],{},"12",[75,1087,975],{},[75,1089,1090],{},"57",[57,1092,1093,1096,1099,1102,1105],{},[75,1094,1095],{},"PR merge distribution (12 mo)",[75,1097,1098],{},"3 people · Top 1: 60% · Top 2: 80%",[75,1100,1101],{},"2 people · Top 1: 95% · Top 2: 100%",[75,1103,1104],{},"1 person · 100% of merges",[75,1106,1107],{},"15 people · Top 1: 47% · Top 2: 72%",[57,1109,1110,1113,1116,1119,1122],{},[75,1111,1112],{},"Issue backlog",[75,1114,1115],{},"30 open · median age 3.4 y",[75,1117,1118],{},"78 open · median age 3.1 y",[75,1120,1121],{},"5 open · median age 1 y",[75,1123,1124],{},"190 open · median age 2.6 y",[57,1126,1127,1130,1133,1136,1138],{},[75,1128,1129],{},"Issue closure rate",[75,1131,1132],{},"N\u002FA — no issues in measurement window",[75,1134,1135],{},"1\u002F3 closed within 30 d · 2\u002F3 within 90 d (small sample)",[75,1137,1132],{},[75,1139,1140],{},"11\u002F27 closed within 30 d · 12\u002F27 within 90 d",[57,1142,1143,1146,1149,1152,1155],{},[75,1144,1145],{},"PR backlog",[75,1147,1148],{},"8 open · median age 61 d",[75,1150,1151],{},"6 open · median age 74 d",[75,1153,1154],{},"7 open · median age 311 d",[75,1156,1157],{},"156 open · median age 135 d",[57,1159,1160,1163,1166,1169,1172],{},[75,1161,1162],{},"Median PR merge time (90 d)",[75,1164,1165],{},"8 d (n=2 — small sample)",[75,1167,1168],{},"7.1 h (n=2 — small sample)",[75,1170,1171],{},"N\u002FA — no merged PRs in window",[75,1173,1174],{},"11.6 h (n=153)",[57,1176,1177,1180,1182,1184,1186],{},[75,1178,1179],{},"Published GitHub security advisories",[75,1181,978],{},[75,1183,978],{},[75,1185,978],{},[75,1187,978],{},[57,1189,1190,1193,1196,1199,1202],{},[75,1191,1192],{},"Responsiveness assessment",[75,1194,1195],{},"🟡 Median open PR age 61 d exceeds 60 d.",[75,1197,1198],{},"🟡 Median open PR age 74 d exceeds 60 d.",[75,1200,1201],{},"🔴 No PRs merged in 90 d, while 7 open PRs have a median age of 311 d.",[75,1203,1204],{},"🟡 Median open PR age 135 d exceeds 60 d; 12 of 27 issues (44%) closed within 90 d.",[57,1206,1207,1210,1213,1216,1219],{},[75,1208,1209],{},"PR merge concentration assessment",[75,1211,1212],{},"🟢 3 people merged PRs in 12 mo; the most active account handled 60% of merges.",[75,1214,1215],{},"🟡 2 people merged PRs in 12 mo; the most active account handled 95% of merges.",[75,1217,1218],{},"🔴 1 person merged PRs in 12 mo and handled 100% of merges.",[75,1220,1221],{},"🟢 15 people merged PRs in 12 mo; the most active account handled 47% of merges.",[10,1223,1224,1225,1228],{},"PR merge distribution counts non-bot ",[212,1226,1227],{},"mergedBy"," accounts, so automated merges may undercount human reviewers. Issue closure rates use issues opened 90–180 days before the snapshot, giving each issue a full 90-day window.",[407,1230,1232],{"id":1231},"public-usage-and-interest","Public usage and interest",[51,1234,1235,1249],{},[54,1236,1237],{},[57,1238,1239,1241,1243,1245,1247],{},[60,1240,920],{},[60,1242,23],{},[60,1244,27],{},[60,1246,31],{},[60,1248,35],{},[70,1250,1251,1268,1285,1301],{},[57,1252,1253,1256,1259,1262,1265],{},[75,1254,1255],{},"Stars",[75,1257,1258],{},"2,542",[75,1260,1261],{},"9,906",[75,1263,1264],{},"2,173",[75,1266,1267],{},"8,769",[57,1269,1270,1273,1276,1279,1282],{},[75,1271,1272],{},"Forks",[75,1274,1275],{},"185",[75,1277,1278],{},"507",[75,1280,1281],{},"174",[75,1283,1284],{},"971",[57,1286,1287,1290,1292,1295,1298],{},[75,1288,1289],{},"GitHub dependents (Used by)",[75,1291,1281],{},[75,1293,1294],{},"1,566",[75,1296,1297],{},"46",[75,1299,1300],{},"None listed",[57,1302,1303,1306,1309,1312,1315],{},[75,1304,1305],{},"Public usage assessment",[75,1307,1308],{},"🟡 Mixed public usage and interest: 174 dependents, 185 forks, and 2,542 stars.",[75,1310,1311],{},"🟢 Strong public usage and interest: 1,566 dependents, 507 forks, and 9,906 stars.",[75,1313,1314],{},"🟡 Mixed public usage and interest: 46 dependents, 174 forks, and 2,173 stars.",[75,1316,1317],{},"🟢 Strong interest: 971 forks and 8,769 stars; GitHub lists no dependents.",[10,1319,1320],{},"GitHub dependents are approximate public-repository counts. They count repositories that declare the project as a dependency, so deployments of Redpanda Connect as a standalone binary or container are not included.",[407,1322,1324],{"id":1323},"overall-repository-signals","Overall repository signals",[431,1326,1327,1332,1337,1342],{},[434,1328,1329,1331],{},[21,1330,23],{}," has modest recent activity, with PR merges spread across three people; its open issues are old.",[434,1333,1334,1336],{},[21,1335,27],{}," has the broadest visible usage of the four, but little recent activity, and one account handled 95% of recorded PR merges in the past year.",[434,1338,1339,1341],{},[21,1340,31],{}," had no commits or merged PRs in the last 90 days, and one account handled all recorded PR merges in the past year.",[434,1343,1344,1346],{},[21,1345,35],{}," is by far the most active, with frequent releases and merges spread across 15 people; its open issue and PR backlogs are large.",[10,1348,1349],{},"These are maintenance and adoption signals, not evidence of runtime quality or performance.",[46,1351,1353],{"id":1352},"test-go-stream-processing-with-kafma","Test Go stream processing with Kafma",[10,1355,1356],{},"A stream processor that passes unit tests can still write the wrong key, lose state after a restart, or stall while it restores. Those problems show up in topics and consumer-group positions, not in the processor's return values.",[10,1358,1359,1362,1363,1367],{},[13,1360,1361],{"href":474},"Kafma"," is a desktop Kafka UI that runs beside your Go service or pipeline. Use its ",[13,1364,1366],{"href":1365},"\u002Ffeatures\u002Fkafka-console","Kafka console"," to drive input topics, inspect output and state topics, and watch the processor's consumer group.",[407,1369,1371],{"id":1370},"drive-input-with-keyed-test-records","Drive input with keyed test records",[10,1373,1374,1375,1379,1380,1383,1384,1388,1389,1393],{},"Use the ",[13,1376,1378],{"href":1377},"\u002Fdocs\u002Fconsole\u002Fproducing-messages","producer panel"," to send records with the keys that should update the same table entry or fall into the same window. ",[21,1381,1382],{},"Loop"," keeps records arriving at a fixed interval, which makes window boundaries visible. Then open the output topic and check keys, headers, and payload encoding. With Schema Registry configured, Kafma ",[13,1385,1387],{"href":1386},"\u002Fdocs\u002Fconsole\u002Fdecoding-messages","decodes"," Confluent-framed records. Its ",[13,1390,1392],{"href":1391},"\u002Ffeatures\u002Fkafka-schema-registry","Schema Registry UI"," lists subjects, versions, and their schema IDs.",[10,1395,1396,1397,1400,1401,1403],{},"Kafma's ",[21,1398,1399],{},"Auto"," partitioning does not follow Goka's default FNV-1a hasher. For topics that Goka reads as co-partitioned input, send test input through a Goka ",[212,1402,423],{},", or set the partition explicitly after computing the one Goka would choose.",[407,1405,1407],{"id":1406},"inspect-state-topics","Inspect state topics",[10,1409,1410,1411,1413,1414,1416],{},"For Goka, open the group's ",[212,1412,340],{}," topic. The latest record for each key is the state a new instance will restore, and a tombstone deletes that key. Compare it with the output topic after the same input. For Redpanda Connect's ",[212,1415,278],{}," cache, inspect the cache topic the same way.",[10,1418,1419],{},"A Goka processor updates the table partition that matches the input partition it processed, so comparing an input topic with the group's own table does not prove keys were hashed correctly. To check co-partitioning, compare the partition of a key in an independently produced input topic with the same key in the table that Goka joins, then confirm the view or join result.",[407,1421,1423],{"id":1422},"restart-and-failure-tests","Restart and failure tests",[10,1425,1426,1427,1431,1432,1436],{},"Open ",[13,1428,1430],{"href":1429},"\u002Fdocs\u002Fconsole\u002Fconsuming-messages#live-and-watch-group-modes","Watch Group"," on the processor's consumer group, then restart an instance or add a second one. A Goka processor blocks input processing until its tables recover, so lag grows during restore and should fall once processing resumes. Member status and committed positions show how long recovery takes. Kafma reads this state without joining the group, triggering a rebalance, or committing offsets. The ",[13,1433,1435],{"href":1434},"\u002Ffeatures\u002Fkafka-consumer-groups","consumer group UI"," shows lag and partition assignments for each member as ownership moves.",[10,1438,1439,1440,1443,1444,1446],{},"To test restarts, send inputs with a unique ID in each payload, such as an ",[212,1441,1442],{},"event_id"," field. Configure the test path to emit one output per input and copy that ID into the output. Stop and restart the processor partway through, wait for the group to catch up in Watch Group, then compare input and output IDs. Duplicate IDs are consistent with at-least-once processing; missing IDs need investigation, and go-streams' early offset marking is one possible cause. Leave ",[212,1445,326],{}," pipelines out of this one-to-one test, because the buffer can drop expired windows.",[10,1448,1449,1450,1452],{},"For Watermill, send a record the handler rejects. With the default ",[212,1451,584],{}," loop, the committed position for that partition stops advancing and later records stay pending in Watch Group.",[10,1454,1455],{},[1456,1457],"img",{"alt":1458,"height":1459,"src":1460,"width":1461},"Kafma watching pending and consumed records around a consumer group's committed offsets",1368,"https:\u002F\u002Fmedia.kafma.app\u002Fdocs\u002Fconsole\u002Fconsuming-messages\u002Fwatch-group.png",1562,[407,1463,1465],{"id":1464},"replay-production-shaped-input","Replay production-shaped input",[10,1467,1468,1472,1473,1477],{},[13,1469,1471],{"href":1470},"\u002Ffeatures\u002Fkafka-data-clone","Kafma Data Clone"," copies a selected range of real records to a test topic, including keys, headers, and tombstones, so a processor can be tested against older schema versions and edge cases that fixtures miss. Copied records receive new offsets and are repartitioned by the target producer, so their placement may not match Goka's hasher. Use clones to test decoding and per-record handling, not Goka's co-partitioned joins. See the ",[13,1474,1476],{"href":1475},"\u002Fdocs\u002Fdata-clone\u002Foverview","Data Clone guide"," for copy behavior.",[10,1479,1480],{},[1456,1481],{"alt":1482,"height":1483,"src":1484,"width":1485},"Kafma cloning selected topics and masked records into a test cluster",1564,"https:\u002F\u002Fmedia.kafma.app\u002Fchangelog\u002Fv1.0.0\u002Fhomepage-data-clone.png",2740,[10,1487,1488,1492],{},[13,1489,1491],{"href":1490},"\u002Fdownload","Download Kafma"," and connect to the same test cluster as the application.",[46,1494,1496],{"id":1495},"frequently-asked-questions","Frequently asked questions",[407,1498,1500],{"id":1499},"does-go-have-apache-kafka-streams","Does Go have Apache Kafka Streams?",[10,1502,1503],{},"No. Kafka Streams is a Java (and Scala) library. Goka is the usual Go stand-in when you need Kafka-backed processor state. It is not an API-compatible port.",[407,1505,1507],{"id":1506},"is-watermill-a-kafka-streams-alternative","Is Watermill a Kafka Streams alternative?",[10,1509,1510],{},"No. Watermill is a messaging and application toolkit. Its Kafka module publishes and subscribes through Sarama. It does not manage Kafka Streams-style state stores or changelog recovery.",[407,1512,1514],{"id":1513},"is-redpanda-connect-a-kafka-streams-alternative","Is Redpanda Connect a Kafka Streams alternative?",[10,1516,1517],{},"Partly. It covers transformation, routing, enrichment, and in-memory windows between Kafka and other systems, and it runs as its own service. It does not provide Kafka Streams' co-partitioned state stores or changelog restore. Use Goka when per-key state must be restored from Kafka.",[407,1519,1521],{"id":1520},"what-happened-to-benthos","What happened to Benthos?",[10,1523,1524,1525,1530,1531,787],{},"Redpanda ",[13,1526,1529],{"href":1527,"rel":1528},"https:\u002F\u002Fwww.redpanda.com\u002Fpress\u002Fredpanda-acquires-benthos",[17],"acquired Benthos"," in 2024 and renamed it Redpanda Connect. Most components remain Apache-2.0, while enterprise features use the Redpanda Community License. The ",[13,1532,786],{"href":784,"rel":1533},[17],[407,1535,1537],{"id":1536},"do-any-of-these-provide-exactly-once-processing","Do any of these provide exactly-once processing?",[10,1539,1540],{},"None of the four documents Kafka Streams-style exactly-once processing. Goka and Watermill's Kafka adapter deliver at least once, as does Redpanda Connect in a normal pipeline whose inputs and outputs support at-least-once delivery. go-streams' built-in Kafka connectors can lose records. For at-least-once paths, make state updates and side effects safe to retry. For go-streams, review when offsets are marked and how sink errors are handled before relying on delivery guarantees.",[407,1542,1544],{"id":1543},"does-goka-support-windowing","Does Goka support windowing?",[10,1546,1547,1548,1550,1551,1553,1554,1557,1558,1561,1562,1564],{},"Not as a built-in operator. Goka provides group tables, ",[212,1549,362],{},", and ",[212,1552,366],{},", but no timer or punctuation API. Store per-key window state in the group table and close a window when something triggers the callback: a later record for that key, a message you emit on a schedule, or a scheduled call to ",[212,1555,1556],{},"VisitAll"," through an experimental ",[212,1559,1560],{},"Visitor"," edge, which can block rebalances and shutdown. For in-memory windows without Kafka-backed state, see go-streams or Redpanda Connect's ",[212,1563,326],{}," buffer.",[407,1566,1568],{"id":1567},"what-if-i-need-managed-windows-and-joins-outside-a-go-service","What if I need managed windows and joins outside a Go service?",[10,1570,1571,1572,1577],{},"Use a dedicated engine such as Flink or Spark Structured Streaming, which manages state and checkpoints for the job. In the Go ecosystem, ",[13,1573,1576],{"href":1574,"rel":1575},"https:\u002F\u002Fgithub.com\u002Fnumaproj\u002Fnumaflow",[17],"Numaflow"," runs streaming jobs on Kubernetes, and Apache Beam's Go SDK runs pipelines on a runner such as Flink or Dataflow.",[407,1579,1581],{"id":1580},"is-there-a-kafka-streams-style-dsl-for-go","Is there a Kafka Streams-style DSL for Go?",[10,1583,1584,1589],{},[13,1585,1588],{"href":1586,"rel":1587},"https:\u002F\u002Fgithub.com\u002Ftryfix\u002Fkstream",[17],"tryfix\u002Fkstream"," and related forks aim at a Java Streams-like DSL. They are much smaller than Goka and are not in this comparison. Treat them as experimental unless you have verified maintenance and operations for your own workload.",[407,1591,1593],{"id":1592},"can-i-keep-using-a-go-kafka-client-with-these-libraries","Can I keep using a Go Kafka client with these libraries?",[10,1595,1596,1597,1599,1600,1602,1603,44],{},"Yes. Goka, Watermill's Kafka adapter, and go-streams' Kafka module use Sarama. Redpanda Connect's ",[212,1598,278],{}," components use franz-go; its deprecated ",[212,1601,271],{}," input and output still use Sarama. Other services can use franz-go, kafka-go, Sarama, or confluent-kafka-go on the same topics as long as serialization matches. Producers writing the same keyed records must also use compatible partitioners; see ",[13,1604,510],{"href":509},[46,1606,1608],{"id":1607},"conclusion","Conclusion",[10,1610,1611],{},"For Kafka-backed aggregates and recoverable per-key state in Go, start with Goka. Choose Watermill for event-driven handlers when Kafka is one of several transports. Use go-streams for an in-process pipeline when the built-in Kafka connectors' record-loss risk is acceptable. Use Redpanda Connect for configured pipelines that move and transform data between Kafka and other systems.",[10,1613,1614,1615,1617],{},"Before connecting these tools through keyed topics, check partitioning: Goka defaults to FNV-1a, Redpanda Connect's ",[212,1616,278],{}," output uses Murmur2, and Watermill's default Kafka marshaler sends no key unless you configure a partitioning marshaler.",[10,1619,1620],{},"If you are porting a Java Kafka Streams topology, keep it on the JVM when API compatibility matters. Otherwise, redesign windows, joins, and recovery for the Go tool you choose, then test delivery across restarts before replacing the existing pipeline.",[10,1622,1623],{},"This guide is maintained by the team behind Kafma.",{"title":1625,"searchDepth":1626,"depth":1626,"links":1627},"",3,[1628,1630,1631,1637,1644,1650,1661],{"id":48,"depth":1629,"text":49},2,{"id":136,"depth":1629,"text":137},{"id":146,"depth":1629,"text":147,"children":1632},[1633,1634,1635,1636],{"id":409,"depth":1626,"text":23},{"id":530,"depth":1626,"text":27},{"id":31,"depth":1626,"text":31},{"id":693,"depth":1626,"text":35},{"id":802,"depth":1629,"text":803,"children":1638},[1639,1640,1641,1642,1643],{"id":824,"depth":1626,"text":825},{"id":910,"depth":1626,"text":911},{"id":1055,"depth":1626,"text":1056},{"id":1231,"depth":1626,"text":1232},{"id":1323,"depth":1626,"text":1324},{"id":1352,"depth":1629,"text":1353,"children":1645},[1646,1647,1648,1649],{"id":1370,"depth":1626,"text":1371},{"id":1406,"depth":1626,"text":1407},{"id":1422,"depth":1626,"text":1423},{"id":1464,"depth":1626,"text":1465},{"id":1495,"depth":1629,"text":1496,"children":1651},[1652,1653,1654,1655,1656,1657,1658,1659,1660],{"id":1499,"depth":1626,"text":1500},{"id":1506,"depth":1626,"text":1507},{"id":1513,"depth":1626,"text":1514},{"id":1520,"depth":1626,"text":1521},{"id":1536,"depth":1626,"text":1537},{"id":1543,"depth":1626,"text":1544},{"id":1567,"depth":1626,"text":1568},{"id":1580,"depth":1626,"text":1581},{"id":1592,"depth":1626,"text":1593},{"id":1607,"depth":1629,"text":1608},"2026-09-27","Compare Goka, Watermill, go-streams, and Redpanda Connect for Kafka stream processing in Go: what each provides and when to use it.","md",{},true,null,"\u002Fblog\u002Fkafka-streams-in-golang",false,{"title":5,"description":1663},{"loc":1668},"blog\u002Fkafka-streams-in-golang",[1674,1675],"Kafka Stream Processing","Golang","QtfJl83cQRmCWZObpHd5uMfPq367fZt747Zqj-I9Bp4",1790518188550]