Mask Sensitive Data in Kafka Messages

Masking changes selected fields before Kafma writes a cloned record to the target. Use it when production or shared test data contains values that should not reach a local cluster unchanged.

Kafma warns when a production source has no masking rules. You still need to identify every sensitive field that applies to your data.

Add a masking rule

Select Add in the Mask section and enter a rule in this form:

topic:field.path

For example, users:contact.email applies to the contact.email field in the message value on the users topic. One rule applies to one topic; add separate rules when the same field appears on multiple topics.

Rules apply to message values, not keys or headers.

Adding a masking rule for a Kafka message value

Hash, Redact, or Drop

Action	Result	Use it when
Hash	Replaces a string with a 16-character pseudonym	You need to correlate records without keeping the original value
Redact	Replaces a string with `***`	The original value is unnecessary
Drop	Sets the selected field value to `null`	The schema permits a null value

Hash output is stable for the same input during one clone run. Kafma uses a new salt for each run, so the pseudonym is not stable across separate clones. The result is a one-way pseudonym, not encryption.

Despite its name, Drop does not remove the field or the whole record. It sets the field to null. If the schema does not allow null, pre-flight warns that the rule cannot be applied safely.

Supported formats and field paths

Masking supports schema-backed Avro, JSON Schema, and Protobuf message values. Kafma decodes the value, applies the rules, and re-encodes it for the target Schema Registry.

Field selectors use dot-separated object paths such as customer.address.street. Array indexes, wildcards, and recursive selectors are not supported.

Masking cannot be combined with Raw bytes because fields are not available until the payload is decoded.

Round-trip limits

Masking decodes and re-encodes the entire message value. During this round trip, Avro int64 values outside JavaScript's safe integer range may lose precision, and Avro bytes or fixed fields may not round-trip exactly. Large JSON Schema numbers outside that range may also be rounded. Kafma reports the relevant codec warning in the execution log.

Validate masking rules in pre-flight

Run pre-flight after adding or changing rules. Kafma samples the selected topics and checks that:

The topic is still selected.
The field path exists.
Hash and Redact target string fields.
Drop targets a field that can accept null.
The message format can be decoded and re-encoded.

Mask rows appear below their topics in the execution plan. Start Clone remains disabled until every mask warning is resolved.

Reviewing masking rules in the Data Clone pre-flight plan

What happens when masking fails

Kafma does not fall back to sending the original value. If a field has the wrong type or the masked value cannot be re-encoded, Kafma drops that record and reports the failure in the execution log. If the field is absent or already null, Kafma skips that mask rule and continues cloning the record.

This prevents a requested field from reaching the target unmasked, but it can produce a partial result. Check the copied and failed message counts, then review the execution log before using the cloned data.