@objectivebyaptean/kafka-consumer-manual-commit 1.0.0

This pallet gives the possibility to consume kafka messages, but without committing them. The messages will be committed manually with another node.

npm install @objectivebyaptean/kafka-consumer-manual-commit

node-red-contrib-kafka-manual-commit

Custom Node-RED nodes for consuming Kafka messages without automatic offset commits, allowing explicit control over when offsets are committed.

This module provides:

• a Kafka consumer node that receives messages
• a Kafka commit node that explicitly commits offsets later in the flow
• a shared Kafka broker configuration node

Features

• Manual offset management (no auto-commit)
• Kafka-native offset behavior:
  • New consumer groups start from latest
  • Existing consumer groups resume from last committed offset
• Explicit commit control using a dedicated node
• Clean separation between message processing and acknowledgment
• Built on KafkaJS
• Designed for Node-RED best practices

Nodes

Kafka Consumer

Consumes messages from a Kafka topic and outputs them into a Node-RED flow without committing offsets automatically.

Behavior

• If the configured group.id has committed offsets, consumption resumes from that offset
• If the group.id has no committed offsets, consumption starts from the latest message
• Messages are replayed after restart until explicitly committed

Outputs Each message contains Kafka metadata required for committing:

msg.payload   // message value (string or null)
msg.topic     // Kafka topic
msg.kafka = {
  consumerId,
  topic,
  partition,
  offset,
  timestamp
}