- Functional Demonstration
The KafkaConnector component is used to send messages to the Kafka Broker and retrieve messages from it. The KafkaConnector component can work either in synchronous or asynchronous mode to send the messages to the Kafka Broker.
Component Configuration panel in the CPS (Configuration Property Sheet) allows customization in the component.
Figure 1: KafkaConnector CPS
The way in which the Kafka Connector acts; either as a Producer or a Consumer.
Selecting this mode makes the connector act as a producer, to send messages as Topics to the Kafka Broker.
Selecting this mode makes the connector act as a consumer, to consume messages from the Kafka Broker.
This mode is enabled only when the Connector Mode is set to "Send.".
When this mode is selected, the Kafka Connector acting as a producer sends requests and waits for the server reply. In this case, errors arising, if any, will be logged in the component and sent to the Exception port as per the configuration set in the Error Handling Configuration panel.
When this mode is selected, the Kafka Connector acting as a producer sends requests and does not wait for the server reply. The callback interface is used to log error messages in error logs and not sent to the Exception port.
The connection URL of the Kafka Connector which is being used to send or receive the messages.
Poll Interval (in ms)
The time interval in milliseconds until which the Kafka Connector acting as the consumer fetches messages from the Kafka Broker.
The name of the topic through which the messages are sent by the producer or through which the messages are consumed by the consumer.
The default properties in the additional properties of the Kafka consumer (Connector Mode - Receive) can be changed as per requirement.
Figure 2: Additional Properties dialog box for Kafka consumer (Connector Mode - Receive) with prepopulated data
Please refer to the link below to add additional properties than the default ones to the Kafka Consumer:
The default properties in the additional properties of the Kafka consumer (Connector Mode - Send) can be changed as per requirement.
Figure 3: Additional Properties dialog box for Kafka consumer (Connector Mode - Send) with prepopulated data
Please refer to the link below to add additional properties than the default ones to the Kafka Producer:
Enable the Expert Properties view to configure these properties.
Figure 4: KafkaConnector CPS with the Expert Properties enabled and highlighted
Error handling configuration
Pre Processing XSL Configuration
Pre Processing XSL configuration can be used to transform request message before processing it. Click the ellipses button against the property to configure the properties.
Refer to the Pre/Post Processing XSL Configuration section under the Common Configurations page for details regarding Pre Processing XSL configuration and Post Processing XSL configuration (below).
Post Processing XSL Configuration
Post Processing XSL configuration can be used to transform the response message before sending it to the output port.
Process Message Based on Property
The property helps components to skip certain messages from processing.
Refer to the Process Message Based On a Property section under the Common Configurations page.
This property is used when there is a need to process messages in parallel within the component, still maintaining the sequence from the external perspective.
Refer to the Threadpool Configuration section under the Common Configurations page.
Configure two KafkaConnector microservices as producer and consumer following descriptions in the Component description and send a message. Connect a Feeder microservice to KafkaProducer and connect the KafkaProducer output and the KafkaConsumer input.
Connect Display microservices to the Kafka microservice output ports to view the respective outputs.
Figure 5: Event process to demonstrate the functions of KafkaProducer and KafkaConsumer
In this example, the input injected using Feeder is sent via the KafkaProducer, which is in turn received by the KafkaConsumer; the message will be displayed on Consumer_Display.
Provide the following input in the Feder text field:
Figure 6: Feeder input sent to the KafkaProducer
KafkaProducer Display window displays the acknowledgment message.
Figure 7: Output received on the KafkaProducer display
KafkaConsumer Display window displays the message recived from the KafkaProducer.
Figure 8: Output received on the KafkaConsumer display