How Can We Help?
When designing a Data StreamConsists of a combination of Stream Objects allowing real-time data to flow through. It is created to address a particular use case., you may decide to run some of the stream’s logic on one host or set of Stream Hosts, and some on another. For instance, if there is a need to monitor sensor readings and escalate any anomalies, it may be desirable to filter out the important data close to the source to avoid constant unnecessary data transfer.
In such cases, the boundaries between the Stream Hosts must be bridged to allow data to flow between them. “Remote” agents are the mechanism that the Data Stream DesignerIs a model-driven approach to connect, transform and action real-time “streaming” data in a visual, drag and drop manner. uses for this task.
Every Category of Stream Hosts can nominate one “Remote Publisher” and one “Remote Receiver.” This will be inserted into a stream at every point a Stream ObjectIs a reusable component that functions as a building block for a stream. There are different types available: action agents, listeners, transformations, context providers and functions. assigned to that Category receives data from a Stream at every point a Stream Object assigned to that Category receives data from a Stream Object in another category. Remote Agents do not need to be configured on all Categories; only on those that expect to receive data from another host.
Remote Agents must be configured using the Category blade and should be paired in such a way that the Category’s Receiver will be able to receive the data from its own Publisher. For instance, if you decided to use the MQTT ListenerIs a Stream Object that is responsible to ingest data from sensors and third party systems. as the Receiver, you should always use an MQTT Action AgentIs a Stream Object that is responsible to consume events in a stream to perform possible internal or external/third-party actions (e.g. data warehouse updates, sending notifications etc.). Once an event is consumed by an Action Agent, its relative response is pushed back onto the stream, upon which further processing is done (if required). as the Publisher and configure both with the same server details. If you do not, any data sent by the Publisher would not be received and the data stream would be broken.
To better understand the process, as well as potential pitfalls, it may help to visualize the process as two separate streams, separated along the boundary. The Data Stream Designer will insert the configured Publisher into the stream to run on the “sending” host, like so:
and will similarly insert the configured Receiver into the stream to run on the “receiving” host:
Comments are closed.