How Does XMPro Data Streams Scale in Azure

To understand how Data Streams scale it is important to look at the Deployment diagram which illustrates various components of the solution and how they interact together:

Azure Development

When the solution is deployed on Azure at an infrastructure level all Web Applications are hosted in Azure Web Apps, all databases are hosted as Azure SQL Databases and Cloud based StreamConsists of a combination of Stream Objects allowing real-time data to flow through. It is created to address a particular use case. Hosts are hosted as Azure Worker Roles. Following is an Infrastructure Diagram for XMPro Data Streams on Azure:

Scalability at Infrastructure Level

At infrastructure level the prime responsibility to scale and load balance shifts to the Infrastructure Provider which in this case is assumed to be Azure.

• Azure Web App
  • Azure auto scales and load balances
• Azure SQL Database
  • Azure auto scales
• Azure Worker Role
  • Azure auto scales (vertical scaling only)

Scalability at Solution Level

Scalability of a solution that is creating using XMPro Data Streams platform is dependent on following aspects

Design of the Use Case

Design of a Use Case and client’s business requirements pay an integral part to the final scalability of the solution. For instance, a Use Case designed to handle bulk processing on a Low powered device will not scale very well; whereas, a distributed Use Case which processes Data listening on a low powered but shifts the rest of the data processing e.g. transformationIs a Stream Objects that alters the shape and form of data., contextualization, analytics and enterprise alert system integration etc. to a Plant or Cloud Stream HostIs a client application which hosts/executes the Use Cases designed by the user in the IoT Portal. will scale a lot better.

Processing Power of Stream Hosts

Processing power of the hardware where Stream Hosts are deployed cannot be ignored in reference to a solution’s scalability capacity. A host with more memory and CPU power will not only be able to process event/messages a lot quicker but also be more resilient to unexpected load or congestion.

Messaging Insfrastructure

When a use Case involves more than one CollectionIs a virtual grouping of Stream Hosts.  All Stream Hosts in a collection are broadcasted the same instructions from the IoT Portal.  This way a Use Case can be mass deployed to all the Stream Hosts in a given Collection. i.e. data is to be transferred from one Host to another over a network, the scalability of the Messaging infrastructure used between the Host is integral to overall performance of the solution. Each Stream Host can be configured to use a certain Messaging Solution e.g. Azure EventHub, Apache ActiveMQ, MQTT etc.