You have a privilege to create a quiz (QnA) related to this subject and obtain creativity score...
Machine Learning, Data Fabric, and IoT
Machine Learning (ML) is a branch of Artificial Intelligence (AI).
The main idea is that a system can process data - a lot of data - to figure out the model or the basic patterns that help analysis of the data. This "learning" can improve efficiency of the decision making processes, for example, predictive analytics, marketing decisions, risk analysis, voice or image processing, etc.
While learning, people often change their goals and priorities of what and how we learn.
Somewhat similar happens - at least we should expect this happening - in ML.
To find the best model ML systems often work simultaneously with several competing models.
Think of them as several different applications processing same data.
A relatively new ML area is ML on graphs. Graphs is perfect to capture the relationships between many things (e.g. the connections between family members) collectively provide vital information, and graph databases make this easy to analyze.
Check this reference.
This requirement of making data available to multiple processes in their raw, original form, set a relatively new concept of data fabric.
Data Fabric provides services for consistent access to data on-premises and in the cloud.
Another requirement is making data persistent, so they can be processed more than once, for example with changeable model parameters. And still another important requirement is dealing with data streams.
Apache Kafka and Apache Spark are two open source projects most commonly used to manage stream data.
There are several vendors, like IBM, NetApp, Talend. and more offering a complete Data Fabric solution. For example, NetApp provides access to Amazon Web Services (AWS) data cloud.
Let us talk about processing data. While Service-Oriented Architecture (SOA) focuses on reuse of services, Microservices continue this line towards service independence, so Microservices can be reused across multiple different applications and environments.
The most common way of service invocation is Web-based REST services.
To process data streams, it is more advantages to use Stream-based Microservices.
This is about Publish-Subscribe messaging model.
In this case the message data are published to a proper Topic and consumers read the data from the Topic whenever consumers need that data.
Here is the sample of the sequence of events during the processing:
Was it clear so far?
1. A data message can be delivered via the REST API to a Kafka Topic.
2. A consumer process will read it from the topic when it needs it for processing.
Internet of Things (IoT) is one of the industries where Machine Learning are often used to make sense from the streams of data coming from multiple sensors.
Think of the streams of stock prices or competing product commercials, weather sensors or multiple video cameras with immediate per-processing by Rasberry Pi and post-processing with the powerful NVDIA processors in the cloud.
