Applying AI to Existing Platforms From AI as a Service by Peter Elger and E\u00f3in Shanaghy This article discusses how to add AI to existing platforms. You can save 40% off AI as a Service, as well as of all other Manning books and videos.\u00a0 Just enter the code psflow40 at checkout when you buy from manning.com.\u00a0 Integration Patterns for Serverless AI You can\u2019t escape the fact that real-world enterprise computing is \u201cmessy\u201d. For a medium to large enterprise, the technology estate is typically large, sprawling and has often grown organically over time. An organization\u2019s compute infrastructure can be broken down along domain lines such as Finance, HR, Marketing, Line of Business systems and so on. Each of these domains may be comprised of many systems from various vendors along with home grown software and they usually mix legacy with more modern software as a service (SaaS-)delivered applications. Concomitant to this, the various systems are typically operated in a hybrid model mixing on premise, co-location and cloud-based deployment. Furthermore, each of these operational elements must typically integrate with other systems both in the domain and outside of the domain. These integrations can be by way of batch ETL jobs, point-to-point connections or through some form of Enterprise Service Bus (ESB) ETL, Point to point and ESB Enterprise system integration is a large topic area which we won\u2019t cover here except to note that there are a number of ways in which system can be connected together. For example, a company may need to export records from its HR database to match up with an expense tracking system. Enterprise Transform and Load (ETL) refers to the process of exporting records from one database, typically in CSV format, transforming and then loading into another database. Another method of connecting systems is to use a point-to-point integration, for example some code can be created to call the API of one system and push data to another systems API. This depends on the provision of a suitable API. Over time the use of ETL and point-to-point integration can accumulate into a complex and difficult to manage system. An Enterprise Service Bus (ESB) is an attempt to manage this complexity by providing a central system over which these connections can take place. The ESB approach suffers from its own particular pathologies and often causes as many problems as it solves. Figure1 illustrates a typical mid-size organization\u2019s technology estate. Figure 1. Typical enterprise technology estate, broken down by logical domain. This image is intended to illustrate the complex nature of a typical technology estate. The detail of the architecture is not important. In this example, separate domains are connected together through a central bus. Within each domain there are separate ETL and batch processes connecting system together. Needless to say, a description of all of this complexity is outside the scope of this article. The question we\u2019ll concern ourselves with is how we can adopt Serverless AI in this environment. Fortunately, there are some simple patterns which we can follow to achieve our goals, but firstly let\u2019s simplify the problem. We\u2019ll use Figure 2 to represent our \u201centerprise estate\u201d in the following discussion, to allow us to treat the rest of our infrastructure as a black box. In the next section we\u2019ll examine four common patterns for connecting AI Services. For example, if part of a companies\u2019 business workflow requires proof of identity through provision of a utility bill or passport, which can be provided as an AI enabled service reducing the manual workload. Another example is that of forecasting, many organizations need to plan ahead to predict their required levels of inventory or staff over a given period of time. AI services could be integrated into this process to build more sophisticated and accurate models saving the company money or opportunity costs. Figure 2. Simplified Enterprise Representation We\u2019ll examine four approaches: \tPattern 1: Synchronous API \tPattern 2: Asynchronous API \tPattern 3: VPN Stream in \tPattern 4: VPN Fully connected streaming Bear in mind that these approaches represent ways of getting the appropriate data into the required location to enable us to execute AI services to achieve a business goal. Pattern 1 Synchronous API The first and simplest approach is to create a small system in isolation from the rest of the enterprise. Functionality is exposed through a secured API and accessed over the public internet. If a higher level of security is required then a VPN connection can be established through which the API can be called. This simple pattern is illustrated in Figure 3. Figure 3. Integration Pattern 1 Synchronous API In order to consume the service, a small piece of bridging code must be created in the estate to call the API and consume the results of the service. This pattern is appropriate when results can be obtained quickly and the API is called in a request\/response manner. Pattern 2 Asynchronous API Our second pattern is similar, in that we expose functionality through an API, but in this case the API acts asynchronously. This pattern, which is appropriate for longer running AI services, is illustrated in Figure 4. Figure 4. Integration Pattern 2 Asynchronous API Under this \u201cfire and forget\u201d model, the bridge code calls the API but doesn\u2019t receive results immediately, apart from status information. An example of this might be a document classification system that processes a large volume of text. The outputs of the system can potentially be consumed by the wider enterprise in a number of ways: \tBy constructing a web application that users can interact with to see results. \tBy the system messaging the results through email or other channels. \tBy the system calling an external API to forward on details of any analysis \tBy the bridge code polling the API for results Pattern 3 VPN Stream In A third approach is to connect the estate to cloud services through a VPN. Once a secure connection is established the bridge code can interact more directly with cloud service. For example, rather than using an API gateway to access the system, the bridge code could stream data directly into a Kinesis pipeline. Results can be accessed in a number of ways; though an API, via outbound messaging, through a web GUI or via an output stream. This is illustrated in Figure 5. Figure 5. Integration Pattern 3 Stream In VPN A Virtual Private Network (VPN) can be used to provide a secure network connection between devices or networks. VPNs typically use the IPSec protocol suite to provide authentication, authorization and secure encrypted communications. By using IPSec, insecure protocols such as those used for file sharing, can be used securely between remote nodes. A VPN can be used to provide secure access into a corporate network for remote workers, or to securely connect a corporate network into the cloud. Although there are a number of ways to setup and configure a VPN, we recommend a serverless approach using the AWS VPN service. Pattern 4 VPN Fully Connected Streaming Our final pattern involves a much deeper connection between the estate and cloud AI services. Under this model we establish a VPN connection as before and use it to stream data in both directions. Although there are several streaming technologies available, we\u2019ve had good results using Apache Kafka. This is illustrated in Figure 6. Figure 6. Integration Pattern 4 Full Streaming This approach involves operating a Kafka cluster on both ends of the VPN and replicating data between the clusters. Within the cloud environment services consume data by pulling from the appropriate Kafka topics and place results back onto a different topic for consumption by the wider enterprise. Kafka Apache Kafka is an open source, distributed streaming platform. Kafka was originally developed at LinkedIn and later donated to the Apache Foundation. Although there are other streaming technologies available, Kafka\u2019s design is quite unique in that it\u2019s implemented as a distributed commit log. Kafka is increasingly being adopted in high throughput data streaming scenarios by companies such as Netflix and Uber. It\u2019s possible to install, run and manage your own Kafka cluster, but we recommend that you take the serverless approach and adopt a system such as AWS Managed Streaming for Kafka (MSK). A full discussion of the merits of this approach and Kafka in general is outside the scope of this article. If you aren\u2019t familiar with Kafka we recommend that you take a look at the Manning book Kafka in Action to get up to speed. Which Pattern? As with all architectural decisions, the approach to take depends on the use case. Our guiding principle is to keep things as simple as possible. If a simple API integration achieves the goal then go with that. If over time the external API set begins to grow then consider changing the integration model to a streaming solution to avoid the proliferation of APIs. The key point is to keep the integration to AI services under constant review and be prepared to refactor as needs dictate. Table 1 summarizes the context and when each pattern should be applied.\u00a0 Pattern Context Example 1 - Synchronous API Single Service, fast response Text Extraction from document 2 - Asynchronous API Single Service, longer running Document transcription 3 - VPN Stream In Multiple services, results for human consumption Sentiment analysis pipeline 4 - VPN Fully Connected Multiple services, results for machine consumption Batch translation of documents Table 1. Applicability of AI as a Service legacy integration patterns That\u2019s all for now. If you want to learn more about the book, check it out on our browser-based liveBook reader here.