November 30th 2020
By Almir Hamza, Software Engineer and Haris Imamović, Software Engineer
Building web services that meet modern demands in terms of robustness, scalability, and resilience is surely a challenging endeavor. Today's web services are expected to serve multiple different applications within an organization, handle large numbers of simultaneous requests, provide near-instantaneous responses, and all of that with almost no downtime. This is especially accentuated in distributed systems built on microservice architecture.
Understanding Reactive Programming
In a microservice architecture, a group of relatively small loosely-coupled services communicate with each other, as well as other system components such as databases, or other external systems. In most cases, these interactions are synchronous and blocking in nature. In Spring Web MVC, it is assumed that applications can block the current thread, (for example, for remote calls). For this reason, servlet containers use a large thread pool to absorb potential blocking during request handling1. Basically, each incoming request will be assigned to a new thread from the server's thread pool for handling the request, thus alleviating the blocking issue.
Figure 1. Thread-per-request concurrency model abstraction
However, this model is subject to the issue that most of the interactions within a single thread are still blocking. A significant amount of time is spent on context switching and waiting for I/O operations to finish. As services face more and more requests, the thread pool can become completely occupied, which would cause the following requests to wait in the queue until the next thread is available. In such cases, a possible next step could be to allocate additional hardware resources and scale horizontally but that also doesn’t come without a cost.
This is the primary motivation for adopting reactive approach and building reactive web services. In Spring WebFlux (and non-blocking servers in general), it is assumed that applications do not block. Therefore, non-blocking servers use a small, fixed-size thread pool (event loop workers) to handle requests1. The default embedded server in Spring WebFlux applications is Reactor Netty which is built on the event loop concurrency model and which is capable of handling concurrency at large scales with a smaller number of threads.
Basically, each request is handled by the event loop which processes events from the event queue, and delegates blocking I/O tasks to new worker threads. Worker threads perform long blocking tasks and after completion, they signal the event loop about completed tasks. The event loop can trigger the callback on the operation completion and return the response back to the caller. The event loop runs in a single thread, although we can have as many event loops as the number of available cores.
Figure 2. Event loop concurrency model abstraction
The key expected benefit of reactive and non-blocking is the ability to scale with a small, fixed number of threads and less memory1. In terms of performance, this will not necessarily make your applications magically run faster, but under great load, you can expect more resilient behavior, provided you have some latency caused by slow running operations (for example, remote calls with slow network I/O). Also, the good news is that there’s no need for an alarm - your application is probably fine. In this context, we are talking about large-scaled systems with thousands of simultaneous requests, so there’s no need to jump and rewrite any relatively small application using some new technologies.
Building Reactive Web Services
Reactive programming with Spring WebFlux is based on the declarative composition of asynchronous logic around data streams and events. Essentially what that means is that we are defining steps that will eventually get executed asynchronously, after some blocking operation completes, via registered callbacks. This requires some getting-used-to because program flow is not the same compared to the traditional imperative programs.
Let's take a look at how we can use Spring WebFlux framework to create a really simple distributed system that observes incoming bank transactions and provides details about each transaction. This example can give us a good idea of how we can combine different components of Spring Reactive Web stack.
Figure 3. Structure of a simple system for handling bank transactions
We use the transactions-generator service to create a stream of transactions and publish records for each corresponding transaction to Kafka topic. Every transaction record is consumed by the transactions-details service. Upon receiving a transaction event this service gets details about source and destination accounts from account-details service and maps all information to the corresponding response. Account information is persisted in MongoDB. Instead of building a reactive client, we will check response data using Postman. You can check out the complete project on GitHub repository.
There are three key components to this system which we will cover:
Why Apache Kafka
Reactive Systems rely on asynchronous message-passing to establish a boundary between components that ensures loose coupling, isolation and location transparency. Employing explicit message-passing enables load management, elasticity, and flow control by shaping and monitoring the message queues in the system and applying back-pressure when necessary5. With Kafka, we are capable of handling an infinite stream of data coming in from an external source, with multiple consumers which can be attached to a topic and removed on an ad-hoc basis.
In this project, we use Reactive Kafka API3 to publish and consume data from Kafka topic using functional API with non-blocking back pressure and low overheads. This basically allows reactive Kafka to be integrated with other reactor systems and provide an end-end reactive pipeline.
We created a simple transaction-generator service to simulate real-world transactions since we are not focusing on how these transactions are actually created within the distributed system, but we just want to show how to integrate reactive Kafka producer within the reactive service. Each transaction holds some basic information like amount and source/destination account IBANs.
Figure 4. Model of Transaction event which is published to Kafka topic
These transaction objects are continuously generated and sent over a defined Kafka topic to simulate a real-world transaction stream. In order to enable Kafka producer, we need to set up some basic configuration properties like bootstrap servers, client id, and key/value serializer and deserializer.
Figure 5. Kafka sender configuration
With config out of the way, we can now use KafkaSender to publish messages to Kafka topics. Reactive Kafka API enables us to simply define all steps in the processing of new transactions such as mapping to Kafka records, defining callbacks for sending and error handling, etc. The transaction generator component creates a stream of Transaction objects which are then mapped to producer records and emitted to the topic.
Figure 6. Kafka sender publishing of transaction records
These transaction records are consumed by transactions-details service which subscribes to the same Kafka topic and listens for the incoming transactions. In order to set up a Kafka receiver, we need to provide similar configuration properties. Without going into much detail the following image shows which configuration properties we used to set up the KafkaReceiver component.
Figure 7. Kafka receiver configuration
Once the required configuration options have been set up, a new KafkaReceiver instance can start to consume inbound messages. The code below creates a receiver instance and creates an inbound Flux for the receiver. Component which implements the logic for receiving and serializing incoming messages is the KafkaReceiverService.
Figure 8. Usage of KafkaReceiver to consume inbound messages
Getting account details
To give more meaning to each received transaction we need to enrich the transaction object with details about both involved accounts. For handling account information and details we created a service called accounts-details.
As shown below, this service offers a simple REST API for fetching Account objects by their IBAN code. Account details controller looks pretty much like a standard REST controller, it supports HTTP operations, takes path and query parameters, and serializes responses as JSON values. But the difference between a classical controller and Reactive controller is that the latter wraps its response data with Flux or Mono components. Spring WebFlux also provides an alternative to the annotation-based programming model using a functional programming model in which functions are used to route and handle requests and contracts are designed for immutability. In our example, we will use annotation-based controllers.
Figure 9. Account-details controller implementation
Spring WebFlux, or better say, it’s fully non-blocking foundation Reactor, provides the Mono and Flux API types to work on data sequences through a rich set of operators. These data types effectively wrap and emit their containing elements, enabling the exchange of data between threads with well-defined memory usage, avoiding unnecessary intermediate buffering or blocking.
After the request is received we need to actually gather the data from some storage. Spring Framework 5 supports reactive drivers for many noSQL databases like MongoDB, Redis, and Couchbase, and also some relational databases with R2DBC. In this example project, we used a very simple approach with in-memory MongoDB.
In order to use the reactive MongoDB repository we first need to include corresponding dependencies. For this example, we will just use embedded MongoDB.
Figure 10. Gradle dependencies for reactive MongoDB
Next step is to define a persistence data model that will represent our bank accounts.
Figure 11. Model for persisting Account information
Finally, in order to perform data access operations, we will create a reactive repository interface.
Figure 12. Reactive repository for Account database model
Now that we have shown how the account data is fetched from the database, let’s take a look at how this service is actually used by transactions-details service. In order to fetch account information, we need to make an API call using some sort of web/http client. Libraries like the RestTemplate or OpenFeign are, by nature, blocking. Consequently, there is no point in using them in a reactive setup. For reactive applications, Spring introduced the WebClient interface representing the main entry point for performing web requests, in a non-blocking manner.
In order to call endpoint on account-details service, we added an AccountDetailsService component which relies on WebClient to execute the call and get the response body as a reactive component.
Figure 13. API call to get account using WebClient
Creating a response to the client
After describing different components and interactions between them, let’s show how it all comes together inside of the transactions-details service. In order to get transaction details we need to make a call to the /transactions endpoint. This endpoint returns a Flux of TransactionDetails data and internally it’s using KafkaReceiver service to observe incoming transactions, and AccountDetails service to load details about interacting accounts. After all these steps, assembled transaction data can be continuously returned to the caller.
Figure 14. TransactionDetails controller implementation
What’s especially interesting is that the response media type of this endpoint is text/event-stream. This media type is an official media type for Server Sent Events (SSE). SSE is a server push technology that allows a client to automatically receive updates from the server over HTTP connection. Unlike WebSockets which offer a full-duplex communication channel, SSE represents only unidirectional communication from server to client. It’s mostly intended for browsers to enable receiving events and updates at any time.
Inside the getAllTransactionsWithAccountDetails method transactions, received from the Kafka receiver, are enriched with their matching source and destination accounts. For each transaction, two separate API calls will be made to fetch both accounts. Previously we showed how a single HTTP call can be executed with WebClient. For executing multiple non-blocking operations and combining their results Reactor API provided a static method zip on Mono and Flux objects.
Figure 15. Non-blocking fetching of multiple account details
What this method does is that it merges all the given monos, which in this case are results of the getAccount calls, into a new Mono that will be fulfilled after all given monos have produced a value. The last parameter is the combinator function which aggregates the values produced by the given monos and returns the new resulting Mono. All this has executed in a non-blocking reactive manner.
The following image shows the responses from the transactions-details service, containing a list of transactions with their respective accounts information.
Figure 16. Transaction-details response example
Within this post we have shown some basic examples of how to apply reactive programming to implement a simple distributed system commonly found in the fintech world.
Spring WebFlux is not a solution to all of your problems, you shouldn’t rewrite your existing Spring Web MVC application just for the sake of it. It makes sense to use a reactive approach in situations when you have a large scale distributed system and you need to drain every last bit of performance and rely on services to handle load via backpressure control, or you can even use it to reinforce just some critical parts of the system - since microservice architecture allows having a mix of technologies.
Reactive programming brings some overhead during development due to the complexity of the program flow and its asynchronous nature, therefore debugging can get really tricky and painful or sometimes near impossible.
In this article we only covered a small subset of features provided by the Spring Webflux framework, there are a lot of different features to discover within the Spring Reactive Web ecosystem, more curious readers can refer to links provided within the resources section.