Reactive Microservice Architecture
Both reactive
service architecture and microservice architecture has certain
aspects that are similar, building a system that are easy to
develop/deploy, loosely-coupled. While reactive service focuses more
on resilient and microservice focus on the services itself. In this
section, I’ll try to go over the 4 traits of reactive systems and
discuss how it would affect us building a microservice.
Responsive
“The system responds
in a timely manner if at all possible. Responsiveness is the
cornerstone of usability and utility, but more than that,
responsiveness means that problems may be detected quickly and
dealt with effectively. Responsive systems focus on providing rapid
and consistent response times, establishing reliable upper bounds
so they deliver a consistent quality of service. This consistent
behaviour in turn simplifies error handling, builds end user
confidence, and encourages further interaction.”
Key note about
rapid and consistent response time:
- Predictable performance at 99%ile trumps low mean latency
- Tail latencies far more important than mean or median
- Client protects itself with asynchronous, non-blocking calls
Using events and
async operations quickly becomes a trend, not just in javascript but
also in java and other languages. However, a lot of async
implementation are clumsy and suffers callback hell, thus the
adoption of Reactive extension in companies like Netflix.
Since we are
talking async and concurrency, let’s clarify the difference between
async, concurrent & parallel.
- Async just mean non-blocking, specifically in reference to i/o operations (not necessarily parallel, can be sequential)
- Concurrent means multiple operations happening at the same time (not necessary parallel)
- Parallel operation is when multiple operations processed simultaneously.
Async programming
is difficult, error handling is complicated and often bugged with
memory leak, race condition, complex state machine, uncaught async
error. And especially when adopting microservice, we essentially
introduce a lot of small services each talking to one or more
services in a graph relationship rather than a tier relationship.
That raises the need to interact with multiple services
asynchronously and some local synchronous operations (e.g. reading
from a memory buffer) to execute a request. One example is Netflix
has an API gateway which talks to UI and various services, instead of
having the UI those services directly. That reduce the amount of
remote API call but put the responsibility to the API gateway which
has way lower network latency and reduce a lot of overhead in
authentication or user metadata.
However, that
creates an abstract concurrency requirement in the API gateway.
Not just async,
some of the operations could be sync. To easily handle these
combination of async operations and sync operations, we’ll need to
have a unified way to handle async and sync operations and Reactive
Extension turns out to be a good match.
Reactive extension
Reactive
extension (Rx) is introduced by Microsoft and adopted by Netflix who
ported it to Java and now it’s available in multiple different
languages. The basic idea of Rx is to combine the Iterator pattern
(which consumer to pull value out of the data source, e.g. looping
over an array) and observer pattern (which publisher to push events
to consumer, e.g. registering an event with a callback.)
Iterator pattern and Observer pattern are
symmetrical
Map, filter &
concatAll can be implemented over any iterators and observer pattern
can be think of as producer iterates consumer. However, the original
gang of 4 misses this symmetry and gave them different semantics,
thus Rx brought the two together so they have the same semantic,
observable. And one thing that the observer pattern from the gang of
4 didn’t specify is for the producer to tell consumer that there’s
no more data, with this, the observable can unsubscribe for us, no
need to have unsubscribe event anymore. An Observable is a
collection span over time. It can model events, animations and async
server requests.
Observable === collection + time
Promise and
Future can’t be cancelled and can only return a single value.
Observable can be cancelled (by using takeUntil, or switchLatest) and
can return any number of values. Netflix is using observable on both
UI & server for async operation. Observable is just like a
promise that can be cancel & retry, can send you 0, 1 or infinite
values but it’s easier to handle concurrency and have no callback
hell or states to maintain.
More importantly,
it decouples the consumption from production as regardless if the
producer is sync or async, local or remote, using event loop, or
actor, they are all wrapped in an async observable and is composed in
the same way. Consumer do not have direct dependency on the
implementation of the producer.
It also clearly
communicates the network cost. In a large team, it’s very easy to
have something looks trivial but it ends up making a lot of expensive
network calls. Also each of the methods could have different
implementation, it could be a BIO network call, NIO network call,
collapsed network call or local cache and we can change the
implementation without affecting the consumer.
Retrieval, transformation, combination are
all done in the same declarative manner.
Solving Async Problem with Rx
Most of the async
problem can be solved by just picking a right flattening strategy:
concatAll, mergeAll, switchLatest. Here’s how they works.
ConcatAll simply
concat all the observable, note that the 4th observable
has to wait for the 2nd observable to finish before
appending to the resultant observable and that can’t be
accomplished without the observable indicating that it’s done.
Also, the empty observable will just disappeared.
TakeUntil is a
way to stop subscribing to the source collection. E.g.
mouseMoves.takeUntil(mouseUp). Both mouseMoves and mouseUp are
observables where mouseMoves contains every mouse move and mouseUp
only contains the mouse up. With TakeUntil, we’ll stop subscribing
to the mouseMoves collection when mouseUp has data.
MergeAll is
merging all collections as the time the events come in. It’s like
lane merge in traffic.
SwitchLatest
flatten the collections but unlike concatAll, it won’t wait for the
completion of a collection before moving on to the next, it simply
unsubscribe from it and when another observable start firing events.
Thus, no more state machine as observable knows when it’s done,
provider can unsubscribe the even and no need to explicitly
unsubscribe event anymore.
Stop thinking in
terms of loops but use maps to solve parallelism or concurrency. In
Javascript 7, probably we’ll have multithreading support, but
instead of letting developer to create a new thread, it’ll be using
map. Since it’s unordered, it can be sent to multiple core and
process them in parallel. And in Java 8, Lambda already support
threading in similar fashion. Below is an example of how RxJava
handles concurrency with Schedulers.
Example: Netflix Search
In this example,
it’s building the type ahead search for Netflix. Key presses
collection is throttled for 250 ms so a short burst of key strokes
are grouped together as a collection. That produce a 2D collection
(or a collection of collections). Then it’s mapped to a function
to call the server to execute the search which will retry 3 times
until the next burst of data comes in to keyPresses (another
collection, as it’s throttled) and ended the current collection.
The result is a 2D collection of search result collection. It’s
then concatAll, which make sure the search results comes in order of
the key presses. E.g. a user search for “apple iphone” with a
little pause between “apple” and “iphone”, so after throttle
and map, the search API is called twice, “apple” and “apple
iphone”. Now assume before all results from searching for “apple”
returns, the user finished typing “iphone”. The result from
searching “apple” will stop immediately after “iphone” is
entered. And once the UI finished showing the “incomplete”
search result for “apple”, it’ll show the search result for
“apple iphone”. However, if the search result for “apple”
return even after searching “apple iphone”, probably because of
too many hits to the search term “apple”, the resultant
collection will contains the result for “apple” before that of
“apple iphone”. Thus, we won’t wrongly show the result of
“apple” as it comes after “apple iphone” and overwritten the
result. Note that if we use switchLatest, the search result for
“apple” will be dropped (cancelled) completely and only show that
of “apple iphone”, which is a more desirable user experience.
Example: Automated batching
In this example,
we can getting the bookmark and rating for each of the video in the
user’s catalog. If we’re to kick off a network call for each of
the invocation, it’ll perform horribly. Under the cover, it take
advantage that it’s async so that the code runs very quickly.
Even those we are
making hundreds of call, we can capture those requests in 1 or 2 ms
window and batch calling the backend in one network call. The
benefit is that the developer can develop the natural way but the
underlining layer can optimize based on how the resource are located
so if it’s over the network, we can do auto batching or streaming,
if it’s in memory, we can simply retrieving it synchronously.
Adopting Reactive Extension
The benefit of
adopting Rx
- Concurrency and async are not-trivial. Rx doesn’t trivialize it. Rx is powerful and rewards those who go through the learning curve.
- Observable async multiple value (future/promise only one)
- Abstract concurrency, non-opinionated concurrency.
- Decouple consumer from production.
- Powerful composition of nested, conditional flows
- First-class support of error handling, scheduling & flow control1.
1
Flow control will be discussed in scaling.
No comments:
Post a Comment