Friday, December 26, 2014

Reactive Microservice Part 2

Reactive Microservice Architecture

Both reactive service architecture and microservice architecture has certain aspects that are similar, building a system that are easy to develop/deploy, loosely-coupled. While reactive service focuses more on resilient and microservice focus on the services itself. In this section, I’ll try to go over the 4 traits of reactive systems and discuss how it would affect us building a microservice.

Responsive

The system responds in a timely manner if at all possible. Responsiveness is the cornerstone of usability and utility, but more than that, responsiveness means that problems may be detected quickly and dealt with effectively. Responsive systems focus on providing rapid and consistent response times, establishing reliable upper bounds so they deliver a consistent quality of service. This consistent behaviour in turn simplifies error handling, builds end user confidence, and encourages further interaction.”
Key note about rapid and consistent response time:
  • Predictable performance at 99%ile trumps low mean latency
  • Tail latencies far more important than mean or median
  • Client protects itself with asynchronous, non-blocking calls
Using events and async operations quickly becomes a trend, not just in javascript but also in java and other languages. However, a lot of async implementation are clumsy and suffers callback hell, thus the adoption of Reactive extension in companies like Netflix.
Since we are talking async and concurrency, let’s clarify the difference between async, concurrent & parallel.
  • Async just mean non-blocking, specifically in reference to i/o operations (not necessarily parallel, can be sequential)
  • Concurrent means multiple operations happening at the same time (not necessary parallel)
  • Parallel operation is when multiple operations processed simultaneously.
Async programming is difficult, error handling is complicated and often bugged with memory leak, race condition, complex state machine, uncaught async error. And especially when adopting microservice, we essentially introduce a lot of small services each talking to one or more services in a graph relationship rather than a tier relationship. That raises the need to interact with multiple services asynchronously and some local synchronous operations (e.g. reading from a memory buffer) to execute a request. One example is Netflix has an API gateway which talks to UI and various services, instead of having the UI those services directly. That reduce the amount of remote API call but put the responsibility to the API gateway which has way lower network latency and reduce a lot of overhead in authentication or user metadata.
However, that creates an abstract concurrency requirement in the API gateway.
Not just async, some of the operations could be sync. To easily handle these combination of async operations and sync operations, we’ll need to have a unified way to handle async and sync operations and Reactive Extension turns out to be a good match.

Reactive extension

Reactive extension (Rx) is introduced by Microsoft and adopted by Netflix who ported it to Java and now it’s available in multiple different languages. The basic idea of Rx is to combine the Iterator pattern (which consumer to pull value out of the data source, e.g. looping over an array) and observer pattern (which publisher to push events to consumer, e.g. registering an event with a callback.)
Iterator pattern and Observer pattern are symmetrical
Map, filter & concatAll can be implemented over any iterators and observer pattern can be think of as producer iterates consumer. However, the original gang of 4 misses this symmetry and gave them different semantics, thus Rx brought the two together so they have the same semantic, observable. And one thing that the observer pattern from the gang of 4 didn’t specify is for the producer to tell consumer that there’s no more data, with this, the observable can unsubscribe for us, no need to have unsubscribe event anymore. An Observable is a collection span over time. It can model events, animations and async server requests.
Observable === collection + time
Promise and Future can’t be cancelled and can only return a single value. Observable can be cancelled (by using takeUntil, or switchLatest) and can return any number of values. Netflix is using observable on both UI & server for async operation. Observable is just like a promise that can be cancel & retry, can send you 0, 1 or infinite values but it’s easier to handle concurrency and have no callback hell or states to maintain.
More importantly, it decouples the consumption from production as regardless if the producer is sync or async, local or remote, using event loop, or actor, they are all wrapped in an async observable and is composed in the same way. Consumer do not have direct dependency on the implementation of the producer.
It also clearly communicates the network cost. In a large team, it’s very easy to have something looks trivial but it ends up making a lot of expensive network calls. Also each of the methods could have different implementation, it could be a BIO network call, NIO network call, collapsed network call or local cache and we can change the implementation without affecting the consumer.
Retrieval, transformation, combination are all done in the same declarative manner.

Solving Async Problem with Rx

Most of the async problem can be solved by just picking a right flattening strategy: concatAll, mergeAll, switchLatest. Here’s how they works.
ConcatAll simply concat all the observable, note that the 4th observable has to wait for the 2nd observable to finish before appending to the resultant observable and that can’t be accomplished without the observable indicating that it’s done. Also, the empty observable will just disappeared.
TakeUntil is a way to stop subscribing to the source collection. E.g. mouseMoves.takeUntil(mouseUp). Both mouseMoves and mouseUp are observables where mouseMoves contains every mouse move and mouseUp only contains the mouse up. With TakeUntil, we’ll stop subscribing to the mouseMoves collection when mouseUp has data.
MergeAll is merging all collections as the time the events come in. It’s like lane merge in traffic.
SwitchLatest flatten the collections but unlike concatAll, it won’t wait for the completion of a collection before moving on to the next, it simply unsubscribe from it and when another observable start firing events. Thus, no more state machine as observable knows when it’s done, provider can unsubscribe the even and no need to explicitly unsubscribe event anymore.
Stop thinking in terms of loops but use maps to solve parallelism or concurrency. In Javascript 7, probably we’ll have multithreading support, but instead of letting developer to create a new thread, it’ll be using map. Since it’s unordered, it can be sent to multiple core and process them in parallel. And in Java 8, Lambda already support threading in similar fashion. Below is an example of how RxJava handles concurrency with Schedulers.

Example: Netflix Search

In this example, it’s building the type ahead search for Netflix. Key presses collection is throttled for 250 ms so a short burst of key strokes are grouped together as a collection. That produce a 2D collection (or a collection of collections). Then it’s mapped to a function to call the server to execute the search which will retry 3 times until the next burst of data comes in to keyPresses (another collection, as it’s throttled) and ended the current collection. The result is a 2D collection of search result collection. It’s then concatAll, which make sure the search results comes in order of the key presses. E.g. a user search for “apple iphone” with a little pause between “apple” and “iphone”, so after throttle and map, the search API is called twice, “apple” and “apple iphone”. Now assume before all results from searching for “apple” returns, the user finished typing “iphone”. The result from searching “apple” will stop immediately after “iphone” is entered. And once the UI finished showing the “incomplete” search result for “apple”, it’ll show the search result for “apple iphone”. However, if the search result for “apple” return even after searching “apple iphone”, probably because of too many hits to the search term “apple”, the resultant collection will contains the result for “apple” before that of “apple iphone”. Thus, we won’t wrongly show the result of “apple” as it comes after “apple iphone” and overwritten the result. Note that if we use switchLatest, the search result for “apple” will be dropped (cancelled) completely and only show that of “apple iphone”, which is a more desirable user experience.

Example: Automated batching

In this example, we can getting the bookmark and rating for each of the video in the user’s catalog. If we’re to kick off a network call for each of the invocation, it’ll perform horribly. Under the cover, it take advantage that it’s async so that the code runs very quickly.
Even those we are making hundreds of call, we can capture those requests in 1 or 2 ms window and batch calling the backend in one network call. The benefit is that the developer can develop the natural way but the underlining layer can optimize based on how the resource are located so if it’s over the network, we can do auto batching or streaming, if it’s in memory, we can simply retrieving it synchronously.

Adopting Reactive Extension

The benefit of adopting Rx
  1. Concurrency and async are not-trivial. Rx doesn’t trivialize it. Rx is powerful and rewards those who go through the learning curve.
  2. Observable async multiple value (future/promise only one)
  3. Abstract concurrency, non-opinionated concurrency.
  4. Decouple consumer from production.
  5. Powerful composition of nested, conditional flows
  6. First-class support of error handling, scheduling & flow control1.

1 Flow control will be discussed in scaling.