Post-MVC part 6: Reactive Programming

Intro

Last week I ended on a cliffhanger saying React is often criticized for not being truly reactive. So this week I want to define what Reactive Programming is all about.

Reactive Programming

To define Reactive Programming we must first look at the form of programming most of us are used to, in the reactive world they call it 'Passive Programming'. The passive and reactive come from the way two 'components' communicate with each other.

In Passive Programming the relationship between two components is that one component usually controls the other component. For example if we have a Car and an Engine component, the Car object triggers the Engine component. An example in code:

Car.prototype.turnKey = function() {
  this.engine.fireUp();
}

The Engine is passive, it never actually starts until some other object explicitly starts it. In this case this is the Car.

In Reactive Programming the relationship is inversed. The Engine starts itself based on certain events. In this case the event would be the Car's turnKey. In pseudo code:

Engine.listenToKey = function(car) {
  car.onKeyTurned(() => {
    this.fireUp();
  });
}

The difference between Passive and Reactive is that the Engine is now responsible for starting itself. This maps back nicely to the the second blog post MVC and JavaScript in which we learned that Component should be isolated and responsible for themselves. This means that to understand how the Engine works we only have to read the Engine's source code.

In in Passive Programming you would have to hunt each line of code that  called fireUp() to understand when the state of Engine was mutated and by whom.

Reactive Programming also has a drawback: when you change the 'onKeyTurned'  event you will have to hunt down all 'observers' to see how the change affects them.

Reactive Programming and Passive Programming invert each other in this regard. The benefits of the one are the cons of the other and vice versa.

The proponents of Reactive Programming advocate that this is a good  trade-off because it leads to a better Separation of Concerns. It is better to write a self contained Component than a Component which is influenced non transparently by other Component.

Observables

At the heart of Reactive Programming lies the "Observable". An Observable can be looked at as a souped up observer pattern as defined by the GoF. It adds two capabilities to the observer pattern the capability to complete, and to ability to signal an error.

With the "Observable" you can listen to events, events can basically be anythings: mouse clicks, keyboard presses, new tweets, time passing, form submits, Ajax calls etc etc. The Component can then act on these evens as is appropriate for that Component.

What makes Reactive Programming so powerful is the ability to manipulate these events. I can list the operations which can be performed but that will be very abstract. I'll try to explain this concept by telling a story.

Imagine you are working at a distribution center for packages. Orders come in and packages come out to the loading bay over a conveyor belt,  were the delivery man loads them into their trucks and delivers them.

The delivery man is an observer of a conveyor belt. Whatever package  rolls of the belt he needs to drive to an address. He observes the 'end' of the conveyor belt, lets look at the start of the conveyor belt.

The first thing that is put on the conveyor belt is an order specification on paper. A robot reads that paper and performs an operation on it, it grabs all items from the order and places them on the conveyor belt. It transforms the order into the real items the customer ordered.

Further down the line another robot looks at the conveyor belt, it sees the ordered items. To fill up the trucks efficiently it tries to put the items into boxes to save time and space.

Then another robot sees the boxes and puts a label on them with the address where they need to be delivered.

Then there is the final robot takes the labeled boxes and gets them off the conveyor belt, and places them into a store. When there are enough boxes that go to the same city he will grab those boxes and send them to a truck. This prevents one driver to have to hop from city to city and makes the delivery process faster.

The package then finally arrives via the conveyor belt to the delivery man. He loads them onto his truck and delivers them.

Every robot in this story was an 'operator' which manipulated the 'stream' aka the conveyor belt. You can use a bunch of chained operators to transform and manipulate the stream to something which is consumable by the Component.

Libraries

The grand-daddy of all libraries is the ReactiveX API. It specifies what an observable stream is supposed to look like, and provides implementations in various libraries. Here's a handy website which interactively shows the operations: http://rxmarbles.com/.

The JavaScript implementation of ReactiveX is called: RxJS.

Other libraries include: Bacon.js, xstream, and Kefir.

Example: Konami Code

Lets look at an example of how to make a observable Konami Code stream. This example is written with the RxJS:

// The famous sequence of input's representing the KONAMI cheat code.
var KONAMI_CODE = ['UP', 'UP', 'DOWN', 'DOWN', 
                   'LEFT', 'RIGHT', 'LEFT', 'RIGHT', 
                   'B', 'A'];

/* 
  Takes a KeyBoardEvent and returns either a KONAMI_CODE key
  or the value null.
  
  It will return a KONAMI_CODE key when the KeyBoardEvent's key
  matches a value in the KONAMI_CODE.
  
  It will return null when the key is not part of the KONAMI_CODE.
*/
function eventToKonamiCode(event) {
  switch (event.which) {
    case 38: return 'UP';
    case 40: return 'DOWN';
    case 37: return 'LEFT';
    case 39: return 'RIGHT';
    case 66: return 'B';
    case 65: return 'A';
  }
  
  return null;
}

/*
  Checks if the codes parameter, which is an array, matches the KONAMI_CODE exactly.
*/
function isKonamiCode(codes) {
  return _.isEqual(codes, KONAMI_CODE);
}

/*
  Appends the 'code' parameter to the 'codes' parameter, if the
  code is the next part of the KONAMI_CODE sequence.
  
  If the code is not the next sequence of the KONAMI_CODE an
  empty array is returned.
*/
function codeAccumulator(codes, code) {
  // Not a normal KONAMI_CODE key so return an empty array.
  if (code === null) {
    return [];
  }
   
  codes.push(code);

  // Check if 'codes' still matches the KONAMI_CODE.
  var subKonamiCode = _.take(KONAMI_CODE, codes.length);
  if (_.isEqual(codes, subKonamiCode)) {
    return codes;
  } else {
    return []; // No match back to square one.
  }
}

var konamiStream = Rx.Observable.fromEvent(document, 'keydown')
  .map(eventToKonamiCode)
  .scan(codeAccumulator, [])
  .filter(isKonamiCode);

var livesSpan = document.querySelector('span');

konamiStream.subscribe(function() {
  alert('You entered the Konami code!');
  livesSpan.textContent = '99';
});

The full example can be found at: http://codepen.io/anon/pen/ZOxGAq?editors=0010

Benefits of Reactive Programming

The Konami code snippet displays the great benefit of the streams approach.  Every Component can listen to the entry of the Konami Code and do something with it. The Component itself decides to 'react' to the Konami Code and not the other way around.

Having such a loose relationship between the consumer and producer of a streams allows for a greater separation of concerns. A classic example from the Reactive world is an autocomplete component. The user enters some text, an Ajax request is fired, some data comes back and is displayed.

This seems really trivial to implement but is actually quite hard. Lets look at a naive implementation:

  var $input = $('#search-input');

  var keyup = Rx.Observable.fromEvent($input, 'keyup')
    .map(e => e.target.value) // Project the text from the input
    .map(search);             // Search does an 'ajax' request
    .subscribe(function(data) {
      // Update the ui here with the data
    }, function(error) {
      // Update the ui here saying what the error was.
    });

After a while a back-ender tells you the server cannot handle the massive amount of request the front-end sends to the server. It turns out every keystroke is sent to the server.

If the user types in: "Battlestar" really fast a request for "B", "Ba", "Bat", "Batt" etc etc is made. Luckily RxJS has an  operation just for this called debounce. It will filter events that happen close to each other:

  var $input = $('#search-input');

  var keyup = Rx.Observable.fromEvent($input, 'keyup')
    .map(e => e.target.value) // Project the text from the input
    .debounce(500) // Wait for 500 milliseconds after the last event.
    .map(search) // Search does an 'ajax' request
    .subscribe(function(data) {
      // Update the ui here with the data
    }, function(error) {
      // Update the ui here saying what the error was.
    });

Turns out that the search doesn't quite find things accurately when only having three or less characters as the search query. So we filter only the queries that have more than three characters:

  var $input = $('#search-input');

  var keyup = Rx.Observable.fromEvent($input, 'keyup')
    .map(e => e.target.value) // Project the text from the input
    .filter(query => query.length > 3) // Only when more than three characters
    .debounce(500) // Wait for 500 milliseconds after the last event.
    .map(search) // Search does an 'ajax' request
    .subscribe(function(data) {
      // Update the ui here with the data
    }, function(error) {
      // Update the ui here saying what the error was.
    });

Now we discover that sometimes the same query is sometimes to the server two times in a row even with the debounce. This happens when the user starts typing again after a query but changes his mind and hits backspaces to his original query. No need to send the request then.

RxJS has the distinctUntilChanged operator for this usecase. It will make sure that events are only fired when they are different from the last event:

  var $input = $('#search-input');

  var keyup = Rx.Observable.fromEvent($input, 'keyup')
    .map(e => e.target.value) // Project the text from the input
    .filter(query => query.length > 3) // Only when more than three characters
    .debounce(500) // Wait for 500 milliseconds after the last event.
    .distinctUntilChanged(); // Only if the query has changed
    .map(search) // Search does an 'ajax' request
    .subscribe(function(data) {
      // Update the ui here with the data
    }, function(error) {
      // Update the ui here saying what the error was.
    });

Can anything still go wrong? Sure a rather insidious bug can still occur: when two Ajax calls are made and the server is slow to respond on the first Ajax calls response can come after the second calls response.

So if the user types in 'Battle' and waits before finishing 'Star' there will be two requests: one for 'Battle' and one for 'BattleStar'. If the server delivers 'BattleStar' response first and 'Battle' later the user will see results for 'Battle' which is unexpected.

RxJS comes to the rescue yet again with 'flatMapLatest' it will make sure only the latest search query's response will be used:

  var $input = $('#search-input');

  var keyup = Rx.Observable.fromEvent($input, 'keyup')
    .map(e => e.target.value) // Project the text from the input
    .filter(query => query.length > 3) // Only when more than three characters
    .debounce(500) // Wait for 500 milliseconds after the last event.
    .distinctUntilChanged() // Only if the query has changed
    .flatMapLatest(search) // Use only the response for the last query.
    .subscribe(function(data) {
      // Update the ui here with the data
    }, function(error) {
      // Update the ui here saying what the error was.
    });

With each improvement we did not have to update the code that 'displays' the data aka the subscriber / consumer of the observable.

This is a very powerful benefit. In essence this allows us to change anything about the way we deliver the end result, as long as we in the end deliver something the consumer expects.

The consumer of the data is not concerned with how the data is produced.

The producer of the data is not concerned with how the data is consumed.

Cons of using Reactive Programming

We are not in Kansas anymore (again)

Reactive Programming has a weakness that it shares with Redux. It is not always easy to learn: Observables, Operators, Streams, Hot / Cold etc etc.

While all of these things are not that difficult to learn, there are many of them. Thinking about Components and applications with these new concepts requires you to learn a new philosophy, and a different way of doing things.

Conceptually Difficult

Reactive Programming can sometimes be conceptually mind-bindingly difficult.

From my personal experience lets take the dynamic Counter's list. Next week I will demonstrate how to create this in Cycle.js. It took me quite a long time to program this particular example. I could not
for the life of me figure out how to create the total count.

I had Observable array of Counters where each item contained a Counter which had an observable inner count$. I had to find a way to combine all count$ streams into one array and do a simple reduce on it.

After some Googling I found this solution:

const totalCount$ = counters$.flatMapLatest(counters => {
    return Rx.Observable.combineLatest(counters.map(counter => counter.count$))
                        .map(arr => arr.reduce((total, count) => total + count, 0));
  }).startWith(0);

In retrospect this solution is obvious. But coming from an imperative world you really have to rewire your mind. Thinking in streams instead of values takes some getting used to.

It doesn't help that the documentation for RxJS / ReactiveX is highly conceptual. It does not always provide clear use cases for some operators making it difficult to know if you found the correct one.

Having said that it was an immensely satisfying experience that has taught me a lot about the nature of some problems. Once your brain makes the 'click' you will the use of streams everywhere.

The the 'autocomplete' example is a very hard problem to solve elegantly without observables. With observables it is very easy to add lots of functionality just  by combining the right operators.

Resources

I really recommend reading André Staltz's excellent introduction to Reactive Programming.

Components and Observables

This weeks post explained the basics of Observables but did not go into any detail on how this powerful concept can be used in combination with a Component based architecture.

Next week we will do just that by looking at Cycle.js.