The case for separating front- and back-end

At 42 we have an ongoing discussion about the separation of the front- and back-end of an application. The back-end being a RESTful service, and the front-end being a modern MVC JavaScript application written in AngularJS.

There are two camps within our ranks: the first camp believes the front- and back-end should be completely separated, where both applications have separate version control, build processes, and deployments.

The second camp believes that the back-end should provide the REST API, serve the JavaScript application, and that there should be one deployment and build process that delivers the whole application (front- and back-end) in one package.

Disclaimer: I'm in the camp that believes strongly in separating the two. So I want to argue the case for separating front- and back-end completely in the post.

History

In ye old days there was only one way to create a website: Using a text-editor you started writing HTML, and when you were done, you uploaded it to a web server. The HTML was static and every time something needed to change you edited the HTML file, and uploaded it again. Easy peasy.

Smart programmers thought: "if we can generated the HTML dynamically, we can do way more cool stuff with the web." Thus was the back-end created and programmers rejoiced. You picked a language and a database and off you’d go. We didn't create websites anymore but web applications.

A funny thing happened to HTML and CSS along the way. They seemed to become less important to web development, almost a necessary evil to make things work. Take job descriptions from this period: Big bold letters: "Looking for Java developer" somewhere at the bottom in a list in a way smaller font: "Knowledge of HTML and CSS are a plus".

We saw HTML and CSS as a thing to be generated but not the actual application. The actual application was written in Java, PHP or C# the HTML was just a build artifact waiting to be served. Heck, some platforms such as "ASP.NET Web Forms" went out of their way to hide any sort of HTML from the developer.

Then the Ajax revolution happened. Before that we thought of JavaScript as a horrible monstrous little language for displaying animated snowflakes around Christmas. Suddenly we could do so much more, we could provide an interactivity and responsiveness never seen before.

I remember seeing 'Ajax' for the first time on Gmail, it was loading new emails whilst the page was kept open. I didn't believe my eyes, I was sure I was seeing things. I checked my computer to see if I had installed some Google application that refreshed my browser somehow, or that I installed some plugin.

Nope, JavaScript was doing an XMLHttpRequest within the browser. It didn't take long before I started using Ajax to impress my coworkers. Then jQuery came along with excellent support for Ajax, and Ajax became commonplace.

JavaScript became another 'plus' in job descriptions. Books describing its good parts came out, and it was really growing up. People started to see its power and what it meant for our end users.

Then the iPhone happened and later Android. Suddenly we found ourselves in need of writing APIs to provide these apps with the same data and CRUD functionality our traditional web apps needed. Since our web apps didn't use Ajax for everything, we started writing REST APIs next to our traditional endpoints. Effectively writing some functionality twice.

Then came a second JavaScript breakthrough: MVC frameworks started popping up, such as Backbone.js and later Angular, Ember, Knockout, etc. JavaScript, it seemed, had learned its lessons from the back-end world.

But what if the 'MVC' front-end would also only use the RESTFul back-end like the iOS and Android apps already did? Then all 'apps' use the exact same calls, and the web apps would no longer be a 'special' case. REST calls returning JSON all the way down.

Our MVC JavaScript apps now rivaled native iOS and Android apps in code size and complexity. With the coming of Node.js, which allows you to run JavaScript outside of the browser, there even came build tools and task runners written in JavaScript.

The front-end part of the equation has finally come onto its own, no longer needing a special 'relationship' with the language the back-end side was programmed in.

Now that the history is out of the way I'll start making my case.

Separation of Concerns

When I came to work at 42 my first assignment was assisting on a Java project that needed to provide a RESTful interface for an iOS app. The project already had a web based front-end, the idea was that the iOS app would eventually support all features the web app did.

The web app front-end used JSP with some jQuery to make it more dynamic. Not every functionality used Ajax, but most did, so another goal of the project was to use Ajax for everything and gut JSP from the project. This way the web app and the iOS app became equals using the exact same API.

We went from a project that had its front- and back-end interwoven, to one where the two only communicated only via Ajax calls. The back-end's job was to provide data, expose ways to manipulate it, and to secure access to the data. Its last job was to serve the HTML for the web app.

When we were done I looked at the relationship between the front- and back-end. The only concrete relationship the two had was that the back-end served the front-end. If I had ripped out the HTML from the project, had it served by another server under the same domain, the Java back-end and even the end user would be none the wiser, everything would still work as before.

Looking at the relationship between the iOS app and the web app I realized they were brothers, they were familiar. Both used the back-end in the same way to authenticate a user and send the same HTTP GET and POST request to get things done.

I realized something: When the back-end and front-end only communicate with each other, but don't need each other, they are separate entities.

From the perspective of the back-end all I need to do is send valid HTTP requests. I can use the entire application from cURL if I want to. The back-end simply doesn't care as long as I send valid HTTP requests in the correct format. The back-end is completely unaware on what consumes its service.

The front-end has the same perspective, it calls some service at some URL. As long as the response is valid, and the service does as promised, the front-end doesn't care what the back-end is. If it's written in Java, PHP or Ruby it doesn't matter, heck, even a monkey sitting behind a keyboard would work as long as the communication works via the same protocol.

The front-end, whether it’s an iOS app or web app, is a separate entity from the back-end. They are separate concerns.

Version Control

In the project described above, the web app and the back-end shared the same version control repository. The iOS app had its own repository.

I asked myself the question if the recently decoupled HTML and JavaScript should move in its own repository. The answer was yes, they needed to be in separate repositories. I came to this conclusion by looking at the 'git' history of the project. The history, after the decoupling, seemed to be a chimera, two entities living in the same body.

On one hand there was the history of the back-end: "changed HTTP call", "Optimised login and made it faster". The other commits were front-end: "Moved navigation to top of screen", "Increased the size of the font".

These commits were often interwoven, worse yet some front-end changes were considered too 'minor', so they got packed together with 'back-end' changes.

When looking at the history from a back-end perspective, the commits looked littered with stupid minor stuff some 'front-end' hipster frontender made. Ruining the view of the API.

When looking from the eyes of the front-ender, the exalted commits to enrich the users experience were interspersed with commits from a neckbeard ranting on and on about "REST" and "HTTP" and "optimizing". Whatever that meant. It made looking how the UI changed between versions more difficult.

What does a background color of a website has to do with an optimization in the login routine? Nothing.

I also thought about my changes to the front-end and back-end. What if only the changes to the front-end needed to be reversed, that would be very uncomfortable since the history is so interwoven.

So from a pure front- and back-enders perspective having one repository is very annoying. With the move towards separate jobs for front- and back-enders not every front-ender is going to know how to solve Java merge conflicts and vice versa for JavaScript merge conflicts.

Building

I've mentioned Node.js and it has transformed the way we think about JavaScript. The ability to run JavaScript without a browser from the command line has lead to some pretty exiting options for building JavaScript apps within JavaScript itself.

Grunt is a JavaScript task runner which makes it easy to run specific tasks in series. Everyone can create Grunt tasks, the community has created tasks for: minifying HTML, JavaScript, CSS and images, it has tasks for compiling LESS down to CSS and many more.

Bower is a dependency manager for the web, which can be used to download third party JavaScript by version. Much like Maven does in the Java world. Bower can even be integrated with Grunt so it can write the correct HTML tags to include the external files for you.

Using JavaScript to build JavaScript applications means that for the first time JavaScript developers are in control of their own destiny. No longer do you have to depend, as a front-end developer, on the language chosen on the back-end side.

When you are in control of your own tooling it becomes easier to extend them and you can use them everywhere. Gone are the days of switching between a build processes for an AngularJS app when switching from a Ruby back-end to and a Java back-end.

Deployment

The deployment on a server of a pure JavaScript application is simple because JavaScript apps are static by nature. In the old model we dynamically generated the HTML on the server to make the app dynamic. Now we use JavaScript to retrieve data via Ajax and use JavaScript itself to dynamically render the app. The JavaScript and HTML and CSS are static, between versions of the app they don’t change each time you serve them.

So all you need is a web server that is good at serving static files. Which is easy because serving static resources means you can cache the app more easily.

Plus knowing that an application is static means you can use content delivery networks to serve the application more easily.

Via Grunt it is possible to generate a package that has made unique names for each resource, such as images, CSS and HTML, and which is ready for deployment. Updating the app becomes as simple as uploading a folder.

Cultural

Suggesting moving the front-end and back-end into their own repositories, and stopping to serve the front-end from the back-end invokes some negative responses:

"Why put them in separate repositories, I'm a fullstack engineer I do both front- and back-end work. It will just make my life harder."

"It's so convenient to have to deploy only one package."

"Maven can do that too, all you need to do is (some magic that a non Java enabled front-ender cannot possibly understand)."

I think these reactions stem from the fact we've been used to seeing HTML, CSS and JavaScript as byproducts of the actual Java application. It is difficult to change something we've all been doing for so long.

I think it is a "culture" thing. Once we start seeing web apps as mature applications with their own build processes, conventions and culture, it becomes more natural to separate them from the back-end.

Here's a thought experiment: You're a developer making a web application via AngularJS calling a REST service you've created with Spring MVC. Suddenly a wild Pointy-Haired Boss appears with an iOS developer in tow, he's going to make the iOS version of the application.

Would you include the Objective-C and Swift code in your git repository?

Would you configure Maven to kick-off a Mac-mini to start the iOS build process, every time you add change something on the server?

Would you like to be emailed when the iOS programmer makes mistakes when the build has failed?

Would you mind it if the iOS programmers mistakes send you emails that the build has failed?

Would you change your release task to submit a version to the App Store?

Would you hold your release until it has completed the App Store approval process?

Did you vomit a little bit just thinking about these unholy questions?

Conclusion

Separating the front- and back-end into their own repositories makes it easier to revert changes.

Building JavaScript apps with JavaScripts empowers the front-end to create great reusable tools, and create conventions regardless of back-end language.

Deploying static files is what servers are good at, using Grunt it becomes really easy to generate these static files.

JavaScript apps are on the same level as iOS or Android apps. They don't belong or are part of the 'server'. Let start treating them as first class citizens of the REST API.

14 comments:

  1. Deyna Cegielski29/1/15 21:09

    Great article and well articulated.

    How do you manage the release process with multiple artifacts? Lets say you're developing a new feature and adding some APIs, and then adding a new view in the front end. The front end release has a dependency on a version of the back end, and the back end.

    How would you orchestrate the build and release of these new artifacts so that they are delivered together?

    ReplyDelete
  2. Hi, i have the same problem, i need create a web app but in the future i need an api to serve a mobile app; what do you think about:
    1. I should develop one generic api to use "api-key" auth in every request and then develop a web app with a front end using something like andujarjs and then mi mobile app should consume the generic api
    2. I should develop mi web app with the front end (with angularjs) separately from the back-end but with the api living in this web app and after create the api for mi mobile app.

    I like the first option because is less code and i use the same business logic on all but i don't have every clear with the correct infrastructura and patters to use.

    ReplyDelete
  3. Andrey Falaleev5/2/15 10:25

    Thanks a lot for such a good article! My own thoughts on the topic are very similar to yours.

    ReplyDelete
  4. Hi we do this at work by having the multiple projects set up under a parent Pom whose packaging type is Pom, the projects are all maven modules in the master Pom. Each project has its own Pom defining it's artefact version etc and when jenkins builds for test all artefacts are deployed together.

    In svn each artefact has its own directory.

    ReplyDelete
  5. I really like to do the seperating work, but I'm stuck in one problem, that HTML doesn't have an 'include' mechanism.

    ReplyDelete
  6. And another problem:How to pass a parameter from one html page to another html page?

    ReplyDelete
  7. Maarten Hus15/7/15 13:43

    I would write a single REST API that services both the native and web front-end applications. You can have multiple authentication mechanisms and pick the one that is easiest to use per front-end app.

    The goal should always be to have one single API for all applications.

    ReplyDelete
  8. Maarten Hus15/7/15 13:46

    I recommend picking a Single Page Application framework for you web front-end, such as AngularJS: https://angularjs.org/. With a SPA you can create complex UI's purely in the front-end.

    ReplyDelete
  9. please tell me some general rules to follows for connecting front-end with back-end design by different teams.

    ReplyDelete
  10. We are truly thankful for your blog entry. You will discover a great deal of methodologies in the wake of going to your post. I was precisely scanning for. A debt of gratitude is in order for such post and please keep it up.
    remote net developer jobs

    ReplyDelete
  11. Thanks for sharing this information. I really like your blog post very much. You have really shared a informative and interesting blog post with people.. Acupuncture Pen HQ

    ReplyDelete
  12. Thank you so much for the post you do. I like your post and all you share with us is up to date and quite informative, i would like to bookmark the page so i can come here again to read you, as you have done a wonderful job. 北九州監禁殺人事件

    ReplyDelete
  13. I am really very happy to visit your blog. Now I am found which I actually want.I check your blog every day and try to learn something from your blog.
    Please keep it up to date like this. Thanks for sharing.
    McAfee is the ultimate cyber security software range that keeps your devices protected round the clock. Although set up for any application is a time consuming and complicated process. However,
    McAfee Activate activate setup is much simpler and convenient than expected

    ReplyDelete
  14. I read this article. I think You put a lot of effort to create this article. I appreciate your work. https://mycrochetpattern.com/home/crochet-toys/grey-totoro-amigurumi-crochet-pattern/

    ReplyDelete