Composable architecture

 

The choice of architecture has an impact on the organisational, technical and financial aspects of a digital project. New alternatives are emerging on the market, but they sometimes mean rethinking the design of the project, the know-how and skills available in the teams, and the organisation of the project team.

What solutions are available, and how do you make the right choice to ensure the profitability, scalability and longevity of your project?

The end of monoliths?

Since the 2000s, companies and e-tailers have been favouring the use of monolithic CMS and eCommerce solutions, i.e. “all in one”: capable of providing highly functional administration interfaces, as well as a front-end, search, authentication, forms, newsletters… in short, everything you need to launch a new Web project. While these solutions have enabled a large number of players to accelerate their digitisation, they have then often reached a glass ceiling and been unable to adapt to new mobile uses (native or PWA), to take advantage of the promises of the Cloud (scalability), and have gradually become a legacy that is very heavy to maintain, undesirable for engineers, sometimes vulnerable and functionally outdated by tools specialising in a single function: search, newsletter, authentication, etc.

MACH architectures as alternatives?

At the opposite extreme from monoliths, microservices, and more specifically MACH (Microservices, API-First, Cloud-Native, Headless) architectures, have emerged as a dogmatic and unavoidable alternative to monoliths. And a number of high-profile Cloud/SaaS players and start-ups that cannot do without this level of scalability, and know how to manage the complexities and new expertise that go with it, have successfully integrated these architectures. However, for many other players, the engineering gap to be bridged is insurmountable, and the drawbacks of MACH architectures outweigh the expected benefits. So how do you reconcile the best of both worlds?

Composable architectures: the best of both worlds?

Headless – Front-end & back-end decoupling

The simple decoupling of front-end and back-end architectures has enabled, and continues to enable, many digital roadmaps paralysed by legacy monoliths to be unblocked.

• being able to modernise the front-end and mobile uses (PWA, offline, performance) without the issue of back-end;
• to be able to use the FrameWorks, libraries and modern productivity tools expected and desired by the greatest number of developers, currently Vue, React or Svelte;
• to decouple the front and back roadmaps without blocking each other;
• to be able to keep all or part of the monolithic back-end, replacing it gradually, without the need for a tunnel effect over several years.

However, this work is not without consequences for the adaptations and investments required around monolithic CMSs, which often expose level,at front-end level that needs to be deported to APIs to avoid recreating front- and back-end dependency, and ultimately distorting one of the interests of headlessness: the decoupling of responsibility.

Atomic design and component logic

The promise of a composable architecture, as its name suggests, is to be able to ‘compose’ a digital product using elements that are watertight, allowing each element to be upgraded or replaced without affecting the other elements of the system. Decoupling the back from the front is not enough to guarantee this promise; it is also essential not to recreate a monolith on the scale of the front end, where any visual or functional change would require rethinking, retesting, duplicating and distorting the existing system.

Atomic design, documented in a storybook, makes it possible to construct a visual and functional language made up of atoms (an icon), molecules made up of atoms (a button), organisms made up of molecules (a footer), and so on. This language makes it possible to compose all types of pages without duplication, exoticism or macro dependency.

HTML rendering paradigm

In a headless world, where the back-end exclusively exposes APIs and data in JSON, the question arises of the HTML rendering strategy:

• Which page or portion of page should be rendered on the browser side, for example content that is personalised by user, not predictable (search autocompletion) and therefore not cacheable?
• Which page or portion of page should be rendered server-side, because it needs to be hidden for performance reasons and exposed to indexing robots?

For pages generated on the server side, there is also the question of what can be generated statically (Server Side Generating, a key concept in a JamStack). This paradigm is by far the most efficient because it only exposes static content hidden on a CDN, but it is also the most demanding and complex to implement in a context of large volumes of content and personalisation. This is why Server Side Generating has become popular as the best compromise between performance and simplicity, by generating the HTML on the server side when a web user requests a page that has expired on the CDN.

Why a Back For Front?

Decoupling front-end and back-end indirectly recreates a dependency when the front-end directly requires the back-end API(s), for example the CMS API, the search API and other information system APIs. The front-end then becomes monolithic again, playing the role of orchestrator and link between the APIs, with the attendant problems of business logic, authentication and performance.

This is why composable architectures require the presence of a new back-end between the front-end and the other APIs in the system, making it possible to :

• exposing a single back-end API for front-end calls, dedicated and adapted to the front-end user experience, preferably in GraphQL to limit the back-and-forth between front and back ;
• never expose business or CMS APIs directly, for reasons of security, performance and decoupling (being able to connect and disconnect new APIs depending on the roadmap);
• load data from business back-end systems (REST APIs, SOAP, files, etc.) and publishers’ solutions (headless CMS) to link them together, adapt them and transform them into the API served at the front-end;
• store the necessary data persistently (data that does not exist on the business back-end) or temporarily (volatile cache, search index), on a case-by-case basis depending on the functional need;
• unify JWT authentication with the various systems, to offer a single JWT token for exchange with the front-end;
  use input (business API responses) and output (HTTP cache) caching systems to protect the business APIs and the back for front itself from too many calls.

When to choose REST and when to choose GraphQL?

GraphQL has become widely popular because of its ability to let the front-end take control of the queries and select the fields, sorting and data filters it needs for display without depending on the back-end. With a minimum of calls and maximum flexibility, the front-end can obtain the JSON data it needs to populate the components.

GraphQL remains the best choice for this dialogue between visual interface and back for front data provider. However, for the exposure of more atomic and generic APIs, not directly coupled to the front-end, REST remains a better choice, less permissive and easier to stabilise in a server-to-server context, or for exposing a business API not influenced by a user interface.

Example of a composable architecture diagram

Composable architectures, a replatforming solution?

Monoliths have the ability to make it difficult to replatform to a composable architecture. Not always knowing where to start, and sometimes afraid of the “house of cards” effect, decision-makers can invest indefinitely in version upgrades until the solution and associated skills are exhausted or disappear.

The scenarios generally undertaken as the first steps towards a composable architecture are :

• Complete decoupling of front-end and back-end (100% API), which is sometimes difficult when the monolith does not offer API functional coverage. However, investment in developing the missing APIs is often profitable in the long term, as long as you can take advantage of an independent front-end.
• Progressive decoupling of front-end and back-end by using devices to include the new front-end progressively via javascript widget, ESI (edge-side-include), or user experience tricks (full page and back button, side panel).
• Progressive decoupling of the back-end in parallel with the front-end, by gradually replacing some of the monolith’s functions with external tools (search, newsletter, forms, etc.).

Composable architectures and the associated challenges

Composable architectures make it possible to unblock paralysed situations and offer many advantages. But they also bring with them new complexities, new know-how and new challenges:

• This means learning new skills and a new vocabulary: Full Stack JavaScript developers, TypeScript, graphQL, HTML rendering paradigms, JamStack, continuous deployment in the Cloud, Kubernetes or even Infrastructure as Code (Terraform)…
• You need to raise your level of expertise: in DevOps, in performance, in code architecture, in APIs, in the Cloud…
• This can lead to a cultural and generational clash: the PHP generation feeling disconnected from a new world that seems inaccessible to them, and the school leavers mastering this new world as the only possible and desirable one.

Myths and legends surrounding composable architecture

The generational shock brings with it its share of fears, apprehensions and resistance, and in the end a number of myths and legends emerge that are and will continue to be the subject of debate over the next few years. Here is an extract of questions and answers, all valid or not.

“It’s harder to recruit a full stack JS developer than a PHP developer.”
TRUE: Composable architectures require a level of engineering that puts companies and e-retailers on the same playing field as digital pure players and start-ups. Competition for recruitment is tough.
FALSE: In terms of volume, the vast majority of school leavers are familiar with this engineering culture, which has become the new standard. 90% of the CVs received by ESNs like Kaliop and its competitors are front-end, back-end or full stack developers with a JavaScript, Cloud and Headless culture.

“JavaScript technologies are fads.”
TRUE: New frameworks and libraries appear every week, and very few survive to the benefit of now mature ecosystems such as Vue/Nuxt, React/Next, Gatsby, Apollo, Nest, etc. It’s hard to keep up with trends and bet on the right horse.
FALSE: JavaScript and the main frameworks that support it have reached a level of adaptation and diffusion in the world of the Web (front and back) and the Cloud that constitutes a point of no return, pending a major revolution that would render the concept of the Web browser obsolete. Furthermore, the volatile nature of the Web mitigates against the volatility of frameworks and their versions.

“Headlessness is a functional impoverishment.”
TRUE: By its very nature, a headless solution focuses on a single function and deliberately moves away from an “all-in-one” ambition.
FALSE: By combining the best headless solutions specific to each function (CMS, identity, forms, newsletter, payment, CRM, etc.), you get a package that is difficult to compete with from a single monolithic solution provider.

“The complexity of headless architectures makes maintenance difficult.”
TRUE: The skills of a publisher’s solution integrator or webmaster are becoming less exploitable in these ecosystems, which call on the skills of engineers who until now have been less interested in the Web than in other disciplines.
FALSE: The skills are new to the Web, but not to the world of IT and IT development. What may seem ‘complex’ or inaccessible to a Web integrator is commonplace to an engineer.