the Merchant Centerwhich is our main web application for administering the project data in our Composable Commerce APIswas no exception to that growth.
When we started developing the Merchant Center application we were a small team of frontend developers and designers. We had our own repository, our own process, our own deployment pipeline, etc.
The Merchant Center was a Single-Page Application using React.js and we developed most of the UI components on our own, following our design guidelines.
Eventually the team and requirements started growing to the point where working as one team, in one codebase, was not feasible anymore.
Some of the organizational changes in the company were related to the structure of the teams, more specifically to having vertical teams. A vertical team is an independent team that focuses and owns a specific domain of the company's product(s) and consists of different roles required to work on that domain. Most of the time teams have the following roles: Product Manager, Engineering Manager, Designer, Frontend Engineer, Backend Engineer, Agile Coach.
For the Merchant Center frontend application it meant that multiple teams (specifically the Frontend Engineers of those teams) will contribute to the development following their own requirements and processes.
So how can multiple contributors work on the same codebase without stepping on each other's toes? Who decides when to deploy? What about merge conflicts?
Obviously our repository setup, our development and deployment processes, our tooling, all of that needed to be changed as well to align with the structural changes of the teams.
At the same time, our enterprise users and customers were showing interest in being able to customize parts of the Merchant Center to tailor some of their specific business requirements.
With similarities between internal and external requirements we saw a chance of addressing both problems with one solution. This is the story of how Custom applications came to be.
With our basic requirements around teams and customization we started envisioning the future of the Merchant Center and more generically of frontend development.
We identified the following points:
- Teams should develop and own independent parts of the Merchant Center.
- Teams should deploy and release parts of the the Merchant Center without deployment dependencies.
- Teams should focus on building user features by using the same set of shared development tools.
- Teams should focus on implementing consistent and usable user interfaces by using the same set of shared UI components and by following the same UX guidelines.
- Our external users and customers should be able to do all of the above.
One thing was clear: we needed to split up the Merchant Center application into multiple little ones.
Wait, isn't all of this what Micro front-end architecture is about? Yes, conceptually very similar but also not, as there are different ways to implement it.
In a nutshell, the idea is to have different parts of the application developed and deployed independently while the end-user consumes and interacts with the application as if it was one.
This split also fits with the organizational changes to vertical teams, where a team owns one or more parts of the application. For example, our Product and Catalog Management team works on the products and categories applications while the Checkout team works on orders and customers.
Each application is developed and hosted independently and on production there is a routing component that sits in front of every application and determines which application should be rendered using the following URL scheme:
https://mc.<domain>.com/:projectKey/:entryPointUriPath
the projectKey identifies the project of the user while the entryPointUriPath identifies the application to be.
To ensure an overall consistency across the different applications, there are common parts that must be used by every application. For example, the main application layout and the basic rendering logic are implemented as a shared component that each application must use. We call it the <ApplicationShell> .
After identifying the long term vision of our frontend infrastructure we needed to execute the plan. At the same time all the technical and structural changes had to be performed in a backwards compatible way to avoid disrupting the existing application.
Going monorepo
As a first chunk of work, we started splitting up the codebase into multiple packages and created a proper monorepo. This required also to move many of the UI components and utilities used across the applications into shared packages within the same monorepo.
Naturally, our CI/CD pipeline also needed to be adjusted and, after many iterations, we ended up with a complex but effective multi-pipeline workflow to test, build and deploy each application independently.
Sharing tools and components
The second phase of the plan was focused on enabling this kind of development to external users. To do so we needed to do two main things:
- Open source many of our UI components to develop the applications.
- Open source the tooling to develop, test and build the applications.
Why open source?
For one thing, we wanted our users to have easy access to the same tools and components that we were using internally but also to establish a common place around developing Merchant Center applications and encouragement contributions.
And so we created two public repositories: UI Kit other Application Kit.
The UI Kit is the place where we develop our design system and implement all basic UI components such as buttons, inputs, notifications, table, etc. All these components are the building blocks for developing the UI in the Merchant Center.
The Application Kit includes all the tooling for developing, testing and building applications.
All these packages are then published to the NPM registry and can be consumed by anyone. Obviously we use all of them internally as well.
With the second phase reaching a mature point, it was time to officially enable the development of these UI applications to external users. This product feature is known as Custom Applications.
A Custom Application is a set of UI tools and components to extend the existing functionality of the commercetools APIs and the Merchant Center. Merchants can implement specific business requirements in a Custom Application and use it in their Merchant Center Projects by configuring and installing Custom Applications into their commercetools Organizations. A custom application is developed and managed by merchants.
To allow users to integrate their Custom Applications into the Merchant Center we had to build a bunch of other things like storing the configuration of Custom Applications via a dedicated UI workflow. A user will then develop and host a custom application to a hosting service of their choice and provide the URL link in the configuration that points to the hosted application.
The Merchant Center routing service (as mentioned before) ensures that the matching application is served correctly, no matter if it's an application developed within commerce tools or by external users.
Custom Applications evolved a lot over the years but the main vision remained the same: to empower teams to independently contribute to the Merchant Center and to focus on delivering customer value.
Scaling our frontend ecosystem with our multiple internal teams (and external users) was definitely a lot of work but a necessary effort to ensure that teams remain productive and focus on what's important within their business domain. Similarly, external users can do the same and focus on building and delivering custom solutions tailored to their business requirements.
dog food the tools and components that our customers use allows us to constantly keep working and improving them, even though at times the process might seem a bit cumbersome. However, with proper automation tools and processes things usually go smooth. In addition to that, we also plan and release features and changes carefully and well tested as both our internal teams and external users would be affected the same way.
As for our teams, the flexibility of working on different parts of the Merchant Center independently is a great benefit. Not only that but all the tools and UI components that we provide allow them to focus on delivering useful and usable features fast. For example, by using our UI Kit components library there is less to no need to implement custom CSS styles.
However, it's not all sunshine and roses. Working with this kind of setup, especially in a monorepo, has it's own pitfalls and downsides as well.
For instance, an ever growing monorepo comes with its own inherited issues. More packages and dependencies to maintain, more teams working on it, more development activity, more CI load.
Updating dependencies can be at times frustrating as there is a larger surface area where things could suddenly fail, in particular to the parts of the codebase without a regular maintenance.
On the other hand, introducing new tools, new libraries or updates to existing ones can be as challenging or even impossible due to the fact that the entire codebase is affected and so are the teams. For example, trying out the latest React version in one application would likely cause version conflicts and weird errors in the other applications.
Another important point to consider is about ownership of the parts of the codebase in the monorepo. For example, applications are easily assigned to teams as owners but for other parts is not always clear who the owner is.
When implementing a shared component used between multiple applications, who owns it?
Ownership is important in terms of maintenance and also helps with reviewing contributions and changes to that code.
Scaling the frontend architecture with your teams does indeed require technical and organizational changes and you need to find a balance about the setup, the tooling and the team's processes.
Using a monorepo has many benefits and for us it was and still is a valuable setup for many of our teams and projects. It also helps with consistency and to easily share and keep things up-to-date.
However, as the organization and codebase grow, there might be issues and limitations to tackle in order to find that balance. For us it meant to invest in things like improving our CI/CD pipeline, having better automation tools, improving our integration and acceptance tests, but also to define clear ownership, to have cross-team collaborations via the Front-Line Dev and the Frontend Chapters.
We are also looking at other ways to facilitate that growth and collaboration between teams. For instance, we are currently evaluating and trying “breaking up” our main monorepo to allow teams to have more focused spaces to work on without some of the limitations of a monorepo mentioned before.
On the other hand, this forces us to invest more into tooling and sharing parts of the stack that is required across repositories.
At the end of the day find the balance to what works best for your structure and teams and invest in tooling and processes to help you with that.
0 Comments