Ledger Live Monorepo Project: Part 1 - Problematics (Make it Pain)

In this blog post series, Valentin De Almeida, a Ledger Live developer, talks us through the Ledger Live codebase evolution over the years. In this blog post, you will learn that it was multi-repository based at first, the issues we encountered along the way, and why it needed to evolve to a mono-repository architecture. In the next blog posts, we will explain how this major migration project was conducted.

A little bit of history

Ledger’s growth in 2020 and 2021 was significant. We aggressively recruited new talent, expanding our team from just a handful to over 20 developers. This means a lot of new engineers were onboarded on existing projects. As our team rapidly grew, we encountered new challenges that we had to quickly address. Despite these new difficulties, we remained focused on our goal and continued to deliver exceptional work.

We took a step back and looked into the new problematics that arise when more and more people got onboarded on the project. Among those new challenges, we can list the following needs:

Simpler flows.
Better guidelines for incoming and external contributors.
A unified set of tools.
Better dependency management.
Centralised open source contributions.

State of the Art: before the migration

Initially, and until last year, Ledger Live was based on a poly-repository architecture, for both mobile and desktop front-ends, as well as all the logic behind it. It was not a conscious decision to work in this manner, but it was only the result of an expanding project with no real architectural lead. Ledger Live is a project that gathers various components into one to deliver the best and most secure app to our Nano users, and it was reflected in the codebase.

The flows we had in place were flaky at best, mostly due to the fact we were 6 or 7 developers a couple of years ago. Since less parties were involved, the communication was way easier and we got away with it. Still, there were some pain points in the way we were working, especially around the developer experience and the release process.

Dev Experience Bottlenecks

Dependencies

Due to the nature of our projects, we work on multiple repositories at the same time, with dependencies between them. Here is a quick overview:

The Ledger open source libraries are used by the business logic, which is then used by both the desktop and mobile apps. But those apps also use the open source libraries, and using two different versions of the same library (like @ledgerhq/errors for example) would break the app.

We needed to bump the version on one side, then publish the libraries (yes, to npm!!!), then try again if it did not work. We started to rely on yalc to symlink projects, which was mostly okay as long as we did not have several layers of dependencies (for example, from the open source libraries to the business logic, and then from business logic to the apps). We tentatively tried to work with yarn link as well, but it seems it was doomed with React Native.

Testing

It was nigh impossible to do integration tests with all the latest code from the different projects. Since we needed to publish the libraries to the registry, testing all components with the latest up to date code was a nightmare

We also had to maintain several CI with duplicated logic.

Context Switching

Always moving around several code editors / projects / directories made the dev experience look really weak.

Release Process Bottlenecks

Versioning

Handling the versioning of different projects is hard, especially when there are several layers of dependencies.

Releasing

Release tracking was complicated due to the fact that projects were split, and we had to manage the releases of the different projects

It was impossible to automate the release process, again due to the fact that projects were split into different repositories.

And of course, Continuous Delivery was unthinkable at this point.

Possible Solution ?

Looking around for inspiration, it seems a mono repository architecture is the central piece we were missing. It would enable a much better development process. There are tools built around this architecture that would help us achieve the missing parts (release, automation, versioning…).

But, what is a mono repository ?

At its core, a mono repository is a project that encapsulates all other related projects (applications, libraries, tools) under one folder / git project. It allows better dependency management, uniformization of tools (like code styles and typescript configs), centralized Continuous Integration, integration testing, uniform version of used packaged (react for example)…

Since it’s a pretty agnostic system, some features were left for us to discover and implement. Hopefully, there are some great community tools that could help us add orchestration to the builds (sequential builds, helpful for CI), versioning, changelog generation.. which would complete what we were missing in our release process.

Cons

Mono repositories make sense in a context where several developers, or a team of developers work on several projects at the same time, with dependencies between them. However, it adds some layer of complexity during the setup phase (especially with already up and running projects that have 4 years of history and active development). Worth to mention, the project gets much bigger (like way bigger) in terms of disk space. All projects are now under the same folder and all dependencies. Which tests are mandatory? When to trigger them?

Pros

After evaluating the time, the cost, and the feasibility of our ambitions, here were some of the expected benefits of this transition:

Improved dependency management: With a monorepo, it’s easier to manage dependencies between different projects, as they are all stored in the same repository. This can reduce the need for workarounds like yarn link or yalc, and make it easier to ensure that all projects are using the correct versions of dependencies.
Better code organisation: A monorepo can make it easier to organise code, as all projects and their dependencies are stored in a single repository. It’s easier to understand how different projects fit together and how they depend on each other.
Improved developer experience: A monorepo can make it easier for developers to work on multiple projects, since they don’t need to switch between different codebases or repositories. It can also make it easier to run integration tests, as all the code is stored in the same repository.
Enhanced automation and continuous delivery: With a monorepo, it’s easier to automate tasks like building, testing, and releasing code. This can help to streamline the release process and make it easier to implement continuous delivery.
Increased development speed. Since different teams work in the same repository, they don’t need to wait until the release to verify the result, accelerating the integration.

Conclusion

Overall, the implementation of a monorepo structure can help to improve the development process, streamline the release process, and enhance the developer experience.

In the next blog posts of this series, we will walk you through how this major migration project was conducted, the tools we used, the choices we made, the result and more. Stay tuned for Part 2: The tools!

Valentin DE ALMEIDA
Developer Experience & Core Tech – Ledger Live