Yarn workspaces and mono repos

Posted on 2022-02-06 in Programmation

I currently work in a small startup with two other developers. We have three projects: an API written in Python with the Django web framework, a big React app and a website written with NextJS. All these project are in the same git repository. This allows us to easily share code, to do changes in all projects easily at once when needed and to move fast. We still can choose what to deploy and we only tests part of the project that works.

So, this monorepo architecture is currently the best choice for us. You can learn more on this subject on monorepo.tools and Misconceptions about Monorepos: Monorepo != Monolith.

Today I'd like to dwell a bit on how we share our TypeScript code. Initially, we had some duplications between the app and the website. It's not a good idea and we were starting to have problems because of this. We wanted to keep the mono-repository approach since it's a very good fit for us: the workflow is more simple and we can move faster. So, we decided to use Yarn workspaces as a way to share code across our repository.

With workspaces, you can declare folders on your file system as packages that can be installed and used directly in other project. It's an efficient, easy and standardized way to share code in a mono-repo. Since our needs are still basic, we only use Yarn and no extra tools (at least for now).

In a nutshell, with workspaces, you will have a package.json at the root of your project. It will have a workspaces key, set to an array of relative folders. So, you put "frontend" to have a workspace at /frontend. In each of these folder, you need a package.json to declare the dependencies specific to the workspace and the name of the workspace under the "name": "my-workspace". That's the name you will use in Yarn commands and not the folder or path. All your dependencies will be locked at the root of your project with only one yarn.lock file.

Here is how we did the migration:

We migrated to Yarn 3. It wasn't required but was on our TODO list and it's easier to work with workspaces. We mostly can install only the dependencies of one workspaces with the focus plugin. It's good to reduce the size of Docker images. See the official documentation for the migration procedure. We didn't update to PnP modules yet.

We created a top level package.json the workspace configuration:

"workspaces": [
    "shared/ts",
    "frontend",
    "website"
]

We removed all the node_modules folder and yarn.lock and run yarn install at the root of the project to create a new (and clean) yarn.lock.
Since all dependencies will now be installed in a single node_modules folder at the root of the project, we updated our CI caching strategy so in only restored it.
We mutualized our common dependencies on the root package.json. This was mostly done to ease update of these dependencies. They were deleted from the workspace specific package.json.
We moved our lint script and its dependencies to root. This way, we can apply the same rules everywhere and lint all the project at once. For this, you need a script like this: "lint": "eslint \"{frontend/src,website,shared/ts}/**/*.{js,jsx,ts,tsx}\"",.

Because we mutualized them, eslint was reporting error about missing dependencies with its import/no-extraneous-dependencies rule. To avoid them, we updated eslint configuration and included both the root and child folder in its packageDir option. For the frontend child folder:

"import/no-extraneous-dependencies": [
  "error",
  {
    "devDependencies": ["**/*.test.tsx", "**/*.test.ts"],
    "optionalDependencies": false,
    "peerDependencies": false,
    "packageDir": [".", "./frontend/"]
  }
]

For the root folder:

"import/no-extraneous-dependencies": [
  "error",
  {
    "devDependencies": ["**/*.test.tsx", "**/*.test.ts"],
    "optionalDependencies": false,
    "peerDependencies": false,
    "packageDir": [".", "./frontend/", "./website/"]
  }
]

We updated our Dockerfile to include the shared code folder and our vendored Yarn version (that's the recommended way to do things to be sure to always us the proper version of Yarn) so the install would work correctly with the proper version (an older versions aren't compatible with the newest yarn.lock format). We had to include all package.json files at their respective place in the file system so Yarn could install all the dependencies correctly. Without it, it complained about missing a package: the workspace we didn't have. If you only need to install a subset of the dependencies with focus, it's not required.
We updated our docker-compose.yaml file to make sure the correct version of Yarn was available in our containers. This was done with volumes.
For the code to compile, we had to add extra dependencies to our project. By default, only the code from the current module is transpiled. To ease our work, we wanted our library to be written in TypeScript and transpiled by the main project. This way, we can import our common code like it was nested in the project and not worry about compiling it before importing it.
- This can be done with NextJS and the next-transpile-modules module. We had to update our next.config.js so it would look like this:
```
 // Use the name of the dependent and shared module.
 const withTM = require('next-transpile-modules')(['shared']);
 let moduleExports = withTM({
   reactStrictEnv: true,
 });

 moduleExports = {
   ...moduleExports,
   async headers() {},
};

module.exports = moduleExport;
```
- For the React frontend, we use react-app-rewired and already had a config-overrides.js file. We use react-app-rewire-yarn-workspaces to compile our shared code. We updated our config-overrides.js like this:
```
const rewireYarnWorkspaces = require('react-app-rewire-yarn-workspaces');

module.exports = {
    webpack: function override(rawConfig, env) {
    const config = rewireYarnWorkspaces(rawConfig, env);
    // Extra configuration
}
```

That's it. A week after our transition, all working fine and we can share code without any issues. If you have a question, please comment below!

Extra resources that helped me do the transition: