I've taken over the role of Architect on a large project made of several applications all sharing a lot of common assemblies.
The project is composed of several sub teams, every team is responsible for an application and share multiple common assemblies (sort of framework). Because the project is still in an early stage, the common assemblies incurs a lot of modifications on implementation but also on contract level. Therefor common assemblies are generated by custom CI (Continous Integration)builds that drops the assemblies on a network folder. The developers working on the applications, references the common assemblies through the network folder so that when a new version of the assembly is build, it is automatically refreshed by Visual Studio.
This chaotic architecture and wrong way of working brings a lot of trouble. First of all, the developers are not isolated from each others changes. Because the applications are directly referencing the assemblies stored on the network share, every time a CI build generate a new version of the common assemblies - these are automatically picked-up by Visual Studio. When breaking changes are introduced in the common assemblies, dependent applications suddenly breaks.
e.g. -> When the developer A check’s in a file belonging to a shared project this trigger a CI build and refreshes the shared assembly on the build output folder – Visual Studio of developer B detect that the referenced assembly has changed and refreshes the local bin folder - potentialy breaking the build of B and this without performing a get-latest version.
The teams also experiences a lot of runtime errors caused by not matching assembly manifests. The problem is that the common assemblies are versioned and also dependent on other common assemblies.
e.g. -> App A depend on Common1-V1 and Common2-V1. Common1-V1 depend on common3-V1 & Common2-V1 depend on Common3-V2. This leads to runtime errors because only 1 assembly version of Common3 can be in the app bin folder at the same time. When the application creates a new instance of Common3 (not the right version) the runtime detect that the assembly manifest of Common3 is different from the expected version and throws a fatal error.
Despite all the pain, the teams didn't want to perform a big architectural refactoring right now – they are behind schedule and needs to release a first version of the applications next month. So I searched for an efficient way to bring some order in this chaos without impacting the project too much.
The first thing was to change the references in the VS soltions. I created a local “_Reference” folder on the root of every VS solution and copied all tools and shared assemblies into this folder. I deleted and re-created all references so that they now point to the local "_Reference" folder. This improve the build time and isolate the teams from each other changes - but the spaghetti of dependencies was still causing a lot of trouble. The teams still needed to constanly update their local _Reference folder with the latest version of the common assemblies otherwise they could experience integration issues later on. But copying constantly the common assemblies is error prone and tedious.
For this project automating the syncing of the local references was a must, therfore I made a small tool: JoPack. I grouped the shared assemblies in packages and created a central xml file representing our project catalogue.
The catalogue is basically an xml file defining the packages and the files included in those packages and the path (source) where they can be found. It also describes the dependencies between the packages. I added a local dependency.xml file to each solution root path. This file lists the packages used by a particular solution. The catalogue is versioned on our source controller and available through a network share.
Based on the dependency.xml, JoPack synchronizes the local reference folder with the shared files.
JoPack download all the packages present in the dependency.xml file but also other packages the root packages are dependent on.
Our shared assemblies listed in the catalog.xml file are still created by CI builds and put on the network shares but now each time a new version of an assembly is created a developer can use JoPack to synchronize his local reference folder. Once the solution is build and all tests are passing the local reference folder can be checked-in making it available to all other developers working on the same application.
We also automated this process by running JoPack as part of our CI build:
We setup a pre-build target that launches JoPack to sync the solution local reference folder on the CI server. When the build succeed the CI build check the _Reference folder back in. Now the developers don’t need to run JoPack on their local machine anymore.
This strategy decreased the complexity due to the number of dependenices because the number of shared parts went from 100 assemblies to 15 packages. The dependencies are also more obvious because we now dispose of a catalogue that provides an overview of all the dependencies between packages. As only 1 version of a particular assembly can be listed in the catalogue the problem of non matching assembly manifest was also solved. What was very apreciated by the developers is that the build time decreased drastically and that they didn't experienced suddenly breaking builds anymore.
Sources, binaries and docs for JoPack can be downloaded here – contribution can be made through the jopack google code repository.
