Merging OpenFOAM versions in git
A question came up recently on this forum
about how to merge the separate change histories from different OpenFOAM
versions. The simplest solution is to merge with an
ours strategy, which
I’ll present directly.
Of course, you can also question why disconnected git histories exist in the first place and immediately notice that the same type of issues will simply re-occur with future OpenFOAM-1.7.x, OpenFOAM-1.8.x, etc releases. I’ll thus also take the opportunity to explain why I think the current OpenFOAM development model is perhaps not as bad at it might first appear and might indeed be the only practical development model at the moment.
The quick solution
This is literally a quick solution if you are using exising local files.
The merge operations are safe to use within your normal OpenFOAM directories (provided they are in a clean state), but it is also quite reasonable to use a new temporary directory for the operations – either for safety or just as a sandpit to play around a bit. Using a new temporary directory also nicely illustrates the distributed nature of git – each repository being its own centre of the universe.
Starting from a completely empty directory:
mkdir /tmp/OpenFOAM-merge cd /tmp/OpenFOAM-merge
We initialize a git repository:
Since there is obviously nothing there yet, we’ll add some repository content. In this case from the current OpenFOAM version (1.6.x):
git remote add foam16x /SOME/PATH/OpenFOAM-1.6.x/.git git fetch foam16x
We can reuse the
fetch commands to add any number of other
repository references. We’ll also add in the previous OpenFOAM version
git remote add foam15x /SOME/PATH/OpenFOAM-1.5.x/.git git fetch foam15x
git branch -a -l you can display all the known branches.
Since OpenFOAM-1.6.x supercedes OpenFOAM-1.5.x, this is where we should be
when conducting an
ours merge. We’ll select the current topic branch
(usually master) from the current OpenFOAM version (which we added as
foam16x in this example):
git checkout -b master foam16x/master warning: You appear to be on a branch yet to be born. warning: Forcing checkout of foam16x/master Branch master set up to track remote branch refs/remotes/foam16x/master. Already on "master"
To keep things straight, and to avoid accidentally working on a tracked branch, we’ll make sure that we are working on a new branch that doesn’t currently exist in the remote repository. The name is fairly arbitrary, we’ll take merged for the example:
git checkout -b merged HEAD Switched to a new branch "merge"
git branch -a -l again, you can verify that you are indeed on the
correct branch. Now that everything is prepared, the merge itself can be
git merge --strategy ours foam15x/master Merge made by ours.
Here we’ve merged in the foam15x/master branch and instructed git that the current branch (ie, the master from foam16x) always has precedence. The two histories are now joined as can be seen in the log information:
git log --graph
Keeping the merged branch
Even although we worked in a temporary directory to create the merged
branch, we may wish to also have a copy back in our original repositories.
This is easily done by
push-ing the new branch back to a particular
git push foam16x merged To /SOME/PATH/OpenFOAM-1.6.x/.git * [new branch] merged -> merged
Changing back to the original OpenFOAM-1.6.x directory, you can
the new branch:
cd /SOME/PATH/OpenFOAM-1.6.x git checkout merged
Since the directory and file contents were resolved as
ours there are no
differences between the previous master branch and the new merged
branch. The subsequent
checkout merged thus shouldn’t affect any file
Exploring what has changed
Now that both repositories have been united, it is possible to explore what
has changed. For the sake of clarity, I’ll use foam16x/master and
foam15x/master to refer to the respective branches. The astute reader will
notice that the following operations didn’t actually require the
operation at all, only that both repositories be accessible as remotes from
the same repository.
From the usual palette of git commands at our disposal, the most useful at
this point is
diff (and to a certain extent
diff-tree as well).
To get a general idea of where most of the changes occurred:
git diff --stat foam15x/master foam16x/master
which we can also restrict to a particular path:
git diff --stat foam15x/master foam16x/master src/OpenFOAM/db
Or look for differences that contain a particular change. For example, to see all changes to files that use the new argsList::optionFound method:
git diff -SoptionFound --stat foam15x/master foam16x/master
For a more detailed view, the
--stat option is simply dropped. Again, a
path can be specified to restrict the view. For example,
git diff foam15x/master foam16x/master etc/bashrc
Why are there separate repositories / histories anyhow?
I must admit that this annoyed me quite a bit for a while. Why have completely separate repositories for 1.6.x and 1.5.x when separate branches would suffice? It helps to turn the question around:
How would it look like with everything in a single repository?
The real issue is actually about how the development should proceed. From an existing development line, from which the 1.5.x version initially stems, the development continues after a release:
o---o---o---o---o development \ o 1.5.x
The changes in development should normally not find their way into the stable 1.5.x branch unless they are clearly necessary bugfixes or feature requests with limited impingement on the overall code. Even many seemingly innocent cosmetic changes should probably not be applied to the stable branch – do you really want to recompile everything just because someone tweaked the documentation formatting in some key header?
Thus in short, the development and 1.5.x branches will rapidly diverge in their git histories. Bugfixes will be applied to both the development and the 1.5.x branches, but the development branch will obviously have many more commits:
o---o---o---o---o---o---o---o development \ o---o---o 1.5.x
When the time comes for an OpenFOAM-1.6.x release, a tested version of the development branch would be taken:
o---o---o---o---o---o---o---o---o---o---o---o development \ \ o---o---o 1.5.x o 1.6.x
For consistency with the previous example, we can use merge
ours on the
newly created 1.6.x branch to preserve the history (Note: this is
functionally equivalent to a normal upstream merge for this example).
o---o---o---o---o---o---o---o---o---o---o---o development \ \ A---o---o---o 1.5.x B---M---o 1.6.x \ / ----------------------
in which the ‘A’ and ‘B’ represent the branch points for 1.5.x and
1.6.x, respectively, and the ‘M’ represents the subsequent merge point
ours to rejoin the histories.
Thus while separate development and maintenence branches are indeed desirable and necessary, it isn’t strictly necessary that the various branches be split across separate repositories. However, there are also advantages to having separate repositories:
- No confusion for users about which version they are using – one repository per OpenFOAM version.
- The git repositories have fewer changes in them and are thus smaller and faster.
- The git history of the internal development repository is missing from the released versions.
The only remaining difficulty (or irritation) might be that the git history of the internal development repository is missing from the released versions. I don’t know if we really need to know every step of OpenFOAM’s internal development in order to use it. It might help, it might be interesting, but probably isn’t strictly necessary.
On the other hand, I can quite understand why there is no OpenFOAM-dev repository published between official releases. Just imagine the flood of posts on the forum:
I’m using the bleeding edge version and the new class ‘X’ worked last week, but now I get lots of compiler error messages and it won’t compile. What is wrong?
I think you get the idea.