Feature image courtesy of Wikipedia.
As mentioned earlier, this is one of those very technical blog posts about version controls.
I just recently migrated all of my Fossil repositories to Mercurial, and I wanted to post some notes about why I used the version control from the first place, and why I’m moving away from it. Believe it or not, I still love what Fossil does, and highly recommend it to anyone who needs to setup their own personal version control server. There were a few flaws with it, though, I couldn’t accept that ultimately lead to this decision.
When Fossil is awesome
A colleague once told me, if you’re going to start your own software company, you’re going to need at least 3 things to go with your company server: a version control, task manager, and wiki. Lo and behold, Fossil is all three bundled into a portable package. Fossil’s developers even show off this capability by serving their webpage with a Fossil wiki!
So how does one go about setting up a Fossil website? If you have a personal Linux web server with Fossil installed and serving CGI scripts, it’s incredibly simple. Create a repository with this familiar command (where [filename] is the name of the repository):
fossil init [filename]
Make sure you give this new file read, write, and executable permissions for the web server. Lastly, write a quick CGI script containing the following lines (where [path] is the full path to the new repository):
#!/usr/bin/fossil
repository: [path]
Now that the hard part is over, you can access this CGI script with your favorite web browser to get to the project’s wiki. From that website, you can configure the repository’s name, users, permissions, wiki, task manager, etc. It’s amazingly simple.
What about using Fossil? After all, an easy configuration is nothing if the application is difficult to use or hardly functional. Well, Fossil is a distributed revision control, much like Git and Mercurial. Compared to a centralized revision controls like Subversion, distributed revision controls has several differences, including better merging, easier branching and tagging, and offline support at a cost of requiring two commands instead of one to commit or update files from the server. The last fact tends to make it a bit intimidating for users who come from Subversion (currently the industry standard). Fossil, however, finds a very good medium between these 2 systems. By default, Fossil has autosync mode on: that means simply running fossil commit or fossil update will automatically push or pull files to the server. This works great even when you’re offline: when you run fossil commit, Fossil will notice that the connection isn’t there, and will prompt you whether to continue the commit. Saying yes will save the changes locally, in which you can just run fossil sync later to synchronize your changes to the server. Of course, for Git and Mercurial users, they can simply turn off autosync with fossil settings autosync off to have their fossil push and fossil pull back. It’s a very elegant solution that I’m surprised I haven’t found any other version controls copying this idea.
Having offline support is boon with the wiki and bug tracker as well. Running fossil server on your own computer lets you open and edit both the wiki and bug tracker on your own computer, even if it’s offline. You can later sync those changes when you get online back again, making the all-in-one application incredibly portable.
Other than that, branching, tagging, and merging are all comparable to Git: excellent. I haven’t had a single conflict at all using those features (to be fair, I’m the only person accessing the repository, so conflicts are unlikely) and it all just worked. The recursive fossil addremove command is also very useful. It searches for any unversioned and missing files to add or remove respectively. The command only got awkward when used before fossil rename, which renames a file (critical for merging files later). Finally, the program performance quite quickly, which is great. Overall, my experience with Fossil has been a very pleasant one…until it crashed.
When Fossil is not awesome
I use Unity, and I import a lot of things from the Unity Asset Store. Importing things from the asset store sadly proved to be a gigantic problem for Fossil. See, Fossil is not only terrible at dealing with large files (like most distributed revision controls are), but also terrible at dealing with very large commits. Perhaps the problem occurs because Fossil puts every revision into a single file, and when you have a ton of file to pack into a single file, it exceeds the computer memory. Regardless of reason, dealing with this problem was aggravating. Sometimes, committing a bunch of small image files would make Fossil crash. That often forced me to make commit in parts. Sometimes, pushing the cloned repository to the server will crash because a commit in your history was too large. This was especially problematic because Fossil has no rebase command to change its history. While I favor the lack of the command, it makes the problem practically unsolvable. This problem alone was enough for me to start looking into alternatives that handles large files.
Other problems I had with Fossil include a bizarre way of creating new wiki pages, which required going to the “Wiki” tab and selecting “Add Wiki”. Creating a list of tickets was awful as well, since the only way to filter and sort your tickets was actually passing in the SQL command to retrieve them. There also isn’t a single-button way to switch the default settings of permissions to the repository from open-source (default) to private, corporate use. Lastly, there isn’t a Tortoise equivalent to Fossil…yet.
When Fossil is…weird
I’ve mentioned earlier that Fossil repositories are all compressed into a single file. This makes cloning in Fossil…weird. To clone a repository served by a CGI script, http://mycompany/repo.cgi, you have to type in the following commands:
fossil clone http://mycompany/repo.cgi repo.fossil
fossil open repo.fossil
It feels weird to have a version control where running fossil clone doesn’t automatically uncompress the files contained in that repository. Still, it does have its advantages. For example, on the the same repo.fossil, you can run fossil close to compress the repository back to a single file, and move the file into a flash drive along with the tiny Fossil executable. From there, just plop the flash drive onto a different computer, move the repository and executable locally, and run fossil open repo.fossil again to continue your work from that computer. I have actually utilized this a few times with Touch Yoga to borrow my colleagues tools. It’s a neat feature, but I question the validity of it.
Hello Mercurial
In any case, I’ve decided to use Mercurial with Redmine wiki and bugtracker moving forward. It handles offline well (useful for traveling), has extensions including one pre-installed extension called largefiles, and even has TortoiseHg for happy Windows users. Converting from Fossil to Mercurial has been pretty easy (though required a bizarre work-around), and making large commits from imported assets is working very smoothly. So far, I’m a happy camper.
P.S. Speaking of Fossil, from what I understand Veracity is a very similar alternative that, on top of a version control, wiki, and task manager, it also comes with an automated build system as well.
Re: Ticket queries and reports. I agree that a “point and click” query builder would make creating/editing ticket queries and reports easier.
Re: “open source” vs “corporate” default settings. While I can imagine where having a second set of default settings that are more oriented toward a corporate environment, this is not an issue I have had to deal with. At work, my teammates and I manage Fossil ourselves and the company firewall prevents outsiders from accessing our Fossil repositories, so has not been an issue for us.
Re: Large files. Don’t know how large Unity asset files can be, so I don’t know.
Re: cloning. This “weirdness” is because in Fossil, a repository and a “working copy” are 2 different things. In this respect, Fossil is similar to Subversion. One repository can have several working copies associated with it.
In theory, “fossil clone” could automatically do a “fossil open”. I don’t know why the designers of Fossil chose not to, but since I was using Subversion when I discovered Fossil, this made sense to me. It also allows Fossil users to, for example, put repositories on one hardrive and working copies on a different harddrive.
“fossil open” does not uncompress anything. It associates the new working copy with the repository, then automatically runs “fossil update” to populate the new working copy. You can create as many working copies as you want, all “connected” to the same repository, so anything you commit from one working copy is immediately available via “fossil update” in the other working copies – no push or pull needed.
(In GIt and Hg (and most other DVCs), every working copy has an embedded repository, so the only way to create more working copies is to clone the repository, which is why you have to push (or pull) changes for them to be available in other working copies.)
In fact, “fossil close” is not needed. As long as all your pending changes are committed (or reverted), you can copy the repository file to another PC, makes changes on that PC, commit them, copy the repository back to the first PC then do “fossil update” in your existing working copy. “fossil close” does, of course, check for pending changes in the working copy, but does not compress anything.
FYI, the “single file” Fossil stores the repository in is a database. Fossil uses a “mini” RDBMS called SQLite. But even if Fossil used, for example, Oracle, this database could be “just” a single file.
Hey, RonW. Thanks for the clone details. That’s pretty awesome that you only need to clone once to have multiple working copies. I really like how Fossil operates.
As mentioned in the blog post, my main issues are crashing on large files and commits. While Unity itself doesn’t generate super-large files (I think the worst I’ve got was a 2MB scene file), there are a lot of images and sound files I deal with that inflates that number. It’s not uncommon for me to make commits of 600 new files, most of them small images around 0.3 MB. I’ve seen Fossil crash multiple times when making these commits, enough so for me to quit. If there’s a way to get around that, that would be all sorts of wonderful in my book.
You’re welcome.
There has been some discussion on making Fossil scale better. Primary areas raised so far are recovery from interrupted pull/push/sync/clone operations and supporting “partial history” clones. That is, a clone operation could request a subset of branches and/or limited date range so that the user will have only whats really needed to look at what she/he is interested in.
Maybe this will help. I don’t know about the crashing. Possibly there is a bug in how Fossil rolls back an interrupted operation.
It’s been a while since I’ve used Fossil, so who knows. Maybe it’s already fixed.
Anyway, to provide more details on the issue, here’s what I typically see on a Windows 7 Powershell:
1) Import a package in Unity.
2) Call “fossil addremove” on the project to add all 200+ files.
3) Call ‘fossil commit -m “importing stuff.”‘ on the project to commit.
4) Wait for a while. If memory serves correct, it hangs for a few minutes.
5) If memory serves correct (it’s fuzzy here), the console throws a bunch of errors.
I’ll try to see if I can replicate this behavior with the latest version of Fossil. If not, I may jump back to it on my next project!
I’ve tried this yesterday, committing 280 MB of files at once, and didn’t seem to find any problems. I know I was using the same machine, so maybe it’s fixed. I’ll give it a go on my next project.
Good to hear that worked.
Re: “open source” vs “corporate” default settings
I just remembered 2 things that partially mitigate the absence of this:
1. “fossil new” has a “–template” option to obtain settings from. It claims to require a path to the repo file.
2. “fossil config” has “export”, “import”, “merge” and other commands to save/load settings to/from a file or even another Fossil server.