Created: 2009-05-11 by IanClatworthy
This spec recommends adding explicit support for managing versioned metafiles (like ignores and rules) in an out-of-tree way. Metafiles will be stored in a .bzrmeta directory. These directories will be supported at all levels of the tree, though some metafiles may only be permitted at the root level.
Metafiles will be added, stat'ed, diff'ed, committed, merged, resolved, propagated and deleted like normal user files but will not have content filtering applied and will not (by default at least) be exported. If a metafile is in conflict after a merge, Bazaar will fall back to using the content from the basis tree, rather than break. Renaming of metafiles will be supported but only between meta directories, i.e. changing the basename will be an error.
Having Bazaar-specific files like .bzrignore in the user tree is undesirable. However, storing them inside the .bzr control directory also has its share of problems, e.g. users need to go digging into a directory that is conceptually opaque, normal Bazaar commands like status, diff, merge and commit can't easily be used to manage these files and GUI tools can't easily show that metafiles have been modified and need committing.
A primary design decision is whether to support metafiles at just the root level or at any level in a tree. Providing support at just the root level is simple and minimises performance issues. On the other hand, there are advantages to allowing metafiles at multiple levels including:
Cleaner partitioning of the metadata in large projects, e.g. if filtered views are used so that technical writers see just the doc directory and test engineers see just the test directory, then each filtered view can naturally have its own rules file as required. That's good if only the technical writers want to use keywords, say, in their part of the tree.
- Less temptation to abuse nested trees simply in order to achieve this.
- Easier round-tripping of metadata from foreign systems including Subversion and Git.
- Easier joining/splitting of projects.
Of course, some metafiles may only make sense at the root level, e.g. a metafile containing shared configuration settings. The decision about supporting just-root vs any-level needs to be made on a per metafile basis but the generic facility ought to support both. It should also be noted that Git successfully supports .gitattributes at any level without performance problems so any performance issues on practical datasets are most likely solvable.
At the root level, the metafiles could be stored in .bzr/meta instead of .bzrmeta. I'm against this for several reasons:
- It's not consistent with how metafiles will be supported at non-root levels.
It makes the UI more complex, e.g. TortoiseBzr and similar tools would need to show .bzr as being modified when metafiles change when they ought to hide this directory instead. If that directory were to only be shown occasionally, drilling down into there would need to mask all the files and directories that we don't want users to care about and that mask would be dependent on what plugins were installed, the version of Bazaar used and perhaps the storage format used.
It makes the implementation more complex: supporting a .bzrmeta directory is simple in inventories (for example) vs supporting .bzr/meta.
In summary, supporting metafiles at multiple levels is more flexible than just supporting them at the root level. However, it is likely that most projects will only have metafiles versioned at 2 or 3 levels so the performance impact, if any, ought to be acceptable. Storing metafiles in a .bzrmeta directory conceptually takes these files out of the tree, is easy to explain, has limited impact on the UI and is relatively easy to implement.
Proposed UI enhancements
Very few changes are required fortunately. The following changes are suggested:
- If a metafile is in conflict and needs to be used, a warning should be output but Bazaar should fall back to the basis tree content and continue to operate.
The mv command should not permit the basename of a metafile to be changed. It should also either error or produce a warning if the new parent directory is not .bzrmeta or nested inside a directory of that name.
- A warning should be output if a metafile is found but is not yet versioned.
It should be noted that a similar idea to this was proposed and rejected previously on that basis that only two metafiles existed: .bzrignore and .bzrrules. See http://article.gmane.org/gmane.comp.version-control.bazaar-ng.general/41959. Since then, I think the motivation for doing so has extended, e.g. the idea of using .bzrxxx has been rejected, content filtering has now landed and needs to be disabled for these files.
Some new features (e.g. nested trees) may benefit by having a sanctioned solution to this generic problem. Some old features (e.g. tags) may also benefit.
Upgrading existing branches that already have a .bzrignore file could work as follows. If the user runs bzr ignore XXX, then .bzrignore could be implicitly renamed to .bzr/meta/ignore and then be modified as normal to include the XXX pattern. If bzr ignores was run, a warning could be output suggesting that the user run bzr mv .bzrignore ./bzr/meta/ignore and commit the change. I think handling the preferred location migration this way is safer that using the upgrade command and the subsequent merge challenges that may arise after multiple users have upgraded independently.
It may be necessary down the track for metafiles at the root level to be fetched before other files, e.g. if some configuration settings (like stacking policy?) are stored there that impact the rest of the fetching process. For ignores and rules, that's not necessary: the files are needed before working tree building but not data fetching.
The export command needs to be changed to delegate metafile masking to a lower level rather than doing it itself.
Content filtering needs to be changed to not be applied to things within a .bzrmeta directory.
An API is required for looking up the contents of a metafile. It should check that the metafile is versioned and not in conflict and behave intelligently otherwise.
Perhaps InventoryDirectory should be extended with a method/property for looking up a metafile by name. I don't think we need to change how inventories are serialised, just smart cache the existence of metafiles during instantiation so that checking for the existing of a metafile in a directory performs well.
It's not clear to me yet whether a new branch format is required to support this or not. If so, we need to get that new format into 2.0 if we can.