| |
| Aufs3 -- advanced multi layered unification filesystem version 3.x |
| http://aufs.sf.net |
| Junjiro R. Okajima |
| |
| |
| 0. Introduction |
| ---------------------------------------- |
| In the early days, aufs was entirely re-designed and re-implemented |
| Unionfs Version 1.x series. After many original ideas, approaches, |
| improvements and implementations, it becomes totally different from |
| Unionfs while keeping the basic features. |
| Recently, Unionfs Version 2.x series begin taking some of the same |
| approaches to aufs1's. |
| Unionfs is being developed by Professor Erez Zadok at Stony Brook |
| University and his team. |
| |
| Aufs3 supports linux-3.0 and later. |
| If you want older kernel version support, try aufs2-2.6.git or |
| aufs2-standalone.git repository, aufs1 from CVS on SourceForge. |
| |
| Note: it becomes clear that "Aufs was rejected. Let's give it up." |
| According to Christoph Hellwig, linux rejects all union-type filesystems |
| but UnionMount. |
| <http://marc.info/?l=linux-kernel&m=123938533724484&w=2> |
| |
| |
| 1. Features |
| ---------------------------------------- |
| - unite several directories into a single virtual filesystem. The member |
| directory is called as a branch. |
| - you can specify the permission flags to the branch, which are 'readonly', |
| 'readwrite' and 'whiteout-able.' |
| - by upper writable branch, internal copyup and whiteout, files/dirs on |
| readonly branch are modifiable logically. |
| - dynamic branch manipulation, add, del. |
| - etc... |
| |
| Also there are many enhancements in aufs1, such as: |
| - readdir(3) in userspace. |
| - keep inode number by external inode number table |
| - keep the timestamps of file/dir in internal copyup operation |
| - seekable directory, supporting NFS readdir. |
| - whiteout is hardlinked in order to reduce the consumption of inodes |
| on branch |
| - do not copyup, nor create a whiteout when it is unnecessary |
| - revert a single systemcall when an error occurs in aufs |
| - remount interface instead of ioctl |
| - maintain /etc/mtab by an external command, /sbin/mount.aufs. |
| - loopback mounted filesystem as a branch |
| - kernel thread for removing the dir who has a plenty of whiteouts |
| - support copyup sparse file (a file which has a 'hole' in it) |
| - default permission flags for branches |
| - selectable permission flags for ro branch, whether whiteout can |
| exist or not |
| - export via NFS. |
| - support <sysfs>/fs/aufs and <debugfs>/aufs. |
| - support multiple writable branches, some policies to select one |
| among multiple writable branches. |
| - a new semantics for link(2) and rename(2) to support multiple |
| writable branches. |
| - no glibc changes are required. |
| - pseudo hardlink (hardlink over branches) |
| - allow a direct access manually to a file on branch, e.g. bypassing aufs. |
| including NFS or remote filesystem branch. |
| - userspace wrapper for pathconf(3)/fpathconf(3) with _PC_LINK_MAX. |
| - and more... |
| |
| Currently these features are dropped temporary from aufs3. |
| See design/08plan.txt in detail. |
| - test only the highest one for the directory permission (dirperm1) |
| - copyup on open (coo=) |
| - nested mount, i.e. aufs as readonly no-whiteout branch of another aufs |
| (robr) |
| - statistics of aufs thread (/sys/fs/aufs/stat) |
| - delegation mode (dlgt) |
| a delegation of the internal branch access to support task I/O |
| accounting, which also supports Linux Security Modules (LSM) mainly |
| for Suse AppArmor. |
| - intent.open/create (file open in a single lookup) |
| |
| Features or just an idea in the future (see also design/*.txt), |
| - reorder the branch index without del/re-add. |
| - permanent xino files for NFSD |
| - an option for refreshing the opened files after add/del branches |
| - 'move' policy for copy-up between two writable branches, after |
| checking free space. |
| - light version, without branch manipulation. (unnecessary?) |
| - copyup in userspace |
| - inotify in userspace |
| - readv/writev |
| - xattr, acl |
| |
| |
| 2. Download |
| ---------------------------------------- |
| There were three GIT trees for aufs3, aufs3-linux.git, |
| aufs3-standalone.git, and aufs-util.git. Note that there is no "3" in |
| "aufs-util.git." |
| While the aufs-util is always necessary, you need either of aufs3-linux |
| or aufs3-standalone. |
| |
| The aufs3-linux tree includes the whole linux mainline GIT tree, |
| git://git.kernel.org/.../torvalds/linux.git. |
| And you cannot select CONFIG_AUFS_FS=m for this version, eg. you cannot |
| build aufs3 as an external kernel module. |
| |
| On the other hand, the aufs3-standalone tree has only aufs source files |
| and necessary patches, and you can select CONFIG_AUFS_FS=m. |
| |
| You will find GIT branches whose name is in form of "aufs3.x" where "x" |
| represents the linux kernel version, "linux-3.x". For instance, |
| "aufs3.0" is for linux-3.0. For latest "linux-3.x-rcN", use |
| "aufs3.x-rcN" branch. |
| |
| o aufs3-linux tree |
| $ git clone --reference /your/linux/git/tree \ |
| git://git.code.sf.net/p/aufs/aufs3-linux aufs-aufs3-linux \ |
| aufs3-linux.git |
| - if you don't have linux GIT tree, then remove "--reference ..." |
| $ cd aufs3-linux.git |
| $ git checkout origin/aufs3.0 |
| |
| o aufs3-standalone tree |
| $ git clone git://git.code.sf.net/p/aufs/aufs3-standalone \ |
| aufs3-standalone.git |
| $ cd aufs3-standalone.git |
| $ git checkout origin/aufs3.0 |
| |
| o aufs-util tree |
| $ git clone git://git.code.sf.net/p/aufs/aufs-util \ |
| aufs-util.git |
| $ cd aufs-util.git |
| $ git checkout origin/aufs3.0 |
| |
| Note: The 3.x-rcN branch is to be used with `rc' kernel versions ONLY. |
| The minor version number, 'x' in '3.x', of aufs may not always |
| follow the minor version number of the kernel. |
| Because changes in the kernel that cause the use of a new |
| minor version number do not always require changes to aufs-util. |
| |
| Since aufs-util has its own minor version number, you may not be |
| able to find a GIT branch in aufs-util for your kernel's |
| exact minor version number. |
| In this case, you should git-checkout the branch for the |
| nearest lower number. |
| |
| For (an unreleased) example: |
| If you are using "linux-3.10" and the "aufs3.10" branch |
| does not exist in aufs-util repository, then "aufs3.9", "aufs3.8" |
| or something numerically smaller is the branch for your kernel. |
| |
| Also you can view all branches by |
| $ git branch -a |
| |
| |
| 3. Configuration and Compilation |
| ---------------------------------------- |
| Make sure you have git-checkout'ed the correct branch. |
| |
| For aufs3-linux tree, |
| - enable CONFIG_AUFS_FS. |
| - set other aufs configurations if necessary. |
| |
| For aufs3-standalone tree, |
| There are several ways to build. |
| |
| 1. |
| - apply ./aufs3-kbuild.patch to your kernel source files. |
| - apply ./aufs3-base.patch too. |
| - apply ./aufs3-proc_map.patch too, if you want to make /proc/PID/maps (and |
| others including lsof(1)) show the file path on aufs instead of the |
| path on the branch fs. |
| - apply ./aufs3-standalone.patch too, if you have a plan to set |
| CONFIG_AUFS_FS=m. otherwise you don't need ./aufs3-standalone.patch. |
| - copy ./{Documentation,fs,include/uapi/linux/aufs_type.h} files to your |
| kernel source tree. Never copy $PWD/include/uapi/linux/Kbuild. |
| - enable CONFIG_AUFS_FS, you can select either |
| =m or =y. |
| - and build your kernel as usual. |
| - install the built kernel. |
| Note: Since linux-3.9, every filesystem module requires an alias |
| "fs-<fsname>". You should make sure that "fs-aufs" is listed in your |
| modules.aliases file if you set CONFIG_AUFS_FS=m. |
| - install the header files too by "make headers_install" to the |
| directory where you specify. By default, it is $PWD/usr. |
| "make help" shows a brief note for headers_install. |
| - and reboot your system. |
| |
| 2. |
| - module only (CONFIG_AUFS_FS=m). |
| - apply ./aufs3-base.patch to your kernel source files. |
| - apply ./aufs3-proc_map.patch too to your kernel source files, |
| if you want to make /proc/PID/maps (and others including lsof(1)) show |
| the file path on aufs instead of the path on the branch fs. |
| - apply ./aufs3-standalone.patch too. |
| - build your kernel, don't forget "make headers_install", and reboot. |
| - edit ./config.mk and set other aufs configurations if necessary. |
| Note: You should read $PWD/fs/aufs/Kconfig carefully which describes |
| every aufs configurations. |
| - build the module by simple "make". |
| Note: Since linux-3.9, every filesystem module requires an alias |
| "fs-<fsname>". You should make sure that "fs-aufs" is listed in your |
| modules.aliases file. |
| - you can specify ${KDIR} make variable which points to your kernel |
| source tree. |
| - install the files |
| + run "make install" to install the aufs module, or copy the built |
| $PWD/aufs.ko to /lib/modules/... and run depmod -a (or reboot simply). |
| + run "make install_headers" (instead of headers_install) to install |
| the modified aufs header file (you can specify DESTDIR which is |
| available in aufs standalone version's Makefile only), or copy |
| $PWD/usr/include/linux/aufs_type.h to /usr/include/linux or wherever |
| you like manually. By default, the target directory is $PWD/usr. |
| - no need to apply aufs3-kbuild.patch, nor copying source files to your |
| kernel source tree. |
| |
| Note: The header file aufs_type.h is necessary to build aufs-util |
| as well as "make headers_install" in the kernel source tree. |
| headers_install is subject to be forgotten, but it is essentially |
| necessary, not only for building aufs-util. |
| You may not meet problems without headers_install in some older |
| version though. |
| |
| And then, |
| - read README in aufs-util, build and install it |
| - note that your distribution may contain an obsoleted version of |
| aufs_type.h in /usr/include/linux or something. When you build aufs |
| utilities, make sure that your compiler refers the correct aufs header |
| file which is built by "make headers_install." |
| - if you want to use readdir(3) in userspace or pathconf(3) wrapper, |
| then run "make install_ulib" too. And refer to the aufs manual in |
| detail. |
| |
| |
| 4. Usage |
| ---------------------------------------- |
| At first, make sure aufs-util are installed, and please read the aufs |
| manual, aufs.5 in aufs-util.git tree. |
| $ man -l aufs.5 |
| |
| And then, |
| $ mkdir /tmp/rw /tmp/aufs |
| # mount -t aufs -o br=/tmp/rw:${HOME} none /tmp/aufs |
| |
| Here is another example. The result is equivalent. |
| # mount -t aufs -o br=/tmp/rw=rw:${HOME}=ro none /tmp/aufs |
| Or |
| # mount -t aufs -o br:/tmp/rw none /tmp/aufs |
| # mount -o remount,append:${HOME} /tmp/aufs |
| |
| Then, you can see whole tree of your home dir through /tmp/aufs. If |
| you modify a file under /tmp/aufs, the one on your home directory is |
| not affected, instead the same named file will be newly created under |
| /tmp/rw. And all of your modification to a file will be applied to |
| the one under /tmp/rw. This is called the file based Copy on Write |
| (COW) method. |
| Aufs mount options are described in aufs.5. |
| If you run chroot or something and make your aufs as a root directory, |
| then you need to customize the shutdown script. See the aufs manual in |
| detail. |
| |
| Additionally, there are some sample usages of aufs which are a |
| diskless system with network booting, and LiveCD over NFS. |
| See sample dir in CVS tree on SourceForge. |
| |
| |
| 5. Contact |
| ---------------------------------------- |
| When you have any problems or strange behaviour in aufs, please let me |
| know with: |
| - /proc/mounts (instead of the output of mount(8)) |
| - /sys/module/aufs/* |
| - /sys/fs/aufs/* (if you have them) |
| - /debug/aufs/* (if you have them) |
| - linux kernel version |
| if your kernel is not plain, for example modified by distributor, |
| the url where i can download its source is necessary too. |
| - aufs version which was printed at loading the module or booting the |
| system, instead of the date you downloaded. |
| - configuration (define/undefine CONFIG_AUFS_xxx) |
| - kernel configuration or /proc/config.gz (if you have it) |
| - behaviour which you think to be incorrect |
| - actual operation, reproducible one is better |
| - mailto: aufs-users at lists.sourceforge.net |
| |
| Usually, I don't watch the Public Areas(Bugs, Support Requests, Patches, |
| and Feature Requests) on SourceForge. Please join and write to |
| aufs-users ML. |
| |
| |
| 6. Acknowledgements |
| ---------------------------------------- |
| Thanks to everyone who have tried and are using aufs, whoever |
| have reported a bug or any feedback. |
| |
| Especially donators: |
| Tomas Matejicek(slax.org) made a donation (much more than once). |
| Since Apr 2010, Tomas M (the author of Slax and Linux Live |
| scripts) is making "doubling" donations. |
| Unfortunately I cannot list all of the donators, but I really |
| appreciate. |
| It ends Aug 2010, but the ordinary donation URL is still available. |
| <http://sourceforge.net/donate/index.php?group_id=167503> |
| Dai Itasaka made a donation (2007/8). |
| Chuck Smith made a donation (2008/4, 10 and 12). |
| Henk Schoneveld made a donation (2008/9). |
| Chih-Wei Huang, ASUS, CTC donated Eee PC 4G (2008/10). |
| Francois Dupoux made a donation (2008/11). |
| Bruno Cesar Ribas and Luis Carlos Erpen de Bona, C3SL serves public |
| aufs2 GIT tree (2009/2). |
| William Grant made a donation (2009/3). |
| Patrick Lane made a donation (2009/4). |
| The Mail Archive (mail-archive.com) made donations (2009/5). |
| Nippy Networks (Ed Wildgoose) made a donation (2009/7). |
| New Dream Network, LLC (www.dreamhost.com) made a donation (2009/11). |
| Pavel Pronskiy made a donation (2011/2). |
| Iridium and Inmarsat satellite phone retailer (www.mailasail.com), Nippy |
| Networks (Ed Wildgoose) made a donation for hardware (2011/3). |
| Max Lekomcev (DOM-TV project) made a donation (2011/7, 12, 2012/3, 6 and |
| 11). |
| Sam Liddicott made a donation (2011/9). |
| Era Scarecrow made a donation (2013/4). |
| Bor Ratajc made a donation (2013/4). |
| Alessandro Gorreta made a donation (2013/4). |
| POIRETTE Marc made a donation (2013/4). |
| Alessandro Gorreta made a donation (2013/4). |
| lauri kasvandik made a donation (2013/5). |
| pemasu from Finland made a donation (2013/7). |
| |
| Thank you very much. |
| Donations are always, including future donations, very important and |
| helpful for me to keep on developing aufs. |
| |
| |
| 7. |
| ---------------------------------------- |
| If you are an experienced user, no explanation is needed. Aufs is |
| just a linux filesystem. |
| |
| |
| Enjoy! |
| |
| # Local variables: ; |
| # mode: text; |
| # End: ; |