FIDL Aliases (draft)

Posted on March 17th, 2010 by Kris Gybels; last modified on March 18th, 2010.
Copyright 2010 Kris Gybels. All rights reserved.

A FIDL alias (pronounced as “fiddle”) is similar to a standard Mac OS X Finder alias. It acts as a placeholder for another file which can be in a different folder and even on another volume. Opening the placeholder opens the real file. At least as long as the Finder can find it; which it typically does not after the real file has been moved to yet another volume.

The major difference between standard aliases and FIDL aliases is that FIDL aliases have better support for moving files across volumes. A FIDL (or “File Identity-Based Link”) is based on unique identifiers for a file's contents. The real file can be renamed, moved to another volume, burned to a DVD, stored in a compressed archive and so on, but it can always be easily found again as long as its contents have remained the same. Because a FIDL uses the same identifiers as used on some popular peer-to-peer networks, you might even be able to delete the file altogether and use the FIDL to re-download it from such a network later.

FIDL aliases are not meant to be a replacement for standard Finder aliases. The two complement each other. A FIDL alias is, by its contents-based nature, only useful for files that you can set to be read-only. Typical examples are files you consider archived and are no longer working on, photographs, reference material in PDF format, software installer packages, movie clips and so on.

Utilities Download

Supporting utilities are currently only available through a Subversion repository. The following command can be executed in a Terminal window:

svn checkout 'https://fidler.svn.sourceforge.net/svnroot/fidler/FIDLer Utilities'

You will also need to have RHASH installed. No binary for Mac OS X is available, but the source code package “rhash-1.1.6-src.tar.gz” can be built and installed on Mac OS X.

You can also browse the Subversion repository if you merely want to take a look at the source.

The utilities have been tested by the author on a G4-based Mac using Mac OS X 10.5.4.

For documentation, see the section “Utilities Documentation” below.

Technical

The implementation for FIDL aliases is very straightforward and uses symbolic links with a dual dereferencing scheme. As such it integrates easily in the Mac OS X Finder and could be applied to other Unix-like operating systems. A useful feature of the Finder for these purposes is that the Finder pretends to the end-user that symbolic links are the same as Finder aliases (but technically, they are not, see the related Wikipedia article.)

A FIDL is intentionally similar to a magnet link. Some of the same hash algorithms are used to identify files by content (specifically, MD5, ED2K-style MD4, SHA1 and TTH). An example of a magnet link is:

magnet:?xt=urn:md5:d84c7e2fb778bbe0d3ed2cce1d0eb55e¬
&xt=urn:sha1:OM2X637IFOUJGQXSGXLYVYF3A5V2XBRR¬
&xt=urn:tree:tiger:OAECNNI2DQKXWVBU4637GUAMSVM7QOZDGCX2WJA¬
&xt=urn:ed2k:369f76a75cc2269e9f547c7a6748d1a7&xl=3707897

The corresponding FIDL would be:

/fidl/draft0/3707897_3BGH4L5XPC56BU7NFTHB2DVVLY_¬
G2PXNJ24YITJ5H2UPR5GOSGRU4_OM2X637IFOUJGQXSGXLYVYF3A5V2XBRR_¬
OAECNNI2DQKXWVBU4637GUAMSVM7QOZDGCX2WJA

The main important property of a FIDL is that it can be usefully stored verbatim as (the path contents of) a symbolic link. Such a symbolic link is then called a “FIDL alias.” A symbolic link containing the above FIDL would obviously point to a directory “fidl” at the root of the filesystem. This directory serves as the “FIDL resolving structure” which resolves a FIDL by mapping it to an actual file whose contents match the FIDL's identifier. Technically, this directory consists merely of secondary symbolic links. The mapping is maintained by a few command-line scripts (at least in the current implementation).

The following is an illustration of this scheme, using the above example FIDL: (the file identified by the example FIDL is available for download from travisdart.com.)

Left: The example FIDL stored as a symbolic link named “Apple Switch Ad, Ellen Feiss.mov” on the main hard disk.

Middle: The symbolic link “Apple Switch Ad, Ellen Feiss.mov” points to the FIDL resolving structure in the folder “fidl”. Note that this folder is hidden in the Finder.

Right: The symbolic link in the FIDL resolving structure points to a file on an external disk matching the FIDL.

The two crucial points of this dual dereferencing scheme are that: (1) if the file is moved to another volume only the second symbolic link, the one which is in the fixed known location of the “FIDL resolving structure”, needs to be updated, and (2) the first symbolic link, the “FIDL alias” contains the necessary information based on cryptographic hashes to unambiguously identify a file by content (at least with a certain theoretical near guarantee).

Utilities Documentation

A few basic script utilities to create and manage FIDL aliases are available (download details provided above). Here's some documentation on what they do:

• fidler_setup
  Creates the /fidl directory, also sets this directory to be hidden in the Finder.
• fidler_identify [PATH]
  Computes the content identifier for the file at PATH and maps the identifier to it from the /fidl directory. Also sets the file at PATH to be read-only and locked.
• fidler_linkto [PATH_FILE] [PATH_FIDL]
  Computes the content identifier for the file at PATH_FILE and puts a FIDL alias at PATH_FIDL. Note it does not create a mapping from the /fidl directory, nor does it set the the file at PATH_FILE to read-only.
• fidler_cleanup
  Removes broken links from the /fidl directory.
• fidler_open_bitzi_ticket [PATH_FIDL]
  Opens the corresponding bitzi ticket for the FIDL in the FIDL alias at PATH_FIDL. See the example bitzi ticket for more information on bitzi.
• fidler_offload [PATH]
  A combination of some of the above: moves the file at PATH to some folder, puts a FIDL alias in its place, sets the moved file to be read-only and locked, and sets up the mapping from the /fidl directory. The folder is configurable by editing the script. The idea is to “offload” a file from the main hard drive to some secondary storage device.

Future Work and Issues

Here are a few potential issues and areas of future work:

• The utility scripts currently do not deal with the possibility of the user making files writeable again and modifying their contents, in which case the link from the “/fidl” directory will incorrectly link to a file which does not have the right identity.
• It might be nice to automatically populate the “/fidl” directory with the necessary links when a hard disk is connected to the Mac, or when a DVD is inserted and so on; and conversely to remove the links when the volume becomes unavailable.
• There's a potential security hole in that the “/fidl” directory reveals that a user likely has or had a certain file in possession. For example, a user may wish to store both the “FIDL alias” and the real file on encrypted disk images thinking that this hides the file from view to others. The “/fidl” directory will still reveal the identity of the contents (but not automatically also the contents; this is mostly an issue if the user's copy of the file is not the only copy that has ever existed).
• The current scheme has been designed for use on a computer that is used by a single user only, though it would also seem to work on a multi-user computer if everyone is given access to the “/fidl” directory. Using an additional user-specific prefix to “/fidl” or a similar solution might work to circumvent everyone being able to access the “/fidl” directory, but would break transportability of “FIDL aliases” to other computers and other users. In the current scheme, storing a “FIDL alias” in say a TAR archive would work and allow it to be transported.

Comments and suggestions for improvement are welcomed, the author can be contacted at: “rinzwind AT users DOT sourceforge DOT net”.