Next: , Previous: Error handling, Up: Top


10 Miscellaneous additional functionality

ASDF includes several additional features that are generally useful for system definition and development.

10.1 Controlling file compilation

When declaring a component (system, module, file), you can specify a keyword argument :around-compile function. If left unspecified, the value will be inherited from the parent component if any, or with a default of nil if no value is specified in any transitive parent.

The argument must be a either nil, a fbound symbol, a lambda-expression (e.g. (lambda (thunk) ...(funcall thunk) ...)) a function object (e.g. using #.#' but that's discouraged because it prevents the introspection done by e.g. asdf-dependency-grovel), or a string that when read yields a symbol or a lambda-expression. nil means the normal compile-file function will be called. A non-nil value designates a function of one argument that will be called with a thunk for calling the compile-file function with proper arguments.

Note that by using a string, you may reference a function, symbol and/or package that will only be created later during the build, but isn't yet present at the time the defsystem form is evaluated. However, if your entire system is using such a hook, you may have to explicitly override the hook with nil for all the modules and files that are compiled before the hook is defined.

Using this hook, you may achieve such effects as: locally renaming packages, binding *readtables* and other syntax-controlling variables, handling warnings and other conditions, proclaiming consistent optimization settings, saving code coverage information, maintaining meta-data about compilation timings, setting gensym counters and PRNG seeds and other sources of non-determinism, overriding the source-location and/or timestamping systems, checking that some compile-time side-effects were properly balanced, etc.

Note that there is no around-load hook. This is on purpose. Some implementations such as ECL or GCL link object files, which allows for no such hook. Other implementations allow for concatenating FASL files, which doesn't allow for such a hook either. We aim to discourage something that's not portable, and has some dubious impact on performance and semantics even when it is possible. Things you might want to do with an around-load hook are better done around-compile, though it may at times require some creativity (see e.g. the package-renaming system).

10.2 Controlling source file character encoding

Starting with ASDF 2.21, components accept a :encoding option. By default, only :default, :utf-8 and :autodetect are accepted. :autodetect is the default, and calls *encoding-detection-hook* which by default always returns *default-encoding* which itself defaults to :default. In other words, there now are plenty of extension hooks, but by default ASDF follows the backwards compatible behavior of using whichever :default encoding your implementation uses, which itself may or may not vary based on environment variables and other locale settings. In practice this means that only source code that only uses ASCII is guaranteed to be read the same on all implementations independently from any user setting.

Additionally, for backward-compatibility with older versions of ASDF and/or with implementations that do not support unicode and its many encodings, you may want to use the reader conditionals #+asdf-unicode #+asdf-unicode to protect any :encoding encoding statement as :asdf-unicode will be present in *features* only if you're using a recent ASDF on an implementation that supports unicode. We recommend that you avoid using unprotected :encoding specifications until after ASDF 2.21 becomes widespread, hopefully by the end of 2012.

While it offers plenty of hooks for extension, and one such extension is being developed (see below), ASDF itself only recognizes one encoding beside :default, and that is :utf-8, which is the de facto standard, already used by the vast majority of libraries that use more than ASCII. On implementations that do not support unicode, the feature :asdf-unicode is absent, and the :default external-format is used to read even source files declared as :utf-8. On these implementations, non-ASCII characters intended to be read as one CL character may thus end up being read as multiple CL characters. In most cases, this shouldn't affect the software's semantics: comments will be skipped just the same, strings with be read and printed with slightly different lengths, symbol names will be accordingly longer, but none of it should matter. But a few systems that actually depend on unicode characters may fail to work properly, or may work in a subtly different way. See for instance lambda-reader.

We invite you to embrace UTF-8 as the encoding for non-ASCII characters starting today, even without any explicit specification in your .asd files. Indeed, on some implementations and configurations, UTF-8 is already the :default, and loading your code may cause errors if it is encoded in anything but UTF-8. Therefore, even with the legacy behavior, non-UTF-8 is guaranteed to break for some users, whereas UTF-8 is pretty much guaranteed not to break anywhere (provided you do not use a BOM), although it might be read incorrectly on some implementations. In the future, we intend to make :utf-8 the default value of *default-encoding*, to be enforced everywhere, so at least the code is guaranteed to be read correctly everywhere it can be.

If you need non-standard character encodings for your source code, use the extension system asdf-encodings, by specifying :defsystem-depends-on (:asdf-encodings) in your defsystem. This extension system will register support for more encodings using the *encoding-external-format-hook* facility, so you can explicitly specify :encoding :latin1 in your .asd file. Using the *encoding-detection-hook* it will also eventually implement some autodetection of a file's encoding from an emacs-style -*- mode: lisp ; coding: latin1 -*- declaration, or otherwise based on an analysis of octet patterns in the file. At this point, asdf-encoding only supports the encodings that are supported as part of your implementation. Since the list varies depending on implementations, we once again recommend you use :utf-8 everywhere, which is the most portable (next is :latin1).

If you're not using a version of Quicklisp that has it, you may get the source for asdf-encodings using git: git clone git://common-lisp.net/projects/asdf/asdf-encodings.git or git clone ssh://common-lisp.net/project/asdf/git/asdf-encodings.git. You can also browse the repository on http://common-lisp.net/gitweb?p=projects/asdf/asdf-encodings.git.

In the future, we intend to change the default *default-encoding* to :utf-8, which is already the de facto standard for most libraries that use non-ASCII characters: utf-8 works everywhere and was backhandedly enforced by a lot of people using SBCL and utf-8 and sending reports to authors so they make their packages compatible. A survey showed only about a handful few libraries are incompatible with non-UTF-8, and then, only in comments, and we believe that authors will adopt UTF-8 when prompted. See the April 2012 discussion on the asdf-devel mailing-list. For backwards compatibility with users who insist on a non-UTF-8 encoding, but cannot immediately transition to using asdf-encodings (maybe because it isn't ready), it will still be possible to use the :encoding :default option in your defsystem form to restore the behavior of ASDF 2.20 and earlier. This shouldn't be required in libraries, because user pressure as mentioned above will already have pushed library authors towards using UTF-8; but authors of end-user programs might care.

When you use asdf-encodings, any further loaded .asd file will use the autodetection algorithm to determine its encoding; yet if you depend on this detection happening, you may want to explicitly load asdf-encodings early in your build, for by the time you can use :defsystem-depends-on, it is already too late to load it. In practice, this means that the *default-encoding* is usually used for .asd files. Currently, this defaults to :default for backwards compatibility, and that means that you shouldn't rely on non-ASCII characters in a .asd file. Since component (path)names are the only real data in these files, and non-ASCII characters are not very portable for file names, this isn't too much of an issue. We still encourage you to use either plain ASCII or UTF-8 in .asd files, as we intend to make :utf-8 the default encoding in the future. This might matter, for instance, in meta-data about author's names.

10.3 Miscellaneous Exported Functions

— Function: coerce-pathname name &key type defaults

This function (available starting with ASDF 2.012.11) takes an argument, and portably interprets it as a pathname. If the argument name is a pathname or nil, it is passed through; if it's a symbol, it's interpreted as a string by downcasing it; if it's a string, it is first separated using / into substrings; the leading substrings denote subdirectories of a relative pathname. If type is :directory or the string ends with /, the last substring is also a subdirectory; if type is a string, it is used as the type of the pathname, and the last substring is the name component of the pathname; if type is nil, the last substring specifies both name and type components of the pathname, with the last . separating them, or only the name component if there's no last . or if there is only one dot and it's the first character. The host, device and version components come from defaults, which defaults to *default-pathname-defaults*; but that shouldn't matter if you use merge-pathnames*.

— Function: merge-pathnames* &key specified defaults

This function is a replacement for merge-pathnames that uses the host and device from the defaults rather than the specified pathname when the latter is a relative pathname. This allows ASDF and its users to create and use relative pathnames without having to know beforehand what are the host and device of the absolute pathnames they are relative to.

— Function: system-relative-pathname system name &key type

It's often handy to locate a file relative to some system. The system-relative-pathname function meets this need.

It takes two mandatory arguments system and name and a keyword argument type: system is name of a system, whereas name and optionally type specify a relative pathname, interpreted like a component pathname specifier by coerce-pathname. See Pathname specifiers.

It returns a pathname built from the location of the system's source directory and the relative pathname. For example:

          > (asdf:system-relative-pathname 'cl-ppcre "regex.data")
          #P"/repository/other/cl-ppcre/regex.data"
— Function: system-source-directory system-designator

ASDF does not provide a turnkey solution for locating data (or other miscellaneous) files that are distributed together with the source code of a system. Programmers can use system-source-directory to find such files. Returns a pathname object. The system-designator may be a string, symbol, or ASDF system object.

— Function: clear-system system-designator

It is sometimes useful to force recompilation of a previously loaded system. In these cases, it may be useful to (asdf:clear-system :foo) to remove the system from the table of currently loaded systems; the next time the system foo or one that depends on it is re-loaded, foo will then be loaded again. Alternatively, you could touch foo.asd or remove the corresponding fasls from the output file cache. (It was once conceived that one should provide a list of systems the recompilation of which to force as the :force keyword argument to load-system; but this has never worked, and though the feature was fixed in ASDF 2.000, it remains cerror'ed out as nobody ever used it.)

Note that this does not and cannot by itself undo the previous loading of the system. Common Lisp has no provision for such an operation, and its reliance on irreversible side-effects to global datastructures makes such a thing impossible in the general case. If the software being re-loaded is not conceived with hot upgrade in mind, this re-loading may cause many errors, warnings or subtle silent problems, as packages, generic function signatures, structures, types, macros, constants, etc. are being redefined incompatibly. It is up to the user to make sure that reloading is possible and has the desired effect. In some cases, extreme measures such as recursively deleting packages, unregistering symbols, defining methods on update-instance-for-redefined-class and much more are necessary for reloading to happen smoothly. ASDF itself goes through notable pains to make such a hot upgrade possible with respect to its own code, and what it does is ridiculously complex; look at the beginning of asdf.lisp to see what it does.

— Function: run-shell-command

This function is obsolete and present only for the sake of backwards-compatibility: “If it's not backwards, it's not compatible”. We strongly discourage its use. Its current behavior is only well-defined on Unix platforms (which include MacOS X and cygwin). On Windows, anything goes.

Instead we recommend the use of such a function as xcvb-driver:run-program/ from the xcvb-driver system that is distributed with XCVB: http://common-lisp.net/project/xcvb. It's only alternative that supports as many implementations and operating systems as ASDF does, and provides well-defined behavior outside Unix (i.e. on Windows). (The only unsupported exception is Genera, since on it run-shell-command doesn't make sense anyway on that platform).

run-shell-command takes as arguments a format control-string and arguments to be passed to format after this control-string to produce a string. This string is a command that will be evaluated with a POSIX shell if possible; yet, on Windows, some implementations will use CMD.EXE, while others (like SBCL) will make an attempt at invoking a POSIX shell (and fail if it is not present).