Note on the Structure of Lispix Source Code, as of October 1999, about the time of version Lx02.

Port of MacLispix to PC

David S. Bright (NIST 837.02)

Kevin Milans (CMU)

Problem: 

MacLispix (MLx) has run only on the Macintosh.  Many users of particular special purpose tools (widgets) have access to PC’s only.

LISP is a programming language with many variants available on various platforms.  Common Lisp (CL) is the result of an attempt on behalf of the LISP community to consolidate these variants into one uniform, portable specification of the LISP language.  CL is an ANSI standard and programs written in CL run under many different LISP environments, including those available for Macintosh, UNIX, and PC based platforms.  MLx was not originally written in CL, but rather in the variation of LISP known as Macintosh Common Lisp (MCL).  Therefore, many of MLx’s useful image handling and graphics features used  LISP functions specific to MCL and thus were only usable on a Macintosh.

Alternative Solutions:

Rewrite MLx completely in C, or in PC-compatible LISP (Allegro Common Lisp for PC - ACL)?

A complete rewrite of MLx in the C language was dismissed because such a rewrite would not solve the portability problems of MLx.  Although C and C++ are portable languages, neither provide mechanisms to work with a Graphical User Interface (GUI).  After such a rewrite, MLx would merely have C portability problems rather than LISP portability problems.

A complete rewrite of MLx in Allegro Common Lisp (ACL), a LISP variant available on PC and UNIX platforms, was similarly dismissed because MLx would suffer the consequences of a fork.  A fork occurs whenever the instructions that make up a single program are duplicated and changes are made independently to the two code bases.  After the fork occurs, both code bases must be maintained separately; changes/improvements to one code base do not benefit the other.

Use a common graphics package?

The use of a third party portable graphics extension to the CL language was rejected for a variety of reasons.  First, these portable graphics extensions to the CL language tend to be very expensive.  Second, MLx aims to be an Open Source program, released under a license similar in spirit to the GNU (http://www.gnu.org/) General Public License (GPL) (http://www.gnu.org/copyleft/gpl.html).  (The GPL is a legal document that protects the rights of any individual to study and improve the source code to programs provided under this license.)  Such programs are much more valuable to the Open Source community when they do not depend on expensive third party libraries.  Finally, most of the benefits afforded by such a third party extension could be realized through the design a new, relatively small, and  independent extension to the CL language.

Solution:

Write as much as possible in CL.

Although MLx is a graphical program and, by necessity, employs GUI functionality outside the scope of CL, much of MLx’s utility  to the scientific community is based upon special purpose widgets;  ultimately, these widgets perform their job on an image by mathematically manipulating the numbers which represent the image. MLx works on images in CL by manipulating the numeric arrays which represent the image.

Rewrite of MLx in CL

Some of the main goals in porting MLx were to abstract certain elements of the GUI functionality MLx required, and implement them using both ACL and MCL specific functions under a CL interface.  For example, in addressing how MLx should deal with images, the first step towards a solution is to abstract the concept of an image, that is to say, one must explicitly define what an image is and what operations may be preformed on it.  Next, the newly abstracted image is implemented; the code for using the image is written  in both ACL and MCL separately.  This entire process progresses with the idea that the two separate implementations of the image are to be used through the same CL interface; in fact, when all the allowed operations on an image were defined in the abstraction step, care was taken to ensure that MLx could request any of them using CL.  (For example, while MLx cannot directly move a window to a given location on the screen because CL lacks this functionality, MLx can request that a window be moved to a given location on the screen in CL.  Such a request is known as a function call to the CL interface.)

CP Library

In such a situation where non-CL operations are requested through a CL interface, it is only natural for a library to emerge.  MLx’s library, nicknamed CP for “Compatibility Package”, is the collection of all the code needed to perform non-CL tasks such as displaying images or resizing windows.  Although it contains a mixture of LISP code written in ACL, MCL, and pure CL, from the point of view of MLx, it serves as a black box with a simple CL interface.

The CL portion of the library contains code which deals largely with the interface of the library; it provides the CL descriptions and functions of various non-CL objects such as images and windows.

The architecture specific portion of the library contains the code which receives the CL requests for operations on the various objects and translates these requests into system specific instructions which perform the requests.  Currently, the CP library contains architecture specific portions for ACL (Windows) and MCL (Macintosh) LISP environments.

Issues

The Mac and PC have very different Operating Systems (OS) and graphics conventions.  Both the ACL and MCL LISP programming environments help  hide some of the differences by providing high level interfaces to the GUI and OS.  However, some of these differences were still quite noticeable.

File names

Traditionally, the Mac, PC, and UNIX systems have had different file naming conventions.  Although CL provides a file naming convention itself, many of MLx’s files were not originally named according to this convention; the offending files were renamed to conform.  (The CL file naming convention was specifically designed to ensure compatibility with traditional Mac, PC, and UNIX filesystems, so adopting the CL file naming convention solved further filename problems.)

UNIX server for CVS.

The development of MLx requires that text files be shared and edited concurrently on PC and Mac systems.  Unfortunately, some problems immediately arose:

Conflicts can occur when two people work on the same file.  For example, one person may work on one function in a file and save changes.  Unfortunately, another person’s work on that same file could have been overwritten unbeknown to the programmer who saved his/her work.  Secondly, the Mac, PC, and UNIX systems all differ in how they mark the end of a line in a text file.  Any file saved on one system must be converted to the other system’s end of line conventions before use.

The Concurrent Versions System (CVS) from Cyclic Software (http://www.cyclic.com/) is an OpenSource, GPLed versioning system used widely in the development of programs.  CVS, a robust system designed to help coordinate the work of hundreds of programmers, easily addressed both of the problems of concurrent development on different platforms.  A modest Intel based 586 running Linux, an OpenSource, GPLed UNIX style Operating System, was deployed as the CVS server for MLx. 

Port to other platforms - Unix boxes & X windows in particular.

Ports of MLx to other platforms are quite possible and should prove relatively easy once the initial rewrite of MLx is complete.  All that would be required to port MLx to a new system is a CL environment as well as some means by which a programmer might implement the architecture specific portions of the CP library under the new environment.

In particular, UNIX systems have a variety of CL LISP implementations, including Allegro Common Lisp, Lucid, CLISP, and CMUCL.  The latter two are available freely to the public with source code.  Further, extensions to the CL language which allow use of the X windows graphic system such as CLX and Garnet are also available for these LISP environments.

Documentation

Alongside the port effort, many new functions have been fully documented with examples for easier use of  MLx and for designing new widgets.

Breaking old code

As with any major source code overhaul, while the code undergoes massive changes, features break.  In the case of MLx, that means many widgets currently only work in versions of MLx released before the changes began.  (These versions of MLx, which are Macintosh only, are still available.)  These widgets, such as the Poly Plot Package, will soon work in the new versions on both the Mac and PC platforms (as well as any other to which MLx has been ported.)

Benefits

Cleaner source code.

MLx had been under development and use for over 15 years.  There was a lot of fat and deadwood.  As a side effect of the port, much of this code has been improved, replaced, or deleted altogether.

The structure of the port cleanly separates the CL portions of MLx and the architecture specific portions, greatly facilitating the job of porting MLx to any other platforms.

Improved basic tools

Zooming and scrolling image windows are fully implemented and supported in the new MLx.  MLx also features a Navigator (much like that in Adobe Photoshop) which allows viewing of large images by moving a rectangle, representing the visibile portion, over a thumbnail of the same image.

In the new MLx, related windows put into a group and receive certain instructions (such as zooming and scrolling) together.  Group membership is changeable.

Behind the scenes, generalized image operation functions allow MLx to perform any operation on an image in a consistent manner, making it easier to understand and modify the source code, as well as reducing the size of the source code.

User interface improvements, such as the Single/multiple mode button, reduce the number of buttons used in MLx and allow for easier use with the same or enhanced functionality.