A Thorny Situation Mac OS

USENIX 2000 Invited Talks Presentation

This site contains user submitted content, comments and opinions and is for informational purposes only. Apple may provide or recommend responses as a possible solution based on the information provided; every potential issue may involve several factors not detailed in the conversations captured in an electronic forum and Apple can therefore provide no guarantee as to the. Mac OS was developed in the early-1980s around the idea of providing the best possible user experience. Apple succeeded in making Mac OS a best-of-breed operating system for personal computers: Mac OS has set the standards against which modern graphical user interfaces are now modeled.

The Challenges of Integrating the Unix and Mac OS Environments

The OS remains in control Preemptive scheduling is bad Opens up many thorny issues having to do with process synchronization If a process is in the middle of doing something critical and gets preempted, then bad things could happen What if a process is preempted in the middle of a system. From 2014 to 2018 the main development of Thonny took place in Institute of Computer Science of University of Tartu, Estonia. Since September 2018 development of Thonny is partially supported by Cybernetica AS. Development of several features in Thonny 3.0 and 3.3 was proposed and supported by Raspberry Pi Foundation. We are also grateful for the help of several contributors.

History and Topical Focus

Wilfredo Sanchez, Apple Computer

Mac OS was developed in the early-1980s around the idea ofproviding the best possible user experience. Apple succeeded inmaking Mac OS a best-of-breed operating system for personal computers:Mac OS has set the standards against which modern graphical userinterfaces are now modeled. The primary alternative to Mac OS at thetime was Microsoft's DOS, which ran on IBM PCs. Mac OS was able toprovide the functionality that DOS provided via a visual model whichhas proven to be much easier for the lay user to understand than thecommand-driven interface employed by DOS and other systems such asAT&T's Unix. Tight integration with the underlying hardware hasallowed the Macintosh platform to extend the experience beyond thesoftware and into the actual management of the machine, withplug-and-play device support and 'out of the box' support for manydevices. The Macintosh led the way to such technologies as desktoppublishing.

Unix has evolved from a different set of goals. Unix has beenpopular in academic and engineering environments, often used onhigh-end servers and workstations. Developers at U. C. Berkeley andother academic institutions have used Unix as a testbed for manyresearch endeavors which have resulted in important computingtechnologies such as TCP/IP. Unix machines were typically shared bymany users, requiring support for individual user accounts which couldbe utilized simultaneously on the same machine. Unix's early exposureto all of these advances lead to a deep understanding of how varioussubsystems interact, resulting in a relatively robust and modularsystem. Unix systems served as the basis and continue to support muchof the infrastructure for today's Internet.

Unix derivatives such as Sun's Solaris, Cal's BSD Unix, and laterLinux have grown to serve key roles as network servers. DOS waseventually replaced by Windows, which sports some of the userinterface features pioneered by Mac OS such the desktop, as well assome of the underlying ideas behind Unix such as preemptivemultitasking. While Unix systems remain in the lead for reliabilityand scalability, and Mac OS continues to push user interfaceinnovation forward, Windows has successfully narrowed the gap in bothareas.

Mac OS X is Apple's effort to create a best-of-both-breedsoperating system which directly leverages the advances and experienceApple has in the user experience arena and the well-established powerof BSD. Mac OS X begins with a BSD foundation called Darwin. OnDarwin we build the programming toolkits from Mac OS which we callCarbon, which in turn enables us to layer on the new user interfaceknown as Aqua. Carbon also provides a straightforward transitionstrategy for our existing application base. We also provide a toolkitcalled Cocoa, which is an object-oriented API derived from NeXT'sOpenStep.

This makes for an interesting system architecture. The idealresult is a system that can be the sort of reliable server platform weuse today to host high-volume web sites, file services, networkgateways, and engineering computation services while also being thesimple to use home computer platform of choice. This is a dauntingtask, because the server platform goals mandate a certain level ofcomplexity (high security, performance tuning parameters, variousnetwork servers, etc.), while for most home users, simplicityoverrides other concerns.

Unix and Mac OS have evolved independently of each other, and thereare fundamental assumptions made in one system which cause numerousfailures in the other. The capabilities of the available filesystems,the networking protocols, and the user/system model are profoundlyvaried, often in ways which cannot be rectified without changing thearchitecture of system components. Additionally, people use thesystems in very different ways and expect different behavior fromthem.

This paper will discuss some of the issues which have arisen atApple in the process of combining these disparate environments andsome of the solutions that were used to make things work well in MacOS X. There have been many such issues, and in the interest ofbrevity, we will visit two key areas here: filesystems and multipleuser support. Examples given do not go into a great deal of detail,but enough information is provided to give the reader an idea of theproblems and our attempt at a solution.

Filesystems

The use of filesystems demonstrates both implementation andsemantic problems between the Unix and Mac environments. We'll startwith the implementation issues, which is the obvious material.

The filesystems used in Mac OS are the Mac OS Extended filesystem,also known as HFS+ and its predecessor the Mac OS Standard filesystem, or HFS (Hierarchical File System). Both filesystems aresupported in Mac OS X, though HFS is support is only provided forcompatibility and users are encouraged to use HFS+ for newvolumes.

The HFS+ volume format, unlike the HFS volume format, fortunatelyprovides storage for Unix-style meta-data (eg. owner and mode bits).This made it a lot easier to enable the use of HFS+ in the Darwinenvironment. However, a few incompatibilities still exist.

HFS+ is a case preserving, but case insensitive, filesystem. Thatis, the case of file names is remembered, but access to file nameswith varying case will yield the same file, and file names which varyonly in case are not allowed in any given directory. Typical Unixfilesystems are, in contrast, case sensitive. At the start of theRhapsody project, which preceded the current Mac OS X work, we hadanticipated that this would be a big problem. Later, when we startedusing HFS+ as the primary filesystem in Darwin, we found surprisinglyfew problems resulting from this behavior, and those which we do findtend to be trivial to fix. We have yet to encounter a problem in thisarea which requires a complex solution.

Another obvious problem is the different path separators betweenHFS+ (colon, ':') and UFS (slash, '/').This also means that HFS+ file names may contain the slash characterand not colons, while the opposite is true for UFS file names. Thiswas easy to address, though it involves transforming strings back andforth. The HFS+ implementation in the kernel's VFS layer convertscolon to slash and vice versa when reading from and writing to theon-disk format. So on disk, the separator is a colon, but at the VFSlayer (and therefore anything above it and the kernel, such as libc)it's a slash. However, the traditional Mac OS toolkits expect colons,so above the BSD layer, the core Carbon toolkit does yet anothertranslation. The result is that Carbon applications see colons, andeveryone else sees slashes. This can create a user-visibleschizophrenia in the rare cases of file names containing coloncharacters, which appear to Carbon applications as slash characters,but to BSD programs and Cocoa applications as colons.

HFS+ lacks support in the volume format for hard links, a standardfeature of UFS. Initially, the attempted creation of a link to a filewould yield a 'not supported' error. We had discussed some '80%'solutions, such as creating symbolic links instead, but the semanticsof symbolic links are significantly different. For troubleshootingreasons it is preferable to fail at link creation time than at somelater time due to problems related to these semantic differences. Theproblem is that there is a significant amount of software which breaksif hard link creation fails, and some of that software needs to beredesigned if hard links cannot be used. In order to accommodate thissoftware, we now emulate hard links by creating a 'kernel-level'symbolic link which is visible only to and interpreted by the HFS+filesystem. This was necessary due to the lack of support in thevolume format. The resulting behavior is very similar to that of hardlinks when viewed from above the kernel, though they are relativelyinefficient in comparison.

UFS, on the other hand, does not support file IDs, which is afeature of HFS+. File IDs are persistent handles to a file, and canbe used to access a file in a manner similar to which one uses pathsin Unix. The nice thing about file IDs is that once the ID for a fileis obtained, the file can be renamed or moved anywhere on disk andstill be found and opened by the holder of the ID, and file access byID is faster than access by path, as it avoids the path lookups. MacOS aliases are similar to symbolic links, with the addition ofalternative means of finding the file. Because aliases include thetarget file's ID, moving the target file does not break the alias asit would a symbolic link.

One of the biggest problems is the lack of complex file support inmost Unix filesystems. HFS+ allows for a file to have multiple datastreams: one, the data fork, contains the actual file data, andanother, the resource fork contains additional resource data. (At thefilesystem level, this is really just two files which happen to sharea name.) The volume format also allows for arbitrary named fileattributes (similar support is in UDF, NFSv4), though the neither MacOS 9 nor Mac OS X support the use of this feature in the fileformat.

This has often been a source of compatibility issues between Mac OSand other platforms in the past. When transferring files via theInternet, they usually need to be encoded in a format such asMacBinary, which will combine both streams into one, so that it can beserved by a web or FTP server. (Servers which run on Mac OS can dothis as needed.)

There were several proposals for how to present complex files toBSD clients which open() them. One was to combine thetwo streams into AppleDouble format. This has the advantage ofcontaining all of the data in the complex file, and programs likecp could work properly. The problem then is that if onewere to open a complex C source file in emacs they'd see not only theprogram text, but also some weird encoding information and whateverresource data that was put there by a Carbon application. A fatalproblem with this scheme is that part of the file data (the encodinginfo) is creates by the filesystem, and isn't really part of the filedata, and should therefore not be editable. The non-lossy aspect ofthis solution is what drew us to is, but the complications turned outto be too substantial.

The present mechanism presents the data fork to the BSD client afile is opened by name. If the resource fork is desired, it can beread by appending a known string to the file path. For example'foo/bar' gets you the data for file bar, but'foo/bar/.namedfork/resource' will get you the resourcefork for bar. (The actual appended string may change.) Note that thefile is not somehow acting as a directory; it is not listable, andstat() calls still show it as a file. This mechanism canbe carried to other filesystems with complex files, such as UDF. (UDFalso has allows attributes for directories.)

The are flaws in this plan as well. Most Unix software will onlysee the data fork of a file, so 'cp foo bar' will resultin a new file bar which is missing the resource data from foo (ifthere was any). Backup and archiving software may also lose data. Westill have the problem of sharing files with systems that don'tsupport complex files. Mac OS X will continue to support complexfiles, but we are moving away from their use in order to increase ourability to operate in a heterogeneous environment.

A third option which we did not explore is to show the two forksare separate files altogether. This is similar to some AppleShareimplementations on Unix, which export a Unix filesystem as anAppleShare volume. When a file 'foo', written by a MacOS client is viewed from the Unix side, there is another file (usuallynamed in a convention similar to '.foo.resource' or'.resources/foo>') which contains the resource data. Ifthese servers are also serving Unix clients (including local access),which is often the case, files tend to get moved around in the Unixside, and the resource file is left in the old location. The resultis an effective loss of the resource data and a need somehow to manage(periodically clean up) these disassociate resource files. The HFS+filesystem can probably address this problem by magically moving theresource file along with the data, but strange behavior of this sorttends to produce bugs, and this approach didn't seem to offersignificant advantages over the alternatives, and it got dropped.

Interestingly, the Carbon team has run into exactly this problem onUFS, where there is no obvious place to store resource forks, thepresent implementation of Carbon uses a similar solution as the oneemployed by Unix AppleShare servers.

Other filesystem problems are more subtle, and are due to differentsemantic expectations made by Mac OS and Unix systems as well asimplementation details.

One example of this is the delete semantics of Mac OS as comparedto the unlink semantics of Unix. In Mac OS, if you try to delete afile which someone else has open, the delete fails. This ensures thatif another application is using your file, the file is not deletedfrom under that application. Unix achieves this by going ahead andremoving the link, but the file continues to exist while it it open(though there is no longer a way to open that file). There are goodarguments to be made for either method, and some Mac OS programs relyon the Mac OS behavior (files are sometimes used as a semaphores),while many Unix programs expect unlink to always succeed (one trick inthe Unix world is to open a file for writing and then unlinking it soas to obtain a private bit of filesystem-backed memory to scribbleon).

It is worth noting that the difference is behavior is in part dueto the implementation of the respective filesystems. In UFS,meta-data is store with the file's inode, and inodes are storedseparately from file data. In HFS+, the file meta-data is stored withthe file data; there is no inode. Removal of the file in HFS+therefore means removal of all access to the file. The unlinkbehavior in UFS come naturally from the separation of data and inodes;you simply delete the file, but keep the inode around as long as it isin use. On the other hand, one can do really fast searching of avolume for files with a given name in HFS+ because of how its storageis layed out. What is interesting here is that implementation detailscan lead to ideas and extensions which may not have been part of theoriginal design.

In order to handle the differing semantics, a delete()system call was added to BSD so that we have both semantics available.Typically, Unix programs will call unlink() and Carbonapplications will call delete(). An unlink will alwayssucceed, so it is still possible for a Carbon application which has afile open to lose that file when it is closed, if a BSD programunlinked it. This is acceptable because the goal is to allow Carbonprograms to continue to cooperate with each other as designed, and BSDprograms don't usually delete files belonging to a Carbon program, sothe software tends to work as expected.

We should also be aware that while the primary goal here is to getMac OS X to operate properly with these differing software stacks, westill have millions of Classic Mac OS users and these users need toshare files and media with our new Mac OS X users. One example oftrouble is Unix permissions on HFS+ volumes. Mac OS 9 will not reador write Unix meta-data; though HFS+ supports it, Mac OS 9 will ignorethose bits completely. That's acceptable until you start moving adisk between Mac OS 9 and Mac OS X systems (or if you dual boot yoursystem and also, to a lesser degree, if you run the Classicenvironment in Mac OS X).

This may be a problem if you expect permissions to be enforcedacross systems, but if you are handing someone your disk or bootinginto another OS, you are bound to get dissapointed eventually. A moreinteresting situation, however, occurs when files get created in MacOS 9 and are seen later by Mac OS X. The files have no Unix meta-dataset, and therefore no permissions or owner. User 0(root) and group 0 (wheel) are reasonable defaults, but afile mode of 0000 is almost certainly not what the userwants. Fortunately, we are able to distinguish between 'no meta-dataset' and zero values for mode and owner info. What we do in this caseis present the owner, group, and mode as some reasonable default,which is determined by examining the permissions on the directory nodeon which the filesystem is mounted. Note that the actual permissionson disk remain unset unless someone sets them explicitly; an unmountand remount with a different default will effect the mode of suchfiles. This allows for a flexible administration of the default modesuch that some disk default to more open permissions than others, asdesired.

As mentioned before, HFS is also supported. This is primarily toallow for the transfer of data to and from removable media andexisting hard drive volumes which are formatted for HFS. Furtherconstraints are found here, but this is similar to DOS filesystemsupport in Linux, and for the file transfer, HFS presents fewproblems. Because HFS does not store mode bits, a default is used inthe same manner that we use for unset modes in HFS+ files. Thesignificant new problem here is that files transferred to and from anHFS volume will lose permission information.

Files

Above the kernel, there are file name conventions in Unix whichhave manifested as limitations and bugs in various Unix software. UFSallows any printable character in a file name, but many Unix shellscripts and programs have problems with files that contain 'special'characters such as whitespace and punctuation. In Mac OS, users havenot been subject to this limitation in the system and should not besubject to them in Mac OS X. One good example of this is themake command, which uses whitespace as a file namedelimiter, making it difficult to produce program output called, forexample, Netscape Navigator®. There is no reason whythis has to be the case, but it is an unfortunate historical artifactof how Unix grew up.

As mentioned above, Unix has a heavy reliance on file paths. Oneof many such examples is the Bourne shell: if the shprogram does not exist in /bin, the system won't evenboot properly. While Mac OS is not immune from such problems, it doesallow the user to relocate or rename even such important items as theSystem Folder. This is a particularly significant issue because,although we use BSD as the core system software, we do not want torequire our users to understand how BSD works. Ideally, the typicalMacintosh user does not even know that BSD is there. The verypresence of such folders as 'usr' and 'etc'on disk is therefore awkward, and we hide those directories and theircontents at the application level in order to avoid confusion as towhy they are there and cannot be moved or deleted. One idea is to putall system files into a Mac OS-style System Folder, and make thatfolder opaque to the user. This requires that there be other methodsfor managing the system software than allowing users to poke in therethemselves, and we still have the problem that standard Unix paths arerequired to exist at the root level of the boot volume. This is stillan area which we are actively working to improve upon.

The original Mac OS had a wonderful feature that applications werekept in one file (though both forks of that file were sometimes used).You could therefore drag one icon from your install floppy onto yourhard disk or another floppy, and you were done. You could drag theone icon to the trash, and get rid of that program. Over the years,we have gotten into a model where there are several files to dealwith. Most of the application files are kept in one folder, butapplications also put various files into parts of the System Folder,such as preferences, system extensions, and fonts. A product likeAdobe Photoshop® installs many files onto your system: there isthe actual executable program; a folder with plug-ins such as filters,file translators, and effects; another with color palette, and brushesand patterns; help files; and samples.

By BSD conventions, which are as reasonable as any Unix conventions,files are located according to function; you might put the program in/usr/bin, the icons in /usr/share/photoshop,the plug-ins in /usr/libexec, and so on. While it hascertain advantages, the problem with this approach is that files arescattered around the disk, making it somewhat difficult to install anduninstall software. This is addressable by using a package installerscheme with receipts and so on, but that is easily broken if the useruses different installers or bypasses the installer by unrolling a tararchive.

One of the neat ideas we inherited from NeXT is the notion ofopaque directories, which at the user level behave more like filesthan folders. We call these 'bundles,' and use them in a variety ofways whenever we need to encapsulate a set of files into oneuser-visible object. You can put all application data into one suchbundle with several subdirectories and files in it. The programexecutable goes in there, as well as any additional resources andplug-ins. The Finder treats the bundle as one opaque object which canbe drag-installed, just like the good ol' days.

We follow a similar scheme for system-wide libraries. Instead ofputting headers in /usr/include, libraries in/usr/lib, and additional resources wherever they go, wecreate a single framework bundle which encapsulates everything neededfor that library. The compiler knows how to find specific files inthe framework, because each type of bundle has a well-definedstructure.

Multiple Users

Unix has its roots in environments where computing machines wereexpensive and rare, which mandated that they be shared. It wasimportant that computational resources be utilized as well as possiblein order to amortize cost, and give everyone involved a chance to getsome cycles in for their respective projects.

Therefore, Unix had an early notion of timesharing, and featureslike multitasking on a single processor, user accounts, and securitybecame important early on. Note that security has many aspects. Oneimportant aspect is some level of protection for one process fromanother process' mistakes. Because time was often critical, it wasreadily apparent that one running program's ability to (usuallyaccidentally) impede or destroy the progress of another wasunacceptable. This lead to the need for preemption and separate,protected address spaces for each process. Other security mechanismssuch as file permissions to some degree extend naturally from thisbasic inter-process security; each process runs on behalf of a givenuser, and is restricted to that user's permissions for access to filesand certain system calls. The private memory allocated to eachprocess ensures that one running program cannot hijack another'smemory and affect its execution.

When the Macintosh was introduced, it was to be the first trulypersonal computer. As such, it has very different needs. Thesemachines were designed so that one person could buy and use them(hence the term personal). There was no need for time sharing asthere would not be more than one console, and users would only need torun one program at a time. The hardware was much more modest. Theoriginal Mac OS could barely squeeze into the 128K of availableRAM.

It later became clear that running a few programs side-by-sidecould be very useful, but this wasn't really true until personalcomputers had gained enough power and memory for this to be feasible.(Aside from computational power, you also needed such things aslarger-screen displays.)

It is easy in hindsight, given the spiffy computers we have today,to look at Mac OS today and think that some of the original decisionswhich we still carry with the system were not well considered, butthat would be an grossly unfair assessment of the situation. Today,we find greater computational resources in 'embedded systems' than wasin the first Macintosh. Mac OS was designed to devote the limitedresources of the personal computer to one user.

Mac OS has had some notion of file permissions since theintroduction of file sharing, but that model does not extend to localvolumes. Mac OS 9 added support for multiple user logins, so thateach user can have his or her own preferences remembered betweenlogins, independent of the rest, and it has some file security.However, this support is rather superficial compared to Unix. Thereis still a single shared address space, and all programs havedevice-level access to the hardware. Even so, it is sufficient forsome uses, particularly in light of the fact that only one user canuse the machine at a time.

In Unix, users are a concept understood even by the kernel. Mac OSsoftware has never had to deal with such a pervasive user modelbefore, and in Mac OS X the challenge is to introduce this featurewithout breaking existing software products that our users relyon.

It is also important to note that users are accustomed to having agreat deal of control over the system without many barriers, but inBSD, even something as seemingly trivial as setting the time of dayrequires 'superuser' access. Now there are files on your disk whichyou can't move around, and folders you can't even open to look in.This appears to require a great deal of user education, which is notdesirable for a system that is supposed to be intuitive to use.

Some of this, however, is intentional. We want to move away fromthe model where the casual user fiddles around with the systemsoftware on a day-to-day basis. This was necessary in Mac OS 9because many software packages install (or ask the user to install)software such as system extensions into the system folder. Mac OS Xdiscourages this sort of manipulation of the system software andinstead tries to provide other (less destabilizing) methods by whichdevelopers can extend the system software.

In general, the issue of trading off user convenience vs. systemsecurity is a complex one. A lot of this should probably be left forthe user to decide, depending on what the machine is being used for,and such factors as its network connectivity. However, a lot of thesetradeoffs can be avoided by writing smarter software, which lets thesystem's security model fit more naturally into how the user interactswith the system. It is important to present security as a feature andnot an annoyance, as the annoyed user is likely to disable securityaltogether, perhaps despite the risks. This issue is system-wide, andneeds to be revisited periodically as we move forward.

Another problem we have run into is a Unix problem which is notspecific to Mac OS X at all, but is one that Unix systems have not yetfelt compelled to address.

Unix filesystems store owner and group information on disk bywriting the user's and group's system ID (typically a natural number)with each file's meta-data. On any given workstation, this name-to-IDmapping is always consistent (though not always one-to-one). If onewere to take a disk from one computer and attach it to another, somehard problems arise. What do you do with the user/group IDs which areon the disk? If the two machines are on the same NetInfo network, andthe user and group IDs on the disk are in the parent NetInfo server,you are in good shape, since they will be consistent across the hosts.However, if the machines are in different NetInfo domains (more oftenthere will probably only be the local domain, so this is the mostlikely scenario), then these IDs may have no useful meaning at all onthe second machine. Even more difficult, some IDs may be consistent(eg. system user IDs), while others are not, and there is no reliableway to know which IDs are correct.

There are many permutations of how this problem may affect filepermissions. An ID on one host may map to a different user on theother, or to no user at all. A person may have an account on bothmachines, but the user name and/or ID may be different on each.

Our solution here is truly nontrivial. We keep track of volumeswhich are 'local' to the running system. When a new volume isintroduced, we ask the user to decide what action to take. Wecan:

Accept IDs at face value. This is the Unix default. Add the volume to the list of locally known volumes.
Ignore IDs on that volume. A new user and group pair called 'unknown' will be returned by the filesystem. Files written to the volume get written by user and group unknown, and chown() and chgrp() calls are ignored. Ignore setuid bits. This is similar the the behavior of DOS filesystems or HFS (not HFS+) volumes, which don't have any storage for users and groups anyway.
Write user and group unknown to each file on disk, unset setuid bits, and then mount the volume normally and add it to the list of locally known volumes. This is similar to option 2, except that new files owners will be local user IDs, and chmod() and chgrp() settings will stick.

The Classic Environment

The problems described above are all solvable in Mac OS X becausewe can change both the Carbon and BSD implementations to accommodateeach other, but one of the reasons that we need Carbon in the firstplace is that the existing Mac OS Toolbox cannot live well on the newCore OS. This brings about a much harder problem: what do we do aboutthe thousands of software titles, already in use by millions of ourcustomers, which were written and built against the old Toolbox?

The Classic environment in Mac OS X creates a virtual machineinside of Mac OS X which boots a largely unmodified version of Mac OS9. Applications which are built for Mac OS 9 and have not been'Carbonized' run in this environment. The Classic environmentreplaces the hardware abstraction layer in Mac OS 9 with a series ofshims that pass requests to parts of Mac OS X. For example, a memoryrequest in Mac OS 9 gets fulfilled by a memory request in the Darwinkernel. Mac OS 9 can thereby use resources managed by Mac OS X.

The resource sharing has varying limitations. For example, if MacOS 9 wants to mount a disk, Classic can accomplish that by creating ashim at the SCSI level which are then handled by I/O Kit in the Darwinkernel (arrow A in figure 1). Because Classic is making raw SCSIrequests, that disk cannot be mounted by BSD on the native side, orthere will be serious problems, so the disk cannot be shared by bothnative and Classic applications. This is, in fact the way disks wereinitially accessed by Classic.

Another option is to store a disk image in a file on the Mac OS Xfilesystem. This provides Mac OS 9 with disk storage without taking adrive away from the native environment (arrow B in figure 1). Oneadvantage here was that while Mac OS 9 only knows about HFS and HFS+filesystems, this disk image could live on any filesystem provided byBSD in Darwin. A third implementation shimmed the Mac OS 9 diskdriver to that in Darwin (arrow C in figure 1), removing direct SCSIaccess and enabling the use of a single partition on a disk, ratherthan taking over the whole device.

Lastly, filesystem calls in Mac OS 9 could be passed to the Mac OSX HFS+ filesystem, enabling the sharing of files on HFS+ filesystemsbetween native and Classic applications, which is now the preferredmode (arrow D in figure 1). We could take things a step further, andhave Classic call into the Carbon library (at this point, we are stillmaking system calls to the kernel), which can emulate HFS+ semanticson several filesystems. It might be possible, then, to have Mac OS 9in Classic using files on UFS and NFS volumes while remaining ignorantof these filesystems.

A Thorny Situation Mac Os 11

There are tradeoffs made as you move along. Most applicationsaccess files via standard API, but not all do. For example, NortonDisk Doctor wants to talk to the disk driver. When we've move up toarrow D, the Mac OS 9 disk driver is out of service, and Disk Doctorwill not work. Similarly, Apple's Drive Setup application goesstraight down to the SCSI level, which is absent in the B, C, and Dcases.

Ironically, the Macintosh's historically closed architecture playeda key role in making this possible. Because details about hardwarewere not public, and direct hardware access has been closely guardedin Mac OS 9, application writers never found it convenient to writearound the standard APIs, and this use of the layering of the systemworks for the vast majority of applications.

The same strategy worked for other devices. USB devices can beshimmed at the USB driver level, which gives Mac OS 9 exclusiveaccess, or at the HID layer, which allows sharing with nativeapplications (arrows A and B in figure 2). Screen usage is done viaCore Graphics (Quartz), which works rather nicely: Mac OS 9 can claimwindow regions, which are then managed by the Mac OS X windowserver.

About the Presenter

Wilfredo Sánchez is a 1995 graduate of the MassachusettsInstitute of Technology, after which he co-founded an Internetpublishing company, Agora Technology Group, in Cambridge,Massachusetts; he then worked on enabling electronic commerce anddynamic applications via the world wide web at Disney Online in NorthHollywood, California. Fred is presently a senior software engineerat Apple Computer in Cupertino, California. He works primarily on theBSD subsystem in Mac OS X as a member of the Core Operating Systemgroup, and as engineering lead for Apple's open source projects. Fredis also a member of the Apache Software Foundation, a NetBSD andFreeBSD developer, and a contributor to various other projects.

Patrick W. Dirks, the Mac OS X filesystems technical lead at Apple,and Brent Knight from the Classic team contributed significantly tothis paper. Errors, however, are Fred's doing.

An updated version of this paper will be available online athttps://www.mit.edu/people/wsanchez/papers/USENIX_2000/.

Contact Information

Wilfredo Sánchez
Senior Software Engineer / Scientist
Apple Computer, Inc.
1 Infinite Loop / 302-4K
Cupertino, CA 95014
408.974-5174

Themes Soft