History of the Graphical User Interface (GUI)

Monday, November 12, 2007

When viewing various forum debates, for some reason, it seems important for some to give credit for various innovations to one company or another. In a similar vein, others try to disprove such claims, etc. There is probably no greater example of this than the origins of the graphical user interface. On one hand, I hesitate to write about this issue because so many people have such strong opinions on the matter. On the other hand, there is an awful lot of misinformation posted in forum discussions.

Finally, the one common theme I’ve noticed when people debate this issue is that they all try to make a case whereby one company gets full credit for the innovation and everybody else copied them. Unfortunately, the origins of such technologies are rarely “black and white” issues and there are many “shades of grey”. Most innovations are built off of the works of others. The graphical user interface is no exception.

Legend has it…

For many years, it was commonly accepted that Apple created the GUI and for good reason. Outside of a few people working in labs, most consumers and IT professionals alike were not familiar with the concept of the GUI prior to the debut of the Apple Lisa. At the time, I recall reading the coverage of this product in technical magazines such as BYTE, etc. Clearly, this technology was treated as something entirely new and there was no reference or comparison to Xerox’s work. Of course, back then, the internet wasn’t what it is today. There were online services and bulletin boards, etc. but for the most part people relied on magazine articles to keep current.

As difficult as it may seem today, the notion of using a mouse with icons and drop down menus, etc. was completely foreign. At the time, all interaction with computers was done with the command line interface (CLI). As I recall, the GUI concept was ridiculed by much of the technology community at the time and the Lisa / Mac platform was painted as a toy by some. It wasn’t for “serious” users.

As we all know, the GUI eventually did become popular to the masses. Radical changes like this often take time to digest. More importantly, it often takes the market leader (someone like Microsoft) to push this as the future direction and basically not give the masses a choice before they see the light.

Once the concept of the GUI became a good thing, some began to challenge Apple’s role in this innovation. Microsoft was smart enough to see which way the wind was blowing and likewise followed Apple with the pursuit of a GUI of their own. Since Microsoft blatantly copied Apple’s work (more on that later), the common sentiment from Microsoft Windows advocates was to cite prior work from Xerox PARC and claim Apple did the same.

Some Microsoft advocates seem to wonder why “what Microsoft did to Apple” was different from “what Apple did to Xerox”. There were several differences. For starters, both Xerox and Apple were pioneering GUI concepts simultaneously. Clearly some concepts were developed at Xerox first and some were developed at Apple first. The problem for Apple was that they actually visited with the Xerox team to see what they were doing. This and this alone is what led some to the perception that Apple just stole their GUI from Xerox. Apple paid in stock (worth millions) for this brief visit. The agreement was up front. Apple didn’t see how things were done in any detail and the programming environment Xerox was working in was so different from Apple’s development environment that it wouldn’t have mattered if they did see the details behind Xerox’s work. Microsoft was not just a competitor to Apple, but they were also a developer for the Mac platform. They had intimate access to Apple’s APIs and frameworks to see how to create a GUI. Not only did Microsoft copy Apple’s work in concept, but as a developer for Apple, they had the blueprint from which to copy. Further, there have been no real significant GUI conventions that Apple is using which originated at Microsoft. With each release of Windows, it became clear that Microsoft was not interested in innovation since it was easier just to copy. With the Windows 95 release, it was clear that Microsoft wasn’t even trying to hide their intentions of copying Apple.

What’s GUI? Who’s a WIMP?

In order to determine who gets credit for something, it’s important to have a reasonable definition. The basic requirements of a GUI are defined in an acronym called WIMP. WIMP stands for Windows (bit mapped graphics required), Icons, Menus and Pointing devices (usually mice).

Xerox’s contribution

Without a doubt, Xerox’s Palo Alto Research Center (PARC) was responsible for many things we take for granted today such as the development of Ethernet, the Laser printer, etc. and certainly, they helped unify some of the elements found in GUIs today. However, just as it is important to give credit where credit is due, it’s equally important to make clear where credit is not due. For example, if you recall the elements (WIMP) necessary for a GUI, we have to ask which were developed at Xerox. Windows? Yes and No. Yes, Xerox developed a windowing system, but no, it did not invent the bit mapped graphics model necessary to display the windows. Icons? No. Menus? Yes, but not in the form we know them today. Pointing devices? Mice? No and no. The point here isn’t to belittle the accomplishments of Xerox. Rather, the point is to illustrate that the concept of the GUI didn’t just pop up at Xerox. Most of the items mentioned such as the bit mapped graphics, the mouse, icons, etc. were prior works that are attributed to Douglas Englebart and his work at the Augmentation Research Center which was funded by various US Government agencies including DARPA, NASA, Air Force, etc.

Xerox did successfully create an advanced object oriented development environment written in Smalltalk. This is where the majority of Xerox’s work went into for this product. Xerox had technology demonstrations of rudimentary windowing systems and the assumption has generally been that Xerox was further along than Apple was when Apple visited with Xerox. In terms of pure GUI concepts, Xerox’s work was certainly an evolution over Englebart’s previous work. However, today’s modern GUI bares little resemblance with what Xerox finally came up with in the Star/Alto product.

Apple’s contribution

In 1979, Jef Raskin started the Macintosh project at Apple. He identified a need for a computer that was easier to use than anything developed to date. Both the Macintosh project and the Lisa project were works in progress prior to Apple’s infamous visit to Xerox. Apple and Xerox were in simultaneous development of a GUI. Apple was aware of Xerox’s work because the founder of the Macintosh project, Jef Raskin had lectured at Xerox on the topic prior to joining Apple. Jef Raskin was something of an authority on the subject at the time. He had written his Master’s thesis on a WYSIWYG graphical interface back in 1967. Likewise, many of the same ideas that fueled Xerox’s effort originated from the creator of the Macintosh project.

At the same time, it would be unfair to suggest Apple visited Xerox and didn’t come away with any ideas. Clearly they did. The fact that Apple’s Lisa GUI was different from Apple’s Macintosh GUI should make that clear enough. But, the devil is in the details. Apple wasn’t developing in a Smalltalk environment, Apple had resource limitations that Xerox didn’t. That is, Apple had to make a GUI work on an affordable piece of hardware. Xerox was working purely in a research environment without the same hardware limitations.

Testimonials from those involved…

There was an interesting essay written by Bruce Horn back in 1996 about this topic. Bruce was of the people recruited from Xerox to work for Apple on the Macintosh project. Clearly, he’s one of the few people that can say with authority what work was developed at which company. Below are a few excerpts from his essay.

“For more than a decade now, I’ve listened to the debate about where the Macintosh user interface came from. Most people assume it came directly from Xerox, after Steve Jobs went to visit Xerox PARC (Palo Alto Research Center). This “fact” is reported over and over, by people who don’t know better (and also by people who should!). Unfortunately, it just isn’t true – there are some similarities between the Apple interface and the various interfaces on Xerox systems, but the differences are substantial.”

Again, Bruce goes on to talk about the Smalltalk environment created at Xerox as this was more significant than the actual GUI pioneering they did.

“Smalltalk has no Finder, and no need for one, really. Drag-and- drop file manipulation came from the Mac group, along with many other unique concepts: resources and dual-fork files for storing layout and international information apart from code; definition procedures; drag-and-drop system extension and configuration; types and creators for files; direct manipulation editing of document, disk, and application names; redundant typed data for the clipboard; multiple views of the file system; desk accessories; and control panels, among others. The Lisa group invented some fundamental concepts as well: pull down menus, the imaging and windowing models based on QuickDraw, the clipboard, and cleanly internationalizable software.”

The list goes on. In fact, he talks about Bill Atkinson’s windowing model and how it was designed. Bill wasn’t even aware of the fact that Xerox’s model didn’t even have self-repairing windows as the Mac did. The point here is that there is a big difference between broad concepts and the actual implementation. Clearly, the Macintosh architecture solved problems that the Xerox team wasn’t even aware of.

The concept of pull down windows is one of the most basic functions of the modern GUI. Again, this is yet another thing that didn’t exist at Xerox. Drag and Drop? Yes, another concept developed at Apple. Control panels? Yup, Apple. Clipboard metaphor / concept? Yup, Apple. Desk Accessories? We know them now as Widgets or Gadgets, but the concept originated at Apple. Most people don’t realize even how icons were used on the Xerox Star. Icons were used as verbs. Like an icon to “save” a file. On the Macintosh, you could use icons as objects, like a trash can or you could drag a document onto an application icon to open it, etc.

What’s interesting is that even things that have been attributed to Xerox such as the selection based text editor, apparently did not originate there at all. Below is a response from the late Jef Raskin in response to Bruce Horn’s essay:

“Horn makes it seem that the selection-based editor came with Tesler from PARC. It may have been a case of convergent evolution, since we already had that paradigm at the Mac project. In this case it dates at least back to an editor I designed much earlier, while at Bannister & Crun. In ’73 I discussed my editor concepts with many people at PARC, so I do not know whether Tesler’s design was influenced by my work, I know it was not the other way around.”

In the following paragraph, it becomes clear that in terms of general concepts, Jef Raskin’s work predated any of the fundamental concepts that were further developed by Xerox.

“My thesis in Computer Science, published in 1967, argued that computers should be all-graphic, that we should eliminate character generators and create characters graphically and in various fonts, that what you see on the screen should be what you get, and that the human interface was more important than mere considerations of algorithmic efficiency and compactness. This was heretical in 1967, half a decade before PARC started. Many of the basic principles of the Mac were firmly entrenched in my psyche. By the way, the name of my thesis was the “Quick-Draw Graphics System”, which became the name of (and part of the inspiration for) Atkinson’s graphics package for the Mac.”

Conclusion

In the end, who created the GUI? There is no single entity that can take credit for that. Apple advocates want to say Apple did. Microsoft advocates would love to say Microsoft did, but there has never been any evidence to support that. Instead, if they can’t attribute this innovation to Microsoft, they refuse to acknowledge Apple’s work so they claim Xerox invented it all.

The problem with the Xerox takes all claim is that most of the basic elements required for a GUI predate Xerox’s work as well. As I mentioned earlier, they didn’t create bit mapped graphics, they didn’t create icons, they didn’t create the mouse pointer, etc. At the same time, they did evolve the concept into a rudimentary working model.

Moreover, if we look at the modern GUI today, it more closely resembles the work done with the original Macintosh than anything else, including Apple’s earlier Lisa system. That’s probably true for several reasons. For starters, Xerox was never able to successfully commercialize their work. Apple’s GUI was certainly the first commercially successful GUI and certainly the first GUI that anyone outside of the Xerox PARC lab that anyone would have ever seen. Also, the market leader, Microsoft basically did just steal from Apple. As such, it is understandable why Apple’s GUI and whatever we call “today’s modern GUI” to be very similar in convention. Really, although there have been many refinements to the GUI, when you think about it, not much as changed in the past 25 years in terms of how we interact with our computer.

In terms of the Xerox Star, its one thing to look at a screen shot of an operating system and it’s another thing to see how it’s used. Not only was the Xerox Star rudimentary in function compared to the Macintosh, but all accounts I’ve read claim it was clumsy and slow to use. That is, there was elegance in the Smalltalk environment they created, but it should be considered an unfinished product at best. It was more of a technology demonstration than a product that’s ready for prime time. The point here isn’t to belittle the work Xerox did. Rather, when considering the time line of events, it’s important to realize how long it takes to create a product like this that actually is ready to ship to the masses.

Finally, Apple was the company that brought the GUI concept to the public and made them aware of it. The majority of today’s GUI conventions date back to Apple’s work more so than any other single entity. For that reason, in my opinion, Apple deserves the lion’s share of the credit for this innovation. But, in the end, neither Apple nor Xerox can claim they alone created the GUI. Clearly, the work from those such as Douglas Englebart, Jef Raskin, etc. predates either works from either Xerox or Apple and this foundation was a necessary building block for both companies. Though both can claim partial credit, when you look at the conventions we associate with a GUI today, the majority of credit would have to go to Apple.