[hand with pencil]
Stuff For Sale
2004 Summer Tour
About
Blog
Class Stuff
Email Me
Events
Gallery
Home
In The Press
Newsletter
Services
Smalltalk
Veggie Van Gogh

Credits
© 2002,
Bytesmiths

[this is simply a banner and menu bar]

Please patronize sponsors of this page!

Bytesmiths no longer is involved in software consulting. Maintenance of this web site is currently subsidised by unrelated business activities. Please pass the word to other interested folks, so I can continue to host this page!
Your site could be listed here, for as little as $12 per month! Go to Bytesmiths Press for details.

This site has been selected by PC Webopaedia as one of the best on this topic!
This site has been awarded a Links2Go Key Resource Award in the Smalltalk category!

Originally published in The Smalltalk Report, September 1997.

As If Progress Mattered

by Jan Steinman

Twenty years ago, the Friends of the Earth produced a seminal book called "Progress As If Survival Mattered." Its major premise, pioneered by E. F. Schumacher in his book "Small Is Beautiful" (Harper and Row, 1973), applies equally well to software: if you can accomplish some goal while using less resource, we are all better off.

In the environmental model, using one liter of gasoline instead of two not only means saving the cost of one liter of gasoline, it means saving the costs of the known and unknown "externalities" associated with that liter of gasoline. These savings may include the resource not spent cleaning up the pollution from that fuel, but also the resource not spent cleaning up the pollution created in producing that fuel, as well as other, more speculative savings, such as the incremental cost of rising sea levels due to global warming from the carbon dioxide released from burning that fuel.

Regardless of your environmental beliefs, there are obvious parallels to the world of computers. In software, this idea of "less is more" means less memory used, faster operation, and the ability to run on cheaper hardware, as well as speculative benefits, such as improved well being of software users.

Of course, the analogy holds true in other ways as well. Just as environmental detractors claim that the resource expended in saving that liter of gasoline is too costly in a world awash in cheap fuel, there are many forces acting against "resource responsible" software: it costs too much, memory is cheap, hardware is getting faster every day, and cable modems or Digital Subscriber Line technology will soon make bandwidth essentially free Any Day Now.

As with environmental issues, software development has its neglected "externalities." Just as the cost of a liter of gasoline primarily reflects the cost to produce it, organizations plan based on the cost of producing software: the salaries and benefits of the staff, the amortized cost of office space, the depreciation of computing resource. It is by minimizing these costs that programmers become team leaders, team leaders become managers, and managers become VPs.

Yet many studies show that the production phase is far less than half the total life cycle cost of software. We believe even these figures are entirely too optimistic, because they do not accurately assess the hidden "externalities" by which low quality software impacts total life cycle costs.

For example, if a manager minimizes production cost by refusing to spend any resource on re-use, the project gets done relatively quickly and goes into maintenance, where about 70% of its total life cycle cost will be consumed. This is according to the "closed box" models used in software maintenance studies -- such studies assume a single product, with no influence on other projects.

When re-use is introduced in such a model, it acts as a powerful amplifier -- both positive and negative. If another group is expected to re-use the hastily produced software that got our example manager a promotion, their productivity will suffer, not improve! Conversely, if the manager gave up the promotion by spending extra time obtaining re-use, some other manager may have a very easy time of it, leveraging from the first manager's efforts -- life isn't fair! Evaluation and reward systems need to change before re-use becomes the norm, rather than the exception.

A Re-Use Sample

The biggest re-use leverage comes from creating a successful framework, because proper frameworks raise re-use to the conceptual level. Re-use can happen on many levels, in order of re-usefulness:

  • "Copy and paste" (AKA "copy and waste") barely qualifies as re-use, but until recently, it was the best there was in most languages. It actually increases the maintenance cost, because the copied code should be maintained in synch.
  • "Is a" re-use is what you get when you subclass. While vastly superior to "copy and waste," it is actually not a very effective re-use strategy, because the intimate relationship between classes and subclasses favor specific use, rather than general use.
  • "Uses a" re-use happens when a controlling object commands a slave object to perform some service. This re-use typically has reduced coupling and increased independence of components, and is therefore more general, and more likely to prove re-useful in widely different circumstances.
  • "Collaborates with" is a special case of "uses a." When you "use" something, you have control over it, whereas a willing collaboration of peers has even less coupling and greater independence. In the extreme case, collaboration can even mean negotiating a protocol to use.

The most successful frameworks provide conceptual re-use through object collaboration, rather than through less re-useful methods. The emphasis should be on protocols, rather than inheritance. In our previous column, we emphasized the following points about frameworks:

  • frameworks solve a class of problems, not a single problem,
  • frameworks generally evolve, and are rarely successfully decreed,
  • frameworks can be small; they don't have to be grand,
  • corporate culture must support the evolution of frameworks,
  • before something can be re-used, it has to be used,
  • frameworks that aren't publicized don't get re-used.

In this column, we're going to illustrate many of these points by stepping you through the evolution of a progress indicating mini framework.

The User Management Problem

Developers hate long operations; that pesky user is so impatient! Even worse are operations of indeterminate length, because the developer looks silly if she puts up a "this will take a long time" dialog, and then it doesn't.

The tried and true solution is to present a progress indicating dialog of some kind to the user. These are most often coded up from scratch in the typical "minimal production cost" shop. If the developer has any sense of the future at all, she won't tie such a dialog too tightly to the application at hand, but will make it useful in other situations.

For example, ENVY/Developer has the class EtProgressDialog, which is a special purpose dialog that has a configurable message and the familiar "sideways thermometer" bar that is filled in with color by sending it the proper message. You use one of these via the rather unwieldy "relay" message in Model:

  Model new
execLongOperation: [:dialog |
1 to: 10 do: [:i |
(Delay forSeconds: 1) wait.
dialog fractionComplete: i / 10]]
message: 'just a few seconds, I promise...'
allowCancel: true
showProgress: true

In this example, a dialog appears with our message, and each second, another 1/10th of the progress bar is filled in. After the tenth time through the loop, the dialog dismisses itself.

But is this a framework? While it might be said to "solve a class of problems," it doesn't allow much flexibility in how the problem is solved. Also, it requires considerable coupling between itself and its sender. The progress dialog user must embed their long operation inside an arcane message (execLongOperation:message:allowCancel:showProgress:), and must actively manage the progress bar by sending another, slightly less arcane message (fractionComplete:) to a "black box" object.

Rather than a framework, EtProgressBar is a good example of "uses a" re-use. We don't mean to pick on EtProgressBar ; it serves its purpose well, but to be considered a framework, a progress indicator must be more versatile -- what happens if you want to indicate progress in some other way, perhaps in a progress bar embedded in some other window, rather than a dialog?

Evolution of a Progress Indicating Framework

We evolved our progress indicating framework from a different direction. Instead of creating one or more classes that could be controlled in a "uses a" relationship, we first thought about operations that were likely to require progress indicating services: enumerations.

Our first progress indicator encapsulated all the mechanism of progress indication behind a single message: progressDo: . This message simply changed the cursor to a number that dynamically indicated the percentage of the loop that had been completed. To show progress during a loop, you merely told your collection progressDo: with the normal enumeration block -- there were no other messages to remember, because the number of iterations itself determined the percentage complete. We quickly expanded this pattern with similar protocol for all the enumeration messages, such as progressSelect:, progressDetect:, etc. 

Using our new method, the first example looks like this:

 1 progressTo: 10 do: [:i | (Delay forSeconds: 1) wait]

Encapsulated behind this simple interface was a new Cursor subclass called ProgressCursor that did all the dirty work. The first cut of ProgressCursor was fairly specific to the problem at hand: like the ENVY example above, it had a specific message for advancing progress called percent:, and the special enumeration messages we mentioned took care of switching cursors back and forth and doing other housekeeping.

Figure 1: a ProgressCursor, indicating progress.

At this point, what we had was a little simpler and easier to use than the ENVY example, but it lacked the latter's user message and a means of canceling. It was no more a framework than the ENVY example. However, from an evolutionary point of view, it had a big advantage over the ENVY example: it was based on a protocol, rather than a class. 

Generalizing Progress

If our initial goal had been to develop a progress indicating framework, we might have decreed that all progress indicators descend from a single class -- this is a common mistake when building frameworks -- but we now had something much more powerful, we had a "conceptual class." Any object that obeyed certain protocol could function as a progress indicator!

We called our conceptual class progressor, and defined it as any object that implemented the following messages:

startProgressing
Do whatever is necessary to be used as a progress indicator, and answer whatever might be needed to recover when done.
 
revertTo: oldSituation
Do whatever is needed to stop being used as a progress indicator. This might involve oldSituation, which was returned from startProgressing when I began being used as a progress indicator.
 
value
Answer the current percentage complete.
 
value: newValue
Set the current percentage complete to newValue, update my parameters, and notify dependents.

We implemented "do nothing" methods for startProgressing and revertTo: as an extension to ValueModel , therefore allowing any instance in the ValueModel hierarchy to serve as a progressor. This illustrates another thing about frameworks -- why invent when you can re-use?

By harnessing the power of an existing framework, we gained the stability of a decade of ValueModel use, and also made our new progress indicating framework feel familiar to anyone who had ever worked with ValueModels. This last point is very important -- no matter how beautiful your framework is, if it has a steep learning curve, its re-use will be less than if it works like something a developer already understands.

Another benefit of re using the ValueModel framework is that we could safely communicate with an arbitrary progressor from the developer's domain model using the changed/update mechanism. The developer could tell the progressor changed: with a request, and if the progressor did not support the request, it simply did nothing. We specified the following aspects as "desirable, but not required" parts of a progressor:

changed: #label: with: textOrString
Ask progressor to change its label to textOrString, if it has one.
 
changed: #label:while: with: (Array with: textOrString with: aBlock)
Ask progressor to change its label to textOrString during evaluation of aBlock.
 
changed: #progressEnd
Terminate the monitored operation and evaluate its cancel block.
 
changed: #frontColor: with:aColorValue
Ask the progressor to change its foreground color to aColorValue.
 
changed: #backColor: with: aColorValue
Ask the progressor to change its background color to aColorValue.

This allows extensible, dynamic access to features of progressors that have not even been created yet, while maintaining compatibility with old ones. For example:

  progressor changed: #label: with: 'not much done'.
(0 to: 100) progressor: progressor do: [:i |
i = 50 ifTrue: [progressor changed: #label: with: 'half done'].
(Delay forMilliseconds: 100) wait]

asks progressor to change its label half way through the enumeration, which it can do if it knows how.

Figure 2: a modal progressor with dynamic text and cancel. 

Remember the enumeration protocol we added? We then "split" the selectors to take an additional argument, so in addition to progressDo:, there was a progressor:do: method that allowed you to pick and choose which progressor you wanted to use during your lengthy enumeration. We followed this pattern with all the enumerators, adding progressor:select:, progressor:detect:, etc.

But the old methods were so handy -- we couldn't just throw them away! (We really couldn't throw them away because they were being successfully used, and had become legacy code.) What we really wanted was a way to associate a progressor with a collection, then you could simply bind them together and use progressDo: instead of having to pass an explicit progressor every time as an argument to progressor:do:.

At first, this seemed impossible. You don't want to go adding another instance variable to OrderedCollection, for example, simply for the few times one might need to indicate progress, and certain collections, such as Array or ByteString won't let you add instance variables at all, even if you wanted to.

However, associating two objects so that one looks from the outside as if it were an instance variable of the other turns out to be not so difficult. Remember, state should only be known outside an object through protocol, so as long as there are "get" and "set" methods, the state can be held anywhere.

In our case, we borrowed storage from an object's dependents fields. (We could have used a separate global just as easily.) Except for a speed penalty that is not noticeable when doing GUI things like progress indication, the following two methods make every object in Smalltalk behave as though they have a new instance variable named progressor .

Object

progressor: progressor\
"Assign a progress indicator to me, or remove any existing one if progressor is nil. progressor need only respond to value:, value, startProgressing, and revertTo:. (The last two methods have been added to ValueModel, so any VisualWorks widget should be able to function as a progressor.) If I previously had a progressor, answer it."

 

  | assn oldOne |
assn := self dependents
detect: [:obj | obj class == Association and: [obj key == #progressor]]
ifNone: [].
progressor == nil
ifTrue: [assn == nil
ifFalse: [self removeDependent: assn]]
ifFalse: [assn == nil
ifTrue: [self addDependent: #progressor -> progressor]
ifFalse: [oldOne := assn value. assn value: progressor]].
^oldOne
progressor
"Answer the progressor that is assigned to me, or nil if none. It is typically a ValueModel, ProgressCursor, or AnimatedCursor."
 
  ^self dependents
detect: [:obj | obj class == Association and: [obj key == #progressor]]
transform: [:progressorAssociation | progressorAssociation value]
ifNone: []

The method detect:transform:ifNone: is one we created because it is so very common to be detecting an object in order to tell it to do something. The column "Documents on the Web" in The Smalltalk Report, June-July 1996, shows our implementation.

Progress Beyond Enumeration

Assigning a progressor to arbitrary objects actually solved two problems. First, the set of progress indicating enumerators we created that do not take a progressor as an argument, such as progressDo: , now merely ask the collection being enumerated for a progressor, supplying a ProgressCursor as a default if the collection has not been assigned a progressor. (In a "headless" server image, the default progressor is an empty ValueModel instead, therefore avoiding server crashes when a progress indicating enumerator is sent.)

The second problem that assigning progressors solved was to fix a weakness in our original assumption: the only long operation during which you'd want to indicate progress was enumeration. By having long operations ask an object for a progressor to use, it opened up the whole area of progress indication during streaming operations, which after enumeration is the second biggest cause of lengthy operations.

PositionableStream

progressor: progressor untilEndDo: aBlock
Use progressor to indicate progress while repeating aBlock as long as I am not at my end. Answer the evaluation of the last expression in aBlock on the final loop.
 
progressUntilEndDo: aBlock
Indicate progress while repeating aBlock as long as I am not at my end. Answer the evaluation of the last expression in aBlock on the final loop.

These messages allow accurate progress indication on most read streams and on write streams that are accurately pre-allocated, but they are a bit more clumsy and hard to remember than the simple pattern we used for naming our progress indicating enumerators. Also, streams tend to be used in many more ways than enumerators, so we added public access to a method that simply updates the progress indicator:

updateProgressor
Update my progressor to indicate my ratio of completeness. If I do not yet have a progressor, assign me the progress cursor.

Because most write stream use is not pre-dimensioned, a progressor behaves a bit strange when used on a write stream. The progressor climbs steadily toward 100% complete, but then at some point it "jumps back" to half of its previous value, graphically showing the doubling in size of the underlying collection as it grows. Similar behavior is seen when reading buffered external streams, such as streams on sockets, since all the progressor can show is how much of the buffer has been read.

These oddities caused user confusion, so we added a new AnimatedCursor class that when used as a progressor, simply shows something is happening. The key method is value: which ignores its argument, but advances the animation one frame. 

Besides being entertaining for the user, the visual behavior of an AnimatedCursor progressor on an operation of indeterminate length can be enlightening to developers -- it speeds up and slows down in relation to the amount of work being accomplished, serving as an "eyeball profiler" to help detect performance bottlenecks.

Progress Diversity

We presented the progressor framework at various times to developer groups, and its simplicity and flexibility struck a chord. Soon there were many unanticipated things being done with it, and requests for additional "plug ins" for it. We made a RatioView that could easily be dropped into a VisualWorks canvas, and extended ApplicationModel with progressor, which answered an embedded progress bar. We made a parameterized text holder, so developers could easily embed text like "Your operation is now 47% complete." in any place where they could display text. We made a modal dialog using a slider. We made a proxy progressor that could indicate progress in a client for an operation on a remote server.

As an example, the following expression indicates progress while selecting all global objects that are not classes. Compare this to the original ENVY example, and you'll find that collection is in charge, rather than the progress indicating mechanism:

  ^Smalltalk
progressor:
(Dialog progressMessage: 'Discovering all globals that are not classes...'
legend: (0 to: Smalltalk size by: Smalltalk size / 5.0 "five legend labels")
onCancelDo: [:stoppedAt | ^stoppedAt])
reject: [:globalVar | globalVar class isMeta]

Figure 3: a modal progress indicator with a legend. 

We even turned Netscape Navigator into a progressor! Using the web server framework we described in a three part series last year (June through September, 1996), we made a short message for the client ("busy..."), but told the client browser we were actually sending ten times the length of that message. Our "Netscape progressor" then dribbled out spaces to the client corresponding to percentage complete messages received via value:, causing the Netscape browser's progress indicator to smoothly move from beginning to end as the enumeration progressed on the server. 

Progressing Towards Frameworks

So what does all this mean? A simple, special purpose progress indicator evolved into a general purpose progress indicating framework by:

  • not pretending we knew all we needed to know at the very beginning,
  • letting message protocol, rather than class hierarchy, drive the conceptual design,
  • allowing ourselves the luxury to periodically re-visit and improve,
  • collaborating with initial users, and incorporating their needs in a general way, and
  • publicizing our work, so that it could be re-used.

With these points in mind, you can grow your "goody bag" of mini frameworks, and earn the respect of fellow developers. (Getting the respect of your boss might not happen until your organization's reward systems evolve, but that's the topic of a whole different column!)


Go to the previous column in the series, or the next column in the series.

160 Sharp Road, Salt Spring Island, British Columbia, V8K 2P6, Canada