Software development has a few of these:

• red, green, refactor
• Individuals and interactions over processes and tools
• Don’t break the build
• Perfect is the enemy of the good

That last one really started off a train of thought for me. How do we know when we’ve reached “good” and when we’re trying too hard to reach perfect? This is particularly interesting to me as I often struggle with “leaving well enough alone,” to pull out another cliché. I’m not sure of the exact set of circumstances that led to it, but somehow I started thinking about value versus cost. All of that coalesced to this:

The Best You Can Afford

That’s how you know when to stop. When you can’t afford to keep making it better anymore. That is, when the value of what you’re working on isn’t worth the effort compared to the other things you could spend that effort on. (I’m sure that’s one of the worst-written sentences ever, but it makes the point.)

At first blush this seems obvious. We all apply this every day to an extent right? Well, not necessarily. The best you can afford is very easy to confuse with the worst you can afford. That is putting in the least effort possible to just get by. We see and do this all the time.

What, really, is the difference? The difference is in what these two tactics get you in the long run. Aiming for the worst is attractive because it is, somewhat obviously, the cheaper option. However, that obvious cheapness results in, well, results that are obviously cheap. In physical products this can mean flimsy plastic parts and a shorter life span. In software, this can mean more defects, a hard-to-use product, and upset end users. Worse yet for software, it often also means a harder time performing the maintenance tasks that can be more than 80% of the cost and time spent working on the software.

Aiming for the best is harder to justify up front because so many of those costs are unknown. However, the attention to detail means physical products that people love to use, use for a longer time, and will recommend to others. It mostly means the same in software. Another benefit for software is that maintenance takes less time and is easier for those who must do it.

So now we have a guideline that can tell us when we are spending too much on heading for perfect: when the value of what you are doing is less than the value of doing something else.

I’m going to finish up with an example that a lot of people have used when making similar points. I’m sure a number of readers will roll their eyes as soon as I mention the product, but I urge you to not focus on that, but instead pick some other product or product category and make some comparisons of your own. When has the “worst” product won? How did it win? When has the “best” product won? How did it win? Which of those products started out ahead an then lost?

The product that I think I can most easily make a case for being built using “the best you can afford” mentality is: the iPhone. To see why, we have to go back to before the iPhone was announced or really speculated about. Back in the early 2000s, there were rumors of Apple working on a tablet computer. These rumors persisted ultimately until the iPad was released. It seems, based on all accounts, that these rumors were true. And again based upon what we have heard, Apple was working on what would become the iPad *before* the iPhone.

This is significant because it points out that Apple, whether you decide that means Steve Jobs, Jonny Ive, a host of other folks, or the combination of all of them, Apple was focused on making the best thing they could. At some point, who knows exactly when, someone realized that what they had—a touchscreen computer with a simplified OS—would make a great phone. At that point, the team felt that the iPad was the “best” product, but the “best they could afford” to make at the time was the iPhone.

Further, it is obvious that Apple doesn’t and never did release a version of the iPhone (or iPad) without work already being in progress on the next version. This implies that the “best we can afford” is a philosophy deeply ingrained into Apple. They are continually working on something better, but they release something that is “good enough”.

There are a number of people out there who will point out that the iPhone isn’t the number one phone any longer or that it won’t stay that way for long. I’m sorry, but I don’t buy it. For one thing, there are at most two kinds of iPhones (not counting storage size differences) available for sale at any one time. This isn’t true for any other type of phone. Also, while Apple only has a certain share of the number of devices sold, its share of the profits is much larger. That is, Apple can afford to sell fewer units because it can also manage to charge more for them. Profit is the goal of a company. Selling the most units is worthless if you don’t also have profit.

I humbly believe another example of the application of “the best you can afford” is the Improving Podcasts that I co-host with Allen Hurst. This was an idea that Allen and I had around two years ago. We wanted to create a software development podcast and have it associated with our company, Improving Enterprises. We spent several months getting some momentum built and finally realized that it was at a point where we had to release something or risk never publishing an episode. Because of that, the first episodes were more work, and the audio quality was suspect. Over time the best we could afford got better until we found, espeically due to the time involved, recording all audio over Skype was a solid balance of creating something of value without detracting from our other valuable works. This reduced the editing time significantly, allowed for remote participation, and was repeatable by either one of us. However, it also means that occaisionally we are subject to the audio difficulties and other problems associated with remote audio.

I could go on, but at this point this article is as long as I—and probably you as well—can afford. It is also the best I can afford right now. I’m sure I could fix some grammar, make some points more clearly, etc. However, I have other work to do and family to be with. I leave you with this question: How might your life and the lives of those around you improve if you stopped doing the worst you can afford and instead did the best you can afford?

Thanks to Eric Herber’s comment on that post, the PDF [PDF no longer there. I have not yet found a replacement.] it mentions, and a bit more research, I was able to make a two significant updates to my installation.

Group Settings

Reading through the PDF, it notes that the informix user and group are required in order for things to work properly. Examining my install, it seems that the IDS installer tried to create the informix group, but somehow assigned it the same group ID as the _lpoperator group. I believe _lpoperator is the group of people who can use the Mac’s printers. Regardless, it wasn’t the informix group that it should have been.

Important! If you don’t understand what is meant by user and group here, my best advice would be to get someone who does to help you. I don’t want to insult anyone, but making changes to the things I am about to mention could render your machine unusable. Here be dragons!

To fix this issue, I had to do two things. First, create the informix group and make it the informix user’s “primary” group. Second, I had to change the group ownership of everything in my Informix install directory from _lpoperator to the new informix group. The PDF talks about using the Workgroup Manager that is available in the OS X Server Admin Tools. I will let you read that for the details except to say that the correct URL for acquiring the tools depends on the version of OS X you have. Regardless, the URL in the PDF is outdated, as is one I found in a search on MacOSXHints. The best means I have found is to go to the Apple Support site’s Advanced Search, enter “server admin tools”, select the radio button to match all keywords, use the drop-down to restrict the document type to “Downloads”, and pick the version you want from the results. I am far from certain, but it would make sense to pick the one closest to your version of OS X.

Once you have the Server Admin Tools download, it is a simple, and typical Mac install. Once installed, you can run the Workgroup Manager tool more or less as stated in the PDF. Do not skip the step that asks you to select “Show All Records” from the “View Menu”. This may be “Show System Records” depending on the version of the tool. If you know a little about Unix user management, the tool is fairly intuitive. If you know a lot about Unix user management, you may prefer the command line tools that can do the same thing. Regardless, I simply had to create the informix group and make it the informix user’s primary group.

It is worth noting that if I had created the informix user and group using this tool before running the installer, I probably would not have had the _lpoperator nonsense nor would I have had to do the next step: changing the group ownership of all the files in my Informix install.

To change the group ownership, I went back to the command line and did the following:

$ cd /Applications/IBM/informix
$ ls -l
total 11656
drwxr-xr-x    3 root      staff         102 Aug 15  2008 CSDK
drwxr-xr-x    5 root      staff         170 Aug 15  2008 DBLD
drwxr-xr-x    3 root      staff         102 Aug 15  2008 ICONNECT
-rw-r--r--    1 root      staff        5904 Aug 14  2008 README.html
drwxr-xr-x    7 root      staff         238 Aug 15  2008 SERVER
drwxrwxr-x    4 informix  _lpoperator      136 Jan 13 15:50 aaodir
drwxr-xr-x  109 informix  _lpoperator     3706 Jan 13 15:54 bin
...
-rwxr-xr-x    1 root      _lpoperator    29713 Aug 14  2008 uninstallserver
$ sudo chgrp -R informix aaodir
...

Note: The “…” on line 11 means there were a lot of other entries. The one at the end means that I repeated that “sudo chgrp” for all of the files and directories that had group ownership of _lpoperator.

The “sqlhosts” File

That group information was annoying and tedious to fix, but it was not really the problem. If you recall from the earlier post, there are some environment settings that need to be loaded. One points to a file that is very important to enabling clients to connect to the server: INFORMIXSQLHOSTS. The whole purpose of the file is to define the mechanisms by which clients can connect, this means whether they must be using shared memory space or can use sockets and what port they use to connect. The one created by the installer for the demo database has a single line that looks like this:

demo_on    onsoctcp    yoda.local    9088

The first is the dbservername and is correct for the demo database. The second is a three-part string that identifies how clients can connect. In short, “on” means a regular database server (short for on-line as in OLTP maybe), “soc” means sockets as opposed to “shm” for shared memory, and “tcp” means the network protocol. So, this seems right as well. The other two entries are the machine hostname and the port. However, although my Mac is named “Yoda” (yes, I’m a Star Wars geek), that is not listed in the /etc/hosts file. I changed this to “localhost” which is in the hosts file. Finally, that last entry is meant to be the service name from the /etc/services file, not the port number. To figure out how to change this, I used the following command:

$ cat /etc/services | grep Informix
sqlexec         9088/tcp    # IBM Informix SQL Interface
sqlexec         9088/udp    # IBM Informix SQL Interface
sqlexec-ssl     9089/tcp    # IBM Informix SQL Interface - Encrypted
sqlexec-ssl     9089/udp    # IBM Informix SQL Interface - Encrypted

My final file still contains one line, which looks like this:

demo_on    onsoctcp    localhost    sqlexec

And now, everything is working smoothly. Whew!

I have recently started a project with a group that uses Informix as the database. To set up my development environment to match production as closely as possible, I have installed IDS for Mac. The instructions that you can find on the web caused me some difficulty. There is a video available from IBM that would be great, but it seems to be outdated slightly. (Viewing it may still give you some benefit, so you can find it here.) To save myself time in the future if other Mac users need to join the project, I have documented what I believe to be the final process here.

Caveats:

• Do not start this process without a current system backup. System settings are going to be modified.
• I tried to document this as I went, however I did make some discoveries late and then inserted them where I believe I should have done them. Those have not been retested.
• It is likely there are better/faster ways to do some of these things.
• There may well be additional steps needed to add additional databases or if your usage is heavier than mine to date.

If you try this out and find some things that don’t work or ways to make it faster or easier, please let me know in the comments. I will happily incorporate them.

One quick formatting note, when command examples are given, the “$” at the beginning of a line is in place of the command prompt. By default in the Mac Terminal app, this is usually of the form “hostname:current_directory username$”. If you copy and paste the commands, remove the “$” and the space after it.

Installing IDS

1. Download IDS here
2. Mount the downloaded IDS disk image (iif.11.50.FC2DE.macosx64.dmg)
3. Double-click the install package (iif.11.50.FC2DE.macosx64.pkg) and follow the prompts.
4. Enable root user and log in as root. Note: It might be possible to follow the steps using only sudo, but most on-line instructions say to use the root account method. As such, this is what has been tested. Instructions provided below are for Snow Leopard 10.6.2.
   1. Open System Preferences
   2. Click “Accounts”
   3. Click “Login Options” (just above the lock in the bottom-left)
   4. Click the button next to “Network Account Server:” (near bottom-middle)
   5. Click “Open Directory Utility…”
   6. Click the lock (bottom-left) and enter admin password
   7. In the menu-bar, choose “Edit” then “Enable Root User”
   8. Enter a password for the root user
   9. Quit Directory Utility.
   10. Close System Preferences
   11. Log out
   12. Choose “Other” from the log-in menu and provide the root as the username and the password entered previously.
5. Kernel options:
   1. Kernel options should be as follows per the IDS README files. (To avoid issues, system settings can be higher, but not lower):

      kern.sysv.shmmax=4398046511104
      kern.sysv.shmmin=1
      kern.sysv.shmmni=512
      kern.sysv.shmseg=512
      kern.sysv.shmall=1073741824
      kern.sysv.semume=10
      kern.sysv.semmsl=87381
      kern.sysv.semmnu=87381
      kern.sysv.semmns=87381
      kern.sysv.semmni=87381
      kern.maxfiles=2147483647
      kern.maxfilesperproc=40000
      kern.maxvnodes=150000

   2. To check them, use the following command:

      $ sysctl kern.sysv.shmmax kern.sysv.shmmin kern.sysv.shmmni kern.sysv.shmseg kern.sysv.shmall kern.sysv.semume kern.sysv.semmsl kern.sysv.semmnu kern.sysv.semmns kern.sysv.semmni kern.maxfiles kern.maxfilesperproc kern.maxvnodes
      

   3. Compare the output to the list above.
   4. To adjust any that need to be changed, use the following command (may not work for kern.sysv.shmmni):

      $ sysctl -w
      

      And copy the key-value from the list above (ex. kern.sysv.shmmax=4398046511104)

   5. Those command-line changes will disappear after a reboot. To make them permanent, edit /etc/sysctl.conf and simply paste the desired key-values. This is an important system file. Make a backup of the file first!
6. Two options (the first runs a CLI installer, the second may run a GUI installer):
   1. Open a terminal window, navigate to /Applications/IBM/informix and run “ids_install”
   2. Open the finder, navigate to /Applications/IBM/informix and double-click “ids_install”
7. Follow the installer prompts. Accept defaults as much as is possible except for the demo server installation and terminal emulator. Be sure to select “Yes” for the demo server and do not use the terminal emulator option.
8. Launch a terminal window.
9. Execute the following command (assuming a Bash shell):

   $ . /Applications/IBM/informix/demo/server/profile_settings
   

10. If desired, copy the settings in that profile_settings file to your profile (as well as the profile of root and/or the informix user created by the installation).
11. Run the following command to load the demo database, which will not be in your path (actual location: /Applications/IBM/informix/bin):

    $ dbaccessdemo
    

12. Run the following command to connect to the database:

    $ dbaccess
    

13. The interface is navigable using the arrow keys on the keyboard, go to “Table”
14. Select the database “storesdemo@demoon”
15. Select “Info”
16. Select the “‘root’.stock” table
17. Select “Columns”
18. View the information (no more detail will be gone into here on using the tool)
19. Select “Exit” and repeat until the tool quits.
20. You can now log out as root and back in as your normal user.
21. If you want to use the informix user to start/stop your database, continue. Otherwise, you can use the root user. Both will work from a terminal.
22. Open Terminal and navigate to the /Users directory.
23. By default, the install creates the informix user’s home directory but leaves it owned by the root user. To change the owner use (enter your password when prompted):

    $ sudo chown informix informix
    

24. Use “su” command in the Terminal to log in as the informix user (enter the informix user’s password when prompted):

    $ su - informix
    

25. Use the command below to print the contents of the needed settings to the terminal:

    $ cat /Applications/IBM/informix/demo/server/profile_settings
    

26. Select the text (using the mouse) and copy it using Command-C.
27. Edit (or create) a .profile file in the informix user’s home directory:

    $ cd ~
    $ vim .profile
    

28. Enter VI’s insert mode by pressing “i”, and paste the copied settings using Command-V.
29. Save and exit VI by pressing Escape, then entering “:wq” (without the quotes).
30. Log out as informix and setup is complete.

Starting IDS

1. Open a terminal and use the “su -” command to log in as root or the informix user. The “-” is important, as it ensures the user’s profile is loaded. If the informix user was enabled to start/stop the database according to the optional steps during installation, this loads the appropriate settings and “informix$” will now be the command prompt. Otherwise, the settings in /Applications/IBM/informix/demo/server/profile_settings must be loaded manually using this command:

   $ . /Applications/IBM/informix/demo/server/profile_settings
   

2. Run the commands below. The output should match:

   $ oninit
   $ onstat -
   
   IBM Informix Dynamic Server Version 11.50.FC2DE — On-Line — Up 00:00:09 — 41756 Kbytes
   

Stopping IDS

1. Open a terminal and use the “su -” command to log in as root or the informix user. (Note: “informix$” is now the command prompt.)
2. Make sure the settings in /Applications/IBM/informix/demo/server/profile_settings are part of the environment.
3. Run the commands below. The output should match.

   $ onmode -ky</blockquote>
   $ onstat -</blockquote>
   
   shared memory not initialized for INFORMIXSERVER ‘demo_on’
   

As a bit of history, the folks at Sun (soon to be Oracle) tried to fix this by all but requiring that Date objects be wrapped with Calendar objects in JDK 1.1. They couldn’t just fix the date implementation directly because of another long-standing tradition: new Java versions must be backward-compatible with previous versions. Regardless, this fix has long been considered a kludge at best, but most of us learned to live with it. Preamble out of the way…

I was working with someone today who had a Calendar object. They wanted to compare it with another object to see which was earlier. So, they called Calendar.after(Object). Makes sense right? Only this method returned false even when the other object was definitively a representation of a time after that represented by the Calendar object. This seemed odd to the developer in question.

The observant among you may have noticed that I did not say they were comparing with another Calendar object. They were, in fact, comparing against a Date object. Knowing that, I immediately suspected the problem. A quick trip to the API docs showed me this gem:

Returns whether this Calendar represents a time after the time represented by the specified Object… if and only if when is a Calendar instance. Otherwise, the method returns false.

So, rather than throwing an IllegalArgumentException or having the method specifically only take a Calendar object, this method and the corresponding before(Object) method both just return false when you send in… anything else. So Calendar.after("dates in Java suck") returns false.

There are no good reasons for this that I can think of, so I went to the Bug Parade. There, I found a few tickets that were marked as fixed or otherwise closed. However, this is still obviously a problem, right? Refining my search, I found what I think is the most recent one. It is marked as “Closed, Will Not Fix”. Why? Here is the relevant part of the comment:

It’s hard to change the implementation at this point. Please use compareTo(Calendar) added in 1.5.0.

I’m sorry, can’t we at least deprecate things that are this broken? Do we not do that after the heckling caused by deprecating 60% of the methods in Date?

A follow up question… Where is the replacement library? We had java.nio, why not create a new java.date? For that matter, why hasn’t a third party library really taken off here? My suspicion is that JPA integration is the main challenge. Especially since, in addition to the problem of them having the same class name, java.sql.Date extends java.util.Date.

Maybe I’ll try out Joda Time or another 3rd party replacement. Does anyone know how their Hibernate sub-project works in the field? (Update: The goal of JSR-310 is actually to get Joda Time into core Java. It had been tentatively scheduled for Java 1.7. A quick look around shows that there was some struggling to get it completed and tested for inclusion. I was not able to determine an outcome. Anyone know?)

I’ll try to make my next post about something more positive.

Therefore, it was with a great deal of happiness that I was able to take on a much more developer-oriented role in my latest Java project. I was back in the groove. There were some annotations I had to learn or re-learn, but they mostly made sense. I was concretely producing value again rather than just enabling others to do so.

Over time, I found myself stumbling over difficulties that were entirely Java-centric. I kept running into problems with Generics. Java reflection was a morass of generics and class-casts. I found myself every single day having to say, “I hate Java Generics.”

// a sample usage of generics
// based on a real usage in my current project

Map<Class<ReasonableClassName>, ReasonableClassName> objectByType =
     new HashMap<Class<ReasonableClassName>, ReasonableClassName>();

// pre 5.0
Map objectByType = new HashMap();

I was never a fan of the Generics proposal for Java 5. It is just overly verbose as the example above shows. The Java 5 version provides type safety, but at what cost? Fortunately, back then, I was able mostly to ignore this. I only really used generics when using collections, and I tend to wrap those in more domain-friendly objects. That practice also meant I didn’t really need generics in the first place, but c’est la vie. I was happy enough.

In the current project things are somewhat different. I am building upon legacy code, using a lot of collections in a single class, and using a lot of reflection. I found APIs that were inconsistent in their use of Generics. For example:

// from java.lang.Class

public Constructor<T> getDeclaredConstructor(Class<?>... parameterTypes) {
    // ...
}

public Constructor<?>[] getDeclaredConstructors() {
    // ...
}

Note that when getting a single declared constructor, it returns a properly specified constructor that will create something of the type of the class. However, if I want the full list of constructors, the array returned now doesn’t know what type the created objects would be. There is a reason for this: having a “genericized” array is “bad”. Don’t ask me why. Folks much more familiar with the generics implementation struggle to understand the reason for this.

Regardless, it means I need to cast the result. Is that not what generics were supposed to save me from? This also could lead (depending on the context) to entirely unnecessary type checks or even a compiler warning for “unchecked type cast”. In short, what I have found is that for every benefit generics provide, there are multiple hoops to jump through or other problems that detract from it. Yes, some of this is caused by the implementation and not the concept, but that makes me think worse of Java, not better.

There is also a push to add more “safety” to the language through annotations and other means. Java cannot take more overhead pushed into doing simple things. I am far from the only one to think this. As an example, see this post from almost a year ago.

I am pretty fluent in Java (sans generics) at this point. It is by far the language I know best. However, I now realize that while Java may continue to be what I am paid to do, my personal first choice will have to be something else. Using the language is too much work; it is too cumbersome. I like Ruby already, but maybe I’ll delve deeper into Scala. At least both of them run on the JVM.