Urania

Scisoft OSX Intel 2008.9.1 was released yesterday. Nor noted in his blog post about it that this is a bug fix release that

should resolve a few outstanding problems created when changing the location of scisoft from /scisoft to /Applications/scisoft/. Packages such as MIDAS and Gnuplot should now work properly.

This version removes some remaining dependencies on the HPC OSX compilers, which I had on my machine but which most users do not have (and do not want to have).

I compared the /Application/scisoft/i386/Packages directory from this release of Scisoft OSX to the last one and there are no new packages installed. The few issues I noted with the Scisoft OSX 2008.8.1 release all appear to still be valid. Notably:

1. Permission Problems Persist: When the installer installs /Applications/scisoft, it appears that a bug in Apple’s Installer program triggers a change in ownership of the /Applications directory to that of the second user on the system. I strongly suggest checking the ownership of the /Applications directory afterward and if isn’t owned by an administrative user, set it as such using:
   

   sudo chown username:admin /Applications
   

   
   (where “username” is the primary administrative users username, in most cases, your username) to perform the repair.
2. ds9 command line executable missing: This release has the same glitch I noticed in version 2008.8.1 in that /Applications/scisoft/i386/bin doesn’t appear to have a ds9 binary installed. You can fix this by installing the X11 version of ds9 there or by linking to the Aqua version of ds9 that was installed using the command line
   

   ln -s "/Applications/scisoft/i386/Applications/SAOImage DS9.app/Contents/MacOS/ds9" /Applications/scisoft/i386/bin/ds9

I have placed a copy of this release in my Scisoft OSX Mirror in case the main Scisoft OSX repository gets bogged down.

Scisoft OSX Intel 2008.8.1 was released about a week ago. I have been working with Nor Pirzkal for the last few months beta-testing this version and trying to make sure my concerns about the previous version were addressed. Nor’s blog post about the update states

There are a few changes in this version and all packages have been updated to the latest available versions. Starting with this version, Scisoft is installed in /Applications/scisoft (it previously was installed in /scisoft). Make sure that you remove any old /scisoft installation and properly update your startup files to source the Setup.csh or Setup.bash from their new locations.

In the README file you are told

Once the collection is successfully installed csh and tcsh users should invoke the command:

source /Applications/scisoft/all/bin/Setup.csh

to gain access to all the software and configure their environment correctly.

Users of the “bash” shell should instead use:

. /Applications/scisoft/all/bin/Setup.bash

Personally, to avoid problems in tcsh with a script breaking or with modifying my PATH multiple times by repeated execution of the Setup.csh script, I use the following line in my .tcshrc to first check for the existence of the Setup.csh script and SCISOFT environmental variables before executing it.

if (! $?SCISOFT) then
test -r /Applications/scisoft/all/bin/Setup.csh && source /Applications/scisoft/all/bin/Setup.csh
endif

Permission Problems Persist
This version addresses all the issues I noticed with Scisoft OSX 2008.5.1 except for one annoying one, the reassignment of ownership of the enclosing directory on installation. If you installed the old version of Scisoft OSX, it would reassign ownership of the root (/) directory to the second user on the system (in my case, since their is none, it showed the user as “502″). I beta-tested various versions of Scisoft OSX but Nor was not able to stamp out this particular problem. I believe Nor has come to the conclusion after extensive testing that this is a problem with Apple’s software for constructing installer packages. By moving the Scisoft OSX installation to /Applications/scisoft, it is the /Applications directory that gets its ownership changed instead of the root directory, which is less damaging. However, I would strongly suggest checking the ownership of the /Applications directory afterward and if isn’t owned by an administrative user, set it as such using:

sudo chown username:admin /Applications

(where “username” is the primary administrative users username, in most cases, your username) to perform the repair.

A Quick-Fix for any Legacy Scisoft-related scripts
If you had a few scripts that relied on Scisoft OSX being located in /scisoft and you don’t want to edit them all is you can make /scisoft point to /Applications/scisoft. This can be accomplished by first moving the old version of Scisoft OSX before installing 2008.8.1 via the command line:

sudo mv /scisoft /scisoft_old

and then once you have installed the new Scisoft OSX in 2008.8.1, create a symbolic link from the old location to the new by typing (again from the command line)

sudo ln -s /Applications/scisoft /scisoft

This will allow any scripts that refer to items in /scisoft to continue to work for the most part.

What’s New?
An investigation of the /Application/scisoft/i386/Packages directory as well as the NEWS file reveals the following changes to this version of SciSoft OSX over the 2008.5.1 version.

• DS9 updated from 5.1 to 5.3beta
  
• FV updated from 5.1 to 5.2.1
• ATLAS updated to version 3.8.2
• MIDAS updated to 08FEBpl1.1
• cdsclient was updated to version 2.87
• OpenMotif updated to 2.1.32_compat
• cfitsio library updaed from 3.090 to 3.090 (the current version is 3.100)
• pango library updated from version 1.20.2 to 1.21.3
• pixman library updated from 0.10.0 to 0.11.2
  
• expat library updated from 2.0.0 to 2.0.1
  
• fontconfig library updated from 2.3.2 to 2.6.0
• freetype library updated from 2.2.1 to 2.3.6
• gettext library updated from 0.14.5 to 0.17
• gtk+ updated from 2.12.9 to 2.12.10
• libpng library updated from 1.2.10 to 1.2.29
• netCDF library version 3.6.2 added
• pkg-config updated from 0.20 to 0.23
• TclTk package updated from 8.4.13 to 8.4.19
• wcstools library updated from 3.6.4 to 3.7.3 (current version is 3.7.5)
• IRAF package rvsao updated t version 2.5.7 to 2.6.4
• Python was updated to 2.5.2 and the following Python libraries were updated:
  
• • ScientificPython updated to 2.6.1

Minor Glitches
The only other minor glitch I noticed was that /Applications/scisoft/i386/bin doesn’t appear to have a ds9 binary installed. You can fix this by installing the X11 version of ds9 there or by linking to the Aqua version of ds9 that was installed using the command line

ln -s "/Applications/scisoft/i386/Applications/SAOImage DS9.app/Contents/MacOS/ds9" /Applications/scisoft/i386/bin/ds9

A few folks have been aware that I am involved in an interesting project to determine the “Shape of our Galaxy”… specifically the nature of the asymmetries observed in the Thick Disk stars in the galaxy. Part of that project involved obtaining multi-fiber spectrograph observations of many stars in selected fields of the sky using the Hectospec multi-fiber spectrometer on the MMT on Mount Hopkins.

Unfortunately, there are very few “external” users of the Hectospec, so the software pipeline available for reducing Hectospec observations, called SPECROAD, was only geared to run on SAO computers. I spent a considerable amount of time last summer and this summer getting that software into a much more portable form, documenting how to use it, and removing bugs from the code.

All this work resulted in the creation of a new version of the Hectospec data pipeline I dubbed “External SPECROAD” or “E-SPECROAD” for short. There were several major problems with the original SPECROAD code that I addressed in E-SPECROAD:

1. Nonportability: The SPECROAD scripts were only designed to run on a Solaris computer at the SAO, with files in specific hard-coded locations using a specific IRAF environment with all the IRAF parameters set before the scripts were run. This has been addressed by trying to assume almost nothing about a user’s IRAF environment from the start except that they have the proper packages installed.
2. Lack of Documentation: The documentation for SPECROAD is frankly inadequate. It was an in-house tool for the SAO, so the publically-available documentation consists mostly of one webpage describing what the scripts do. I have now written up both an installation guide (because there are a lot of pre-requisite pieces of code to install) and a E-SPECROAD user’s guide.
3. No Installation Instructions: There were several IRAF packages to install (available here). The shell scripts required not only the installation of several IRAF packages and the compilation of several C programs, they used korn shell (ksh) which I was unfamiliar with. And, as I would also soon discover, the ksh installed on the Mac had serious bugs in its list handling. There is now a fairly compete installation guide laying out all the pre-requisites and addressing installations on both Macs and Linux boxes.
4. Undocumented Bugs: The available SPECROAD scripts had major bugs whose workarounds were undocumented and only available when I would email someone at the SAO to inquire about them. In one case, you had to know to actually break out of the script with Control-C, then run some of the reduction in IRAF, and then restart the script! E-SPECROAD has been fairly extensively debugged. It’s not bug free, but the exterminator has made some serious passes at the code.
5. Assumed Availability of Spectral Templates: The SPECROAD scripts typically relied on pre-set spectral line databases for different grating/central wavelength configurations instead of having the user exploit the IRAF identify command to build a database. E-SPECROAD is geared toward the user who is going to wavelength calibrate their observations by building the spectral line database for themselves.
6. Assumed Use of Low Resolution Spectra: The SPECROAD scripts were mostly pre-configured for working with 270gpm gratings and required some tweaking for 600gpm. E-SPECROAD can work with either configuration without editing the code.
7. Inconvienent Backups As Data Reduction Progresses: If one of the scripts executed by SPECROAD crashed, you had to dig around subdirectories to “backtrack” the changes before restarting the script. With disk space not an issue, I prefer monolithic backups of the entire data directory at certain key points in the data reduction pipeline. Since you may not like this form of backup (as they consume a lot of disk space), they are optional.

Today, I am posting E-SPECROAD online as a beta release along with installation instructions and a user’s guide. All the other external users of Hectospec I am aware of, all two of you, can enjoy or complain to me about this code. The ones I am unaware of are free to use the code as well.

I can attest the Aqua version of SAOImage DS9 version 5.3beta does indeed play nice with Apple’s dopey firewall behavior (see here for notes on version 5.2’s incompatibility with the Leopard firewall). However, the command line version that uses X-windows DOES NOT play nice with the Leopard Firewall. If you run the X-windows version of “ds9″ on a Mac running Leopard’s built in Firewall in “Set access for specific services and applications”, you will end up with a completely hosed ds9 executable which will not launch ever again.

As such, for now, since I prefer the X-windows version of SAOImage DS9, I am leaving the the Firewall off for now, I’m not too concerned.

Today I noticeed two interesting software updates for Mac-based professional astronomers.

The first one I noticed was the updating of Xquartz to version 2.3.0. Xquartz is the updated version of X11 for the Mac OS (even ahead of Apple’s own installed versions) that I prefer to use, largely because bug-fixes get rolled in here before they appear in Mac OS. This version requires Mac OS 10.5.4 and has a couple of caveats attached to it for programmers, notably:

The software supporting the deprecated imake build system is not provided in this package. If you need imake and xmkmf, please install the X11 package that came with your Leopard DVD before installing this version. Alternatively, you can compile these packages on your own or get them from a third party such as Fink or MacPorts. The darwin configuration files used by the imake build system are outdated and not supported. Developers using this build system are advised to migrate to autoconf.

[Added July 24, 2008: Apparently, this version of Xquartz changed the X11.app Icon so it now X11 looks like this

when it is on the Dock. Interesting, but it threw me for a second. The only documentation I found of this change is in a Ticket filed with XQuartz's bug reporting system. Still, I think this is a good idea, as it gives a visual cue that you are using XQuartz as opposed to the default X11 installation.]

Along with a bunch of library changes, the key update appears to be having the Xserver updated to the 1.4 branch of Xorg. There is also “support for adding new $DISPLAY sockets after the server is running” (which I think means using the DISPLAY variable will not break things) and “/usr/X11/bin/Xquartz is just a stub that will ‘do the right thing’,” whatever that means. I have upgraded to it and as a reminder, if you upgrade MacOS after installing Xquartz, you will need to reinstall Xquartz to get it back.

The other interesting software release for Mac-based astronomers I noticed today was SAOImage DS9 which has released version 5.3beta, which appears to be, based on the statement on their homepage that “MacOSX 10.5 users with firewall enabled, please use version 5.3 beta”, geared toward addressing the issue I noticed this April with version 5.2 where launching SAOimage DS9 with a certain firewall setting on the Mac could result in the the application becoming irreparably damaged at launch.

When I moved the my Scisoft OSX mirror to a new server, I messed up the permissions on the files, preventing users from downloading the Scisoft OSX packages from the mirror. This has now been fixed.

I was trying to upgrade wine within MacPorts when I realized I had forgotten to upgrade Xquartz after my upgrade to MacOS 10.5.3 on my Mac Pro. So I checked, Xquartz has been upgraded to version 2.2.3. Since version 2.2.1 (which I talked about in my blog here).

• Upgraded the freetype library to version 2.3.6, which fixes “A bunch of potential security problems have been found [and fixed in this release”
• Upgraded to pixman library to version 0.11.4.
• Xquartz fixes from xorg-server-1.3.0-apple21, the key fix being support for monitor hotplugging, although several security fixes also occurred.

Again, if you upgrade MacOS (say to version 10.5.4, which is supposed to be released soon in order to support iPhone G3 and MobileMe), you will likely need to reinstall Xquartz (unless Apple has upgraded their X11 installation to match Xquartz).

SciSoft OSX Intel 2008.5.1 was released today. Nor’s blog post about the update states

The latest update to Scisoft OSX is now available. This includes an update to Pyraf (v. 1.6), GSL, pygsl, Gunplot, and a few backend libraries.

I went ahead and decided to install it today to investigate the improvements. As is usually my procedure, I first moved the directory containing my functional SciSoft OSX install temporarily out of the way via the command line:

sudo mv /scisoft /scisoft_old

Having done that I double-clicked on the installer package and let it do its thing, installing everything in the /scisoft directory. I poked around a bit an realized almost immediately that /scisoft/i386/Applications directory was empty! This one is kind of a show stopper as it means DS9 and FV are unavailable. I copied the files back from the previous installation without too much of a hitch. I did check, the installer package file with Pacifist and it does appear to contain those files in it, so I am not sure why they didn’t appear to install. I hope this is an isolated incident and not a recurring issue with the installer.

Other minor glitches I noticed:

• I discovered that /scisoft/i386/Packages, /scisoft/NEWS and /scisoft/i386/share are set to have owner 502:502 instead of root:admin. This glitch is easily fixed by issuing the following command from the Terminal
  

  sudo chown -R root:admin /scisoft/

• I also noticed that in addition to the x11iraf-1.5DEV installation, the entire x11iraf-1.3.2 installation is still sitting in the /scisoft/i386/Packages directory. All the x11iraf binaries in /scisoft/i386/bin/ are linked to x11iraf-1.5DEV instead of the older 1.3.2 binaries. I suspect this is an oversight.

An investigation of the /scisoft/i386/Packages directory as well as the NEWS file reveals the following changes to this version of SciSoft OSX over the 2008.3.1 version.

• GSL updated from 1.9 to 1.11
• DS9 updated from 4.13 to 5.1 (The current version of SAOImage DS9 is actually version 5.2, you can read about how to update the SciSoft version of DS9 in this post)
• GNUPlot updated from 4.0.0 to 4.2.3
• Swarp updated from version 2.15.7 to 2.17.1
• WeightWatcher updated from version 1.7 to 1.8.7
• pyraf updated from version 1.3 to version 1.6
• The following Python libraries were updated:
• • pygtk updated from 2.8.6 to 2.12.1
  • matplotlib updated from 0.90 to 0.91.2
• atk library updated from 1.10.3 to 1.22.0
• cairo library updated from 1.1.6 to 1.6.4
• cfitsio library updaed from 3.040 to 3.080
• glib library updated from 2.8.6 to 2.16.3
• glib-1.2.10 library added as well (possibly for a particular package needing older version of library)
• gtk+ library updated from 2.8.19 to 2.12.9
• gtk+-1.2.10 library added as well (possibly for a particular package needing older version of library)
• pango library updated from version 1.10.4 to 1.20.2
• pixman 0.10.0 library added
• libtiff library upgraded from 3.7.4 to 3.8.2

Hats off to the SciSoft OSX folks for keeping this package up to date. I have placed a copy in my SciSoft OSX mirror in case there are any access issues.

The Xquartz project is a useful one for Leopard users of astronomical applications because of the dependence of most astronomical applications on X11. A few days ago (what can I say, its finals and I am swamped with exam writing and grading) they released version 2.2.1. It contains all the tweaks that made it into the previous version and also includes

• All packages updated to versions intended to ship as part of X11R7.4 (as of 2008.04.21).
• Fixed multiple crash-causing bugs in the server.
• Fixed cmd-tab to properly move all windows forward when entering X11.app.
• Cleaned up multi-monitor support (still not completely bulletproof) [I have 2 monitors, so this is a big one for me].

As I have noted before, Apple includes some of the work done in this project in OSX updates, so it is suggested that you install the latest XQuartz release after updating to 10.5.2 (and any future 10.5.x or security updates).

Immediately after installing SAOImage DS9 5.2, I had a major failure of the application and initially I just thought it was some sort of build bug. This is what I posted at that time:

[HOLD OFF ON THIS UPDATE! I have discovered that at least on one of my systems, this version of ds9 is refusing to run properly. It launched once, but when I attempted to check the “About SAOImage DS9”, it triggered the following error:

 “An internal error has been detected local header mismatch couldn't open “zvfsmntpt/doc/sun.gif”: no such file.

(this occurred in both Aqua and X11 versions). Furthermore, all future attempts to launch ds9 (again, either Aqua or X11) fail with the following error:

Error in startup script: couldn't read file “./zvfsmntpt/src/ds9.tcl”: no such file or directory  

Even removing the preferences file at ~/.ds9.prf didn’t help.]

Apparently, my problems with SAOImage DS9 in Leopard are a known issue. If you configure the built-in Firewall to “Set access for specific services and applications” so that you can approve “holes” in your firewall on an Application by Application basis, your first launch of SAOImage DS9 will irreparably damage the application!  Unfortunately, Apple implements the application firewall in part by modifying the Application package of the Application you are running by digitally signing it if it was not digitally signed by the developer (adding a file called CodeResources to the Application package). According the Apple’s documentation on this:

If you run an unsigned application not in the Application Firewall list, you will be presented with a dialog with options to Allow or Deny connections for the application. If you choose Allow, Mac OS X 10.5 will sign the application and automatically add it to the Application Firewall list. If you choose Deny, Mac OS X 10.5 will sign the application, automatically add it to the Application Firewall list and deny the connection.

So basically,Apple doesn’t warn you in the dialog box that comes up that it has whatever decision you make, it will modify the application by digitally signing it and it doesn’t give you a way to avoid this. This is, in my opinion, is an incredibly boneheaded move on Apple’s programmer’s part. They readily admit that

  Some applications check their own integrity when they are run without using code signing.

They suggest the application firewall will try to automatically detect these and avoid modifying them, but they should give you, the user, the option instead of making the decision via some internal algorithm.  MacOS X shouldn’t assume its OK to change an application. In the case of SAOImage DS9, they are irreparably damaging the application without leaving you a way to avoid the damage once you trigger the application firewall. Shame on you Apple. The only way to fix it is to reinstall the application!

So when I figured this out (a tip of the hat to this post on IRAF.net). I reinstalled the SAOImage DS9 executables (both Aqua and X11 versions) and before launching them, I set the Firewall (via the Security Pane of the System Preferences) to “Allow all incoming connections” (this is the default mode, so it is as secure as MacOS Tiger was). Everything now appears to work just fine.

Personally, I believe an application that fails its checksum should present a message indicating that is the problem instead of just crapping out, but in this case, the fault lies mostly with Apple. Apple is damaging applications by making this critical decision in the background, without user intervention!

[See my more recent post warning about MacOS X Firewall settings and how they can destroy the SAOImage DS9 executable during its first launch! This problem is avoidable by tweaking the Firewall settings, but once you have launched SAOImage DS9 with the bad settings, the application is damaged can can't be relaunched again. A reinstallation is the only solution, so it is a good idea to avoid this problem.]

The folks in Cambridge have kept busy. They have released SAOImage DS9 version 5.2. The versions for MacOS X include the following:

• Aqua 10.4 Tiger (Universal)
• Aqua 10.5 Leopard (Universal)
• X11 10.4 Tiger (Universal)
• X11 10.5 Leopard (Universal)

The rather extensive changes are detailed in the release notes here, but the notable ones to me include:

• ANALYSIS: for MacOSX tiger, wrap cmds with shell and PATH.
• GUI: change default directory for standard dialog to $HOME.
• ANALYSIS: add /sw/bin to default path for MacOSX. While unstated in the release notes, this is clearly an attempt to support Fink, which places its installation in the /sw directory.
• GUI: ds9 will now start in the users home directory for MacOSX Aqua users when invoked from a double click and the default dialog box is Motif or Windows.
• MACOSX: fixed a problem with printing non standard colors.
• MACOSX: restore postscript printing.
• REGIONS: apply WCS to fits regions if present.
• GUI: add support for user configured button bar.
• CATALOGS: add support for simbad.
• IMEXAMINE: added support for key stroke events.
• Although unstated in their release notes, they are now apparently providing universal binaries instead of PPC and Intel binaries for MacOS X.

I have previously posted notes for integrating upgrades of DS9 into the Scisoft OS X installation and they still work just fine.

One of my favorite little programs is LaTeXit.  It allows you to typeset LaTeX equations outside of a text editor and then drag the results into programs like Keynote or Pages.  It was not fully compatible with Leopard and my fix was a kludge that could break other programs.  Pierre Chatelier has released updated LaTeXit to version 1.15.0, which restores Leopard compatibility.  Notably, you can now use the default0 /etc/profile file without fear.