Urania

Nor Pirzkal has released another minor update to Scisoft OSX. He states in his blog that:

A minor revision of Scisoft is available for downloading. Many people are experiencing many problems with the Apple Package Installer and I am distributing this release as a simple tar.gz file.

This version of Scisoft is distributed as a simple tar file that can be un-tarred in /. Scisoft expect to be located in /Applications/scisoft/

I can confirm the CONTENTS file distributed with the install is identical to the file for Scisoft 2009.9.1 so this release may just be a way to avoid some issues with the Apple Package Installer program.

This change to a tarball format is good, but it changes how to install the software. However, there is at least one major glitch in this distribution

• WCSTools is missing from the installation! There are symbolic links to the package, but the /Application/scisoft/i386/Packages/wcstools-3.7.3/ directory is empty! If you are upgrading from a previous version of Scisoft OSX, there is an easy way to recover from this problem. I am outlining my suggested installation routine below.
• As I noted for Scisoft 2009.9.1, “on Snow Leopard, Gordon Richards discovered that if you attempt ‘import pylab’ in python, you get a bus error. I can confirm the same error occurs on my Snow Leopard machine using either this SciSoft OSX release or the previous one. Furthermore, I can confirm the error DOES NOT occur in Leopard. I am not a heavy Python user, so I will leave it to Gordon, Nor, and others to investigate this issue. Gordon notes that this blog posting contains instructions for getting pylab installed under a vanilla Snow Leopard install, in case you need them.”

My suggested routine for upgrading to this version of Scisoft OSX from a previous version is the following:

1. Backup the previous version of Scisoft by renaming the /Application/scisoft/ directory to /Application/scisoft_old/.
2. Download the current gzipped tarball of Scisoft OSX. The current version of Scisoft OSX is available for download from the Scisoft OSX website, but I have made the package available on my Scisoft OSX mirror as well, in case it is faster for people.
3. Using the command line in an administrative account, you can untar the tarball using the command:

   sudo tar -C / -xzvf Scisoft_OSX_macintel_2009.10.1.tar.gz

   You will be asked for your account password to allow ’sudo’ to run the tar command as root.
4. The files are untarred with their ownership intact from when Nor created the tarball, so be sure to change the ownership to match your root account using
   

   sudo chown -R root:admin /Applications/scisoft/

5. Finally, if you want to copy over the WCStools package from the previous Scisoft OSX installation, you can use the command:

   sudo cp -r -p /Applications/scisoft_old/i386/Packages/wcstools-3.7.3/ /Applications/scisoft/i386/Packages/wcstools-3.7.3/

   That should get your back the WCStools.

I have alerted Nor to the missing wcstools directory as well as some other minor issues with this release. Hopefully, if Nor has some time, he will make

Fast on the heals of their release of version 5.3 about two weeks ago, the folks at the SAO have released version 5.4 of SAOImage DS9. Here are the links to the downloadable Mac-related SAOImage files

• Aqua 10.4 Tiger (Universal)
• Aqua 10.5 Leopard (Universal)
• X11 10.4 Tiger (Universal)
• X11 10.5 Leopard (Universal)
• X11 10.5 Leopard (Universal) with Firewall

The release notes for SAOImage DS9 don’t necessarily suggest dramatic changes in this version relative to version 5.3.

1. CATALOGS: removed support for Chandra Source Catalog at request of Ian Evans of CXC (only added on October 3, 2008).
2. MASKS: add support for mask transparency.
3. MASKS: add new mask properties.
4. GRID: add grid title support.

My previously posted notes on integrating upgrades of DS9 into the Scisoft OS X installation still work. Just note that newer releases of Scisoft OS X place the binaries in /Applications/scisoft/i386/bin/ instead of /scisoft/i386/bin/ and if you are installing the X11 binary that is compatible with the firewall, you have to install both the ds9 and ds9.zip file in the bin/ directory of Scisoft OSX.

Scisoft OSX Intel 2008.9.1 was released yesterday. Nor noted in his blog post about it that this is a bug fix release that

should resolve a few outstanding problems created when changing the location of scisoft from /scisoft to /Applications/scisoft/. Packages such as MIDAS and Gnuplot should now work properly.

This version removes some remaining dependencies on the HPC OSX compilers, which I had on my machine but which most users do not have (and do not want to have).

I compared the /Application/scisoft/i386/Packages directory from this release of Scisoft OSX to the last one and there are no new packages installed. The few issues I noted with the Scisoft OSX 2008.8.1 release all appear to still be valid. Notably:

1. Permission Problems Persist: When the installer installs /Applications/scisoft, it appears that a bug in Apple’s Installer program triggers a change in ownership of the /Applications directory to that of the second user on the system. I strongly suggest checking the ownership of the /Applications directory afterward and if isn’t owned by an administrative user, set it as such using:
   

   sudo chown username:admin /Applications
   

   
   (where “username” is the primary administrative users username, in most cases, your username) to perform the repair.
2. ds9 command line executable missing: This release has the same glitch I noticed in version 2008.8.1 in that /Applications/scisoft/i386/bin doesn’t appear to have a ds9 binary installed. You can fix this by installing the X11 version of ds9 there or by linking to the Aqua version of ds9 that was installed using the command line
   

   ln -s "/Applications/scisoft/i386/Applications/SAOImage DS9.app/Contents/MacOS/ds9" /Applications/scisoft/i386/bin/ds9

I have placed a copy of this release in my Scisoft OSX Mirror in case the main Scisoft OSX repository gets bogged down.

A few folks have been aware that I am involved in an interesting project to determine the “Shape of our Galaxy”… specifically the nature of the asymmetries observed in the Thick Disk stars in the galaxy. Part of that project involved obtaining multi-fiber spectrograph observations of many stars in selected fields of the sky using the Hectospec multi-fiber spectrometer on the MMT on Mount Hopkins.

Unfortunately, there are very few “external” users of the Hectospec, so the software pipeline available for reducing Hectospec observations, called SPECROAD, was only geared to run on SAO computers. I spent a considerable amount of time last summer and this summer getting that software into a much more portable form, documenting how to use it, and removing bugs from the code.

All this work resulted in the creation of a new version of the Hectospec data pipeline I dubbed “External SPECROAD” or “E-SPECROAD” for short. There were several major problems with the original SPECROAD code that I addressed in E-SPECROAD:

1. Nonportability: The SPECROAD scripts were only designed to run on a Solaris computer at the SAO, with files in specific hard-coded locations using a specific IRAF environment with all the IRAF parameters set before the scripts were run. This has been addressed by trying to assume almost nothing about a user’s IRAF environment from the start except that they have the proper packages installed.
2. Lack of Documentation: The documentation for SPECROAD is frankly inadequate. It was an in-house tool for the SAO, so the publically-available documentation consists mostly of one webpage describing what the scripts do. I have now written up both an installation guide (because there are a lot of pre-requisite pieces of code to install) and a E-SPECROAD user’s guide.
3. No Installation Instructions: There were several IRAF packages to install (available here). The shell scripts required not only the installation of several IRAF packages and the compilation of several C programs, they used korn shell (ksh) which I was unfamiliar with. And, as I would also soon discover, the ksh installed on the Mac had serious bugs in its list handling. There is now a fairly compete installation guide laying out all the pre-requisites and addressing installations on both Macs and Linux boxes.
4. Undocumented Bugs: The available SPECROAD scripts had major bugs whose workarounds were undocumented and only available when I would email someone at the SAO to inquire about them. In one case, you had to know to actually break out of the script with Control-C, then run some of the reduction in IRAF, and then restart the script! E-SPECROAD has been fairly extensively debugged. It’s not bug free, but the exterminator has made some serious passes at the code.
5. Assumed Availability of Spectral Templates: The SPECROAD scripts typically relied on pre-set spectral line databases for different grating/central wavelength configurations instead of having the user exploit the IRAF identify command to build a database. E-SPECROAD is geared toward the user who is going to wavelength calibrate their observations by building the spectral line database for themselves.
6. Assumed Use of Low Resolution Spectra: The SPECROAD scripts were mostly pre-configured for working with 270gpm gratings and required some tweaking for 600gpm. E-SPECROAD can work with either configuration without editing the code.
7. Inconvienent Backups As Data Reduction Progresses: If one of the scripts executed by SPECROAD crashed, you had to dig around subdirectories to “backtrack” the changes before restarting the script. With disk space not an issue, I prefer monolithic backups of the entire data directory at certain key points in the data reduction pipeline. Since you may not like this form of backup (as they consume a lot of disk space), they are optional.

Today, I am posting E-SPECROAD online as a beta release along with installation instructions and a user’s guide. All the other external users of Hectospec I am aware of, all two of you, can enjoy or complain to me about this code. The ones I am unaware of are free to use the code as well.

SciSoft OSX Intel 2008.5.1 was released today. Nor’s blog post about the update states

The latest update to Scisoft OSX is now available. This includes an update to Pyraf (v. 1.6), GSL, pygsl, Gunplot, and a few backend libraries.

I went ahead and decided to install it today to investigate the improvements. As is usually my procedure, I first moved the directory containing my functional SciSoft OSX install temporarily out of the way via the command line:

sudo mv /scisoft /scisoft_old

Having done that I double-clicked on the installer package and let it do its thing, installing everything in the /scisoft directory. I poked around a bit an realized almost immediately that /scisoft/i386/Applications directory was empty! This one is kind of a show stopper as it means DS9 and FV are unavailable. I copied the files back from the previous installation without too much of a hitch. I did check, the installer package file with Pacifist and it does appear to contain those files in it, so I am not sure why they didn’t appear to install. I hope this is an isolated incident and not a recurring issue with the installer.

Other minor glitches I noticed:

• I discovered that /scisoft/i386/Packages, /scisoft/NEWS and /scisoft/i386/share are set to have owner 502:502 instead of root:admin. This glitch is easily fixed by issuing the following command from the Terminal
  

  sudo chown -R root:admin /scisoft/

• I also noticed that in addition to the x11iraf-1.5DEV installation, the entire x11iraf-1.3.2 installation is still sitting in the /scisoft/i386/Packages directory. All the x11iraf binaries in /scisoft/i386/bin/ are linked to x11iraf-1.5DEV instead of the older 1.3.2 binaries. I suspect this is an oversight.

An investigation of the /scisoft/i386/Packages directory as well as the NEWS file reveals the following changes to this version of SciSoft OSX over the 2008.3.1 version.

• GSL updated from 1.9 to 1.11
• DS9 updated from 4.13 to 5.1 (The current version of SAOImage DS9 is actually version 5.2, you can read about how to update the SciSoft version of DS9 in this post)
• GNUPlot updated from 4.0.0 to 4.2.3
• Swarp updated from version 2.15.7 to 2.17.1
• WeightWatcher updated from version 1.7 to 1.8.7
• pyraf updated from version 1.3 to version 1.6
• The following Python libraries were updated:
• • pygtk updated from 2.8.6 to 2.12.1
  • matplotlib updated from 0.90 to 0.91.2
• atk library updated from 1.10.3 to 1.22.0
• cairo library updated from 1.1.6 to 1.6.4
• cfitsio library updaed from 3.040 to 3.080
• glib library updated from 2.8.6 to 2.16.3
• glib-1.2.10 library added as well (possibly for a particular package needing older version of library)
• gtk+ library updated from 2.8.19 to 2.12.9
• gtk+-1.2.10 library added as well (possibly for a particular package needing older version of library)
• pango library updated from version 1.10.4 to 1.20.2
• pixman 0.10.0 library added
• libtiff library upgraded from 3.7.4 to 3.8.2

Hats off to the SciSoft OSX folks for keeping this package up to date. I have placed a copy in my SciSoft OSX mirror in case there are any access issues.

Immediately after installing SAOImage DS9 5.2, I had a major failure of the application and initially I just thought it was some sort of build bug. This is what I posted at that time:

[HOLD OFF ON THIS UPDATE! I have discovered that at least on one of my systems, this version of ds9 is refusing to run properly. It launched once, but when I attempted to check the “About SAOImage DS9”, it triggered the following error:

 “An internal error has been detected local header mismatch couldn't open “zvfsmntpt/doc/sun.gif”: no such file.

(this occurred in both Aqua and X11 versions). Furthermore, all future attempts to launch ds9 (again, either Aqua or X11) fail with the following error:

Error in startup script: couldn't read file “./zvfsmntpt/src/ds9.tcl”: no such file or directory  

Even removing the preferences file at ~/.ds9.prf didn’t help.]

Apparently, my problems with SAOImage DS9 in Leopard are a known issue. If you configure the built-in Firewall to “Set access for specific services and applications” so that you can approve “holes” in your firewall on an Application by Application basis, your first launch of SAOImage DS9 will irreparably damage the application!  Unfortunately, Apple implements the application firewall in part by modifying the Application package of the Application you are running by digitally signing it if it was not digitally signed by the developer (adding a file called CodeResources to the Application package). According the Apple’s documentation on this:

If you run an unsigned application not in the Application Firewall list, you will be presented with a dialog with options to Allow or Deny connections for the application. If you choose Allow, Mac OS X 10.5 will sign the application and automatically add it to the Application Firewall list. If you choose Deny, Mac OS X 10.5 will sign the application, automatically add it to the Application Firewall list and deny the connection.

So basically,Apple doesn’t warn you in the dialog box that comes up that it has whatever decision you make, it will modify the application by digitally signing it and it doesn’t give you a way to avoid this. This is, in my opinion, is an incredibly boneheaded move on Apple’s programmer’s part. They readily admit that

  Some applications check their own integrity when they are run without using code signing.

They suggest the application firewall will try to automatically detect these and avoid modifying them, but they should give you, the user, the option instead of making the decision via some internal algorithm.  MacOS X shouldn’t assume its OK to change an application. In the case of SAOImage DS9, they are irreparably damaging the application without leaving you a way to avoid the damage once you trigger the application firewall. Shame on you Apple. The only way to fix it is to reinstall the application!

So when I figured this out (a tip of the hat to this post on IRAF.net). I reinstalled the SAOImage DS9 executables (both Aqua and X11 versions) and before launching them, I set the Firewall (via the Security Pane of the System Preferences) to “Allow all incoming connections” (this is the default mode, so it is as secure as MacOS Tiger was). Everything now appears to work just fine.

Personally, I believe an application that fails its checksum should present a message indicating that is the problem instead of just crapping out, but in this case, the fault lies mostly with Apple. Apple is damaging applications by making this critical decision in the background, without user intervention!

[See my more recent post warning about MacOS X Firewall settings and how they can destroy the SAOImage DS9 executable during its first launch! This problem is avoidable by tweaking the Firewall settings, but once you have launched SAOImage DS9 with the bad settings, the application is damaged can can't be relaunched again. A reinstallation is the only solution, so it is a good idea to avoid this problem.]

The folks in Cambridge have kept busy. They have released SAOImage DS9 version 5.2. The versions for MacOS X include the following:

• Aqua 10.4 Tiger (Universal)
• Aqua 10.5 Leopard (Universal)
• X11 10.4 Tiger (Universal)
• X11 10.5 Leopard (Universal)

The rather extensive changes are detailed in the release notes here, but the notable ones to me include:

• ANALYSIS: for MacOSX tiger, wrap cmds with shell and PATH.
• GUI: change default directory for standard dialog to $HOME.
• ANALYSIS: add /sw/bin to default path for MacOSX. While unstated in the release notes, this is clearly an attempt to support Fink, which places its installation in the /sw directory.
• GUI: ds9 will now start in the users home directory for MacOSX Aqua users when invoked from a double click and the default dialog box is Motif or Windows.
• MACOSX: fixed a problem with printing non standard colors.
• MACOSX: restore postscript printing.
• REGIONS: apply WCS to fits regions if present.
• GUI: add support for user configured button bar.
• CATALOGS: add support for simbad.
• IMEXAMINE: added support for key stroke events.
• Although unstated in their release notes, they are now apparently providing universal binaries instead of PPC and Intel binaries for MacOS X.

I have previously posted notes for integrating upgrades of DS9 into the Scisoft OS X installation and they still work just fine.

To quote iraf.net:

As of September 2007, NOAO has resumed development of IRAF on a limited scale, although primary responsibility for user support remains at the community IRAF.NET web site. IRAF V2.14 is a formalized version of the IRAF.NET V2.13 releases previously made available by iraf.net.

The major version number increase is meant to disambiguate NOAO releases from earlier versions and to provide a clean code base for future development. Unlike earlier ‘major’ IRAF releases that contained substantial new functionality, this release may seem a bit lacking. Support for Intel-based Mac and Cygwin platforms is the key highlight as well as many unseen but required maintenance/viability changes. A number of new tasks and features have been added, either inherited from the V2.13 releases or done as part of external package development.

It appears a full installation is required, even if you have previously installed IRAF 2.13. I won’t be upgrading until SciSoft OS X is upgraded (there is a beta I am playing with of SciSoft, but it has IRAF 2.13 in it). Marcos Huerta is already playing with it so those of you needing package installers of (just) IRAF 2.14 will probably get your wish soon enough.

No… the new Scisoft OSX isn’t out quite yet, but I got the following note from Nor Pirzkal responding to my post regarding the various broken symbolic links in SciSoft OSX 2007.11.1.

Just to let you know that I just implemented your changes in 2007.11.2b. Thanks again.

So it looks like these bugs will be fixed when Scisoft OSX 2007.11.2b is released soon. I have to hand it to them, Nor is pretty quick to respond to these bugs, given he has a full time job which doesn’t involve maintaining SciSoft OSX!

On October 15, the folks behind SAOImage released SAOImage DS9 version 5.0. The big change I noticed is they now have a completely MacOS X native (read “Aqua”) version of ds9 (but only if you use the application package version of ds9, the command line versions remain X11)! I downloaded the following three versions:

• MacOSX 10.4.x (Universal)
• Darwin PPC 10.4.x
• Darwin Intel

The new features lists page tells us that this release includes:

• MacOSX Aqua Support: DS9 has been ported to MacOSX Aqua and is an universal application which no longer requires X11.
• Compressed FITS Support: DS9 supports compressed FITS images using RICE compression.
• Mask Support: DS9 supports overlay masks. A mask is defined as a valid FITS image, in which a non zero value indicates that the selected mask color is to be displayed instead of the data value color.
• SkyView Support: DS9 provides support for HEASARC’s image cutout service, SkyView. This site provides image cutout service for a number of image surveys, including SDSS.
• Multi-Language Support: DS9 provides multi-language support. By default, the language used for menus and dialog boxes is based on the value of the operating system locale variable. The user may override the default value by selecting the desired language in the preferences or by the -language command line option.
• Preferences: Preferences are automatically saved when a user changes an option. Selecting the saving preferences menu item is no longer needed.

More detailed release notes are available here. I was able to get this version of ds9 integrated with SciSoft OSX by doing the following:

1. Decompress the command line version of ds9 via the terminal using tar xzvf ds9.darwinppc.5.0.tar.gz (PPC version) or tar xzvf ds9.darwinintel.5.0.tar.gz (Intel version). When the decompression is done, all you have is an executable called ds9.
2. Next, from the terminal, go to the /scisoft/bin directory (on PPC) or /scisoft/i386/bin/ directory (on Intel) and rename the old ds9 executable to something like ds9_old (using something like mv ds9 ds9_old).
3. Copy your newly decompressed ds9 executable into the SciSoft OSX binary directory. I should note the command line version of ds9 still requires X11.

The IRAF.net forums had a post from someone having issues getting the IRAF package rvsao to compile properly on MacOS X on Intel-based Macs. I’ve commented about this before and had discussions with Doug Mink, the fellow behind rvsao as I have battled to compile it. It’s not actually that tricky, you just have to be careful. The key issue is that the most commonly used g77 compiler on Intel-based Macs is probably g77 version 3.4 as downloaded from the High Performance Computing for Mac OS X site. This version of g77 is a bit more picky about logical statements. Specifically, when you try to compile rvsao within IRAF using mkpkg, you get the following error (as noted by Phil Massey in the IRAF.net Forums earlier today):

I’m trying to install the external package rvsao on my Intel mac. However, when I go to do the mkpkg there is a program “writetemp.f” that won’t compile:

In subroutine `tmpwrf':
writetemp.f:139:
if (.not.(debug .eq. .true.)) goto 130
1 2 3
Use .EQV./.NEQV. instead of .EQ./.NE. at (2) for LOGICAL operands at (1) and (3

I was having exactly these sorts of errors with not only rvsao, but several of the SAO IRAF packages this summer and after banging my head on this a bit, I found a solution that worked for me. It turns out g77 needed to be told to use less restrictive logicals via a “-fugly-logint” flag during the compile. So my solution was to write a perl script that was saves as /usr/local/bin/f77, such that when f77 was called (by these IRAF mkpkg calls) it would issue a g77 call with this flag set on the compile. So I posted about this on that IRAF.net forum thread started by Phil Massey.

I got a response from Mike Fitz where he points out that you can set up the XC compiler used by IRAF to always use these flags via environmental variables:

If you use ‘g77′ all the time and need this flag it can be set for all compilations either in the user environment (e.g. your .cshrc file) using:

Code:

setenv XC_F77 g77 setenv XC_FFLAGS “-fugly-logint”

or for problematic packages you can edit the ‘mkpkg.inc’ file for the package (normally in the pkg$lib directory but in the case of RVSAO it’s in the main rvsao$dir) to add “-/fugly-int” to the ‘XFLAGS’ definition. The ‘/’ tells the XC compiler to pass the flag thru to the underlying compiler unchanged, but note this sometimes causes problems or warning if say g77 knows the flag but gcc doesn’t (i.e. the XFLAGS are passed to all sources being compiled, you’ll need the XC_FFLAGS trick to pass only to fortran code).

Another new version of Scisoft OSX for Intel has been released and is available for download here. Here’s the entire description of the changes in this version of Scisoft OSX from the release notes:

This version contains an updated [ESO-]MIDAS package.         

I checked the list of packages included and sure enough, the only change was in the ESO-MIDAS package (whose current version is actually 07SEPpl1.0).

• MIDAS 07FEBpl1.1 (previously MIDAS 06FEBpl1.0)

I don’t use ESO-MIDAS, which is an image reduction package aimed as users of ESO’s La Silla facilities and the VLT at Paranal, so I can’t really comment as to the usefulness of this update. As with previous versions of Scisoft, the release notes warn if you use Aquaterm devices for PGPLOT, GNUPLOT, or any other packages that you “must manually run /scisoft/i386/Applications/Aquaterm.app once first to enable the aquaterm devices.”

I have also confirmed that this release also fixes the IRAF problem with symbolic linking to mkpkg and xc in /scisoft/i386/bin/ I referred to earlier in my blog.   Thanks for the fix guys!

Finally, as I noted with the last release of Scisoft, my standard method for updating Scisoft OSX is to move my existing /scisoft directory to /scisoft_old and then I unpack the new version. That way, in case anything goes wrong, I can always switch back.