As I mentioned in my blog post earlier today, I have been having issues using the JpGraph graphing package for PHP with Apple’s built-in PHP under MacOS 10.5. It appears Apple’s security efforts have “secured” PHP to the point where JpGraph (and PDFLib) will not function properly under the built-in PHP. [Note added after initial post: The rest of this post describes installing Apache 2 and PHP under MacPorts and configuring it to be similar to Apple's built-in servers. This appeared to solve my problems, but then my Mac locked-up [possibly related] and on reboot, the new MacPorts-based PHP installation started throwing the same errors as Apple’s built-in PHP. More information on this problem is located in my newer post on the issue.]
I had tried to alleviate the solution previously by compiling a version of PHP myself that would be compatible with Apple’s built-in Apache 2 web server. This turned out to be difficult because Apple’s Apache 2 web server is a “universal” binary, meaning it contains four seperate binaries (for 32-bit PowerPC, 64-bit PowerPC, 32-bit Intel, and 64-bit Intel processors). As such I needed to compile a “universal” PHP binary and since I wanted MySQL support, I needed a “universal” MySQL binary. This turned out to be too much for this astronomer, so I gave up on making a new PHP that was compatible with Apple’s built-in web server.
So I bit the bullet and after reviewing the options, I decided to install Apache 2 and PHP under MacPorts. If you have read the other posts on this site, you know I really like MacPorts as the quick and dirty way to get many things running on the Mac. However, despite this, I don’t like installing MacPorts for items Apple provides, instead generally preferring to use Apple’s “pre-installed” versions. Furthermore, there have been a lot of complaints on the MacPorts mailing lists about various issues compiling PHP5. So I wasn’t as quick to jump onto the MacPorts bandwagon for PHP as I am for other problems. However, since I am familiar with MacPorts, I decided this was the best approach for getting my online Finder Charts to work again.
The process proved reasonably painless, especially since I was able to review this blog post, where he lays out pretty much what to do. However, since I wanted to achieve maximum compatibility with Apple’s built-in web server and PHP setup, in case I wanted to switch back, I ended up doing things a little bit differently, so I am outlining my steps here.
I started by installing the MacPorts version of Apache 2 using the command sudo port install apache2
Next, I had to create the Apache 2 configuration files and edit them. I started by copying the sample configuration file
and then editing /opt/local/apache2/conf/httpd.conf with my favorite text editor to change the configuration to match the that of the built-in Apache 2 server a closely as possible. My matching the configuration of Apple’s built-in server, I can switch back to it with relative ease if I choose to do so later. So I made the following changes to /opt/local/apache2/conf/httpd.conf:
I changed DocumentRoot to "/Library/WebServer/Documents" as is the case with Apple’s built-in server.
To allow index.php files to be used as directory indexes, I changed
DirectoryIndex index.html
to
DirectoryIndex index.html index.php
(I don’t know why this isn’t necessary with Apple’s built-in server, but it was necessary here).
I changed to
and in that directory block of code, I added “MultiViews” to the Options line.
I changed to in order to prevent the listing of .DS_Store files in directory listings by the web server.
Before the ErrorLog block of code in this file, I added the following lines copied from Apple’s default Apache 2 configuration:
#
# Apple specific filesystem protection.
#
Order allow,deny
Deny from all
Satisfy All
Order allow,deny
Deny from all
Satisfy All
I changed ErrorLog to "/private/var/log/apache2/error_log"
I changed CustomLog to "/private/var/log/apache2/access_log common"
To match Apple’s Apache 2 server configuration, I changed ScriptAliasMatch to
I uncommended the following lines near the end of the file:
[OPTIONAL] Because I use the WebDAV server on my server, I also uncommented
Finally, I added the following lines to the end of the file in order to allow loading of the PHP5 configuration
Next, I editted /opt/local/apache2/conf/extra/httpd-userdir.conf and added the following to the end of the file
#
# Users might not be in /Users/*/Sites, so use user-specific config files.
#
Include /private/etc/apache2/users/*.conf
[OPTIONAL] Since I use the built-in WebDAV server, I made a backup of the WebDAV configuration, then copied the
Default MacOS X one, because I have spent a lot of time tweaking it previously and I didn’t want to have to reinvent the wheel.
I had to install PHP5 with MacPorts. Since I wanted to add support for Apache 2 and MySQL, I entered the command:
sudo port install php5 +apache2 +mysql5 +pear
which has the side effect of installing MacPorts version of MySQL as well. Assuming everything goes well, after a few minutes (this takes longer than the apache2 install earlier), the installation will end. At this point we can configure the Apache 2 mod_php module by typing:
cd /opt/local/apache2/modules/opt/local/apache2/bin/apxs -a -e -n "php5" libphp5.so
[OPTIONAL] Next, I had to create the PHP5 configuration file and edit it. I started by copying the sample configuration file
sudo cp /opt/local/etc/php.ini-dist /opt/local/etc/php.ini
and then editing /opt/local/etc/php.ini to make it match /etc/php.ini (which Apple’s built-in PHP uses). All the changes I made were optional and related to the specifics of my setup. The only interesting one was that I wanted to continue to use the MySQL.com binary distribution of MySQL server, so I set the following variables in /opt/local/etc/php.ini:
mysql.default_port from “” to 3306
mysql.default_socket from “” to /private/tmp/mysql.sock
mysqli.default_socket from “” to /private/tmp/mysql.sock
I deactivated Apple’s built-in web server by turning off Web Sharing in the Sharing.prefPane.
Finally, I launched the new webserver (and set it up for launching on boot-up in the future) by typing
[OPTIONAL] I like the ability to turn on and off the Apache webserver from the command line using apachectl. I can “emulate” this in /bin/tcsh (my prefered shell) by adding the following command to the ~/.tcshrc file:
alias apache2ctl 'sudo /opt/local/etc/LaunchDaemons/org.macports.apache2/apache2.wrapper'
After which I can bring down the server by typing apache2ctl stop and restart it by typing apache2ctl start.
Applying the latest MacOS X Security Update from Apple today reminded me of the problems I have had with PHP on Leopard. The Security Patch brought Leopard up to version 5.2.6. Unfortunately, it doesn’t fix a problem I have been having with PHP5 under Leopard.
While I’m sure Apple did this for security reasons, since upgrading the Leopard, I have had a problem with scripts that use the JpGraph plotting PHP library crashing. After looking in /var/logl/apache2/error_log, I see the following eror message:
The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.
I’ve been trying to figure out why this has been happening for a while. Johan Persson at JpGraph and I exchanged some emails where he outlined for me how this was a problem with Apple’s pre-built version of PHP5 since JpGraph is not fork()-ing any processes. Well, today I decided to look into this again and discovered that the PDFLIb folks have documented this problem a little better, copying from their documentation here:
PHP on Mac OS X. Apple’s PHP version which is bundled with Mac OS X does not work with PDFlib DSOs. To use PHP with PDFlib on Mac OS X you need third-party PHP packages such as MAMP, XAMP[P] for Mac, or Marc Liyanage’s version from www.entropy.ch. Mac OS X 10.5 (Leopard) adds new complications. As described in developer.apple.com/releasenotes/CoreFoundation/CoreFoundation.html it is no longer possible to use CoreFoundation functions after a call to fork( ) without exec( ). However, CoreFoundation functions are required for PDFlib’s host font feature, and the critical sequence above is used in the combination of Apache and PHP. This may trigger the following error message in the Apache log (and can even crash the Apache process):
The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on _THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.
In order to avoid this problem you can run PHP as a CGI on Apache, or disable the host font feature in PDFlib[.]
Sure enough, JpGraph is using host fonts. Damn.
Unfortunately, the Marc Liyanage’s version of PHP5 for Leopard from www.entropy.ch is still in beta and the last update of MAMP was quite a long time ago. And while XAMPP for MacOS X looks promising (at least it has been updated recently), I would really like to keep on using the MySQL server I already have running. So for now, I try installing Apache 2 and PHP using MacPorts.
A few folks have been aware that I am involved in an interesting project to determine the “Shape of our Galaxy”… specifically the nature of the asymmetries observed in the Thick Disk stars in the galaxy. Part of that project involved obtaining multi-fiber spectrograph observations of many stars in selected fields of the sky using the Hectospec multi-fiber spectrometer on the MMT on Mount Hopkins.
Unfortunately, there are very few “external” users of the Hectospec, so the software pipeline available for reducing Hectospec observations, called SPECROAD, was only geared to run on SAO computers. I spent a considerable amount of time last summer and this summer getting that software into a much more portable form, documenting how to use it, and removing bugs from the code.
All this work resulted in the creation of a new version of the Hectospec data pipeline I dubbed “External SPECROAD” or “E-SPECROAD” for short. There were several major problems with the original SPECROAD code that I addressed in E-SPECROAD:
Nonportability: The SPECROAD scripts were only designed to run on a Solaris computer at the SAO, with files in specific hard-coded locations using a specific IRAF environment with all the IRAF parameters set before the scripts were run. This has been addressed by trying to assume almost nothing about a user’s IRAF environment from the start except that they have the proper packages installed.
Lack of Documentation: The documentation for SPECROAD is frankly inadequate. It was an in-house tool for the SAO, so the publically-available documentation consists mostly of one webpage describing what the scripts do. I have now written up both an installation guide (because there are a lot of pre-requisite pieces of code to install) and a E-SPECROAD user’s guide.
No Installation Instructions: There were several IRAF packages to install (available here). The shell scripts required not only the installation of several IRAF packages and the compilation of several C programs, they used korn shell (ksh) which I was unfamiliar with. And, as I would also soon discover, the ksh installed on the Mac had serious bugs in its list handling. There is now a fairly compete installation guide laying out all the pre-requisites and addressing installations on both Macs and Linux boxes.
Undocumented Bugs: The available SPECROAD scripts had major bugs whose workarounds were undocumented and only available when I would email someone at the SAO to inquire about them. In one case, you had to know to actually break out of the script with Control-C, then run some of the reduction in IRAF, and then restart the script! E-SPECROAD has been fairly extensively debugged. It’s not bug free, but the exterminator has made some serious passes at the code.
Assumed Availability of Spectral Templates: The SPECROAD scripts typically relied on pre-set spectral line databases for different grating/central wavelength configurations instead of having the user exploit the IRAF identify command to build a database. E-SPECROAD is geared toward the user who is going to wavelength calibrate their observations by building the spectral line database for themselves.
Assumed Use of Low Resolution Spectra: The SPECROAD scripts were mostly pre-configured for working with 270gpm gratings and required some tweaking for 600gpm. E-SPECROAD can work with either configuration without editing the code.
Inconvienent Backups As Data Reduction Progresses: If one of the scripts executed by SPECROAD crashed, you had to dig around subdirectories to “backtrack” the changes before restarting the script. With disk space not an issue, I prefer monolithic backups of the entire data directory at certain key points in the data reduction pipeline. Since you may not like this form of backup (as they consume a lot of disk space), they are optional.
Unfortunately, it looks to to be as twitchy as running your own Darwin Calendar Server. I’ve been using CalDAV on my own personal server for a few months and I have to admit it has been a bit twitchy, but getting better as time rolls on. I tried to synchronize this afternoon to Google Calendar and got the following error:
The server responded with
"HTTP/1.1 500 Internal Server Error"
to operation CalDAVWriteEntityQueueableOperation.
I got this error when trying to create a new calendar in Google from iCal. This is actually a documented error, but it didn’t go away for me like Google suggested it would. I am also getting that error when I try to import items into my GCal calendar via an “import” of the ics file I “exported” from my old calendar. I saw this same error on my server some time back. I needed to update my Darwin Calendar Server source code before I could fix it. In any case, I hope this is only a temporary issues since I could be very happy with the idea that I can now drop the maintenance of Darwin Calendar Server on my own computer and hand that task over to Google.
Followup (July 30): I noticed all of the errors occurring during the import were for old classes last semester. So I forced my way through them by clicking “Revert to Server” for every one of the errors. There were probably 20 of them. Once done, I compared my calendar on my computer and Google’s Calendar, they were identical. So for now, it looks like GCal’s CalDAV server may serve my needs after all. I could have avoided all this by directly importing my ics file into Google through their web interface.
Followup (August 27): So I got an iPod Touch and discovered that CalDAV calendars are loaded on the iPod Touch as “Read-only”. If Apple really wants to back CalDAV as a standard, you would think they would support it more fully on their own products. For now, I found that BusySync works quite nicely for syncing my iCal calendars with gCal in such a way that the calendars are hosted on my Mac and the iPod Touch then can treat the calendars are read-write. This, along with the lack of native ToDo list synchronization on the iPods are about the stupidest bits of inconsistency I have seen in using the iPod Touch. Luckily the ToDo list being missing from the iPod Touch doesn’t affect me as much as I am using OmniFocus, which works very well on the Mac and iPod Touch and allows synching between both.
I can attest the Aqua version of SAOImage DS9 version 5.3beta does indeed play nice with Apple’s dopey firewall behavior (see here for notes on version 5.2’s incompatibility with the Leopard firewall). However, the command line version that uses X-windows DOES NOT play nice with the Leopard Firewall. If you run the X-windows version of “ds9″ on a Mac running Leopard’s built in Firewall in “Set access for specific services and applications”, you will end up with a completely hosed ds9 executable which will not launch ever again.
As such, for now, since I prefer the X-windows version of SAOImage DS9, I am leaving the the Firewall off for now, I’m not too concerned.
Today I noticeed two interesting software updates for Mac-based professional astronomers.
The first one I noticed was the updating of Xquartz to version 2.3.0. Xquartz is the updated version of X11 for the Mac OS (even ahead of Apple’s own installed versions) that I prefer to use, largely because bug-fixes get rolled in here before they appear in Mac OS. This version requires Mac OS 10.5.4 and has a couple of caveats attached to it for programmers, notably:
The software supporting the deprecated imake build system is not provided in this package. If you need imake and xmkmf, please install the X11 package that came with your Leopard DVD before installing this version. Alternatively, you can compile these packages on your own or get them from a third party such as Fink or MacPorts. The darwin configuration files used by the imake build system are outdated and not supported. Developers using this build system are advised to migrate to autoconf.
[Added July 24, 2008: Apparently, this version of Xquartz changed the X11.app Icon so it now X11 looks like this
when it is on the Dock. Interesting, but it threw me for a second. The only documentation I found of this change is in a Ticket filed with XQuartz's bug reporting system. Still, I think this is a good idea, as it gives a visual cue that you are using XQuartz as opposed to the default X11 installation.]
Along with a bunch of library changes, the key update appears to be having the Xserver updated to the 1.4 branch of Xorg. There is also “support for adding new $DISPLAY sockets after the server is running” (which I think means using the DISPLAY variable will not break things) and “/usr/X11/bin/Xquartz is just a stub that will ‘do the right thing’,” whatever that means. I have upgraded to it and as a reminder, if you upgrade MacOS after installing Xquartz, you will need to reinstall Xquartz to get it back.
When I moved the my Scisoft OSX mirror to a new server, I messed up the permissions on the files, preventing users from downloading the Scisoft OSX packages from the mirror. This has now been fixed.
Upgraded the freetype library to version 2.3.6, which fixes “A bunch of potential security problems have been found [and fixed in this release”
Upgraded to pixman library to version 0.11.4.
Xquartz fixes from xorg-server-1.3.0-apple21, the key fix being support for monitor hotplugging, although several security fixes also occurred.
Again, if you upgrade MacOS (say to version 10.5.4, which is supposed to be released soon in order to support iPhone G3 and MobileMe), you will likely need to reinstall Xquartz (unless Apple has upgraded their X11 installation to match Xquartz).
I have been an avid Firefox user for quite some time. But when I moved to Intel-based Macs, I discovered that that Schubert’s PDF Browser Plugin didn’t work on Intel Macs (except if I placed Firefox in Rosetta mode, running its PPC code in emulation). hat project appears to have died on the vine, with no updates in abut 2 years. Furthermore, Adobe has insisted on making its inline PDF plugin Safari-only. This always struck me as rather redundant, since Apple has used Mac OS X’s Quartz graphics engine to allow viewing of PDFs inline in Safari. This, combined with Firefox’s longer launch times, made me slowly shift to using Safari about 80% of the time.
Yesterday, Firefox 3 was released. I have been using the betas for the last month and have been happy with its improved speed and functionality. Its part of the reason I am back to about a 50/50 split in Safari vs. Firefox use. Today, the other show fell in the form of this posting on MacOS X Hints:
There is now a Firefox extension named firefox-mac-pdf, available for Firefox 3 under OS 10.5 that utilizes the built-in PDF support in OS X to display PDFs in-browser. In my testing, it appears to work very well. It doesn’t have the nifty fading bezel that the Safari PDF viewer does, but it supports all the same keyboard shortcuts and you get the standard Mac OS PDF contextual menu when you control-click on a displayed PDF.
Its interface is not quite as easy to use as Schubert’s plugin, but it works. I now have inline PDF viewing in Firefox and things are better in the world again.
Saving PDFs: The one issue I noticed is there was no seemingly obvious way to save the PDF once you were viewing it. Nothing in the contextual menu allowing “Save as…” for example. Turns out it was easier to save the PDF than I imagined. In the “Issues” page for firefox-max-pdf I found this exchange which included the solution:
We had a set of hard storms come through yesterday afternoon. I was doing SkyWarn spotting with the local hams. I think (but will leave it to experts to confirm) that the structure under the smooth part of the storm (”the shelf) is a wall cloud. For a sense of scale, this is 8 photos stitched together covering above a fifth of my horizon and the leading edge of this storm when it was about five miles away.. Shortly after I shot this picture, the sirens went off. Rotation was spotted, but luckily, no funnel cloud touched down. Thankfully, while the storm blew through at 60 miles per hour, we had no significant damage… Still, it looked mean…
Addendum: I sent the picture to Greg Gust, who runs some of the SkyWarn stuff for the National Weather Service in Grand Forks. He sent me these additional comments:
Most of the structure is a classic shelf cloud on the leading edge of a line of thunderstorms… this the widespread strong to severe winds as it came through. One the south end was the feature that we believe was a wall cloud… rotating updraft… and was what prompted us to issue the Tornado Warning across the southern part of Fargo and Moorhead.
[Added August 9,2008] It appears at least two people saw this wall cloud and posted videos on YouTube. I have embedded them below.
This first video is from “bluemanhal” and contains some audio. Notice how far he has to pan the camera to see this. My photo was a panorama about 150 degrees, which loses its perspective in the image:
This second video is from “cpilotkid” and appears to have been shot from the West Acres Mall parking lot, but again notice how far he has to pan to get the entire wall cloud:
The latest update to Scisoft OSX is now available. This includes an update to Pyraf (v. 1.6), GSL, pygsl, Gunplot, and a few backend libraries.
I went ahead and decided to install it today to investigate the improvements. As is usually my procedure, I first moved the directory containing my functional SciSoft OSX install temporarily out of the way via the command line:
sudo mv /scisoft /scisoft_old
Having done that I double-clicked on the installer package and let it do its thing, installing everything in the /scisoft directory. I poked around a bit an realized almost immediately that /scisoft/i386/Applications directory was empty! This one is kind of a show stopper as it means DS9 and FV are unavailable. I copied the files back from the previous installation without too much of a hitch. I did check, the installer package file with Pacifist and it does appear to contain those files in it, so I am not sure why they didn’t appear to install. I hope this is an isolated incident and not a recurring issue with the installer.
Other minor glitches I noticed:
I discovered that /scisoft/i386/Packages, /scisoft/NEWS and /scisoft/i386/share are set to have owner 502:502 instead of root:admin. This glitch is easily fixed by issuing the following command from the Terminal
sudo chown -R root:admin /scisoft/
I also noticed that in addition to the x11iraf-1.5DEV installation, the entire x11iraf-1.3.2 installation is still sitting in the /scisoft/i386/Packages directory. All the x11iraf binaries in /scisoft/i386/bin/ are linked to x11iraf-1.5DEV instead of the older 1.3.2 binaries. I suspect this is an oversight.
An investigation of the /scisoft/i386/Packages directory as well as the NEWS file reveals the following changes to this version of SciSoft OSX over the 2008.3.1 version.
Hats off to the SciSoft OSX folks for keeping this package up to date. I have placed a copy in my SciSoft OSX mirror in case there are any access issues.
The Xquartz project is a useful one for Leopard users of astronomical applications because of the dependence of most astronomical applications on X11. A few days ago (what can I say, its finals and I am swamped with exam writing and grading) they released version 2.2.1. It contains all the tweaks that made it into the previous version and also includes
All packages updated to versions intended to ship as part of X11R7.4 (as of 2008.04.21).
Fixed multiple crash-causing bugs in the server.
Fixed cmd-tab to properly move all windows forward when entering X11.app.
Cleaned up multi-monitor support (still not completely bulletproof) [I have 2 monitors, so this is a big one for me].
As I have noted before, Apple includes some of the work done in this project in OSX updates, so it is suggested that you install the latest XQuartz release after updating to 10.5.2 (and any future 10.5.x or security updates).
