Workshop & IDL extensions

Cluster Workshop

This week we had a maxBCG cluster workshop in AA.
It was very productive from my point of view. We clarified
the science we want to do, defined tasks that need to
be completed, and actually did some work. I got to some
of the sub-sampling that had been on the back burner while
I worked on systematics. These were pretty enlightening,
including a feature seen that an L200 luminosity split in a tNgals
2Mpc bin resulted in very weird profiles as you might expect,
while L200 in an N200 bin was much cleaner. At this point
I'm really liking N200 measures and I think the work that
Eduardo and Morad are both doing to improve this will be
fruitful.

I also did some X-ray clusters and, although the sample is
small and S/N not terrible exciting, it points to some interesting
future work, such as going back to the RASS and looking for
lower flux sources like we did back in the 42 clusters paper.

IDL extensions

I've been getting tired of the crappy file reading
support IDL has, and the slowness of fits and its
inability to properly deal with 64-bit integers. A while
back I wrote a suite of routines to read and write structures
to a file. These handle all the basic data types of IDL
and can write/read in either the native byte order
or another. This makes reading at least 3 times faster
than fits on an little endian machine.

But what is really needed is the ability to extract
individual rows and columns for both memory
and speed reasons. You can't do this efficiently
with the build in routines. I wrote binary_read and ascii_read
C routines, linked to IDL with the DLM mechanism, which
allow efficient reading of data from these types of files into
structures. You can extract just the data you need fast.

Also polished off ascii_write which writes a structure to an
ascii file. Many times faster than looping and doing printf statements.
The idlstruct routines can all work without these, but if these are
compiled it makes them much more powerful.

I realized as I was finishing off binary_read that this is
approaching the ability of databases. I thought about how
I could modify those simple Goddard database routines, which
just work with the most basic read/write abilities of IDL,
to make an extremely efficient native IDL database system.
Don't know if it's worth it since I have an interface to postgres,
but it would have the advantage of not requiring another
program. It would never have all the features. And, besides,
I'm supposed to be migrating to Python!

Anyway, the sdss idl (aka sdssidl, umsdss_idl) code, C/C++
and documentation are here:

http://cheops1.uchicago.edu/idlhelp/sdssidl/umich_idl.html

Erin