rmfit Tutorial

 

Also see:

Prerequisites

  • IDL
    alias idl='/afs/slac.stanford.edu/g/ek/rsi/idl_6.1/bin/idl'
    /afs/slac.stanford.edu/g/ek/rsi/idl_6.1/bin/idl

OR

  • Virtual Machine (VM)

See Install IDL VM.

xx_Download and install rmfit Version 3.1.1.

  • GNU Scientific Library

Download and install GNU Scientific Library (e.g., gsl-1.13.tar.gz).

tar -xzvf gsl-1.13.tar.gz
./configure --prefix /afs/slac/g/glast/users/yourDirectory
make
make install

Note: If you have ROOT permission, you can install wherever you like by setting --prefix accordingly.

Note: To compile GSL you will need an ANSI C-compiler. Both static and shared versions of the libraries will be compiled by default. Some platforms do not support shared libraries. (See GSL - GNU Scientific Library: Installation Instructions.)

Notes to self -- Starting point:

  • First, ssh to noric05.
  • Then check "whereis libgsl.so". It won't be there, but this way you can check again after the build.
  • After the build, Mark recommends checking the make script to determine where libgsl.so is installed. (issue "whereis libgsl.so" to get a clue.)

Part 1. Download rmfit

Notes:

  • Instructions in this section are written for rmfit version 3.1.1
  • At the time of this writing, mfit has not been compiled for Windows, so spectral fitting is not possible in a Windows environment; hence comments related to Windows have been edited out of this version of the instructions. -CP
  • All rmfit questions can be addressed to Rob Preece.

Install rmfit

  1. Copy rmfit to your work directory:

cp /nfs/farm/g/glast/u54/omodei/rmfit_v31rc1.tar .

  1. Untar the file:

$ tar xf rmfit_v31rc1.tar
$ ls
bin idl711linux.x itt rmfit_3.1rc1 rmfit_v31rc1.tar

$ cd rmfit_3.1rc1/
$ ls
README.txt RMFit Windows Installation Instructions.doc data hlp lib license mfit print.cfg rmfit.pro rmfit.sav save_parms.sav src util

$ cd mfit
$ ls
dlm mfit_func.info src

$ cd src
$ ls
Docs gsl_interface_routines.c mfit.F95 mfit_idl.c mfit_kinds.f95 mfit_user_func_b.f95 Make.mfit.sh gsl_interfaces.f95 mfit_cparams.h mfit_interface.c mfit_user_func_a.f95

chuckp@noric07 $

Note: While you can run rmfit directly in IDL, we will be running the .sav version on the virtual machine (vm) using the free downloadable IDL virtual machine that we have just installed.

libgsl:

  1. Edit the make script Make.mfit.sh to change the compiler and linker options from:
  • from C to C++
    (/usr/bin/c++ /usr/include/c++ /opt/TWWfsw/bin/c++)
  • from Fortran 95 to gcc
    (/usr/bin/gcc /opt/TWWfsw/bin/gcc /usr/share/man/man1/gcc.1.gz)

    Note: If your versions of gcc or gfortran have non-standard names, you can provide the names to the MFIT make script by defining the enviroment variables CC or F95, e.g., "export CC=gcc-4.3", or "export F95=gfortran-4.3".

  • export GSL_PATH=/usr/lib/libgsl.la /usr/local/lib/libgsl.so /usr/local/lib/libgsl.la /usr/local/lib/libgsl.a

    Note: If libgsl.* is installed in a standard location, such as /usr/lib, the linker will automatically find it. The link will most likely work if libgsl.* is at /usr/local/lib.

If your GSL library is located in a less standard directory, you will probably have to inform the MFIT make script of the directory using the environment variable GSL_PATH, e.g., "export GSL_PATH=/opt/local/" if libgsl.* is located at /opt/local/lib (MacPorts) or "export GSL_PATH=/sw" (fink) if libgsl.* is located at /sw/lib.

The full path is not used because the shell script also must locate the include files. See the "GSL NOTES" section below for an explanation of how to obtain and install the GSL.

  1. To compile the MFIT Fortran shared object, from the rmfit/mfit/src subdirectory, source Make.mfit.sh (e.g., ./Make.mfit.sh).

Troubleshooting Tips:

  • Make sure IDL knows how to find the shared object for mfit. On UNIX, including
    Mac OS X, IDL uses the environment variable IDL_DLM_PATH. The DLM for mfit
    is located in subdirectory "mfit/dlm" of the rmfit directory.

    An example definition of IDL_DLM_PATH is:

    export IDL_DLM_PATH="<IDL_DEFAULT_DLM>:/home/mydir/rmfit/mfit/dlm"

    The first field provides the directories that IDL needs to find its own shared objects. <IDL_DEFAULT_DLM> is not a shell enviroment variable but rather a symbol internal to IDL -- IDL will expand it to the correct directory containing the IDL DLMS. The second field, after the colon, is the "mfit/dlm" subdirectory of your location of the rmfit directory.

Before configuring the environment variable IDL_DLM_PATH, determine IDL's default DLM path. Enter IDL and type "print, !dlm_path". Typical outputs are:

/Applications/rsi/idl61/bin/bin.darwin.i386/ [Mac OS X]
/opt/local/rsi/idl61/bin/bin.linux.x86_64 [Linux]

Use the directory that you found from "print, !dlm_path" in the definition of IDL_DLM_PATH (above) instead of <IDL_DEFAULT_PATH>.

  • Make sure IDL knows how to find the source files. On UNIX, IDL uses the shell environment variable IDL_PATH:

    export IDL_PATH="<IDL_DEFAULT_PATH>\ :+/home/mydir/rmfit\:/usr/local/IDL_AstronUsersLibrary/pro/"

The symbol <IDL_DEFAULT_PATH> is not a shell enviroment variable but rather a
symbol internal to IDL — IDL will expand it to be the directories containing
IDL's own source files. The second field after the colon is the directory where you have located rmfit; the "+" will cause IDL to search all subdirectories.

Before configuring the environment variable IDL_PATH, find out the default location for IDL's source files on your system by entering "print, !path" at the IDL prompt. You will see a huge list of directories, however, they should all have top levels in common, up to /lib or /examples. Typical examples for the common portion of the IDL Default directory are:

/Applications/rsi/idl61/ [Mac OS X]
/opt/local/rsi/idl61/lib [Linux]

Extract this common portion and include it in the definition of IDL_PATH (above),
prepended with a "+" symbol, instead of using <IDL_DEFAULT_PATH>. The '+' allows
IDL to expand the given path to all subdirectories, which we want to do.

  • If you plan to run the save file version, find the rmfit.sav file in the /rmfit directory and copy it into a useful directory for your analyses; i.e., at the root of a directory of FITS files.
  • To run rmfit:
    1. To run the save file version, which doesn't require an IDL license:
      Enter "idl -vm='/path_to_dir_w_rmfit_sav/rmfit.sav'" at your system prompt.
      (Or "idl -32 -vm='/path_to_dir_w_rmfit_sav/rmfit.sav'" to run in 32-bit mode
      on a 64-bit machine.)

    If you will be doing this often, you can make an alias and add it to your login file (in bash): alias rmfit="idl -vm='/path_to_dir_w_rmfit_sav/rmfit.sav'"
    You will need to click through the banner ad for IDL to get to rmfit.

    b) To run rmfit directly, which requires an IDL license: Type "idl" (or "idl -32" to select 32-bit IDL on a 64-bit machine), then "rmfit".

    If you encounter errors running rmfit, especially when trying your first spectral fit, please check the section "RUNTIME ERRORS" below.


    Note: There is a user's guide available in the RMFIT window's Help menu; the same help information can be found in the rmfit/help/help.hlp text file. If you are unsure where to begin, this is the place.

GFORTRAN NOTES

You will need C and Fortran 95 compilers. The Fortran 95 compiler needs to support the Fortran 2003 ISO C Bindings, as most now do. gcc (version not critical, >= 4.0 is reasonable) and gfortran >= 4.3 are suitable. GNU Compilers 4.3.0 were released more than a year ago (2008 March 5). Many Linux distributions provide gfortran 4.3 (e.g., Ubuntu 8.10 provides 4.3.2, information on some distributions is at http://gcc.gnu.org/wiki/GFortranDistros). For Mac OS X, MacPorts
(http://www.macports.org/) provides gcc43, the GNU Compiler Collection 4.3, which includes gcc and gfortran 4.3.3. Fink (http://www.finkproject.org/) package gcc43, GNU Compiler Collection Version 4.3, has gcc and gfortran 4.3.1 (stable branch, currently only in source code form). Recent gfortran binaries for both Linux and Mac OS X are provided at the GCC Wiki (http://gcc.gnu.org/wiki/GFortranBinaries).

Additional information on the binaries available from the GCC Wiki: The Mac OS file is a dmg file which installs gfortran very easily. gfortran will be installed at /usr/local/gfortran with a link "gfortran" at /usr/local/bin/.

For Linux, links to various pages and versions for various needs are available from http://gcc.gnu.org/wiki/GFortranBinaries/. That page also has links to installation instructions. In brief: ungzip and untar the file, for example in directory /opt. It will expand to sub-directory gcc-trunk. Add /opt/gcc-trunk/lib to LD_LIBRARY_PATH. Either run /opt/gcc-trunk/bin/gfortran
or add /opt/gcc-trunk/bin to your PATH.

Check your gfortran installation via:
which gfortran
gfortran -v

Notes on gfortran versions:
4.3.0 Documentation implies should work. Failed on one machine -- did not recognize ISO C Binding.
4.3.1 not tested
4.3.2 Worked on Linux Ubuntu
4.3.3 Worked on Mac OS X
4.4.0 Obtained from GCC Wiki. Worked on Mac OS X and SL Linux.

GSL NOTES

Installing the GNU Scientific Library (GSL) on your system.

Most versions of Linux provide the GSL through their package manager, either as an rpm or as a debian package (apt-get). For Mac OS X, the GSL is available from both Fink http://www.finkproject.org/) and MacPorts (http://www.macports.org/). These Mac versions are likely to be 32-bit.

For Darwin (Mac OS X), if you are running a 64-bit version of IDL (see above), you will need to have a GSL library compiled to be 64-bit (unless you invoke IDL with the "-32" flag). This is so that the GSL shared library will be compatible with the MFIT shareable executable, which needs to be compatible with the IDL executable.

GSL is also very easy to install from the source code (http://www.gnu.org/software/gsl/).

i) ungzip and untar the file.
ii) cd to the expanded directory, such as gsl-1.12
iii) make clean [optional on a fresh install]

For Linux or a 32-bit version of IDL for Mac OS X, or for any IDL invoked with
the "-32" flag:
iv) ./configure

For a 64-bit version of IDL for Mac OS X:
iv) ./configure CFLAGS="-m64" LDFLAGS="-m64"

v) make

If you get an error from the "make" step, try again from step iii), adding "--disable-shared"
after the "./configure" of step iv).

vi) sudo make install

This will install the GSL in directory /usr/local/, e.g., the libraries libgsl.a and libgsl.so (Linux) or libgsl.dylib (Mac OS X) in /usr/local/lib (and only libgsl.a if you have used "--disable-shared") and various include files in /usr/local/include/gsl. Optional additional directions are in the file INSTALL. That file also describes how to run some tests.

MAKE ERRORS

The following errors have been reported during the running of Make.mfit.sh:

M-1. If you see any messages similar to the followng:
/var/tmp//ccvuR0IU.s:12294:operands given don't match any known 386 instruction

Chances are, you do not have an appropriate linker from Apple. The solution is to download and install the latest version of XCode from the Apple Developer's area (free to join). For Mac OS X "Tiger" (10.4x) this is version 2.5. For more recent builds of the OS, Leopard or Snow Leopard, get the latest version of XCode (version 3.1.2 as of June 2009).

M-2. If you see multiple compiler errors related to 'libgsl', see the section on installing the GSL, or set the environment variable GSL_PATH.

M-3. If the MFIT Link step fails with many error messages of the form:

mfit_interface.o: In function `__sigismember':
mfit_interface.c:(.text+0x0): multiple definition of `__sigismember'
mfit_idl.o:mfit_idl.c:(.text+0x0): first defined here

The cause, as far as we understand, is a non-usable gcc obtained as part of the gfortran binary from the gfortran wiki. Use your older gcc instead. The gfortran binary is OK. You can use the environment variable CC (see instructions above) to point to a specific gcc by name or path.

M-4. The following error from the Make step:

ld warning: in /usr/local/lib//libgfortran.dylib, file is not of required architecture

was observed when gfortran 4.3 was installed without deleting gfortran 4.2. Apparently gfortran 4.3 was using the libraries of version 4.2, and those libraries were not compatible. Solution: uninstall or delete the earlier version of gfortran.

RUNTIME ERRORS

The following errors have been reported when MFIT is first invoked, when performing the first
spectral fit of an rmfit session:

R-1. If you get the error message listed below, it likely means that you have compiled MFIT
for 32-bits, while you are running a 64-bit IDL, or vice-a-versa. The instructions above discuss this issue. Either remake MFIT and GSL for the other bit level, or force IDL to run as 32-bit program by invoking IDL with the "-32" flag.

% MFIT: Error loading sharable executable.
Symbol: IDL_Load, File = ..../mfit.so
dlopen(/...../mfit.so, no suitable image found.
Did find: ...../mfit.so: mach-o, but wrong architecture % Execution halted at:

R-2. A very similar error message about the "wrong architecture" results when a Power PC version
of IDL is used on an Intel Mac -- on an Intel Mac, you will normally have a gcc & gfortran that will compile MFIT to an Intel executaable. This will result in IDL and MFIT executables that are incompatible. A solution is to download the current version of IDL and run that version without a license, using the virtual machine version. You can still use your older version of IDL for other purposes.

R-3. If the GSL library is not in your LD_LIBRARY_PATH, then you will get the following message when mfit is called:

% MFIT: Error loading sharable executable.
Symbol: IDL_Load, File = ...../mfit.so
libgsl.so.0: cannot open shared object file: No such file or directory
% Execution halted at: MFIT::FITMODEL 1655

Solution: Add an instruction in your login file to add the GSL location to LD_LIBRARY_PATH.
e.g.: export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
(or DYLD_LIBRARY_PATH for Mac OS X).

Owned by:  
Last updated by: Chuck Patterson 11/18/2009

Notes to self :

  • When working on a SLAC Public machine, you will need to know what type of machine your're running on (e.g., RHEL4-32, RHEL4-64, etc.) To do this, go to Nagios Hostgroup Overview and scroll down to glastlnx01, 02, etc. machines.)
  • To launch firefox: chuckp@noric14 $ /opt/TWWfsw/bin/firefox
  • To download IDL, go to: http://mynasadata.larc.nasa.gov/idlvm.html
  • http://www.ittvis.com/
  • E-mail dated 11/05 from "The System Ranger":

The directory

/afs/slac.stanford.edu/u/gl/chuckp/.mozilla

in your AFS home directory was detected to be readable by all SLAC users by a routine scan. This can expose your private information and perhaps cached passwords to others, who could then use them to log on as you.
If you want to learn more about this and some additional steps you can take to secure your AFS space see

http://www.slac.stanford.edu/comp/unix/homedir.html

The following entries were removed from the Access Control List for the
directory:

system:slac rl
system:authuser rl

---
This message is from the System Ranger, an automated system monitor.
For questions or problems, send email to unix-admin@slac.stanford.edu.
[homes 140]

 

 

Owned by:  

Last updated by: Chuck Patterson 11/18/2009