Handling Fortran binary files with Python

Today’s top tip, straight from the front lines of, err, cosmology, concerns the handling of Fortran binary files with Python. If you have to deal with legacy Fortran code, you might find that it has a penchant for outputting files in an “unformatted” binary format. Exactly how the data is stored in this format can apparently depend on a lot of factors, such as the endian-ness of your machine. Hurrah! The files are non-portable. The necessity to figure what exactly is being dumped into the file can also cause headaches if you’re trying to read Fortran output into a program written in another language (for example, C/C++).

And it gets worse – what if you’re trying to write a file to be read in by the Fortran code, using a different language? Well, this particular problem cropped up for me, and of course I wanted to use Python to solve it.

For Python, fortunately, Neil Martinsen-Burrell has written FortranFile, which is subclassed off the standard Python file object, and which provides some convenient functions for reading and writing Fortran-formatted files. (There’s also an entry in the SciPy Cookbook that is relevant for Fortran I/O.) The pertinent function for me was writeReals(), which writes a NumPy array to a file in the appropriate format. The only subtlety I came across while briefly fiddling with this was that, when constructing a new FortranFile object for writing, you have to pass the mode keyword (e.g. mode="w"), which is inherited from file. (There’s a thread with a couple of tips on using FortranFile here.)

There are probably better ways of doing this than downloading someone else’s code off the internet, but like I said, I didn’t want to spend all afternoon on it.

About Phil Bull

I'm a theoretical cosmologist, currently working as a NASA NPP fellow at JPL/Caltech in Pasadena, CA. My research focuses on the effects of inhomogeneities on the evolution of the Universe and how we measure it. I'm also keen on stochastic processes, scientific computing, the philosophy of science, and open source stuff. View all posts by Phil Bull

4 responses to “Handling Fortran binary files with Python

  • Phillip Helbig

    “If you have to deal with legacy Fortran code, you might find that it has a penchant for outputting files in an “unformatted” binary format. Exactly how the data is stored in this format can apparently depend on a lot of factors, such as the endian-ness of your machine. Hurrah! The files are non-portable. “

    Legacy == stuff that works

    The whole point of the unformatted format is that it depends on a number of factors. The reason to use it is speed. If you have formatted files, then there is conversion when writing them and conversion when reading them. Obviously, this is necessary if they should be read or written by a human, or another program on another platform, or are intended to be portable across architectures. For such cases, use formatted but realize that there is a speed penalty. On the other hand, for files written and read by the same program, unformatted is the way to go: it’s faster, and you don’t have to worry about how many digits to write out etc.

    • Matt

      Note that there is a big difference between unformatted sequential files and unformatted direct access files. Mainly that sequential files add record information (which is most of the unportable portion). The problem with direct access files is that you need to specify the record length yourself and each record must have the same length …

  • Phil Bull

    In many cases, legacy is “a black box that appears to work”!

    Anyway, to address your main point: Of course unformatted files have their uses, but the problem is that because they’re pretty much the default output format for Fortran, they tend to get used in a lot of places where they shouldn’t have been – for example, as input or output files which are likely to be generated/read by other programs. Certainly, in the program I was looking at yesterday, the advantages of using unformatted files (like speed) weren’t relevant, so it was a poor choice by the software author.

  • Joe Zuntz

    I did this a few years back – http://www-astro.physics.ox.ac.uk/~jaz/page4/page4.html . At least you’ve met me in person so you can complain if it doesn’t work!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: