Home

Quick Start

Data

Python

Courseware

Lecture Graphics

Problems

Solutions

Bibliography


Python Links

Python Home

python.org

Basic install

Graphics

Learning Python

 

Overview

Python is an interpreted, open-source, object-oriented language. "Interpreted" means you just type a statement or series of statements, and they execute as soon as the statements make up a complete command, without the need to first compile the statements into machine language. In this regard, Python is like Matlab. "Open-source" means that nobody owns it or sells it, and that it is developed and maintained by a large community of dedicated users. Anybody can see how the thing works, and there are no secrets. Open-source means Python costs nothing, and so nobody needs to deal with the hassle and cost of licensing. Python has a very extensive user base, so it's not going away any time soon. All Unix-like operating systems heavily make use of Python, and Google largely runs on Python. One of the virtues of Python is that it is very easy to extend; for high performance, Fortran libraries can easily be turned into Python commands using f2py (distributed with numpy) and c or c++ libraries can easily be turned into Python commands using swig. "Object-oriented" means that Python supports the definition of structures which combine places to store data with functions that operate on the data, making for a very powerful way of building toolkits. Python provides a very gentle and intuitive way of getting used to this software development technique, though the language can also be used perfectly well in the old-fashioned familiar procedural way.

First you should do a basic Python installation if it isn't already on your system. That is described under Basic install. Then you should start becoming familiar with the Python language, using the resources in Learning Python. If you want to produce graphics within Python, without writing out data and plotting it up in some other program, you should install a graphics package, as described under Graphics. Then, you should install the Courseware and start becoming familiar with its use (beginning with the Computational Toolkit exercises in the Chapter 1 Workbook).

The most basic thing you need to know from the rest of this page is about where to put add-on modules. If you are using Mac OS 10.6 or higher and run into difficulties with the 32bit/64bit distinction, you may want to read up on the technical explanation of this for trouble-shooting purposes. This shouldn't be necessary if you follow my instructions for installing your own Python and use only the 32-bit add-ons I point you to. At present, all the basic software you need is available in 64 bit versions, so the only reason to stick with 32 bit is that the Climt_Lite module which provides the interface to the ccm radiation model is (for now) only compiled for 32 bit. If you are an instructor and wish to set up all the courseware on a server for students to log into, look here.

[**Also link to some ready-builds for Mac including everything (CliMT too, also rest of basic Courseware modules, put in site-packages?); include the PPC version as well. ]

As an alternative to the "roll your own" Python installation I go through here, I highly recommend the Enthought Python distribution. This single-click comprehensive installation comes with all scientific support add-ons you need, including the numpy array package and graphics. It is available in both 32 bit and 64 bit versions for Mac OSX, Windows and Linux, and if you are affiliated with a degree-granting academic institution you can download it for FREE. This is by far the easiest way to get up and running with Python and the add--ons you need. (Even if you need do buy it, it is well worth the money). It does graphics using MatPlotLib rather than the PyNgl package my courseware uses by default, but MatPlotLib is easy to use so you'll have no problem doing the graphs you need by using MatPlotLib commands, if you don't want to install Ngl. For Windows users, MatPlotLib has some real advantages, since you don't need to go through the hassle of installing CygWin to get x11 graphics. My next release of ClimateGraphics.py will include an option to use MatPlotLib. If you use the Enthought distribution, you'll still need to install the Courseware Modules(including climt_lite) yourself.

back to top

Where to put add-on modules

One of the great virtues of Python is that it is very easy to extend the basic capability of the language with new commands and functionality. The expansion packages take the form of modules.

A module is really no different than any other Python script. It is simply a Python script that is intended to define a bunch of functions, constants and other objects that are generally useful in a wide range of applications. A module called MyModule.py would be imported into Python using the command

import MyModule

There are other ways to import modules, described under Learning Python.

If you want Python to be able to find your extension modules no matter what directory you are working from, you need to either put the modules in the some place where Python looks by default, or to tell Python where you want it to look. A script which imports a module will always find that module if the module is located in the same directory as the script.

The other place Python looks for modules by default, is the directory site-packages. which is deep down in the directory tree of the directory where Python itself has been installed. (See Basic install for information on where to find the site-packages directory.) Putting things here normally requires system-administrator permission. This is where you would put things that are intended to be available system-wide to all Python users, for example the numpy array package and the graphics package you are using. A user can examine scripts in site-packages, but this is not the place to put scripts that that the user or instructor will want to modify frequently. There is nothing really wrong with putting the courseware modules in the site-packages directory. Putting courseware modules in site-packages also works in Microsoft Windows.

I prefer to keep special-purpose modules in a separate directory in my home directory. You can have as many such directories as you want, and they can be put anywhere where you have read permission. However, you have to somehow tell Python to look in these directories as well as the default places. On Unix-like systems, including Mac OSX, this is handled by setting the PYTHONPATH environment variable. Doing this takes you into fairly geeky territory, described under About Shells, Paths and Environment Variables. Microsoft Windows also has environment variables, and you can set PYTHONPATH using either a dialog box or the set command. Instructions on how to do this are found here (see Section 3.3.1).

Once you set up your PYTHONPATH to find a modules directory (say, MyModules), any script placed in that directory (say, MyScript.py ) can be imported by name, e.g

import MyScript

However, if you put a subdirectory in your modules directory, Python will not be able to find those modules. For example, if MyModules contains a subdirectory called Package, which contains scripts Package/MyScript1.py and Package/MyScript2.py , then import MyScript1 ,etc. won't work. However if the subdirectory contains a special script __init__.py (note double underscores) which tells Python what to import, then an import of the subdirectory name ( import Package , in the example given) will work. Developers who wish to group many scripts into a single directory you can install typically will do this, but you will probably never need to do this yourself.

back to top

About 32 bit vs. 64 bit on Mac OSX

Before Mac OS 10.6, all applications were 32 bit. With OS 10.6, Apple began moving to a 64 bit world. OS 10.6 does not require all applications to be 64 bit, and 32 bit apps still run fine, but it permits 64 bit apps. This only impacts your world because Python itself is an app, and that means that if you are using a 32-bit version of Python, any add-on modules that use compiled code as opposed to straight Python (e.g. numpy, graphics modules and CliMT_lite) need to be compiled for 32-bit. Likewise, if you are using a 64-bit version of Python you need to use 64-bit compatible add-ons. This distinction does not apply to courseware utilities like ClimateUtilities.py or the Chapter Scripts, since they are in straight Python and do not have compiled components. For now, I recommend sticking with a 32-bit Python world, since precompiled 64-bit versions of the add-ons you need are not yet uniformly available. Be warned that the version of Python that comes shipped with Mac OS 10.6 and higher is a 64-bit Python, so using 32 bit on 10.6 and up means you need to install your own version of Python, which is recommended in any case.

The 32-bit vs. 64 bit distinction also exists on Linux, and on Windows.

On Linux and Mac OS, you can use the Linux command file to determine whether an application or compiled module is 32 bit or 64 bit. For example, the python shipped with my OS 10.6 laptop is /usr/bin/python. If I go into a terminal window and type

file /usr/bin/python

I get the response:

/usr/bin/python: Mach-O universal binary with 3 architectures
/usr/bin/python (for architecture x86_64): Mach-O 64-bit executable x86_64
/usr/bin/python (for architecture i386): Mach-O executable i386
/usr/bin/python (for architecture ppc7400): Mach-O executable ppc

This means that this version is a "fat binary" which will work with either a 64 bit intel operating system (x86_64), a 32 bit intel operating system (i386) or the old Power PC chip (ppc7400). When you run this python, the most "advanced" version for your operating system will run. Under OS 10.6 that will be the 64-bit version. Under OS 10.5 that will be the 32 bit version. You don't get a choice.

Now, on my system I installed the 32 bit Mac Python build. This installs with a link to /usr/local/bin/python . If I type

file /usr/local/bin/python

then I get the response

/usr/local/bin/python: Mach-O universal binary with 2 architectures
/usr/local/bin/python (for architecture ppc): Mach-O executable ppc
/usr/local/bin/python (for architecture i386): Mach-O executable i386

which means that this version is compiled without the 64 bit option. Therefore, it will run as 32 bit (i386) on an Intel chip Mac, even if you are on OS 10.6 (or presumably higher). On an old PPC Mac, it will run for that architecture.

Compiled libraries, which are part of certain Python add-on modules, also come in different architectures. Compiled libraries have the extension ".so" and are usually buried somewhere down in the directory tree for that module. For example, my installation of the module climt_lite , which I put in my modules directory, has a library climt_lite/_grid.so in it. So, if I go to my modules directory, and type

file climt_lite/_grid.so

the response I get is:

climt_lite/_grid.so: Mach-O bundle i386

which means this has been compiled for i386 (i.e. the 32 bit architecture). If I start up /usr/local/bin/python (which is the default on my system, as I can tell by typing which python) and then type import climt_lite into the interpreter, everything will work fine. On the other hand, if I start up the python shipped with Mac OS 10.6, which will run as 64 bit, and do the same import, I get the error message:

ImportError: dlopen(climt_lite/_grid.so, 2): no suitable image found. Did find:
climt_lite/_grid.so: mach-o, but wrong architecture

which is a bit cryptic, but all the important thing is "wrong architecture" which means that the library was compiled for an incompatible system. You'd get the same message if you tried to run an intel-only library on an old Power PC Mac.

back to top

Setting up a shared server for a class

If you are teaching a class using the courseware, you might find it convenient to install and run the courseware on a server that students can log into. This saves the trouble of guiding each student through an installation on their own computer, which can become particularly time-consuming given the range of different machines and operating systems you will encounter. Tech-savvy students (especially ones with Mac or Linux) still have the option of installing their own version if they find that more convenient.

It is possible to use a Mac OSX box as a server but generally speaking it is best to use a Linux box as a server. Any cheap computer of the sort that is usually sold for running Microsoft Windows can run Linux quite easily and cheaply. When running on a server, regardless of whether it's Mac OS X or Linux, students need to have an account on the server. When using a Mac as a server, you also need to go into the "sharing" preferences panel on the Mac you are using as a server and allow remote logins.

Back in the day, before the Web, the only way to access remote resources was to "log in" to another computer using some form of terminal program, and type in commands. This is still how most scientific computing is done. Many students these days are unfamiliar with the process, having never had to access remote resources except through a web browser. If they are going to do science, though, the sooner they learn about logging in and using basic Linux/Unix commands, the better.

Logging in to a remote machine is done using the ssh command, which you type into a terminal window on either Mac or Linux. On a Mac, you must be using an x11 terminal window if you want the server to be able to put up graphics on your screen. That includes use of idle, which must be able to draw graphics to handle its user interface. All Macs come shipped with x11 as a clickable application, ready to use. On Linux, every window is an x11 window, since x11 is the native windowing system for Linux.

Let's suppose the server is called snorkmaiden.umoomin.edu, and that your user id on that machine is hemulen. If you just want a plain text connection, from a Mac or Linux terminal window, you type

ssh snorkmaiden@umoomin.edu

You will then get a prompt for the password. After you enter your password, you will then get the command prompt for the server. Then you start entering commands, just as if you were running on your own machine. If you want a graphics connection, instead (from an x11 terminal window) you type

ssh -Y snorkmaiden@umoomin.edu

The "-Y" flag tells the server that it should put up any graphics in a window on your own machine. On some older systems, you need to use "-X" instead.

Once you have established a connection to the server via ssh, any commands you issue run on the remote machine, and to the user, function just as if the software were running locally on their own computer. The only little wrinkle you need to be aware of is that any files software running on the server accesses must be stored on the server. Similarly, any files you save from software running on the server will be saved in your directory on the server.

Mostly this is no problem, since the whole point of running on the server is that everything you need gets kept there, so you can get at it from any network-connected machine in the world. The one exception where you probably will want to move files to your own machine is when you want to save graphics or Python code for inclusion in a lab report or problem set. There are various ways to download files from your directory on the server. One way is to use the command scp or sftp, or maybe ftp. Or, the instructor can set up a web server on the courseware server that automatically makes files in some special subdirectory of your home directory (typically www or public_html) accessible via a web browser. This is very convenient, since then you can put graphics for your problem set directly on the web, and not need to print them out. But if you just want to get some graphics in your problem set, the easiest thing is to just produce the graphics on your screen, and take a screen shot. Preview on a Mac does this very easily, and similar utilities are available on Linux and MS Windows. For putting Python code or text output into a file on your own machine, you can just copy from an idle window or terminal window, and paste into a word processor document on your own machine.

ssh is available for MS Windows, but on Windows you don't generally start it by typing a command. Instead you double-click the terminal application, and enter the server and userid information in a dialog box. [**Link to MS Windows ssh apps]. This gives you a plain text terminal connextion. If you want to see graphics, you will need a version of x11. Microsoft doesn't make this especially easy, which is one of the reasons Windows is not a suitable platform for serious scientific work. However, there is a fairly convenient (and free) third party x11 package called Cygwin [**INSERT CYGWIN LINK] , which you can download and install.

[**NB set up a separate "about x11 and windowing systems" page, and link this on the Graphics page as well as here]

If you are running on the courseware on a Linux box, MatPlotLib will work fine, though to use MatPlotLib on Linux you need to build it from source since precompiled versions for Linux do not seem to be available.

When using a shared class server, it is a good idea to put a copy of the ChapterScripts in a world-readable directory in the home directory on the server set aside for class resources (like data sets and courseware modules). This way, the students can just use idle on the server, open the ChapterScript they want using the idle open file dialogue box, and then save a copy on their home directory so they can modify it. Alternately, students can directly download chapter scripts from the Courseware portal, open in a text editor, and then copy and paste into an idle editor window.

When running the courseware on a server, you only need one copy of the shared courseware modules (ClimateUtilities.py, etc.), since these are not meant to be modified by the student. The courseware modules could be put in the site-packages directory for the Python installation, but since they are specific to just your course it is cleaner to put them in a directory of their own, located in the instructor's home directory or a course home directory. For example, if you put the courseware modules in the directory

/home/PlanetaryClimate/PlanetaryClimateModules

then to make the modules available to a user, it is necessary to include this in the Python search path by setting the $PYTHONPATH environment variable. If the accounts are set up with the csh or tcsh shell, for example, this is done by each user executing the command

setenv PYTHONPATH /home/PlanetaryClimate/PlanetaryClimateModules

upon login. Actually, to spare the user the trouble of executing this each time, the sysadmin should put this line in the .login file in each user's home directory, creating the file if it doesn't already exist. The procedure is similar when using the sh or bash shells, except you use the EXPORT command instead of setenv, and the name of the initialization file which executes upon login is different. (I like to keep things simple by sticking to csh or its variants)

Using idle on a shared machine: idle uses a trick to allow it to keep the Python interpreter which is running your script in a separate subprocess from the Python interpreter that is running idle itself. This trick, however, makes it impossible to run more than one instance of idle at a time on any given machine. If you are on a shared machine and try to start up idle on the command line while somebody else is running, you'll get an error message. When running idle on a shared machine, you should start up idle using the command idle -n instead, which runs without a subprocess. This has certain limitations that make it harder to work if you are developing complex software yourself, but for running the courseware the differences will be hardly noticable.

Using a Mac as a server:

Mac OS10 is basically a flavor of Linux, so it is possible to use an OS 10 box as a course server. Since PyNgl uses x11 for graphics, it can put up graphics on the user's machine across the network. You log into OS10 just like you would log into any Linux machine. The only real problem with using a Mac as a server is that the version of idle that comes with the pre-built Mac Python distribution (including the Enthought distribution) uses the native Mac windowing system, and therefore can't put it's windows up over a network. There used to be a Python available through the fink[**FINKLINK] project which used an x11 version of idle, but that seems to be no longer available, and in any event fink seems to have become far less up-to date than it used to be. You could try building your own x11 idle from source code if you are an uber-geek. A simpler option if you are in a position where you have to use a Mac as the server is to just give up on using idle and use some other convenient text editor, preferably a user friendly one that puts up a menu-driven editor window over x11. The x11 version of emacs is a very good choice for this. You won't be able to execute your script directly from the editor, like you can with idle, but it isn't so hard to keep a seperate terminal window open and just run your script from the python command line when you want to test it out. Another good option for editing is to turn on file sharing on the Mac, which will allow students to mount their server directory on their desktop as if it were a local disk. Then they can edit using whatever text editor they like (I like BBEdit myself).

Note that MatPlotLib graphics (which comes shipped with the Enthought Python distribution) uses the native Mac windowing system, so it can't put up graphics across the network if you are using a Mac for the server. For this reason, sticking with PyNgl for graphics is recommended if you want to set up the courseware on a server on a Mac.

By default, the Mac OS does not give permission for applications to "forward" x11 commands over the network. To enable this, you have to enable "x11 port forwarding." This isn't done through the System control panel. Instead you have to edit a line in a certain system configuration file. [**Finish instructions. What to edit in inetd configuration file. Check that this is still necessary on 10.6.]

back to top