|
|
The art of Python programming...
Why use a programming language to process data?
Most students do their calculations in Excel or, if
you do not have a license, the alternative open source
spreadsheet programme Calc
from OpenOffice. However,
suppose that you have long time series of
meteorological data. In Excel the maximum row lenght
would then be reached very quickly so that you would
have to divide your data over different sheets. This
would be very inefficient. It is also complicated to
build a computer model in a spreadsheet program to
simulate processes, such as groundwater flow, erosion,
or release of greenhouse gases.
In this case, professionals often revert to writing of a dedicated
program that can do the processing of data and do model calculations
automatically. The most common languages for programming
are C programming language and its
derivatives and Java
and Fortran to name just a
few. These languages are very powerful but have the disadvantage that
you have to write the program and then compile it to work with it.
Another branch of programming languages are the interpreted
languages. Commands can then typed in and are directly evaluated by
the interpreter, without compilation. A number of commands can be
saved in a text file, which is then called a script. Interpreted
languages
are Matlab, Python
and GNU
Octave. As a Matlab license is very costly we prefer to use the
open source Python language to work with. Python is a very versatile
programming langauge that can be used to process data in a rather easy
way and this web page will give you some hints and examples on how to
use Python in your scientific carreer.
How to install Python
If you are working with MS Windows (or Ubuntu Linux) go to
the Python(xy) download page
at www.pythonxy.org and under
the Windows XP/Vista installer/administrator account, download
the full edition. Once downloaded, double-click on the
installer and choose next or I agree a couple of times
until Python(xy) is installed.
Please note that if you intend to print equations or special
characters in any graphs that you want to plot with Python, you will
need to
install LaTeX
first.
Python(x,y) also makes several useful tools and programs available,
including the Scientific PYthon Development EnviRonment (spyder),
which is designed for interactive computing and data visualisation and
has a simple and intuitive user interface.
An alternative is the Anaconda Python distribution from Continuum Analytics. This is also a free Python distribution (including for commercial use and redistribution) that contains about 300 popular Python packages.
A third alternative distribution for MS Windows, MacOS or RedHat Linux
platforms, is
the Enthought
Python Distribution, which contains all the programs and tools
that you will need. However, before you can download this distribution you will have to register and the trial version of this distribution is only of unlimited duration for educational purposes, so you cannot use this commercially.
If you use the Debian
or Ubuntu Linux platforms you can
just install Python from a shell command line using the
command: apt-get install python python-matplotlib.
Learning Python
A short Python tutorial
- Python programming for
Hydrology students manual - was written by M.J. Waterloo for the
Hydrology MSc course "Soil Vegetation Atmosphere Exchange". You can
also download the
accompanying example script
file, which creates an image with Python. For one of the
assignments in this manual you can download
the water level data text file (wl.txt) with
date, time and water level logger data of the Rio Boco River near
Aveiro in Portugal.
There are many online tutorials. See, for instance, the Python
Tutorials web site
at www.awaretek.com.
Python modules and scripts for Hydrologists and Earth Scientists
Several useful scripts are given below. You can in most cases easily adapt these to your needs.
Subject |
Python scripts and function libraries |
| Geophysics |
|
| GPS and GIS scripts |
- Python GIS: various Python modules for working with GIS - Python GPS, MapView, LatLongUTMconversion.
- Paul Inkenbrandt from the Utah Geological Survey has made several scripts that work with ArcGIS. The scripts include Stiff diagrams, Box and Whisker and Piper plots. His scripts scan be downloaded
from https://github.com/inkenbrandt/Arcpy.
|
| NetCDF file operations |
- txt2netcdf: various Python functions for importing text into NetCDF data files (creating files, adding variables, listing structure, etc.), developed by Ko van Huissteden.
|
| Meteorology |
- meteolib: Python library containing meteorological functions for calculation of atmospheric vapour pressures, air density, latent heat of vapourisation, heat capacity at constant pressure, psychrometric constant, day length, extraterrestrial radiation input, potential temperature and wind vector. The documentation for this module is presented at here (meteolib module functions web site). Functions to convert event-based data records to equidistant time-spaced records (event2time) and to convert date values to day-of-year values (date2doy) are now in a separate meteo_util module. Documentation is presented here (meteo_util module functions web site). Module documentation is also available as a PDF document. Author: M.J. Waterloo.
|
| Hydrology |
- evaplib: Python library containing functions for calculation of evaporation rates. Functions include Penman open water evaporation, Makkink reference evaporation, Priestley Taylor evaporation Penman Monteith (1965) evaporation and FAO's Penman Monteith ET0 reference evaporation for short, well-watered grass. In addition there is a function to calculate the sensible heat flux from temperature fluctuation measurements. View documentation of evaplib module functions. Module documentation is also available as a PDF document. Author: M.J. Waterloo.
- Groundwater flow modelling manual for Python written by Vincent post
|
| Hydrochemistry |
|
| Workshop meteorological data processing |
- Meteodata
processing assignment manual: Meteorological data processing
manual for the script and data listed below. It also contains a
section on how to report on these data
using LaTeX.
- meteoscript_yourname_2014.py:
This script file is used for processing of meteorological data
collected during the hydrological field courses in Twente and
Portugal. Author: M.J. Waterloo. The script reads the ASCII data,
calculates state variables and actual evaporation (temperature
variance energy balance method) and potential evaporation. It also
plots time series of several weather variables. The script contains
questions that should be answered by the student to better understand
what the script does. It makes use of
the meteolib
and evaplib modules presented
above. Create an ecohydrology/pyscript directory and download and save
the script and modules in this directory.
- slowtable.dat: File containing
meteorological data collected in Portugal during the hydrological
field course in Aveiro, 2014. This is the output file of the Campbell
Scientific CR1000 data logger. Create an ecohydrology/data directory
and download and save the data file in this directory./li>
|
| Statistical Resources for Hydrologists |
- R
Resources for Hydrologists: This weblog lists a lot of programs
for statistics in hydrology
using R, which is a computer
language for statistical computing and graphics. Statistical functions
of R can be accessed in Python, for instance
through Rpy
|
| Python hydrological modelling tools |
- Vincent Post and others have developed a Python package to create, run and process MODFLOW, SEAWAT, MT3D and other MODFLOW based groundwater models. The software and documentation can be found at the FloPy3 web site and can be downloade from Github.
- Catchment and plant growth modelling frameworks (CMF, PMF), and statistical parameter optimisation tools (SPOTPY) programs were developed for Python at the Justus-Liebig-University in Giessen, Germany. The software and manuals are available for download at the website of Landscape, Water and Biogeochemical Cycles, together with the papers on which the programs are based.
- Sat Kumar Tomer from the Department of Civil Engineering - Indian Institute of Science has written an excellent Python in Hydrology handbook and also made several hydrology and earth sciences scripts available from his Ambhas project that can be downloaded from https://pypi.python.org/pypi/ambhas.
- Jaap Schellekens from Deltares in Delft, The Netherlands, has his WFLOW distributed hydrological modelling platform - part of the Openstreams project - available at http://schj.home.xs4all.nl/html/.
|
|
