Auroral GLOW model by Stan Solomon in Python

plotglow

Thanks to Steve Kaeppler who pointed out this model to me, I have trivially wrapped the GLOW auroral VER model in Python, making it very easy to use the GLOW model from Python. I am still working on this so please leave a comment or email if it’s missing something you need (which is likely).

I would like to complement Solomon’s team who generally wrote the Fortran 77 code without excessive legacy techniques and syntax errors, which made the porting much faster. I only had to make minimal modifications to bring the code up to Fortran 77 standards.

f2py: Running Fortran code in Python on Windows

F2PY is a great package allowing you to run a lot of Fortran code from Python. Fortran 77 code naturally has a high chance of success.
Fortran 90 code can also work, but very elaborate Fortran 90 code may not work.

On Linux, all you need to do is

sudo apt-get install python-numpy gfortran

On Windows, the f2py setup is more involved. You need the MinGW compiler and to modify your system path. I assume here that you have Anaconda Python with Numpy 1.9 installed already. This procedure was used on a 64-bit Windows 10 machine.

1) install MinGW-W64 with the setup settings:
Version 4.9.2 (or newer)
Architecture: x86_64
Threads: posix
Exception: seh
Build revision: 2

2) Insert the mingw bin path to your system path. I did this permanently by going into Windows Advanced System Settings, Environment Variables, System variables, Path, and at the end put:

C:\Program Files\mingw-w64\x86_64-4.9.2-posix-seh-rt_v4-rev2\mingw64\bin

check to see if that’s where your gfortran.exe is at.

test your installation by say using the lowtran7 code.
Following the instructions there, you should get a lowtran7.pyd file that is invoked when running
python pylowtran7.py and you’ll see a plot of atmospheric loss.

Note: If you get the “NotImplementedError: Only MS compiler supported with gfortran on win64″, that is still occurring with Numpy 1.9.2, but has been patched in the master branch (to be released in a future version of Numpy). You can patch this yourself in c:\Anaconda\Lib\site-packages\numpy\distutils\fcompiler\gnu.py on line 337. Comment out the
raise NotImplementedError(...)
and simply replace with the word
pass
and you’ll be working.

Lowtran7 example output in the Arctic looking near zenith.
Lowtran7 example output in the Arctic looking near zenith.

Compiling OpenCV3 with extra contributed modules

This procedure is for OpenCV3 beta.
Some of the functions you might want like cv2.createBackgroundSubtractorGMG are not in the standard OpenCV 3 package at this time. The newer functions by design go into the opencv_contrib repository.

If you’re comfortable with the previous procedure, you might be ready to try adding these packages.

CRITICAL POINT: you must have the space between -D OPENCV_EXTRA_MODULES_PATH= for this to work, or it will silently ignore the opencv_contrib modules.

I will assume you have the prereqs installed already.

After the cmake command, scroll back up and check that you see something like (for example, look for bgsegm, that’s one of the opencv_contrib modules)

OpenCV modules:
 -- To be built: core imgproc imgcodecs videoio highgui xobjdetect adas video bgsegm bioinspired flann ml features2d calib3d ccalib face text datasets line_descriptor objdetect optflow photo reg rgbd saliency shape xfeatures2d stitching superres surface_matching videostab ximgproc xphoto python2 tracking ts
 -- Disabled: java world
 -- Disabled by dependency: -
 -- Unavailable: androidcamera cuda cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaoptflow cudastereo cudawarping cudev python3 viz cvv matlab

cd /tmp

git clone --branch 3.0.0-beta --depth 1 https://github.com/Itseez/opencv.git

git clone --branch 3.0.0-beta --depth 1 https://github.com/itseez/opencv_contrib

cd /tmp/opencv

mkdir release
cd release

cmake -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules/ -DBUILD_TIFF=ON -DBUILD_opencv_java=OFF -DWITH_CUDA=OFF -DENABLE_AVX=ON -DWITH_OPENGL=ON -DWITH_OPENCL=ON -DWITH_IPP=ON -DWITH_TBB=ON -DWITH_EIGEN=ON -DWITH_V4L=ON -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=$(python3 -c "import sys; print(sys.prefix)") -DPYTHON_EXECUTABLE=$(which python3) -DPYTHON_INCLUDE_DIR=$(python3 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_PACKAGES_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") ..
# the -j6 is for compilation only, to use up to 6 threads. It has no effect on the compiled opencv code execution
make -j6
make install

Once installed, you should be able to from within Python type:


import cv2
x = cv2.bgsegm.createBackgroundSubtractorGMG()

Python OpenCV2 vs. OpenCV3 API compatibility

The OpenCV2 API in Python is a vast improvement over the non-Numpy cumbersomeness of OpenCV1. Moving to OpenCV3 in Python, some slight changes were made to the API, that so far I have found some easy workarounds for.

Not so easy was the removal of “legacy” functions from OpenCV3, such as cv.CalcOpticalFlowHS. I don’t plan to find a workaround for those at this time.

Here are some example workarounds that allow the same Python code to work in OpenCV2 in Python 2.7 and OpenCV3 in Python 3.4.

FourCC and SimpleBlobDetector:

try:
    from cv2 import cv
    from cv import FOURCC as fourcc
    from cv2 import SimpleBlobDetector as SimpleBlobDetector
except ImportError:
    from cv2 import VideoWriter_fourcc as fourcc
    from cv2 import SimpleBlobDetector_create as SimpleBlobDetector

drawKeypoints

In OpenCV3, you must pass the outImage argument, and OpenCV2 is happy with this as well.

calcOpticalFlowFarneback

The third argument must be specified as flow= to be OpenCV2/3 compatible.

Anaconda Python: OpenCV3

I wanted to have access to OpenCV3 in Python3, so I compiled OpenCV3 for Anaconda Python3 as follows, using this reference.

I have not tried it, but I think that this would work for Python 2.7 as well by changing the python3 references to python2.

BE SURE when you type in Terminal
python3 that it’s the anaconda python you get. If not, make an alias in ~/.bash_aliases
alias python3="$HOME/anaconda3/bin/python3"


sudo apt-get install git libjpeg-dev libpng-dev libtiff5-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev pkg-config cmake libgtk2.0-dev libeigen3-dev libtheora-dev libvorbis-dev libxvidcore-dev libx264-dev sphinx-common libtbb-dev yasm libfaac-dev libopencore-amrnb-dev libopencore-amrwb-dev libopenexr-dev libgstreamer-plugins-base1.0-dev

cd /tmp
git clone --branch 3.0.0-beta --depth 1 https://github.com/Itseez/opencv.git
cd /tmp/opencv

mkdir release
cd release

cmake -DBUILD_TIFF=ON -DBUILD_opencv_java=OFF -DWITH_CUDA=OFF -DENABLE_AVX=ON -DWITH_OPENGL=ON -DWITH_OPENCL=ON -DWITH_IPP=ON -DWITH_TBB=ON -DWITH_EIGEN=ON -DWITH_V4L=ON -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=$(python3 -c "import sys; print(sys.prefix)") -DPYTHON_EXECUTABLE=$(which python3) -DPYTHON_INCLUDE_DIR=$(python3 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_PACKAGES_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") ..

# make -j4 applies up to 4 threads to compilation only, makes no difference to running opencv code

make -j4
make install

NOTE: I assume you’re using an Ubuntu 14.04 system, within Anaconda Python 3.4 installed to ~/anaconda3/

Note: If you get a lot of errors about 70% of the way through like
error: ‘CODEC_ID_H264’ was not declared in this scope
then try (I had to do this with Ubuntu 15.04:

sudo apt-get install libavcodec-ffmpeg-dev libavutil-ffmpeg-dev libavfilter-ffmpeg-dev libavformat-ffmpeg-dev libavresample-ffmpeg-dev

Note: I have disabled Cuda via “-DWITH_CUDA=OFF”, assuming you don’t have a GPU to use. Because Cuda takes so much longer to compile, even if you have the GPU, maybe first try without CUDA, to see if OpenCV3 is going to work for you, then recompile with CUDA.

Note: To avoid the libopencv_imgcodecs.so.3.0.0: undefined reference to `TIFFOpen@LIBTIFF_4.0′ type errors, I added the -DBUILD_TIFF=ON option

Note: If you get the error like
lib/libm.so.6: version `GLIBC_2.15′ not found (required by /usr/lib/x86_64-linux-gnu/libx264.so.142)

then try


cd ~/anaconda3/lib
mv libm.so.6 libm.so.6.bak

core dump

If you get SIGILL illegal instruction (Core dumped) when typing import cv2 let’s try to get more information by typing in Terminal

sudo sysctl -w kernel.core_pattern=core
ulimit -c unlimited

and rerun the command that gave the core dumped error. Now you’ll have a file “core” in that directory. Type in Terminal
gdb python core
and you’ll get some more hints about the failure. Don’t feel so frustrated, I have enabled a lot of options in that Cmake command above, so let’s try turning some of the optimizations off. Here I was testing on an Ubuntu 15.04 system with a Haswell CPU.

Here’s a minimal cmake to try if you get frustrated by errors:

cmake -DBUILD_TIFF=ON -DBUILD_opencv_java=OFF -DWITH_CUDA=OFF -DENABLE_AVX=OFF -DWITH_OPENGL=OFF -DWITH_OPENCL=OFF -DWITH_IPP=OFF -DWITH_TBB=OFF -DWITH_EIGEN=OFF -DWITH_V4L=OFF -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=$(python3 -c "import sys; print(sys.prefix)") -DPYTHON_EXECUTABLE=$(which python3) -DPYTHON_INCLUDE_DIR=$(python3 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_PACKAGES_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") ..

Importing old OpenCV modules from OpenCV 2 in Python on Windows

Some of the legacy modules like Horn Schunck Optical Flow in OpenCV 2.4 are located in “cv” instead of “cv2″ .

To be cross-platform compatible, instead of doing


import cv,cv2

I suggest doing

import cv2
from cv2 import cv

—————————————————-

Further, in Windows running Anaconda Python 2.7 (or whatever Python 2.7 you choose), OpenCV is not installable through conda.

To install OpenCV 2.4 for Python on Windows, do the following:

  1. From

    http://opencv.org/downloads.html

    download OpenCV for Windows, version 2.4

  2. Extract to a directory of your choice, let’s say c:\opencv and copy the file
    c:\opencv\build\python\2.7\x64\cv2.pyd
    to
    c:\Anaconda\Lib\site-packages\

If you’re using CPython instead of Anaconda, copy to that directory instead.