Installation Guide

This guide provides comprehensive installation instructions for DAO (Durham Adaptive Optics) on Linux, macOS, and Windows platforms.

Get the source code

The DAO source code is hosted on GitHub. The recommended way to obtain the code is to clone the repository with git. Below are a few common options — pick the one that suits your environment.

Clone (SSH)

git clone git@github.com:Durham-Adaptive-Optics/daoBase.git ~/daoBase
cd ~/daoBase

Note

For Windows build instructions, see the dedicated Windows Installation section below.

Prerequisites

Linux (RedHat-based distributions)

For RedHat-based distributions (CentOS, RHEL, Fedora):

yum install openssl-devel protobuf-devel gsl-devel numactl-devel glibc-devel ncurses-devel redis gtest-devel zeromq zeromq-devel

Linux (Ubuntu/Debian)

For Ubuntu and Debian-based distributions:

sudo apt install -y libtool pkg-config build-essential autoconf automake python3 python-is-python3 libssl-dev libncurses5-dev libncursesw5-dev redis libgtest-dev libgsl-dev libzmq3-dev protobuf-compiler numactl libnuma-dev

macOS

You need Homebrew for package management on macOS:

brew install pkg-config zeromq protobuf gsl

If you use Anaconda, there could be a conflict between protobuf versions, so uninstall the conda version:

conda uninstall libprotobuf

Linux ARM64

ARM64 systems may encounter protobuf version mismatches. The recommended solution is to rebuild from source.

First, install Abseil:

git clone https://github.com/abseil/abseil-cpp.git
cd abseil-cpp
mkdir build && cd build
cmake .. -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
sudo make install

Then install Protocol Buffers:

wget https://github.com/protocolbuffers/protobuf/releases/download/v25.3/protobuf-25.3.tar.gz
tar zxvf protobuf-25.3.tar.gz
cd protobuf-25.3
mkdir -p cmake/build
cd cmake/build

cmake ../.. \
  -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
  -Dprotobuf_BUILD_TESTS=OFF \
  -Dprotobuf_ABSL_PROVIDER=package

make -j$(nproc)
sudo make install
sudo ldconfig

Build System (Waf)

DAO uses the Waf build system. Install it as follows:

mkdir ~/bin
cd ~/bin
wget https://waf.io/waf-2.0.26
ln -s waf-2.0.26 waf
chmod u+x *

Add $HOME/bin to your PATH environment variable.

Python Environment

A Python installation is required. We recommend Miniconda, but any Python distribution will work.

Linux x86_64

mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm -rf ~/miniconda3/miniconda.sh
~/miniconda3/bin/conda init bash
~/miniconda3/bin/conda init zsh

Linux ARM64

mkdir -p ~/miniconda3
curl -L https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh -o ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm -rf ~/miniconda3/miniconda.sh
~/miniconda3/bin/conda init bash
~/miniconda3/bin/conda init zsh

macOS ARM64

mkdir -p ~/miniconda3
curl -L https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh -o ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm -rf ~/miniconda3/miniconda.sh
~/miniconda3/bin/conda init bash
~/miniconda3/bin/conda init zsh

Python Packages

Install the required Python packages:

pip install posix_ipc zmq protobuf==3.20.0 astropy python-statemachine statemachine redis numpy sphinx screeninfo

Note

daoServer (the distributed SHM registry daemon) requires redis, zmq, and numpy — all of which are included in the command above. A running Redis server is also required on at least one machine in the cluster; see Distributed SHM Server (daoServer) for installation instructions and usage.

For macOS, check your protoc version and install the corresponding protobuf:

protoc --version
# If version is 29.3, install:
pip install protobuf==5.29.3

Environment Variables

Linux/macOS

Add these environment variables to your shell configuration file (.bashrc, .zshrc, etc.):

export DAOROOT=$HOME/DAOROOT
export DAODATA=$HOME/DAODATA
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$DAOROOT/lib:$DAOROOT/lib64
export PATH=$PATH:$HOME/bin:$HOME/DAOROOT/bin
export PYTHONPATH=$PYTHONPATH:$DAOROOT/python
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$DAOROOT/lib/pkgconfig

For macOS, add these additional variables:

export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/opt/homebrew/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/homebrew/lib
export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/opt/homebrew/include
export C_INCLUDE_PATH=$C_INCLUDE_PATH:/opt/homebrew/include
export CPATH=$CPATH:/opt/homebrew/include
export PKG_CONFIG_PATH=/opt/homebrew/lib/pkgconfig

Important

Ensure that the DAOROOT directory exists:

mkdir $HOME/DAOROOT

Building DAO

Once all prerequisites are installed and environment variables are set:

waf configure --prefix=$DAOROOT
waf
waf install

Build Options

Debug Build

To build with debugging support:

waf build --debug

Address Sanitizer

For development and debugging (developers only):

waf build --sanitizer

Unit Tests

To build and run unit tests:

waf --test

Documentation

To build the documentation (requires Doxygen and Sphinx):

waf build_docs

To clean the documentation build:

waf clean_docs

Windows Installation

These instructions explain how to build and install daoBase on Windows using the Waf script.

Prerequisites

You must ensure the following software is installed:

Microsoft Visual Studio: Download from https://visualstudio.microsoft.com/downloads/
Protocol Buffers v3.20.0: Download “protobuf-cpp-3.20.0” from https://github.com/protocolbuffers/protobuf/releases/tag/v3.20.0
ZeroMQ: Download source from https://github.com/zeromq/libzmq
pkg-config: Use pkg-config-lite 0.28-1 from https://sourceforge.net/projects/pkgconfiglite/files/

Building Protocol Buffers

Extract the Protocol Buffers source code
Open Visual Studio “x64 Native Tools” command prompt in the extracted directory
Run the following commands:

cd cmake
mkdir build
cd build
cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=C:\path\to\protobuf\install ..
nmake

pkg-config Setup

Create pkg-config files manually for the dependencies.

For Protocol Buffers, create protobuf.pc:

prefix=C:/path/to/protobuf
exec_prefix=${prefix}/cmake/build
libdir=${prefix}/cmake/build
includedir=${prefix}/src

Name: protobuf
Description: Protocol Buffers
Version: 3.20.0
Cflags: -I${includedir}
Libs: -L${libdir} -llibprotobuf

For ZeroMQ, create zmq.pc:

prefix=C:/path/to/zeromq/cmake-build
exec_prefix=${prefix}
libdir=${prefix}/lib
includedir=${prefix}/include

Name: libzmq
Description: 0MQ c++ library
Version: 4.3.5
Libs: -L${libdir} -lzmq
Libs.private: -lstdc++
Cflags: -I${includedir}

Save both files in a dedicated folder for later use.

Python Setup (Windows)

Download and install Miniconda from https://docs.anaconda.com/miniconda/
Open “Anaconda Prompt (miniconda3)” from the Start Menu
Install required packages:

pip install zmq protobuf==3.20.0 astropy python-statemachine statemachine redis sphinx screeninfo

Waf Installation

Download Waf from https://waf.io/waf-2.0.26 and place it in a suitable location.

Environment Variables (Windows)

Set these variables in the Miniconda command prompt (create a .bat file for convenience):

set DAOROOT=C:\path\to\your\DAOROOT
set DAODATA=C:\path\to\your\DAODATA
set PATH=%PATH%;%DAOROOT%\bin;C:\path\to\protobuf\bin
set PYTHONPATH=%PYTHONPATH%;%DAOROOT%\python
set PKG_CONFIG_PATH=%PKG_CONFIG_PATH%;%DAOROOT%\lib\pkgconfig;C:\path\to\pkg-config\folder

Building on Windows

python waf configure --prefix=%DAOROOT%
python waf
python waf install

Windows Limitations

Documentation building is not currently supported on Windows
Address sanitization is not available
Unit tests are not yet ported to Windows
libdaoProto compilation is not yet supported
Cannot overwrite shared memory files that are in use by other processes

Troubleshooting

common errors:

daoLogging.pb.h:15:2: error: #error "This file was generated by a newer version of protoc which is"
   15 | #error "This file was generated by a newer version of protoc which is"
      |  ^~~~~
daoLogging.pb.h:16:2: error: #error "incompatible with your Protocol Buffer headers. Please update"
   16 | #error "incompatible with your Protocol Buffer headers. Please update"
      |  ^~~~~
daoLogging.pb.h:17:2: error: #error "your headers."
   17 | #error "your headers."
      |  ^~~~~
In file included from daoLogging.pb.cc:4:
daoLogging.pb.h:29:10: fatal error: google/protobuf/generated_message_tctable_decl.h: No such file or directory
   29 | #include "google/protobuf/generated_message_tctable_decl.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

This is caused by the wrong protoc being used and typcally the one comes with anaconda.

fix by

which protoc

protoc --version

conda uninstall protobuf

rm $(which protoc)

Then install the correct protoc.