HowTo setup an nrec development system

1. Installing the operating system

1. Make sure, that you have got linux compatible hardware.

2. Get a current debian netinst iso from debian.org and burn it on a CD.

3. Boot from this CD and install debian (desktop system).
   • Choosing one partition for all data is fine. If you feel expertic, you can create different partitions.
   • Type in the administrator password and login data for your first default user (e.g. devel).

   • Use a network mirror from your country (e.g. ftp://ftp.de.debian.org/).

   • Choose the "Desktop environment" and "Standard system" software installation profiles.

4. Get the nvidia driver here: NVIDIA-Linux-x86-173.14.12-pkg1.run.

5. Install the gcc compiler and libraries:

    # apt-get install gcc libc6 libc6-dev

6. Find out the running kernel version:

    # uname -a

7. Install the corresponding kernel headers:

    # apt-get install linux-headers-[kernel version]

   If you are unsure what to install, check the available packages:

    # apt-cache search linux-headers

8. Now you can install the nvidia-driver:

    # sh NVIDIA-Linux-x86-173.14.12-pkg1.run

   And tell the installation program to build a new kernel module.
9. When your X11 ist running, you can install sux:

    # apt-get install sux

10. Adjust your display settings:

     # nvidia-settings

    Set the appropriate resolutions for your screens and set up a separate screen for the second monitor (if you have one).
11. After restarting the x-server and enjoying the good resolution, you can install kde:

     # apt-get install kde

    If something goes wrong you may try using aptitude instead:

     # aptitude install kde

2. Building a Linux Kernel with RTAI realtime extensions

1. Get latest stable linux kernel source from http://www.kernel.org/pub/linux/kernel/ (e.g. http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.23.17.tar.gz)

2. get rtai from https://www.rtai.org/RTAI/ (e.g.: https://www.rtai.org/RTAI/rtai-3.6.1.tar.bz2)

3. Unpack kernel-sources and rtai-sources to /usr/local/src
   • (after downloading or moving the packages to /tmp):

    # cd /usr/local/src && tar xzvf /tmp/linux-2.6.23.17.tar.gz  && tar xjvf /tmp/rtai-3.6.1.tar.bz2

4. Change into the build dir

    # cd linux-2.6.19.7 

5. Copy config file of your running kernel into the build dir (for a reasonable default config)

    # cp /boot/config-2.6.x .config

6. Get packages for kernel-compilation:

    # apt-get install kernel-package
    # apt-get install libncurses5-dev

7. Apply rtai patch suitable for your kernel version:

    # patch -p1 --dry-run < ../rtai-3.6.1/base/arch/i386/patches/hal-linux-2.6.23-i386-1.12-03.patch
    # patch -p1 < ../rtai-3.6.1/base/arch/i386/patches/hal-linux-2.6.23-i386-1.12-03.patch

8. Configure and build kernel:

    # make-kpkg --initrd --append-to-version rtai --config menuconfig kernel-image 

   When the kernel menuconfig menu pops up, make sure

   • The right processor type for your system is selected (Processor type and features -> Processor family). You might want to find it out by # cat /proc/cpuinfo.

   • "Loadable module support -> Module versioning support" is disabled.

   • Processor Type and Features -> Symmetric multi-processing support is enabled.

   • Processor Type and Features -> Preemption Model -> Preemptible Kernel (Low-Latency Desktop) is enabled.

   • SATA support is enabled if you have a SATA HD.
   Then "Exit", Answer question "Save kernel config?" with "Yes" and wait for the kernel to compile (Take a break).
9. Leave kernel source dir

    # cd ..

10. Install the built kernel

     # dpkg -i linux-image-2.6.23.17rtai_2.6.23.17rtai-10.00.Custom_i386.deb

11. If you want to be sure, check grub boot manager entry:

     # less /boot/grub/menu.lst

12. Reboot:

     # reboot

13. Known issues:
    • If you get lots of IRQ-related messages after rebooting, insert the parameters noirqdebug and pci=noacpi to the kernel-command of your boot-manager (lilo or grub).

3. Building the RTAI modules

1. Go to the rtai source directory:

    # cd /usr/local/src/rtai-3.6.1

2. Make it:

    # make menuconfig

   • In the menu "General" set the rtai-path (/usr/lib/realtime) and the path to the kernel sources (/usr/local/src/linux-2.6.23.17).
   • Choose "Exit" and "Yes".

    # make install

3. If you want to test rtai, do:

    # export PATH=/usr/lib/realtime/bin:$PATH
    # cd /usr/lib/realtime/testsuite/kern/latency
    # ./run

4. It is useful to link the rtai modules to /lib/modules/$(uname -a)/ because they can then be loaded automatically if e.g. a comedi module needs them:

    # ln -s /usr/lib/realtime/modules /lib/modules/2.6.23.17rtai/rtai
    # depmod -a

4. Comedi

1. Download comedi from http://www.comedi.org/download/ (e.g. http://www.comedi.org/download/comedi-0.7.76.tar.gz) and move it to /usr/local/src.

2. Unpack it:

    # cd /usr/local/src
    # tar -xvzf comedi-0.7.76.tar.gz 

3. Configure it with the rtai package:

    # cd comedi-0.7.76
    # ./configure --with-rtaidir=/usr/lib/realtime

4. Make and install it:

    # make
    # make install
    # depmod -a

5. Create the devices:

    # make dev

6. Now is a good time to download (http://www.comedi.org/download/, e.g. http://www.comedi.org/download/comedilib-0.8.1.tar.gz), compile and install comedilib:

    # tar -xvzf comedilib-0.8.1.tar.gz
    # cd comedilib-0.8.1
    # ./configure
    # make
    # make install

7. Load modules and associate comedi-driver of your card to comedi-device (find appropriate driver at http://www.comedi.org/hardware.html):

    ## RTAI modules
    # modprobe rtai_hal
    # modprobe rtai_fifos
   
    ## Comedi modules
    For National Instruments cards:
    # modprobe ni_pcimio
    For measurement computing cards:
    # modprobe cb_pcidas64
   
    # modprobe comedi_rt_timer
    # modprobe kcomedilib

   Modules can be specified in /etc/modules.
8. Configure comedi with the comedi-driver of your card

   (find appropriate driver at http://www.comedi.org/hardware.html): Measurement Computing:

    # comedi_config /dev/comedi0 cb_pcidas64

   National Instruments:

    # comedi_config /dev/comedi0 ni_pcimio

9. Test comedi installation:

    # comedi_test

10. comedi_config can be called in a startup-script for automation Put the following code into /etc/init.d/comedi-device):

     #! /bin/sh
     #
     # comedi-device         Associate Comedi driver with device file
     #
     # Friedemann Bunjes     <friedemann.bunjes@uni-tuebingen.de>
     # 
    
     case "$1" in
       start)
             echo -n "Associating Comedi driver with device file."
     #        comedi_config /dev/comedi0 ni_pcimio
             comedi_config /dev/comedi0 cb_pcidas64
            ;;
       stop)
             ;;
       restart|force-reload)
             ;;
       *)
             ;;
     esac 
    
     exit 0

    and link to startup script directories:

     # ln -s /etc/init.d/comedi-device /etc/rc4.d/S30comedi-device
     # ln -s /etc/init.d/comedi-device /etc/rc5.d/S30comedi-device

5. Qwt

6. Development tools and libraries

1. Install subversion client

    # apt-get install subversion

2. Install kdevelop (optional)

    # apt-get install kdevelop

3. Install the HDF5 library (required) and tools (optional)

    # apt-get install 'libhdf5-serial*' hdf5-tools

4. Install the movie libraries

    # apt-get install libavformat-dev libavcodec-dev libpostproc-dev 

5. Install the gnu scientific library

    # apt-get install gsl-bin libgsl0 libgsl0-dev

7. Older setup informations

Here is the content of the NrecSetupHowto.txt which has been used so far:

Contents
========
1. Hardware requirements
2. Linux kernel
3. RTAI Modules
4. Comedi
5. Qwt
6. Development tools and libraries
7. NREC
8. Known issues


1. Hardware requirements
=======================
- Modern x86 PC
- Dual head graphics adapter
- For Data Acquisition and Digital I/O development: A Comedi (see www.comedi.org) compatible DAQ card, e.g. a Measurement Computing PCI-DAS6025.
- For the Trigger/Clipping the following connections must be established:
        DOut Channel 0 (trigger subdevice) - AIn Channel 2 (analog in subdevice)
        If subdevice for spike recording is available:
        DOut Channel 0 (trigger subdevice) - DIn Channel 0 (digital in/spikes subdevice)


2. Linux Kernel
===============
- Recent stable 2.6 kernel. Either from kernel.org or from your distribution, e.g. Debian packaged kernel.
- Make sure the following configuration options are selected. (Current Debian Sarge kernel-images have all these options set, so there is probably no need to build a custom kernel.):
        - CONFIG_PREEMPT: Preemptible kernel gives better latencies for realtime tasks.
        
2.1 Linux Kernel with RTAI realtime extensions
==============================================
- Install latest stable kernel sources
        Get latest stable linux kernel source from
        http://kernel.org/
        e.g.:
        $ wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.22.6.tar.bz2
        or:
        $ wget http://www.eu.kernel.org/pub/linux/kernel/v2.6/linux-2.6.19.7.tar.gz 

- Install RTAI extensions

        # get rtai from https://www.rtai.org/RTAI/
        e.g.: rtai-3.6.1.tar.bz2
  
- Unpack kernel-sources and rtai-sources to /usr/local/src 
  (after downloading or moving the packages to /tmp):
        $ cd /usr/local/src && tar xzvf /tmp/linux-2.6.19.7.tar.gz  && tar xjvf /tmp/rtai-3.6.1.tar.bz2
- Change into the build dir
        $ cd linux-2.6.19.7 
- Copy config file of your running kernel into the build dir (for a reasonable default config)
        $ cp /boot/config-2.6.x .config
        
- Get packages for kernel-compilation:
        # apt-get install kernel-package
        # apt-get install libncurses5-dev

- Apply rtai patch suitable for your kernel version:
        # patch -p1 --dry-run <
        ../rtai/vulcano/base/arch/i386/patches/hal-linux-2.6.19-i386-1.7-01.patch
        # patch -p1  < ../rtai/vulcano/base/arch/i386/patches/hal-linux-2.6.19-i386-1.7-01.patch
        
- Configure and build kernel
        $ fakeroot make-kpkg --initrd --append-to-version rtai --config menuconfig kernel-image 
                
        When the kernel menuconfig menu pops up, make sure
        (a) The right processor type for your system is selected (Processor type and features -> Processor family).
        (b) "Loadable module support -> Module versioning support" is disabled.
        (c) Processor Type and Features > Symmetric multi-processing support is enabled.
        (d) SATA supoort is enabled if you have a SATA HD.
        (e) See above (2. Linux Kernel) for additional required config options.
        Then "Exit", Answer question "Save kernel config?" with "Yes" and wait for the kernel to compile (Take a break).
        
- Leave kernel source dir
        $ cd ..

- Install the built kernel 
        $ su -c "dpkg -i linux-image-2.6.19.7rtai_2.6.19.7rtai-10.00.Custom_i386.deb"

- check grub boot manager entry:
        $ less /boot/grub/menu.lst

- If you get lots of IRQ-related messages after rebooting, insert the parameters noirqdebug and pci=noacpi to the kernel-command of your boot-manager (lilo or grub).

3. RTAI Modules
===============

- Get rtai from https://www.rtai.org/RTAI/
- or check out stable branch (vulcano) via cvs to /usr/local/src/rtai/vulcano:
        $ cvs -d:pserver:anonymous@cvs.gna.org:/cvs/rtai co vulcano
- Install like written in /usr/local/src/rtai/vulcano/README.INSTALL:

        # cd /usr/local/src/rtai/
        # mkdir build && cd build/
        # make -f ../vulcano/makefile menuconfig

- In the menu "General" set the rtai-path (/usr/lib/realtime)
  and the path to the kernel sources (/usr/local/src/linux-2.6.19.7).

- Make it:
        # make -f ../vulcano/makefile
        # make install

- If you would like to test rtai, do:
        # export PATH=/usr/lib/realtime/bin:$PATH
        # cd /usr/lib/realtime/testsuite/kern/latency
        # ./run
        
- It is useful to  link the rtai modules to /lib/modules/$(uname -a)/ 
  because they can then be loaded automatically if e.g. a comedi module needs them:
        # ln -s /usr/lib/realtime/modules /lib/modules/2.6.19.7rtai/rtai
        # depmod -a


4. Comedi
=========
- The official comedi-source package in the Debian repository which 
  provides the source for Comedi kernel modules (DAQ drivers) is 
  outdated and lacks support for newer kernels or hardware. 
  More recent versions can be obtained from [http://www.comedi.org/download.html]
  but not in Debian package (.deb) format; so they are not as easy installable and 
  removable as the Debian ones.
  This is why this document explains how to create a Debian package from Comedi CVS 
  (with the tarballs released on [http://www.comedi.org/download/] this is even 
  simpler as there is no need to run autgen.sh).
  
- You will need some Debian administration and development tools.
        # apt-get install build-essential dpkg-dev devscripts debhelper fakeroot \
                        linda apt-src module-assistant

- Get original debian comedi package source:
        $ cd /usr/local/src
        $ mkdir comedi-debian && cd comedi-debian
        $ su -c "apt-src update"
        $ apt-src install comedi-source
        $ cd ..

- Get current comedi release:
        # wget http://www.comedi.org/download/comedi-0.7.74.tar.gz
- Unpack:
        # tar xzf comedi-0.7.74.tar.gz
- Merge with debian source:
        # cp -r comedi-debian/comedi-0.7.70/debian comedi-0.7.74/
        # cd comedi-0.7.74
- Now, we have to generate a list of files which are to be included in the package:
        $ find . -path ./debian -prune -o -type f -print > debian/driver.files
- and add a changelog entry at the top of debian/changelog:
        $ vim debian/changelog
        
        The changelog has to obey a strict format, so copy an existing entry and 
        carefully apply your changes without changing the amount of whitespace 
        in the header and footer lines.

        An example:
        -------------------------------------------------------------------------------
        comedi (0.7.74-1) unstable; urgency=low

          * Debianizing the comedi-0.7.74 release tarball.

         -- Jan Gukelberger <g.u.g.i@gmx.de>  Wed, 19 Sep 2007 19:16:22 +0100

        -------------------------------------------------------------------------------

        The term in parentheses defines the version of the created package. 
        You should choose the format [upstream.version]-[your.version] where 
        [upstream.version] is the last version number released by the 
        Comedi project (can be checked at http://www.comedi.org/download/).

- NOTE1: Currently (Feb 2007) Comedi PCMCIA drivers are broken with kernels >= 2.6.17. 
        Therefore you have to edit debian/rules to not build those drivers or the last 
        installation step (module-assistant) will fail. Add the option '--disable-pcmcia' 
        to the './configure' line. See below [A.2] for a patch.

- NOTE2: If you are using debian/rules from an old debian package (i.e. 0.7.70 or earlier) 
        you also have to add the right rtai path to the ./configure line:
        ./configure [...] --with-rtaidir=/usr/lib/realtime --disable-pcmcia

- Now we can finally build the Debian package:
        $ debuild --linda -us -uc --linda-opts -i
        [Say 'y' if you are asked whether to continue despite a missing original tar file.]
- If everything went well you now have a debian package in the parent directory.
        $ cd ..

- NOTE3: This .deb package is system independent. I.e. having created it once 
        you can copy the package to any Debian system and need only execute 
        the following two "Modules install" commands to install Comedi drivers 
        on this system.

- Install your created Debian package:
        $ su -c "dpkg -i comedi-source_[version]_all.deb"
        e.g. # dpkg -i comedi-source_0.7.74-1_all.deb
        
- Build and install the comedi-modules for your running kernel using module-assistant (m-a):
        $ su -c "m-a a-i -t comedi"

- Make comedilib package:

- Download comedilib-x.x.x.tar.gz from http://www.comedi.org/download

- Unpack:
        # cd /usr/local/src
        # tar xjf comedilib-0.8.0.tar.bz2
        # cd comedilib-0.8.0

- Make package:
        # debuild --linda -us -uc --linda-opts -i
        # cd ..

- Install package:
        # dpkg -i libcomedi-0.8.0_0.8.0-1_i386.deb libcomedi-dev_0.8.0-1_i386.deb
        # 

- Load modules and associate comedi-driver of your card to comedi-device:
        (find appropriate driver at http://www.comedi.org/hardware.html)
        
        # RTAI modules
        modprobe rtai_hal
        modprobe rtai_fifos

        # Comedi modules
        modprobe ni_pcimio
        #modprobe cb_pcidas64
        modprobe comedi_rt_timer
        modprobe kcomedilib

- Configure comedi with the comedi-driver of your card
  (find appropriate driver at http://www.comedi.org/hardware.html):
  Measurement Computing:
        # comedi_config /dev/comedi0 cb_pcidas64
  National Instruments:
        # comedi_config /dev/comedi0 ni_pcimio
        
- Test comedi installation:
        # comedi_test

- Modules can be specified in /etc/modules, 

- comedi_config can be called in a startup-script for automation 
  (put A.3 into /etc/init.d/comedi-device):
        # ln -s /etc/init.d/comedi-device /etc/rc4.d/S30comedi-device
        # ln -s /etc/init.d/comedi-device /etc/rc5.d/S30comedi-device


5. Qwt
==========
- Make sure you have qt3 installed:
        # apt-get install qt3-dev-tools qt3-designer qt3-apps-dev  qt3-assistant  qt3-doc

- Make sure you have the g++ compiler installed:
        # apt-get install build-essential

- Download the latest Qwt 5.0 from 
        http://sourceforge.net/projects/qwt -> "Download qwt" -> "qwt" 
        (currently:  qwt-5.0.2.tar.bz2)
- Unpack the archive:
        $ tar xjf qwt-5.0.2.tar.bz2
- Build the library:
        $ cd qwt-5.0.2
        $ qmake qwt.pro
        $ make
- To use the library you have to 
        o EITHER install Qwt by 
                (a) copying the files in src/ to /usr/local/include/.
                (b) copying those in lib/ to /usr/local/lib/.
                (c) adding /usr/local/lib to the ldconfig config file /etc/ld.so.conf.d/i486-linux-gnu.conf
                (c) running ldconfig as root.
        o OR make sure the Qwt files are found by
                (a) adding the path to include/ to the INCLUDEPATH line in the nrec *.pro file.
                (b) adding the path to lib/ to the LIBS line in the nrec *.pro file (prefixed with '-L').
                (c) adding the path to lib/ to the LD_LIBRARY_PATH environment variable.


6. Development tools and libraries
==================================
- Install subversion client
        # apt-get install subversion
- Install kdevelop (optional)
        # apt-get install kdevelop
- Install the HDF5 library (required) and tools (optional)
        # apt-get install 'libhdf5-serial*' hdf5-tools
- Install the movie libraries
        # apt-get install libavformat-dev libavcodec-dev libpostproc-dev 
- Install the gnu scientific library
        # apt-get install gsl-bin libgsl0 libgsl0-dev


7. NREC
=======

- Checkout module "nrec" from the subversion server
        $ svn co https://jan@134.2.114.11/nrec
- Generate Makefile from qmake project file
        $ cd branches/src-xstimulus && qmake -o Makefile CTestExperiment.pro
- Import NREC project into KDevelop (optional)
        1) Run kdevelop
        2) "Project" -> "Import Existing Project"
                * Directory: Browse to nrec/branches/src-xstimulus you have just checked out.
                * Fetch from: Ignore this.
                * Project Name: "nrec"
                * Project Type: "QT C++ Application (QMake based)"
                * Author: Your Name
                * Email: Your Email address
- Compile NREC. Either in KDevelop "Build" -> "Build Project" or on the command line type
        $ make



8. Known issues
===============

8.1 Frame length is 100ms too large:
Symptoms: When NREC is started from X running on a second graphics adapter the first time, syslog says "Disbling IRQ X" and from now until reboot all frames are 100ms too long.
Cause: For some reason, a flood of unhandled interrupts from the Matrox graphics card occurs and this IRQ is disabled by kernel interrupt debugging routines.
Solution: Append the bootparam 'noirqdebug' to the kernel command line. (lilo.conf: Insert an append = "irqdebug" in the section of your kernel image; run lilo. See 'man 7 bootparam' and 'man 5 lilo.conf')

8.2 RealTime graphics thread runs away:
Symptoms: Matrox graphics and MC DAQ card share the same IRQ (see 'lspci -v'). When a trial is started, the RT graphics thread takes all CPU time and {renders the system unusable | is killed by the watchdog}.
Cause: matroxfb and Comedi cb_pcidas64 drivers interfere the IRQ management of each other. (Bad implementation in Comedi driver?) matroxfb can't enable the VSYNC interrupt so the ioctl( WAITFORVSYNC ) fails. Hence, DirectFB does a busy wait for the IRQ so the thread never sleeps.
Solution: Put the DAQ card into another PCI slot that doesn't share interrupts with the AGP slot. Consult the motherboard manual or try it out. A good guess is a slot physically far away from the AGP slot.

8.3 dfb hangs at startup/initialization (dfbinfo hangs already)
dfb init has problems, if the module has been loaded before X. 
Solution: Load matroxfb after X has been started.
Reload dfb modules:
# rmmod matroxfb_base matroxfb_DAC1064 matroxfb_accel cfbcopyarea cfbimgblt cfbfillrect matroxfb_Ti3026 matroxfb_g450 g450_pll matroxfb_misc mga
# modprobe matroxfb_base
# modprobe mga


==============
|| APPENDIX ||
==============

A.1 RTAI device nodes creation script
=====================================
-----------------8<-----------------------------
#!/bin/bash
mkdir /dev/rtf
for n in `seq 0 9`
do
        f=/dev/rtf/$n
        mknod -m 666 $f c 150 $n
done
for n in `seq 0 9`
do
        f=/dev/rtf$n
        mknod -m 666 $f c 150 $n
done
-----------------8<-----------------------------

A.2 Patch to disable PCMCIA drivers:
=====================================
-------------------------------------------------------------------------------
--- comedi-debian/comedi-0.7.70/debian/rules    2007-02-25 18:12:38.000000000 +0100
+++ comedi-build/debian/rules   2007-02-25 18:56:10.000000000 +0100
@@ -34,7 +34,7 @@
 config.status: configure
        dh_testdir
        # Add here commands to configure the package.
-       CFLAGS="$(CFLAGS)" ./configure --host=$(DEB_HOST_GNU_TYPE) --build=$(DEB_BUILD_GNU_TYPE) --with-rtaidir=/usr/lib/realtime
+       CFLAGS="$(CFLAGS)" ./configure --host=$(DEB_HOST_GNU_TYPE) --build=$(DEB_BUILD_GNU_TYPE) --with-rtaidir=/usr/lib/realtime --disable-pcmcia


 build-arch:  build-arch-stamp
-------------------------------------------------------------------------------

A.3 Startscript for automatically configuring comedi device:
=============================================================

#! /bin/sh
#
# comedi-device         Associate Comedi driver with device file
#
# Friedemann Bunjes     <friedemann.bunjes@uni-tuebingen.de>
#

case "$1" in
  start)
        echo -n "Associating Comedi driver with device file."
#        comedi_config /dev/comedi0 ni_pcimio
        comedi_config /dev/comedi0 cb_pcidas64
       ;;
  stop)
        ;;
  restart|force-reload)
        ;;
  *)
        ;;
esac

exit 0