# Development Instructions ## Help support in random order - Mark Donszelmann - Henk Boterenbrood - Jörn Schumacher - Markus Joos ## Table of Content - [Get the Software](#get-the-sofwtare) - [Requirements](#requirements) - [Setup and Build](#setup-and-build) - [Software Repositories](#software-repositories) - [External Software](#external-software) - [Dependencies](#dependencies) ## Get the Software The checked out top-level software directory only contains a README and some scripts. Every package lives in its own git repository (see [Software Repositories](#software-repositories)). You can clone them all or you can clone individual packages, see dependencies for which ones you need. - To clone all the packages at once use the script: ``` ./clone_all.sh ssh ``` which will create subdirectories for all of the packages listed above. All will be on the master branch. Any package already checked out will be pulled for changes. - To clone individual packages you could issue the command(s): ``` git clone ssh://git@gitlab.cern.ch:7999/atlas-tdaq-felix/regmap.git git clone ssh://git@gitlab.cern.ch:7999/atlas-tdaq-felix/flxcard.git git clone ssh://git@gitlab.cern.ch:7999/atlas-tdaq-felix/felixbase.git git clone ssh://git@gitlab.cern.ch:7999/atlas-tdaq-felix/felixbus.git git clone ssh://git@gitlab.cern.ch:7999/atlas-tdaq-felix/felixcore.git ``` the names of all packages are listed at the top of this document. - To switch to another branch use the script: ``` ./checkout_all.sh ``` which will switch all subdirectories to branch_name if that branch exists. - To update the packages to the latest revision you can run the script: ``` ./pull_all.sh ``` - or update an individual package by changing into its directory and pulling from git: ``` cd flxcard git pull ``` - To see the status of all the packages: ``` ./status_all.sh ``` - To commit and push to a specific package: ``` cd flxcard git commit -m "Your Message" git push ``` ## Requirements ### Required packages for SLC6 (based on SLC6 Base x86_64 2015-09-04) ``` sudo yum install gcc sudo yum install mesa-libGL sudo yum install xkeyboard-config ``` ### Required packages for Centos7 (based on CC7 Base x86_64 2016-02-09) you will need to install the following packages: ``` sudo yum install gcc sudo yum install make sudo yum install mesa-libGL sudo yum install libpng sudo yum install libSM sudo yum install libXrender sudo yum install fontconfig sudo yum install xkeyboard-config ``` (in a future version of CC7 these may be installed by default) ### LCG You need access to LCG on cvmfs or our separate LCG for Felix distribution. #### Use cvmfs Check if you can access /cvmfs/sft.cern.ch/lcg ``` ls /cvmfs/sft.cern.ch/lcg ``` If not, you can install install cvmfs: ``` sudo yum install https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpm sudo yum install cvmfs cvmfs-config-default sudo cvmfs_config setup /etc/cvmfs/default.local (inside cern) CVMFS_REPOSITORIES=atlas.cern.ch,sft.cern.ch CVMFS_HTTP_PROXY="http://ca-proxy.cern.ch:3128" or (outside cern) CVMFS_REPOSITORIES=atlas.cern.ch,sft.cern.ch CVMFS_HTTP_PROXY=DIRECT sudo service autofs restart cvmfs_config probe ``` #### Use a local copy of LCG for Felix from ``` https://atlas-project-felix.web.cern.ch/atlas-project-felix/user/dist/software/cvmfs/ ``` unpack it in and setup the variable: ``` export LCG_BASE= ``` ### lsb_release You need to make sure the command lsb_release works. ``` lsb_release ``` If not install it or ask for it to be installed: ``` sudo yum install redhat-lsb ``` ## Setup and Build Most of our code uses 'cmake', a c/c++ compiler and LCG for external packages. The current minimum version of gcc/g++ we need is 4.9.x, for which we rely on LCG84. You can also use version 6.2, for which we rely on LCG87. To setup the correct compiler, the correct version of cmake and some other parameters, use: ``` source /software/cmake_tdaq/bin/setup.sh x86_64-slc6-gcc49-opt ``` You may want to put this in your .bash_profile to be set up when you login. Possible binary tags are: x86_64-slc6-gcc49-opt, x86_64-slc6-gcc62-opt, x86_64-centos7-gcc49-opt, x86_64-centos7-gcc62-opt. The code builds in a "binary" target directory, which is created inside the "software" directory with the following commands: ``` cd software cmake_config x86_64-slc6-gcc49-opt ``` You can then compile all the code by: ``` cd x86_64-slc6-gcc49-opt make ``` You can also compile a specific package (for instance flxcard) by: ``` cd x86_64-slc6-gcc49-opt/flxcard make ``` ## Software Repositories | Repository | Description | |:-----------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------| | [benchmarks](https://gitlab.cern.ch/atlas-tdaq-felix/benchmarks) | Benchmarks of different applications of the FELIX project | | [cmembuff](https://gitlab.cern.ch/atlas-tdaq-felix/cmembuff) | C++ wrapper for selected functions of the cmem_rcc library | | [data_transfer_tools](https://gitlab.cern.ch/atlas-tdaq-felix/data_transfer_tools) | Tools to create and manipulate Data Files | | [drivers_rcc](https://gitlab.cern.ch/atlas-tdaq-felix/drivers_rcc) | Official version of the drivers (synched with the release of the rpm) | | [elinkconfig](https://gitlab.cern.ch/atlas-tdaq-felix/elinkconfig) | GUI to configure the GBT Emulator and CentralRouter | | [external](#external-software) | External packages NOT in cvmfs | | [fatcat](https://gitlab.cern.ch/atlas-tdaq-felix/fatcat) | Advanced test client for FELIX hosts | | [fdaq](https://gitlab.cern.ch/atlas-tdaq-felix/fdaq) | Command line tools to stream data to disk, upload data, check data and others | | [fel](https://gitlab.cern.ch/atlas-tdaq-felix/fel) | Command line tool to test and verify the GBT data stream and its content | | [felixbase](https://gitlab.cern.ch/atlas-tdaq-felix/felixbase) | Library for all felix development | | [felixbus](https://gitlab.cern.ch/atlas-tdaq-felix/felixbus) | Library to connect all felixcores and swrods together | | [felixcore](https://gitlab.cern.ch/atlas-tdaq-felix/felixcore) | Main FELIX application in charge of DAQ and Slow-Control | | [felix-fpga-flash](https://gitlab.cern.ch/atlas-tdaq-felix/felix-fpga-flash) | Tools for the FPGA firmware version control, firmware remote reconfiguration and update. | | [felix-mon](https://gitlab.cern.ch/atlas-tdaq-felix/felix-mon) | Monitoring tools for FELIX hosts | | [felix-nightly](https://gitlab.cern.ch/atlas-tdaq-felix/felix-nightly) | Nightly build, and test on the hardware of the firmware and software. | | [felix-web-mon](https://gitlab.cern.ch/atlas-tdaq-felix/felix-web-mon) | Web Application to monitor FELIX | | [flxcard](https://gitlab.cern.ch/atlas-tdaq-felix/flxcard) | API and command line tools to configure, control, and test FLX cards | | [gbt_config_scripts](https://gitlab.cern.ch/atlas-tdaq-felix/gbt_config_scripts) | Standalone python based scripts to configure and control the GBT Wrapper | | [gbt_data_emulator](https://gitlab.cern.ch/atlas-tdaq-felix/gbt_data_emulator) | Matlab GBT Emulator data format generator for the FPGA RAM based emulator | | [gbt_standalone_scripts](https://gitlab.cern.ch/atlas-tdaq-felix/gbt_standalone_scripts) | *obsolete* Standalone scripts to configure and control the GBT Wrapper (including example bitfiles) | | [gitlab_ci_runner](https://gitlab.cern.ch/atlas-tdaq-felix/gitlab_ci_runner) | Gitlab CI Runners for slc6 and cc7 | | [gitlab_ci_test](https://gitlab.cern.ch/atlas-tdaq-felix/gitlab_ci_test) | Gitlab CI Testing project | | [hdlc_coder](https://gitlab.cern.ch/atlas-tdaq-felix/hdlc_coder) | HDLC library for encoding and decoding HDLC | | [netio](https://gitlab.cern.ch/atlas-tdaq-felix/netio) | Communication library to communicate between felixcore and swrods | | [packetformat](https://gitlab.cern.ch/atlas-tdaq-felix/packetformat) | API to the packetformat used between the FLX card and felixcore and fdaq | | [pcie_hotplug](https://gitlab.cern.ch/atlas-tdaq-felix/pcie_hotplug) | Script to perform PCIe hotplug of cards | | [pepo](https://gitlab.cern.ch/atlas-tdaq-felix/pepo) | Simple PEek and POke program to write registers in the FPGA register map | | [regmap](https://gitlab.cern.ch/atlas-tdaq-felix/regmap) | Register map header files for the various programs | | [tdaq_for_flx](https://gitlab.cern.ch/atlas-tdaq-felix/tdaq_for_flx) | Folder containing pointers to various TDAQ based packages and libraries for the driver | | [tests](https://gitlab.cern.ch/atlas-tdaq-felix/tests) | Home to some fearsome software experiments | | [wuppercodegen](https://gitlab.cern.ch/atlas-tdaq-felix/wuppercodegen) | JINJA based register automatic generation tool | | README.txt | this file | ## External Software | Repository | Description | |:--------------------|:--------------------------------------------------------------------------------| | [bootstrap] | Web Styling from http://getbootstrap.com/getting-started/ | | [catch] | Testing framework from https://github.com/philsquared/Catch | | [concurrentqueue] | Lock free multi-reader multi-writer queue from https://github.com/cameron314 | | [czmq] | C API to zeromq from https://github.com/zeromq/czmq | | [datatables] | Web DataTables from https://datatables.net/download/packages | | [highcharts] | Web Charting library from http://www.highcharts.com/download | | [jinja] | Jinja2 for Python from http://jinja.pocoo.org/docs/dev/ | | [jquery] | Web JavaScript framework from https://code.jquery.com | | [json] | JSON for Modern C++ from https://github.com/nlohmann/json | | [libcurl] | libcurl https://curl.haxx.se/libcurl/ | | [libfabric] | Open Fabrics Interfaces from https://github.com/ofiwg/libfabric | | [libnuma] | numactl-devel providing libnuma | | [markupsafe] | MarkupSafe for Python from https://pypi.python.org/pypi/MarkupSafe | | [mathjs] | Web JavaScript math library for units from http://mathjs.org | | [patchelf] | Utility to change rpath for the distrbution from http://nixos.org/patchelf.html | | [pyyaml] | YAML for Python from http://pyyaml.org/wiki | | [readerwriterqueue] | Lock free single-reader single-writer queue from https://github.com/cameron314 | | [simplewebserver] | Simple Web Server for C++ from https://github.com/eidheim/Simple-Web-Server | | [spdlog] | Logging library from https://github.com/gabime/spdlog | | [sphinx] | Documentation tool for Python from https://pypi.python.org/pypi/Sphinx | | [zeromq] | ZeroMQ communications library from https://github.com/zeromq/zeromq4-1 | | [zyre] | Beacon system built on zeromq from https://github.com/zeromq/zyre | ## Dependencies The following packages have dependencies, TDAQ dependencies are listed in parentheses, optional dependencies in square brackets, dependencies from the FELIX area are in curly-braces. - cmembuff: drivers_rcc - elinkconfig: flxcard regmap (Qt) - drivers_rcc: - - fdaq: flxcard regmap (Qt) - fel: drivers_rcc flxcard regmap - felixbase: {packetformat} - felixbus: {zyre czmq zmq} - felixcore: felixbus felixbase netio FlxCard drivers_rcc (tbb Boost) - flxcard: regmap drivers_rcc - netio: (tbb) {zmq} [fabric] - packetformat: - - pepo: drivers - regmap: - ### External (LCG) libraries Some packages need external libs. All external libs are taken from the LCG area on cvmfs. Each of them are specified in the CMakeLists.txt files of those packages. We currently rely on the following libs: - Qt (GUI Library) - tbb (Intel Threading Building Blocks) - Boost (C++ library) ### LCG - LCG via cvmfs is available on the test-bed (after sysadmin request). - LCG version 87 is available in ATCN P1, under /sw/atlas/sw/lcg/releases/LCG_87/. Version 84 is not. - LCG tar files for FELIX with only the items needed by FELIX is available - For the binary distributions of FELIX the needed shared objects are extracted and included in the distribution automatically.