Computing Overview

Outline

• Online System Overview

• DAQ Progress
  • Data Collection Modules (DCM’s)
  • Event Builder (EvB)

• Online Control Systems (ONCS) Progress

• Offline Progress

• Simulation Progress

Online System Overview

• Data-driven system consisting of:
  • Trigger
    • LL1
      (Local Level 1)
    • GL1
      (Global Level 1)
  • DCM
  • EvB
    • SEB
      (Sub-Event Buffers)
    • Switch (ATM)
    • ATP
      (Assembly and Trigger Processors aka L3)
  • T & C
    • Hardware implements partitioning
  • ONCS
    • Controls entire system
    • Also ancillary systems

DCM Progress

• Prototyped all major components on daughter-cards:
  • Optical link (HP Glink + Methode)
    • Discovered error in HP’s implementation for 16-bit operation

• FPGA-driven zero-suppression engine

• SHARC DSP

• Laid out entire board consisting of 4 DCM’s + interface DSP + VME

• Specified VME and SEB interface with ONCS

• Added post-doc

DC Member: C.Y. Chi

DCM Status and Schedule

• Status:
  • DCM Main board: layout finished
  • DCM_A daughter card: schematic finished layout started
  • DCM_T daughter card: schematic started

• Future work:
  • July-97: DCM_T delivered to BNML for TEC FEM-DCM-ONCS integration test
  • Nov-97: All DCM prototypes complete
  • Aug-98: DCM production complete

• Schedule slippage from:
  • Transition to optical links
  • Internal interference (TOF, some EvB)
  • External interference (LHC)

EvB Progress

• Oct-96: B.A. Cole appointed as “DAQ” EvB representative

• Fall-96: Evaluation of possible technologies:
  • CDR cross-bar (deprecated)
  • Custom switch based on Columbia LQCD gate array
  • ATM-based solution

• Began informal collaboration with CERN’s RD31:
  • Nov-96: Met with J.P. Dufey at BNL
  • Jan-97, Apr-97: with RD31 at CERN

• Spring-97: Develop scaleable solution based on:
  • Sub-Event Buffers with PCI-interface to NT machine
  • NT host with OC-3 Network Interface Card
  • FORE ASX-1000 ATM Switch
  • NT-based Assembly and Trigger Processors (ATP’s)

DC Member: B. Cole

The PHENIX Event Builder

(Prepared by B.A. Cole)

EvB - Manpower

• Have met with RD-31 twice over last 4 months.
• RD-31 is finished -- group is beginning to split.
• ATLAS Demonstrator-C group has agreed to collaborate with PHENIX for mutual benefit.
• They will provide real work + valuable experience.
• We will continue to work with J.P. Dufey and CERN group but official arrangement likely not possible.

(Prepared by B.A. Cole)

ONCS Overview

Responsibilities:

• Configuration and initialization of entire Online system

• Monitoring data flow through the system

• “Ancillary” systems” = Control and monitoring of:
  • HV
  • Low voltage
  • Gas systems
  • Cooling

• User interface to Online

• Calibration and configuration databases

• Interface to data archiving at RCF

• Sector-test support

ONCS Progress

• Added manpower:
  • TAC96: 3.5 FTE’s
    (E. Desmond, H.J. Keyhias, T. Kozlowski, C. Witzig)
    • Sep-96: M. Purschke
    • Oct-96: J. Haggerty, S. Pate (0.5 FTE)
    • Apr-97: S. Durrant
  • TAC97: 7 FTE’s

• Further development of system model:

• Sector-test support:

• Interface specifications

• Development of DCM-SEB PCI interface card

• Ancillary system development:
  • NMSU+LANL work integrating CEBAF EPICS drivers into ONCS
  • Delivery to BNL for Fall Sector Test

}

Using code

generated

from

formal model

DC Member: C. Witzig

ONCS Work in 96-7

(Selected

highlights only)

ONCS Release 1

• ONCS Release 1
  • First ONCS PROTOTYPE that tests some of the features of ONCS
  • Written with the intent to be able to integrate PHENIX hardware as it becomes available to ONCS
  • Current Release 1 prototype tests part of ONCS design
  • CAMAC and other non-Phenix hardware (M.Purschke)
  • CORBA based event communication mechanism (in UNIX and VxWorks) (E.Desmond)
  • OO model for “run control” (T.Kozlowski,C.Witzig)
  • Concept of event pools for distributing events to processes (M.Purschke, C.Witzig)
  • Defined an analysis framework (“pstuff”) for online monitoring and calibration (M.Purschke)
  • User Interface (UIM/X) (H.J.Kehayias)

• Some Design Characteristics (already implemented in release 1)
  • Distributed object oriented system
  • Use of asynchronous state machines
  • Event-based communication mechanism
  • Separation of user interface from application
  • Release 1 is our first OO prototype from which we learn how to make a better release 2!

Offline Progress

• Release of PEP to alpha testers in mid-96
  • Tested by RICH, MVD and tracking groups
  • Feedback at Jul-96 Collaboration meeting
  • Further developments toward beta release in remainder of CY96

• Began evaluation of STAF late in CY96
  • Proven system
  • CORBA compliant
  • Advantageous for STAR/PHENIX collaboration at RCF
  • Established physics framework + distributed objects

• Decided in Mar-97 to use STAF
  • Cooperate with STAR at RCF and in Grand Challenge
  • PHENIX brings expertise in distributed systems and DB’s

• Added 3 solid FTE’s (two stationed at BNL)

DC Member: S. Sorensen

Offline Work in 96-7

(Selected

highlights only)

STAF Features

• Modular framework in C++

• CORBA-compliant underpinnings

• Users
  • Specify modules in IDL
  • See data as tables

• Framework provides
  • Scripted environment
  • I/O services with UNIX-like internal directories
  • Memory management
  • Error logging
  • etc...

Simulation Progress

Progress on many fronts:

• Physics

(Not presented here)
• Event generators
• Trigger algorithms
• Backgrounds

• Computing Resources
  (Aggregate ~6K SpecInt92’s)
  
  • LLNL ASICE
  • LBNL PDSF
  • BNL RCF

• Massive Simulations
  • 10K events in central arms
  • 100K in muon arms

DC Member: C. Maguire

Sim. Work in 96-7

(Selected

highlights only)

Issues and Concerns

(Roughly prioritized)

• Manpower
  • “Offline” software (I.e., the sub-system specific and global physics code)
    • Discussed at Nov-96 Computing meeting
    • No obvious solution in sight
    • Heavy reliance on students and post-docs “coming on board”
  • DCM’s, especially software: Will saturate one post-doc
  • EvB:
    • Need to hire specialists at Nevis asap
    • RD31 collaboration very useful interim (at least) solution

• Schedule:
  • DCM’s are late
  • Parallel-ization of DCM’s and EvB should help

• Culture:
  • PHENIX is ~101-2 increase in complexity from existing heavy ion experiments
  • Nuclear physics has not developed a cadre of “computing specialists” as has HEP

Role of Computing Coordinator

• To Date:
  • Instituted “Computing Day” as part of every PHENIX week
    • Status of sub-system software
    • Feedback between users and developers
    • Global decisions (e.g., languages, transition to STAF)
  • Conflict “resolution”

• Needed:
  • Identification/recruitment of resources (people)
  • Greater tracking of global software development
    • Progress updates per sub-system on ~2-month cycle
    • Coordination between
      • Sub-systems
      • Service groups