Experience with Windows Based IOCs SNS Diagnostics Presented by Dave

Experience with Windows Based IOCs SNS Diagnostics Presented by Dave Purcell June 17-20, 2003 Diagnostics Team BNL, LANL, LBNL, ORNL

SNS Diagnostics Overview 248 IOCs – 232 PC based (19 Running or Installed and Ready for Beam) – 16 VME based (3 Installed and Ready for Beam) 20 unique system designs 600 devices 100 GByte/s peak 5 GByte/s sustained June 17-20, 2003 Diagnostics Team BNL, LANL, LBNL, ORNL

SNS Diagnostics Layout Operational Next 6 Months FY2004 FY2005 MEBT 6 Position 2 Current 5 Wires 1 Emittance 1 fast faraday cup 1 faraday/beam stop DTL 10 Position 5 Wire 5 Loss 5 Faraday Cup 6 Current D-Plate 3 Position 1 Loss 1 Current 2 Wire 2 Faraday Cup 1 Bunch 1 Video 1 Halo 3 Neutron 1 Beam Stop Faraday Cup 1 Emittance (Slit and Collector) Diagnostics Team IDump 1 Position 1 Wire 1 Current RING 44 Position 2 Ionization Profile 87 Loss 1 Current 5 Electron Det. 2 Wire 1 Beam in Gap 2 Video 1 Tune CCL 10 Position 9 Wire 8 Neutron 48 Loss 3 Bunch 1 Faraday Cup 1 Current SCL 32 Position 58 Loss 8 Laser Wire CCL/SCL Transition 2 Position 1 Wire 1 Loss 1 Current EDump 1 Current 2 Loss 1 Wire RTBT 17 Position 43 Loss 4 Current 5 Wire 1 Harp HEBT 29 Position 11 Wire 49 Loss 4 Current LDump 6 Loss 6 Position 1 Wire June 17-20, 2003 BNL, LANL, LBNL, ORNL

General History – How We Got Here Some key decision points 1999: VXI modules vs. Network Attached Devices – More info on next slide 2000: PC vs Embedded System – Support from industry and other SNS Divisions 2000: Brookhaven’s SNAP Protocol vs. Channel Access – Use extra protocol/layer only if necessary 2002: PCAS vs. IOC Core – 3.14 released for Win32 and successfully tested on diagnostics June 17-20, 2003 Diagnostics Team BNL, LANL, LBNL, ORNL

Evolution of architecture for high bandwidth diagnostic systems Current architecture: Smart network attached devices (NAD) Industry standard processing system (PC servers), and internal bus (PCI) Device-oriented partitioning (mirrors Application Software’s object model) Original architecture: Smart VXI modules Custom processing system (DSP), custom internal bus Functional partitioning EPICS Channel Access Hub U T D D D D D C t i S S S S S P I mP P P P P U l e VXI Beamline devices Diagnostics Team EPICS Channel Access Rack mount Hub Hub Network, timing Laptop Rack mount Rack mount Rack mount Rack mount Network, timing Beamline devices June 17-20, 2003 BNL, LANL, LBNL, ORNL

General Strategies Network Attached Device – Self contained, field replacement Each pickup or sensor has its own resources such as timing, data acquisition and processing. – Avoid brittleness - Loose coupling between devices through simple interfaces: sensor, event links, machine protect, network. – Minimal single points of failure Rapid development, refinement, and replacement by deploying COTS/commodity hardware, drivers, software, and architecture Where possible, continuous online self-test and calibration Deployment strategy: build in flexibility, but plan for minimum feature set during early operations An individual device can fail or be serviced without disrupting other devices. Software is simplified as it doesn’t have to manage single resources among multiple devices. June 17-20, 2003 Diagnostics Team BNL, LANL, LBNL, ORNL

19 IOCs Currently Deployed using Windows 12 Using PCAS – 7 Wire Scanners – 5 Beam Position Monitors 7 Using IOC Core – – – – 1 Beam Position Monitor 2 Beam Current Monitors 2 Faraday Cups 1 Physics Machine (Test System for CA Client and Java Apps) – 1 Neutron Detector June 17-20, 2003 Diagnostics Team BNL, LANL, LBNL, ORNL

Experience During Operations Commissioning Conditions do not allow for good statistics. No “Blue Screen of Death” No Downtime Because of Windows – With PCAS, Some Restart of LabView Required. Suspected problem within ActiveX. But not pursued because R3.14/IOC Core became available for Win32. – Upgrades to Software/Hardware limit long runs – Lack of Beam Time to Play Good Stories – The Berkeley developed neutron detectors were implemented with IOC Core, integrated in a few days, running continuously since December 2002. – The Physics Test Machine developed with IOC Core, running continuously. June 17-20, 2003 Diagnostics Team BNL, LANL, LBNL, ORNL

Performance of the Implemented IOCs Test Hardware with 100Mb/s Ethernet Pentium 4, 2GHz, Server with IOC Shared Memory Pentium 3, 1.4GHz, Client with SNS CA Client Test Program Server Loop through 1000 pre-generated sets of 4096 doubles. Set PV (I/O Intr) and set interrupt with sequence number in a nanosecond field of timestamp Test Program Client Monitor PV and check for sequence number, mark breaks in sequence Results At 300Hz: 300*4096*8 9.8 Mb/s or 78Mbits/s. (Note: packet loss only when using computer for additional tasks.) June 17-20, 2003 Diagnostics Team BNL, LANL, LBNL, ORNL

Lessons Learned – Successful Overall Network Attached Device concept is working Prototypes easily integrated PCAS Worked as Advertised R3.14 Released. Meets Needs – IOC core 3.14 on Win32 has been impressively stable for a first release on this platform – even in prototype form, PC systems have been as stable as VxWorks systems and Linux consoles. Rapid fault isolation Good experience with commercial off the shelf hardware/software Based on this experience, all systems except BLM will now be deployed on PC platform June 17-20, 2003 Diagnostics Team BNL, LANL, LBNL, ORNL

Future Plans - Central Role of Database Oracle Server 1. Design information: names, locations, .db, .cmd, 2. Equipment receiving, acceptance test data: tracked by barcode Diagnostics Team XML, .db, HTML Server 6. Applications 4. Web-based reports, Initialization Files 5. Software in Network Attached Devices 3. Calibration/Maintenance of installed devices: tracked by barcode June 17-20, 2003 BNL, LANL, LBNL, ORNL

Future Plans – NAD Production Replacing or building the software of a NAD is to become a push button operation. – PC management package such as SMS or Altiris to manage and monitor the hundreds of NADs. – Oracle database as the one depository for all files to be installed on a NAD and to use the PC management package to setup and maintain our NADs. A Channel Access Client for LabVIEW, independent from ActiveX for Windows and Linux, is under development. June 17-20, 2003 Diagnostics Team BNL, LANL, LBNL, ORNL