MUST2 SAMURAI_2018 S3-LEB-LPC SUPERNEMO MUGAST EXPAND SCALP GALATRON HiCARI VELO
  MUST2, Page 12 of 48  Not logged in MUST2
New entries since:Thu Jan 1 01:00:00 1970
Fixed, e748, Thu Jul 6 07:23:22 2017, e748, MUST2, Software, CD2, MUST2 : VMECOM : TCP error - TCP/IP : time-out 
"MUST2 : VMECOM : TCP error - TCP/IP : time-out" at 7:19.
RRC shows MUST2 "OFFLINE"
We stop run 331 -> Doesn't stop
We try TK, KNA -> no effect
We try KALL, RC -> RC doesn't start
We enter D6 and reboot MUST2 VXI and AGAVA VME and restart RC
No events... -> time out...
Probably a betwork problem
We're trying to get in touch with an astreinte
Fixed, e748, Thu Jul 6 07:58:09 2017, e748, MUST2, General, CD2, SiLi 1/T5 triped 
SiLi 1/T5 triped at 7:30. We leave it off while trying to restart the run control.
We turn it on again at 8:03.
Fixed, e748, Thu Jul 6 08:22:08 2017, Valérie L, RUN, General, CD2, Problem of DAQ and networks problems ProblèmeDAQ6juillet8h.pngProblemGUIRuncontrolNumExo.pngRebootsoftCrateAGAVApassword.pngProblemDAQ5juilletNumExo.png
After the problem on Run control and the offline of MUST2, the DAQ on numexo could not be started again.
see error messages below.
The soft "astreinte" was called at 8h20 and also the one of G2I at 8h30 for network problems.
The G2I was on responding machine.
A series of junk runs was done after a series of init on both Run controls.
We asked to the hardware to reboot the AGAVA crtae from the Terminal.
Problem: we need to know the password - the software "astreinte" has it.
--------------------------------------------------------------------------------------
Junk runs during the tests of the restart of the DAQ
General DAQ
-rw-r--r-- 1 e748 Exxx 2,0M 6 juil. 08:04 run_0332.dat.06-07-17_07h56m55s
-r--r--r-- 1 e748 Exxx 1,4M 6 juil. 08:16 run_0333.dat.06-07-17_08h15m05s
-r--r--r-- 1 e748 Exxx 1,1M 6 juil. 08:20 run_0334.dat.06-07-17_08h18m59s
-r--r--r-- 1 e748 Exxx 2,0M 6 juil. 08:40 run_0335.dat.06-07-17_08h38m27s
-r--r--r-- 1 e748 Exxx 832K 6 juil. 09:10 run_0336.dat.06-07-17_09h08m47s
-r--r--r-- 1 e748 Exxx 1,3M 6 juil. 09:16 run_0337.dat.06-07-17_09h15m06s
-r--r--r-- 1 e748 Exxx 9,2M 6 juil. 09:26 run_0338.dat.06-07-17_09h24m30s

DAQ chio
-r--r--r-- 1 e748chio Exxx 16K 6 juil. 08:04 run_0332.dat.06-07-17_07h57m06s
-r--r--r-- 1 e748chio Exxx 1,9K 6 juil. 08:15 run_0333.dat.06-07-17_08h15m16s
-r--r--r-- 1 e748chio Exxx 4,1K 6 juil. 08:20 run_0334.dat.06-07-17_08h18m33s

--------------------------------------------------------------------------------------
Fixed, e748, Thu Jul 6 09:54:53 2017, Valérie L, RUN, General, CD2, Run control OK -Runs 339 and 340 CorrelBidimChioandplotCatsRun340.pngRun339scalers.pngScalersAnalyserRun340.png
After the reboot soft of the AGAVA NUMEXO and realignment of the TGS (time stamp Agava) the situation is stabilized.
The "astreintes" for hardware (Gilles W) and software (Clotilde M) have rebooted from the Terminal the crates (AGAVA).
Antoine Lemasson has reloaded the setup of the GTS.
Charles Houarner and Antoine have checked the configuration for the VME AGAVA
cf the ELOG entry Id=233 indicating the procedure for the correct configuration of Agava before restarting.
https://elog.lpc-caen.in2p3.fr/MUST2/233 ]
Now the run control is starting correctly for Numexo.
The correlation plots are checked cf plots below for Chio and Cats.
There were DAQ problems between 7h20 and 9h30. It took ~ 2h to test the restart/init, then after calling the astreintes (at 8h30) it took nearly 1 hour to solve the problem (reboot+ settings of the crates and DAQ. The data runs restarted at 9h32 with run 339.

===========================================================================================
General DAQ e748 /data/e748X/e748/acquisition/run
Run 339 6 JULY start 9h32 STOP 9h43
-r--r--r-- 1 e748 Exxx 65M 6 juil. 09:43 run_0339.dat.06-07-17_09h32m52s

DAQ chio e748 chio /data/e748chioX/e748chio/acquisition/run
-r--r--r-- 1 e748chio Exxx 757K 6 juil. 09:43 run_0339.dat.06-07-17_09h32m40s
===========================================================================================
General DAQ e748
Run 340 6 JULY start 9h47 STOP ...
-rw-r--r-- 1 e748 Exxx 506M 6 juil. 11:07 run_0340.dat.06-07-17_09h47m43s

DAQ chio e748 chio
-rw-r--r-- 1 e748chio Exxx 6,2M 6 juil. 11:12 run_0340.dat.06-07-17_09h47m33s
===========================================================================================
Fixed, e748, Thu Jul 6 10:35:50 2017, Valérie L, MUST2, General, CD2, Sili-1/T5 trip 
The Sili1 of T5 has tripped (again) at 10h30.
On again. 400V; I ~ 20µA at 10h40, still increasing 25.6µA at 11h30.(The limit Io is set at 26µA)
To regulate this detector the Vo set is put at 380V instead of 400V. Now I ~ 24.4µA.
We wil check that the current is stabilized below 24µA, and after the V will be set again at 400V.
Fixed, e748, Thu Jul 6 10:39:09 2017, BéaF & ValérieL - shift U, OFFLINE, General, , Test of merge on Run 339 run339_ICEdiff_LTSDIff_TrigM2.pngrun339_TAC_LTSDIff_TrigM2.png
spectra of the merge checked by Béa.
The merge is correct on the correlations cf plots below.

To Check Merger use:

ssh -X e748chio@ganp976
cd Merge_CHIO
./ConvertRun.sh runnumber

Roughly 30% of events are merge.

Note that we had to include a new condition in the loop of Merge.C in order to avoid the loop to get stuck with the multihits at the end of the run.

>>while( TMath::Abs(t1-t2-ftime_center) <= ftime_window && i1<Entries_numexo && i2<Entries_vme)



Then to check the correlations:
Go to:

cd Merger/merge
Open with ROOT the last file that was created with the runnumber and Draw
merger->Draw("TPLCHIO:LTSDiff>>a(400,-150.,150.,400,10.,8000.)","","colz");

This is to verify that the Driftime is correlated

Also Draw
merger->Draw("IC_E-IC_E_ANA:LTSDiff>>a(400,-150.,120.,400,-18000.,1500.)","","colz");
This is to verify that the difference gives a horizontal line centred at zero.

Save the plots and put them in the elog.


Validated by Antoine Lemasson
Fixed, e748, Thu Jul 6 11:41:20 2017, Valérie L, DAQ, General, N/A, MEMO START/STOP + AGAVA configuration in DAS AfterRebootAgavaRegStatusOx80entréeAfterAlignt.pngDasVMEChioCheckSamplesNumber.pngRunControl748On.pngRunControlNarvalE748chioON.pngParameterListWatcherOnRCe748chio.png
START/STOP of the DAQ
---------------------------------------------------------------------
START acquisition e748chio (DAQ Numexo; in the Run Control display on ganp976)
START acquisition e748 (in the Run Control of 748)
....do your quiet RUN (cf NB1). check it is incremented ll -h
--> on DAQ e748 /data/e748X/e748/acquisition/run
--> on DAQ e748chio /data/e748chioX/e748chio/acquisition/run
........
STOP acquisition e748
STOP acquisition e748chio

DATA RYTHM:
During 3h45 (with the beam rate at 45k /s on CATS)
data run size for e748 is 1Go and for e748chio the corresponding run size is 15Mo.

Plots: Narval Run control for E748, RC for e748chio
Parameter checking of buffer in Numexo (-->RUn Control of e748chio --> clock right on Watcher --> Paremeter list -->window refresh and check the incrementation of the buffers)
---------------------------------------------------------------------
[ Cas où l'acquisition est "tombée en marche" (GAP dixit) il faut procéder à un reset et contrôler ].

Checking of the AGAVA/NUMEXO configuration
In case of a REBOOT of the AGAVA VXI/VME crate (reboot command in the AGAVA terminal for instance), we need:
i) to load again the setup, ii) to check it on the DAS window of the general DAQ.

To load the setup, go and connect to the terminal e748chio (best is to do it directly by going to the computer Numexo, room close to DAQ D6; otherwise do it by remote control)
launch the setup command:
> setupGTS
> configure_gts_numexo2-94.csh

checking on the DAS window
Then go to the DAS of the general experiment e748 and check the status of the VME AGAVA
- Put the DAS ONLINE, the window appears: SOFTWARE / HARDWARE /CANCEL
click on CANCEL if the das sofware was already correctly set (check the MUST2 configuration before going to ONLINE)

1. check the register status, initially we may find it at 0 due to the setup command, it should be written at 0x80 (picture below).
2. the sampling numbers should be correct see first plot, sample number = 120; sample number before stop = 60.

-----------------------------------------------------------------------------------------
run control Numexo is opened by connecting on the numexo machine by:
ssh -X e748chio@ganp976
then start_acq e748chio



NB1: you may check that the header of the run is correct by looking at it with Ganil_Tape:
Terminal
cd /data/e748X/e748/acquisition/run
ll -h
Ganil_Tape run_03***
Fixed, e748, Thu Jul 6 13:21:49 2017, Daisuke, RUN, Hardware, CD2, RUN340 
During RUN340

13h10 The beam intensity dropped from 35 kpps to nearly zero. Iulian called PCP. They said they found the beam alignment was a bit off, so they were trying to realign the beam to get back the axis. We kept running the acquisition.
13h20 Returned to 35 kpps.
Fixed, e748, Thu Jul 6 13:33:23 2017, e748, RUN, General, CD2, Stop of Run 340 due to beam problems 
The run 340 was stopped at 13h32. Before the stop fo the run the beam was unstable and stopped by machine team.

General DAQ e748
r--r--r-- 1 e748 Exxx 1000M 6 juil. 12:33 run_0340.dat.06-07-17_09h47m43s
-r--r--r-- 1 e748 Exxx 264M 6 juil. 13:32 run_0340.dat.06-07-17_09h47m43s.1

DAQ CHIO
-r--r--r-- 1 e748chio Exxx 15083577 6 juil. 13:32 run_0340.dat.06-07-17_09h47m33s
Fixed, e748, Thu Jul 6 13:37:22 2017, Valérie L, RUN, General, CD2, Run 341 ScalersRun341Beginning.pngScalersRun341At14h35.png
Run 341 START AT 13h37 STOP AT 15h21 (beam stop see the ELOG entry Id=239)
At the beginning (cf scalers below) the beam was at 6µA (--> CATS at 30kcps/s), the machine needs to recover slowly to go to 7eµA.
(run checked with Ganil_Tape ok)
-r--r--r-- 1 e748 Exxx 433M 6 juil. 15:21 run_0341.dat.06-07-17_13h35m20s

DAQ CHIO
-r--r--r-- 1 e748chio Exxx 4,9M 6 juil. 15:21 run_0341.dat.06-07-17_13h35m04s

===========================================================================================
Fixed, e748, Thu Jul 6 13:37:37 2017, Adrien Matta, OFFLINE, General, N/A, Energy loss table for d, 3He and 3H 20x
Here it is!
Fixed, e748, Thu Jul 6 14:07:20 2017, Bea, OFFLINE, General, CD2, Convert & Merge Numexo Runs in ganp166 
To Convert run DAQ_general

offline
npp e748
cd offline
Offline /data/e748X/e748acquisition/run/name of run to conver /data/e748X/e748/acquisition/root/name of run after


Then to convert Numexo runs: We have set up a new directory to convert Numexo runs without having to log in into e748chio account.

For regular conversions please use the following commands:

Go to:

/home/e748/analysis/nptool/Projects/e748/GRU_NUMEXO

GRU+

GRU "GruConvTree.C(run_number)"

To Merge:

Go to :

/home/e748/analysis/nptool/Projects/e748/merger


root -l "Merge.C(run_number_numexo,run_number_daq_normal)"
Fixed, e748, Thu Jul 6 15:24:25 2017, e748-shift U, BEAM, General, CD2, Beam stop, end of run 341 
The beam was stopped due to vacuum problem at CSS2.
End of run 341 at 15h21.
Fixed, e748, Thu Jul 6 17:05:51 2017, Valérie L, RUN, General, CD2, run 342 +overview of TAC ScalersRun342at17h10.pngCorrelDatTrigGatconfRun342.pngCorrelTpulserChiorun342.pngRun342_Tcats12.pngRun342_TACcats1.png
Beam recovered after shut down due to the vacuum problem (power supply on the vacuum system was off). Beam was off during 1h30.

Run 342 started at 17h04
run DAQ 748 run_0342.dat.06-07-17_17h04m50s
-r--r--r-- 1 e748 Exxx 561M 6 juil. 18:58 run_0342.dat.06-07-17_17h04m50s
DAQ chio
/data/e748chioX/e748chio/acquisition/run
-r--r--r-- 1 e748chio Exxx 254K 6 juil. 19:01 run_0342.dat.06-07-17_17h04m27s

Stopped at 18:59 to close roof windows of the accelerator...
(they were opened today to give some "fresh air" above the machine sector)
Beam is stopped a priori during 1/4 hour.
Fixed, e748, Thu Jul 6 19:17:12 2017, e748 MA, RUN, General, CD2, RUN 343 
Run 343
same conditions as run 342
beam=7uA

MUST2
-rw-r--r-- 1 e748 Exxx 2,2M 6 juil. 19:16 run_0343.dat.06-07-17_19h15m59s

Chio
-rw-r--r-- 1 e748chio Exxx 1,6K 6 juil. 19:16 run_0343.dat.06-07-17_19h15m50s
Fixed, e748, Thu Jul 6 20:07:46 2017, Daisuke + Thomas R+ Valérie L, CHIO, General, CD2, Problem of counting on Chio ScalersRun343At20h09.png
It seems that the ChIo is counting at a very low rate, around 100/s instead of 5000/s in previous runs.
Thomas R is called at 20h to fix the problem.
See scalers of Run 343 at 20h09.

Run stopped at 20:32, there is no bias on the cathode of the Chio, Thomas is entering the room.
HV on cathode put on again. CHIO should work better now...
MUST2 run
rw-r--r-- 1 e748 Exxx 367M 6 juil. 20:31 run_0343.dat.06-07-17_19h15m59s
Chio Run
-r--r--r-- 1 e748chio Exxx 159K 6 juil. 20:31 run_0343.dat.06-07-17_19h15m50s
Fixed, e748, Thu Jul 6 20:46:15 2017, Thomas R + shift U (Valérie L) +S (Daisuke, Marlène, Shumpei) , RUN, General, CD2, Run 345 DAQ+344 chio after chio problem ScalersRun345at20h54.png
The HV on cathode is now set again at 2900V. At 20h50 we get back the beam for the run.
(Grid still ok at 920V)
Beam problem cf the scalers. Losing beam somewhere.

DAQ e748
-rw-r--r-- 1 e748 Exxx 640K 6 juil. 20:51 run_0345.dat.06-07-17_20h50m01s
DAQ echio
-rw-r--r-- 1 e748chio Exxx 2,8K 6 juil. 20:50 run_0344.dat.06-07-17_20h49m34s
Fixed, e748, Thu Jul 6 21:27:52 2017, Thomas R + shift U (Valérie L) +S (Daisuke, Marlène, Shumpei) , RUN, General, CD2, Run 345 beam intensity loss 
The beam rate was low after intervention with the CHIO HV.

Thomas contacted Jean-Charles and PCP. It turned out that some quadrupoles (Q31, Q34, Q21) of LISE were turned off. Thomas put correct parameters and turned on the magnet.

21h25 full beam.
run 345 daq e748 run 344 e748chio was ongoing during the problem of Qpole. We stop at 21h35 and realign run numbers.
Fixed, e748, Thu Jul 6 21:35:29 2017, Thomas R + shift U (Valérie L) +S (Daisuke, Marlène, Shumpei) , RUN, General, CD2, Run 346 e748+e748chio 6x

Run 346 started at 21h37
Beam around 7µA CATS 2 46k
22h00 beam lost
22h08 beam back 6 µA. CATS 34 kpps.
22h32 beam 7.5 µA. Very stable since then.

We have checked the CATS reconstruction and the Chio spectra (with the CheckPID macro) and they look OK (see attachement).
However the signal from the CHIO looks nice in the oscilloscope but not in the online (see spectra attached). Some events sound good, some are flat... to be checked


DAQ 748
/data/e748X/e748/acquisition/run
e748 Exxx 5,5M 6 juil. 21:39 run_0346.dat.06-07-17_21h37m36s

DAQ e748chio
/data/e748chioX/e748chio/acquisition/run
e748chio Exxx 1,6K 6 juil. 21:37 run_0346.dat.06-07-17_21h37m14s
Fixed, e748, Fri Jul 7 00:25:16 2017, e748, MUST2, Software, CD2, Calibration for CsI 
Rough energy calibration for CsI was performed (only for MM1 and MM4).

Calibration file is at
/home/e748/analysis/nptool/Projects/e748/Calibration/Energy/Calibration/CsI_test/

ELOG Home Page