justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 233052.1@dunegpschedd02.fnal.gov

Jobsub ID233052.1@dunegpschedd02.fnal.gov
Workflow ID9355
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-26 13:20:39
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-10-26 17:35:45
From worker nodeHostnamewn-lb-10.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-26 13:21:26
Input filesmonte-carlo-009355-000001
JobscriptExit code0
Real time4h (15144s)
CPU time4h (14998s = 99%)
Max RSS bytes4030951424 (3844 MiB)
Outputting started2025-10-26 17:33:50
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09355/1/001/reco_stage1.root
Finished2025-10-26 17:35:45
Saved logsjustin-logs:233052.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ing tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 114115 traces, 0 summary
Retagger: tagging trace set: gauss with 87265 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 87265 traces tagged "gauss"
FrameSaver: q=3.06536e+06 n=1533028 tag=gauss
wclsFrameSaver: saving 114115 traces tagged "wiener"
FrameSaver: q=3.22933e+06 n=1356063 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 41153 XUs and 45769 XVs -> 644 XUVs
C:0 T:1 77499 XUs and 68993 XVs -> 8401 XUVs
C:0 T:2 106801 XUs and 121282 XVs -> 19808 XUVs
C:0 T:3 43733 XUs and 41783 XVs -> 1735 XUVs
C:0 T:4 8013 XUs and 9146 XVs -> 80 XUVs
C:0 T:5 30847 XUs and 24906 XVs -> 3674 XUVs
C:0 T:6 89640 XUs and 93644 XVs -> 5933 XUVs
C:0 T:7 34477 XUs and 36947 XVs -> 424 XUVs
C:0 T:8 5810 XUs and 6378 XVs -> 51 XUVs
C:0 T:9 32803 XUs and 50853 XVs -> 2570 XUVs
C:0 T:10 16502 XUs and 12883 XVs -> 3193 XUVs
C:0 T:11 93 XUs and 80 XVs -> 15 XUVs
46528 XUVs total
9390 collection wire objects
39619 potential space points
Neighbour search...
5495265 tests to find 2499426 neighbours
Iterating with no regularization...
Begin: 6.33587e+07
0 5.56835e+07
1 5.50455e+07
2 5.49427e+07
3 5.49109e+07
Now with regularization...
Begin: 4.59624e+07
0 4.59287e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410127534
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410127534
fLifetime: 30000 [us]
26-Oct-2025 18:33:48 CET  Closed output file "reco_stage1.root"
26-Oct-2025 18:33:48 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          346.719       430.719       571.816       418.713       56.4522        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0016621     0.0062383     0.0100269     0.0063057    0.00277158       10     
reco:ophit:OpHitFinder                             0.356335      0.402523       0.43719      0.401987      0.0251834       10     
reco:opflash:OpFlashFinder                         0.216569       0.25545      0.287386      0.250866      0.0259697       10     
reco:opslicer:OpSlicer                             0.682077      0.818572       1.01159      0.819107      0.0996058       10     
reco:caldata:DataPrepModule                         6.35875       6.49737       7.05008       6.4193       0.191011        10     
reco:wclsdatasp:WireCellToolkit                     44.2487       59.8378       66.349        64.4001       8.31308        10     
reco:gaushit:GausHitFinder                          1.08154       1.30449       1.74537       1.29167      0.173049        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000573773   0.00087225    0.00116291    0.000870874   0.000178445      10     
reco:reco3d:SpacePointSolver                        15.8951       22.1669       30.6906       21.7495       5.17952        10     
reco:hitpdune:DisambigFromSpacePoints               1.23977       1.65886       2.56222       1.57211       0.35682        10     
reco:pandora:StandardPandora                        27.1873       39.1009       71.9595       33.185        13.0265        10     
reco:pandoraWriter:StandardPandora                 0.443096       0.50673      0.601821      0.508037      0.0427355       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.74928       6.57999       8.68298       6.36403       1.02269        10     
reco:pandoraShower:LArPandoraShowerCreation         3.92517       7.46973       13.495        7.38746       2.33577        10     
reco:pandoracalo:Calorimetry                        2.49714       3.53859       4.5396        3.51766       0.51922        10     
reco:pandoracalonosce:Calorimetry                   2.53127       3.36649       4.43614       3.2922       0.509924        10     
reco:pandorapid:Chi2ParticleID                    0.00569382    0.00645072    0.00730049    0.00651402    0.000435759      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0464821     0.0556727     0.0632477     0.0566174    0.00474027       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00537721    0.00600808     0.0067022    0.00590457    0.000520526      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.76943       7.61317       13.8774       7.19197       2.47351        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.6001        7.20523       12.1788       6.90322       2.00904        10     
reco:emtrkmichelid:EmTrackMichelId                  169.327       191.884       238.17        191.538       19.061         10     
reco:crttag:SingleCRTMatchingProducer              0.0134849     0.0185797     0.0274571     0.017253     0.00451319       10     
reco:crtreco:TwoCRTMatchingProducer                0.0115033     0.015069      0.0189518     0.0150123    0.00244067       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.883992       1.00809       1.15097       0.98308      0.0852243       10     
reco:pandora2Track:LArPandoraTrackCreation          12.9111       21.6447       33.4081       20.5799       5.22243        10     
reco:pandora2calo:Calorimetry                       6.01191       10.3402       16.3582       10.2478       2.48743        10     
reco:pandora2calonosce:Calorimetry                  5.69801       9.8291        14.3913       9.65819       2.17956        10     
reco:pandora2pid:Chi2ParticleID                    0.0119579     0.0133795     0.0167083     0.0131633    0.00143602       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0566392     0.0700464     0.083495      0.0711744    0.00702104       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0121456     0.0134154     0.0168109     0.0130434     0.0013754       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.81499       7.62277       10.0772       7.11365       1.48715        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.06335       8.75705       13.3473       8.12214       2.30352        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.52042       7.53934       11.291        6.96405       1.90852        10     
[art]:TriggerResults:TriggerResultInserter        4.8873e-05    6.50658e-05   0.000134782   5.52535e-05   2.43424e-05      10     
end_path:out1:RootOutput                           8.242e-06    1.01354e-05   1.6535e-05    9.5245e-06     2.188e-06       10     
end_path:out1:RootOutput(write)                     2.90845       3.31015       4.12549       3.27956      0.351926        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4832.9 MB
  Peak resident set size usage (VmHWM): 4030.95 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.04 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9573368
-rw-r--r--. 1 dune009 dune 2265485081 Oct 26 17:20 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2147915911 Oct 26 17:17 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 1940711469 Oct 26 18:33 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1725584756 Oct 26 14:42 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1650788953 Oct 26 14:28 g4_stage1.root
-rw-r--r--. 1 dune009 dune   72030330 Oct 26 18:29 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243198 Oct 26 18:33 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Oct 26 18:33 mem.db
-rw-r--r--. 1 dune009 dune      62376 Oct 26 18:33 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Oct 26 18:33 time.db
-rw-r--r--. 1 dune009 dune        519 Oct 26 17:20 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 26 14:42 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 26 18:33 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Oct 26 14:21 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Oct 26 18:33 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Oct 26 17:26 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Oct 26 17:20 debugprod.log
justIN time: 2025-11-05 23:36:29 UTC       justIN version: 01.05.01