justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 20183.1@dunegpschedd01.fnal.gov

Jobsub ID20183.1@dunegpschedd01.fnal.gov
Workflow ID205
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-08-01 11:28:01
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-08-01 15:12:27
From worker nodeHostnamecn297.htc.esc.qmul
cpuinfoAMD EPYC 7402 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-01 11:29:20
Input filesmonte-carlo-000205-000001
JobscriptExit code0
Real time3h (13171s)
CPU time3h (12364s = 93%)
Max RSS bytes4281364480 (4083 MiB)
Outputting started2025-08-01 15:08:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/00205/1/001/reco_stage1.root
Finished2025-08-01 15:12:27
Saved logsjustin-logs:20183.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

oolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 131438 traces, 0 summary
Retagger: tagging trace set: gauss with 105367 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 105367 traces tagged "gauss"
FrameSaver: q=3.5724e+06 n=1901115 tag=gauss
wclsFrameSaver: saving 131438 traces tagged "wiener"
FrameSaver: q=3.73595e+06 n=1697655 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 42895 XUs and 47646 XVs -> 470 XUVs
C:0 T:1 80709 XUs and 83633 XVs -> 4081 XUVs
C:0 T:2 116303 XUs and 119339 XVs -> 5199 XUVs
C:0 T:3 49814 XUs and 46956 XVs -> 434 XUVs
C:0 T:4 13873 XUs and 11324 XVs -> 89 XUVs
C:0 T:5 54768 XUs and 31901 XVs -> 3824 XUVs
C:0 T:6 108854 XUs and 90839 XVs -> 3917 XUVs
C:0 T:7 46926 XUs and 37395 XVs -> 348 XUVs
C:0 T:8 8001 XUs and 7067 XVs -> 46 XUVs
C:0 T:9 42013 XUs and 49670 XVs -> 2849 XUVs
C:0 T:10 15500 XUs and 8633 XVs -> 1801 XUVs
C:0 T:11 134 XUs and 81 XVs -> 14 XUVs
23072 XUVs total
11975 collection wire objects
21125 potential space points
Neighbour search...
644519 tests to find 346098 neighbours
Iterating with no regularization...
Begin: 7.63996e+07
0 6.76791e+07
1 6.71462e+07
2 6.70855e+07
Now with regularization...
Begin: 5.67781e+07
0 5.67639e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 408381178
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 408381178
fLifetime: 30000 [us]
01-Aug-2025 16:08:50 BST  Closed output file "reco_stage1.root"
01-Aug-2025 16:08:50 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          297.529       385.27        519.651       382.467       61.7623        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000958176   0.00320788    0.00492966    0.00367068    0.00121976       10     
reco:ophit:OpHitFinder                             0.268374      0.315171      0.363015      0.302912      0.0330474       10     
reco:opflash:OpFlashFinder                          0.15639      0.198304      0.252975      0.183302      0.0342734       10     
reco:opslicer:OpSlicer                             0.648734      0.764405      0.916063      0.745208      0.0790383       10     
reco:caldata:DataPrepModule                         5.45042       5.61001       5.83883       5.58952      0.133854        10     
reco:wclsdatasp:WireCellToolkit                     33.7841       38.5206       42.0397       38.6785       2.77143        10     
reco:gaushit:GausHitFinder                         0.986543       1.24636       1.61247       1.26657      0.162907        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000465838   0.000568101   0.000699283   0.000537773   7.58473e-05      10     
reco:reco3d:SpacePointSolver                        8.02086       16.7936       33.5891       13.187        7.81963        10     
reco:hitpdune:DisambigFromSpacePoints               1.13284       1.68012       2.54043       1.72469       0.36549        10     
reco:pandora:StandardPandora                        24.2323       34.4654       56.1233       31.859        9.03348        10     
reco:pandoraWriter:StandardPandora                 0.382283      0.476059       0.60571      0.469551      0.0624418       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.03508       4.35037       5.83669       4.12029      0.976468        10     
reco:pandoraShower:LArPandoraShowerCreation         2.93423       4.80578       7.85766       4.85108       1.41026        10     
reco:pandoracalo:Calorimetry                        1.56298       2.26303       3.13208       2.13491      0.499877        10     
reco:pandoracalonosce:Calorimetry                   1.34886       2.08296       2.73164       2.06556      0.433211        10     
reco:pandorapid:Chi2ParticleID                     0.005372     0.00633865    0.00732829    0.00636777    0.000529524      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0487647     0.0580054     0.0710592     0.0581392    0.00641079       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00512559    0.00594711    0.00698417    0.00586123    0.000565501      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.51451       4.42667       7.09864       4.48033       1.31029        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.61716       4.41979       7.10491       4.52914       1.26557        10     
reco:emtrkmichelid:EmTrackMichelId                  175.449       215.518       274.547       217.448       28.6471        10     
reco:crttag:SingleCRTMatchingProducer              0.0105301     0.0148742     0.019986      0.0143894     0.0025219       10     
reco:crtreco:TwoCRTMatchingProducer               0.00730472     0.0093039     0.0124786    0.00929527    0.00143265       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.836984       1.0032        1.32879      0.986618      0.133116        10     
reco:pandora2Track:LArPandoraTrackCreation          8.82329       14.0928       21.9394       14.0884       3.71744        10     
reco:pandora2calo:Calorimetry                       3.88915       6.51019       9.77663       6.5251        1.53886        10     
reco:pandora2calonosce:Calorimetry                  3.51964       6.02227       8.67838       6.21189       1.44894        10     
reco:pandora2pid:Chi2ParticleID                    0.013063      0.015699      0.0181622     0.0157813    0.00172394       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0587473     0.0729108     0.0895818     0.0741018    0.00868693       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0129504     0.0155844     0.0179335     0.0158037    0.00166265       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.3787        5.29176       7.40541       5.4191        1.20547        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.26485       5.52892       7.55099       5.72434       1.42172        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.02413       5.07532       7.10356       5.22957       1.34009        10     
[art]:TriggerResults:TriggerResultInserter        4.7578e-05    6.03706e-05   0.000119082   5.4206e-05    2.00423e-05      10     
end_path:out1:RootOutput                           8.716e-06    1.19262e-05   2.4736e-05    9.9785e-06    4.51611e-06      10     
end_path:out1:RootOutput(write)                     2.82898        3.479        4.6026        3.44273      0.528379        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5086.38 MB
  Peak resident set size usage (VmHWM): 4281.36 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.49 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10383992
-rw-r--r-- 1 pildune32 56801 2429016600 Aug  1 15:02 detsim_stage2.root
-rw-r--r-- 1 pildune32 56801 2307872404 Aug  1 14:59 detsim_stage1.root
-rw-r--r-- 1 pildune32 56801 2123043033 Aug  1 16:08 reco_stage1.root
-rw-r--r-- 1 pildune32 56801 1885056983 Aug  1 12:58 g4_stage2.root
-rw-r--r-- 1 pildune32 56801 1811550047 Aug  1 12:35 g4_stage1.root
-rw-r--r-- 1 pildune32 56801   76057650 Aug  1 16:03 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 56801     242851 Aug  1 16:08 jobscript.log
-rw-r--r-- 1 pildune32 56801     217088 Aug  1 16:08 mem.db
-rw-r--r-- 1 pildune32 56801      62376 Aug  1 16:08 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 56801      36864 Aug  1 16:08 time.db
-rw-r--r-- 1 pildune32 56801        519 Aug  1 15:02 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 56801        519 Aug  1 12:58 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 56801        519 Aug  1 16:08 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 56801         52 Aug  1 12:29 all-input-dids.txt
-rw-r--r-- 1 pildune32 56801          7 Aug  1 16:08 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 56801          0 Aug  1 15:07 ProtoDUNE.log
-rw-r--r-- 1 pildune32 56801          0 Aug  1 15:02 debugprod.log
justIN time: 2025-08-04 19:41:59 UTC       justIN version: 01.04.00