justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 233497.0@dunegpschedd02.fnal.gov

Jobsub ID233497.0@dunegpschedd02.fnal.gov
Workflow ID9403
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-28 06:21:51
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2025-10-28 10:23:04
From worker nodeHostnamecn003.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-28 06:22:33
Input filesmonte-carlo-009403-000001
JobscriptExit code0
Real time3h (14223s)
CPU time3h (13971s = 98%)
Max RSS bytes4166791168 (3973 MiB)
Outputting started2025-10-28 10:19:37
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09403/1/001/reco_stage1.root
Finished2025-10-28 10:23:04
Saved logsjustin-logs:233497.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 109055 traces, 0 summary
Retagger: tagging trace set: gauss with 79061 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 79061 traces tagged "gauss"
FrameSaver: q=2.64746e+06 n=1311566 tag=gauss
wclsFrameSaver: saving 109055 traces tagged "wiener"
FrameSaver: q=2.83633e+06 n=1131314 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 17804 XUs and 19522 XVs -> 103 XUVs
C:0 T:1 34087 XUs and 38668 XVs -> 3302 XUVs
C:0 T:2 121363 XUs and 112600 XVs -> 6375 XUVs
C:0 T:3 45812 XUs and 39076 XVs -> 349 XUVs
C:0 T:4 9407 XUs and 9212 XVs -> 78 XUVs
C:0 T:5 29254 XUs and 23376 XVs -> 5705 XUVs
C:0 T:6 170197 XUs and 99335 XVs -> 9338 XUVs
C:0 T:7 48868 XUs and 35579 XVs -> 369 XUVs
C:0 T:8 4206 XUs and 4342 XVs -> 16 XUVs
C:0 T:9 21772 XUs and 31553 XVs -> 1731 XUVs
C:0 T:10 34301 XUs and 11441 XVs -> 3897 XUVs
C:0 T:11 219 XUs and 145 XVs -> 29 XUVs
31292 XUVs total
10184 collection wire objects
28706 potential space points
Neighbour search...
2184396 tests to find 1143158 neighbours
Iterating with no regularization...
Begin: 4.68617e+07
0 3.40824e+07
1 3.31974e+07
2 3.30368e+07
3 3.29812e+07
4 3.29544e+07
Now with regularization...
Begin: 2.52654e+07
0 2.52274e+07
1 2.52073e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410161961
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410161961
fLifetime: 30000 [us]
28-Oct-2025 10:19:34 GMT  Closed output file "reco_stage1.root"
28-Oct-2025 10:19:34 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          295.822       631.437       1675.99       588.806       373.306        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00155314     0.0466392     0.430231      0.0046124     0.127869        10     
reco:ophit:OpHitFinder                             0.276794      0.361502      0.496105      0.331264      0.0675293       10     
reco:opflash:OpFlashFinder                         0.143752      0.214515      0.288645      0.214997      0.0432475       10     
reco:opslicer:OpSlicer                             0.467266      0.793146       1.09689      0.747806      0.178224        10     
reco:caldata:DataPrepModule                         5.59589       7.45735       9.60632       7.82781       1.2535         10     
reco:wclsdatasp:WireCellToolkit                     43.9828       56.1445       69.8063       57.2839       7.75978        10     
reco:gaushit:GausHitFinder                         0.870517       1.50836       2.16112       1.51313      0.422141        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000578439   0.000904209   0.00146293    0.000817402   0.000296553      10     
reco:reco3d:SpacePointSolver                        6.94683       19.8331       61.9085       14.8019       14.9893        10     
reco:hitpdune:DisambigFromSpacePoints               1.07545       2.43871       4.06488       2.50384       1.03805        10     
reco:pandora:StandardPandora                        14.3076       102.814       705.515       41.9519       201.243        10     
reco:pandoraWriter:StandardPandora                 0.347957      0.578032       1.07002      0.545734      0.199179        10     
reco:pandoraTrack:LArPandoraTrackCreation           7.35692       13.9577       28.3239       13.1426       5.9463         10     
reco:pandoraShower:LArPandoraShowerCreation         5.70172       13.3666       25.5849       13.245        6.00302        10     
reco:pandoracalo:Calorimetry                        3.03232       6.21541       12.8143       5.33894       2.99073        10     
reco:pandoracalonosce:Calorimetry                   2.88975       6.00363       12.2057       5.08602       2.85999        10     
reco:pandorapid:Chi2ParticleID                    0.00478849    0.00811147     0.0139844    0.00828146    0.00264617       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0444226     0.0710577     0.107181      0.0640258     0.0204093       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00453402    0.00728201     0.0118875     0.0070406    0.00229963       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.97924       12.521        25.4349       12.0421       6.64052        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.43105       12.1556       24.9275       12.0501       6.36066        10     
reco:emtrkmichelid:EmTrackMichelId                  123.898       256.861       464.933       285.048       97.7331        10     
reco:crttag:SingleCRTMatchingProducer              0.0206548     0.0313555     0.0442417     0.0312415    0.00716215       10     
reco:crtreco:TwoCRTMatchingProducer                0.0179308     0.0281148     0.0380776     0.0271147    0.00657662       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.828409       1.41403       2.21164       1.36668       0.49082        10     
reco:pandora2Track:LArPandoraTrackCreation          19.3538       40.2746       76.7307       39.6117       16.5699        10     
reco:pandora2calo:Calorimetry                       7.3952        17.4194       36.3854       17.4497       8.09691        10     
reco:pandora2calonosce:Calorimetry                  7.04046       16.4085        33.04        16.5914       7.5277         10     
reco:pandora2pid:Chi2ParticleID                   0.00967005     0.0164422     0.0299489     0.0159327    0.00585264       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.054161      0.0963093     0.201885      0.094069      0.0397281       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00944141     0.0159931     0.0276215     0.0156559    0.00530777       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.77193       12.6371       20.5426       13.5723       4.22081        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.05915       14.2656       28.2129       14.6661       6.36797        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.11243       12.339        25.7309       12.6235       5.72157        10     
[art]:TriggerResults:TriggerResultInserter        3.5719e-05    7.01785e-05   0.000173929   5.70885e-05   3.83776e-05      10     
end_path:out1:RootOutput                           7.177e-06    1.01024e-05   1.6031e-05    8.6895e-06    3.02127e-06      10     
end_path:out1:RootOutput(write)                     1.83776       2.9414        5.33001       2.65355      0.977707        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4963.31 MB
  Peak resident set size usage (VmHWM): 4166.79 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.33 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10173648
-rw-r--r-- 1 pildune22 pildune 2395123973 Oct 28 08:31 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2275419192 Oct 28 08:28 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2061506940 Oct 28 10:19 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 1855309616 Oct 28 06:50 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1759212993 Oct 28 06:30 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   70623540 Oct 28 10:15 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     242943 Oct 28 10:19 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Oct 28 10:19 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Oct 28 10:19 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Oct 28 10:19 time.db
-rw-r--r-- 1 pildune22 pildune        519 Oct 28 08:31 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Oct 28 06:50 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Oct 28 10:19 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Oct 28 06:22 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Oct 28 10:19 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Oct 28 08:56 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Oct 28 08:31 debugprod.log
justIN time: 2025-11-04 01:53:59 UTC       justIN version: 01.05.01