justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269488.0@dunegpschedd01.fnal.gov

Jobsub ID269488.0@dunegpschedd01.fnal.gov
Workflow ID11296
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-21 09:33:10
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce04
Last heartbeat2025-12-21 14:00:42
From worker nodeHostnameccwcondor0051
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-21 09:44:03
Input filesmonte-carlo-011296-000001
JobscriptExit code0
Real time4h (15271s)
CPU time4h (14895s = 97%)
Max RSS bytes4441612288 (4235 MiB)
Outputting started2025-12-21 13:58:35
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11296/1/001/reco_stage1.root
Finished2025-12-21 14:00:42
Saved logsjustin-logs:269488.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nning tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 173958 traces, 0 summary
Retagger: tagging trace set: gauss with 136456 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 136456 traces tagged "gauss"
FrameSaver: q=4.73118e+06 n=2497878 tag=gauss
wclsFrameSaver: saving 173958 traces tagged "wiener"
FrameSaver: q=4.99882e+06 n=2233382 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 48904 XUs and 50083 XVs -> 572 XUVs
C:0 T:1 106267 XUs and 115414 XVs -> 8051 XUVs
C:0 T:2 137782 XUs and 160426 XVs -> 7669 XUVs
C:0 T:3 55445 XUs and 58553 XVs -> 599 XUVs
C:0 T:4 13901 XUs and 16415 XVs -> 139 XUVs
C:0 T:5 55370 XUs and 73721 XVs -> 4860 XUVs
C:0 T:6 189304 XUs and 254139 XVs -> 16167 XUVs
C:0 T:7 65425 XUs and 70382 XVs -> 981 XUVs
C:0 T:8 10152 XUs and 8885 XVs -> 125 XUVs
C:0 T:9 79615 XUs and 95370 XVs -> 6667 XUVs
C:0 T:10 81114 XUs and 52443 XVs -> 5618 XUVs
C:0 T:11 576 XUs and 386 XVs -> 116 XUVs
51564 XUVs total
17415 collection wire objects
46252 potential space points
Neighbour search...
4588806 tests to find 2083492 neighbours
Iterating with no regularization...
Begin: 1.00052e+08
0 8.2421e+07
1 8.11055e+07
2 8.09275e+07
3 8.08808e+07
Now with regularization...
Begin: 6.79447e+07
0 6.79024e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411250920
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411250920
fLifetime: 30000 [us]
21-Dec-2025 14:58:33 CET  Closed output file "reco_stage1.root"
21-Dec-2025 14:58:33 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          401.39        501.829       684.838       488.138       90.2766        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00130836    0.00429699    0.00675499    0.00451753    0.00173754       10     
reco:ophit:OpHitFinder                             0.324156      0.366701      0.409424      0.367243      0.0239544       10     
reco:opflash:OpFlashFinder                         0.201105      0.233367      0.272405      0.232824      0.0232363       10     
reco:opslicer:OpSlicer                             0.598431      0.838474      0.974685      0.859811      0.116382        10     
reco:caldata:DataPrepModule                         5.42845       5.85409       6.41108       5.89235      0.294148        10     
reco:wclsdatasp:WireCellToolkit                     32.3056       35.7529       38.4813       35.7388       2.08872        10     
reco:gaushit:GausHitFinder                          1.18493       1.70085       2.04813       1.77263      0.272177        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000496653   0.000654127   0.000853225   0.000636815   0.000113231      10     
reco:reco3d:SpacePointSolver                        15.5713       24.1221       34.1338       23.0038       6.57934        10     
reco:hitpdune:DisambigFromSpacePoints               1.84087       2.56125       3.35417       2.6967       0.521704        10     
reco:pandora:StandardPandora                        36.7688       57.0343       85.4382       52.265        16.9682        10     
reco:pandoraWriter:StandardPandora                 0.527904      0.653722      0.895784      0.665622      0.106411        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.28079       7.59432       13.1065       6.76808       2.38665        10     
reco:pandoraShower:LArPandoraShowerCreation         5.22984       9.46408       20.2908       7.9341        4.63883        10     
reco:pandoracalo:Calorimetry                        2.94043       4.14404       7.70056       3.76241       1.37297        10     
reco:pandoracalonosce:Calorimetry                   2.68781       3.85832       6.97308       3.52821       1.24408        10     
reco:pandorapid:Chi2ParticleID                    0.00578252    0.00732819    0.00893584    0.00738663    0.00109798       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0549383     0.0748224     0.0956612     0.0732692     0.0112464       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00597186    0.00709651    0.00900349    0.00703525    0.000936881      10     
reco:pandoraShowercalo:ShowerCalorimetry            6.30858       8.92022       13.9618       8.2395        2.62192        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.24175       8.81085       13.5421       8.01407       2.50447        10     
reco:emtrkmichelid:EmTrackMichelId                  212.059       251.35        316.213       247.735       32.9249        10     
reco:crttag:SingleCRTMatchingProducer              0.0130393     0.0207142     0.0327161     0.0194757    0.00665293       10     
reco:crtreco:TwoCRTMatchingProducer               0.00867899     0.0128731     0.0165712     0.0127271    0.00249349       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.938771       1.11496       1.36258       1.08812      0.127629        10     
reco:pandora2Track:LArPandoraTrackCreation          17.3472       24.6355       40.4795       22.3366       7.76287        10     
reco:pandora2calo:Calorimetry                       8.36822       11.9969       18.9583       11.8511       3.43311        10     
reco:pandora2calonosce:Calorimetry                  7.74663       11.4419       18.4882       10.3081       3.56229        10     
reco:pandora2pid:Chi2ParticleID                    0.0135215     0.0190107     0.0230439     0.0192162    0.00285159       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0586794     0.0986416     0.136478      0.096988      0.0199429       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0125464     0.0183418     0.0232348     0.0173371    0.00293677       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.03224       8.15639       13.1262       7.15782       2.75872        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.33376       9.22298       14.112        8.66415       2.49049        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.50911       7.92065       12.1419       7.37744       1.96441        10     
[art]:TriggerResults:TriggerResultInserter         5.324e-05    7.86491e-05   0.00018509     6.608e-05    3.69312e-05      10     
end_path:out1:RootOutput                          1.1838e-05    1.61415e-05   3.5464e-05    1.3766e-05    6.64321e-06      10     
end_path:out1:RootOutput(write)                     3.00393       3.61997       4.53052       3.68123      0.498533        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5301.92 MB
  Peak resident set size usage (VmHWM): 4441.61 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.31 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.60 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 12056844
-rw-r--r-- 1 dune001 lbno 2760363155 Dec 21 13:32 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2619789486 Dec 21 13:30 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2520757348 Dec 21 14:58 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 2215689447 Dec 21 11:07 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 2135512736 Dec 21 10:51 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   93481430 Dec 21 14:49 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242894 Dec 21 14:58 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Dec 21 14:58 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Dec 21 14:58 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Dec 21 14:58 time.db
-rw-r--r-- 1 dune001 lbno        519 Dec 21 13:32 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Dec 21 11:07 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Dec 21 14:58 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Dec 21 10:44 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Dec 21 14:58 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Dec 21 13:39 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Dec 21 13:32 debugprod.log
justIN time: 2026-02-07 06:27:40 UTC       justIN version: 01.06.00