justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271059.0@dunegpschedd01.fnal.gov

Jobsub ID271059.0@dunegpschedd01.fnal.gov
Workflow ID11434
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-31 08:47:39
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2025-12-31 12:22:16
From worker nodeHostnametds407.pic.es
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-31 08:49:44
Input filesmonte-carlo-011434-000001
JobscriptExit code0
Real time3h (12702s)
CPU time3h (12114s = 95%)
Max RSS bytes3980484608 (3796 MiB)
Outputting started2025-12-31 12:21:28
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11434/1/001/reco_stage1.root
Finished2025-12-31 12:22:16
Saved logsjustin-logs:271059.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

pService:prepare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 76155 traces, 0 summary
Retagger: tagging trace set: gauss with 59207 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 59207 traces tagged "gauss"
FrameSaver: q=2.07304e+06 n=1094493 tag=gauss
wclsFrameSaver: saving 76155 traces tagged "wiener"
FrameSaver: q=2.17473e+06 n=975806 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 11633 XUs and 9929 XVs -> 73 XUVs
C:0 T:1 18378 XUs and 17314 XVs -> 1328 XUVs
C:0 T:2 43239 XUs and 66181 XVs -> 1644 XUVs
C:0 T:3 25570 XUs and 29260 XVs -> 184 XUVs
C:0 T:4 12653 XUs and 11436 XVs -> 99 XUVs
C:0 T:5 42843 XUs and 35480 XVs -> 3830 XUVs
C:0 T:6 30409 XUs and 31105 XVs -> 969 XUVs
C:0 T:7 17315 XUs and 15788 XVs -> 39 XUVs
C:0 T:8 5467 XUs and 5164 XVs -> 47 XUVs
C:0 T:9 24442 XUs and 29665 XVs -> 2295 XUVs
C:0 T:10 33747 XUs and 40607 XVs -> 6156 XUVs
C:0 T:11 307 XUs and 247 XVs -> 32 XUVs
16696 XUVs total
6865 collection wire objects
15891 potential space points
Neighbour search...
1161189 tests to find 534014 neighbours
Iterating with no regularization...
Begin: 3.58963e+07
0 3.09082e+07
1 3.06287e+07
2 3.05946e+07
3 3.05863e+07
Now with regularization...
Begin: 2.50289e+07
0 2.50169e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411451483
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411451483
fLifetime: 30000 [us]
31-Dec-2025 13:21:22 CET  Closed output file "reco_stage1.root"
31-Dec-2025 13:21:22 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          287.889       436.142       561.29        442.978       76.8185        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00142834    0.00381476    0.00647393    0.00398911     0.0014557       10     
reco:ophit:OpHitFinder                             0.244405      0.323335      0.396018       0.3328       0.0500979       10     
reco:opflash:OpFlashFinder                         0.125981      0.162188      0.222121       0.16212      0.0287703       10     
reco:opslicer:OpSlicer                             0.565773      0.874598       1.20959      0.871215      0.190439        10     
reco:caldata:DataPrepModule                         5.33147       7.25957       8.12301       7.42524      0.747008        10     
reco:wclsdatasp:WireCellToolkit                     42.0337       45.1967       49.6039       45.2394       2.06437        10     
reco:gaushit:GausHitFinder                          0.77981       1.44296       1.72103       1.54284      0.264168        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000355265   0.000646196   0.000834511   0.000656164   0.000138431      10     
reco:reco3d:SpacePointSolver                        4.09768       15.8869       28.8246       16.1894       7.3455         10     
reco:hitpdune:DisambigFromSpacePoints              0.791082       1.5328        2.22159       1.58781      0.389822        10     
reco:pandora:StandardPandora                         15.6         32.5834       56.5587       31.199        10.9403        10     
reco:pandoraWriter:StandardPandora                 0.310857      0.485955      0.708096      0.479993      0.111507        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.7559        4.12133       7.47388       4.08498       1.50341        10     
reco:pandoraShower:LArPandoraShowerCreation         1.52261       4.54129       7.86923       4.53042       1.94305        10     
reco:pandoracalo:Calorimetry                       0.975031       2.36438       4.20911       2.4564       0.907922        10     
reco:pandoracalonosce:Calorimetry                   1.03254       2.21503       3.89697       2.23384      0.855504        10     
reco:pandorapid:Chi2ParticleID                    0.00438613    0.00747387     0.0108609    0.00749262    0.00188205       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0437556     0.0715767     0.0948001     0.0752549     0.0182325       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00483285    0.00733937    0.00912986    0.00721036    0.00131453       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.18462       4.58215       7.43865       4.19752       1.97912        10     
reco:pandoraShowercalonosce:ShowerCalorimetry      0.976945       4.39975       7.70436       4.20251       1.93914        10     
reco:emtrkmichelid:EmTrackMichelId                  183.866       263.554       316.154       268.841       39.3913        10     
reco:crttag:SingleCRTMatchingProducer              0.0095456     0.0154297     0.0203635     0.0146602    0.00328762       10     
reco:crtreco:TwoCRTMatchingProducer               0.00628937     0.0107349     0.0139408     0.0107214    0.00254516       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.721438      0.992017       1.26922       1.02679      0.173943        10     
reco:pandora2Track:LArPandoraTrackCreation          5.42965       13.0211       18.7599       13.057         3.806         10     
reco:pandora2calo:Calorimetry                       2.15558       5.87779       9.75428       5.87828       2.08724        10     
reco:pandora2calonosce:Calorimetry                  2.54966       5.71925       9.93962       5.6311        2.25713        10     
reco:pandora2pid:Chi2ParticleID                    0.0128064     0.0178228     0.0242592     0.0188149    0.00339091       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0569349     0.0854753     0.123561      0.0919056     0.021257        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0098968     0.0171332     0.0234727     0.0182822    0.00392201       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.15693       4.84205       8.14673       4.98206       1.69665        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.00729       5.31712       8.37706       5.23596       1.88533        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.31478       5.00028       7.82491       5.07601       1.62325        10     
[art]:TriggerResults:TriggerResultInserter        4.6711e-05    7.47389e-05   0.000138031   7.05755e-05   2.36419e-05      10     
end_path:out1:RootOutput                            8.9e-06     1.22622e-05    1.596e-05    1.2465e-05    1.97326e-06      10     
end_path:out1:RootOutput(write)                     1.93393       3.47529       4.33994       3.71866      0.690423        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4805.87 MB
  Peak resident set size usage (VmHWM): 3980.48 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.31 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.57 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 8837664
-rw-r--r-- 1 dune001 dune 2125454269 Dec 31 12:05 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2022146412 Dec 31 12:02 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 1752642069 Dec 31 13:21 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1576315712 Dec 31 10:09 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1512161367 Dec 31 09:58 g4_stage1.root
-rw-r--r-- 1 dune001 dune   60412680 Dec 31 13:17 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     242668 Dec 31 13:21 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Dec 31 13:21 mem.db
-rw-r--r-- 1 dune001 dune      62376 Dec 31 13:21 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Dec 31 13:21 time.db
-rw-r--r-- 1 dune001 dune        519 Dec 31 12:05 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Dec 31 10:09 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Dec 31 13:21 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Dec 31 09:50 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Dec 31 13:21 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Dec 31 12:13 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Dec 31 12:05 debugprod.log
justIN time: 2026-02-07 00:51:47 UTC       justIN version: 01.06.00