justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 250015.0@dunegpschedd02.fnal.gov

Jobsub ID250015.0@dunegpschedd02.fnal.gov
Workflow ID10539
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-23 12:38:51
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce508
Last heartbeat2025-11-23 15:09:10
From worker nodeHostnameb9p06p6630.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-23 12:39:46
Input filesmonte-carlo-010539-000001
JobscriptExit code0
Real time2h (8886s)
CPU time2h (8447s = 95%)
Max RSS bytes4019462144 (3833 MiB)
Outputting started2025-11-23 15:07:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10539/1/001/reco_stage1.root
Finished2025-11-23 15:09:10
Saved logsjustin-logs:250015.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

inder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 119710 traces, 0 summary
Retagger: tagging trace set: gauss with 91459 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 91459 traces tagged "gauss"
FrameSaver: q=3.15284e+06 n=1607976 tag=gauss
wclsFrameSaver: saving 119710 traces tagged "wiener"
FrameSaver: q=3.31415e+06 n=1401641 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 24373 XUs and 29206 XVs -> 299 XUVs
C:0 T:1 58314 XUs and 95238 XVs -> 7720 XUVs
C:0 T:2 103480 XUs and 74656 XVs -> 5595 XUVs
C:0 T:3 45767 XUs and 29356 XVs -> 321 XUVs
C:0 T:4 12680 XUs and 12151 XVs -> 137 XUVs
C:0 T:5 63110 XUs and 80796 XVs -> 21191 XUVs
C:0 T:6 60127 XUs and 57462 XVs -> 2463 XUVs
C:0 T:8 9625 XUs and 6741 XVs -> 78 XUVs
C:0 T:9 58733 XUs and 63325 XVs -> 3653 XUVs
C:0 T:10 31734 XUs and 23443 XVs -> 3744 XUVs
C:0 T:11 345 XUs and 258 XVs -> 34 XUVs
45235 XUVs total
11204 collection wire objects
42345 potential space points
Neighbour search...
9494939 tests to find 3883684 neighbours
Iterating with no regularization...
Begin: 5.59329e+07
0 4.45588e+07
1 4.39442e+07
2 4.38667e+07
3 4.38455e+07
Now with regularization...
Begin: 3.50792e+07
0 3.50329e+07
1 3.49995e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410690250
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410690250
fLifetime: 30000 [us]
23-Nov-2025 16:07:51 CET  Closed output file "reco_stage1.root"
23-Nov-2025 16:07:51 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          159.462       213.544       254.514       209.871       28.2033        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00104637    0.00298987    0.00490735    0.00310668    0.00122598       10     
reco:ophit:OpHitFinder                             0.187727      0.212258      0.245216      0.205343      0.0190082       10     
reco:opflash:OpFlashFinder                         0.100738      0.128832      0.177825      0.128988      0.022569        10     
reco:opslicer:OpSlicer                             0.376657      0.448008       0.53568       0.43345      0.0577276       10     
reco:caldata:DataPrepModule                         3.74637       3.8343        4.09372       3.81018      0.0973172       10     
reco:wclsdatasp:WireCellToolkit                     24.2895       25.4954       26.5124       25.3912      0.760792        10     
reco:gaushit:GausHitFinder                         0.555216       0.78973      0.942669      0.784474      0.101492        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000271735   0.000366269   0.000513329   0.000354821   5.85834e-05      10     
reco:reco3d:SpacePointSolver                        7.17746       10.5275       17.7663       9.84463       3.06019        10     
reco:hitpdune:DisambigFromSpacePoints              0.648122      0.993052       1.27985      0.974262      0.190408        10     
reco:pandora:StandardPandora                        10.1356       18.0154       28.2349       17.4237       5.08605        10     
reco:pandoraWriter:StandardPandora                 0.219436      0.275188      0.327531      0.264362      0.0341455       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.43705       2.04313       2.74178       2.01499      0.413203        10     
reco:pandoraShower:LArPandoraShowerCreation         1.14265       2.25162       3.21888       2.13768      0.570099        10     
reco:pandoracalo:Calorimetry                       0.799485       1.27254       1.62912       1.28087      0.265277        10     
reco:pandoracalonosce:Calorimetry                   0.73206       1.17418       1.55355       1.18934      0.265686        10     
reco:pandorapid:Chi2ParticleID                     0.0028736     0.0035662    0.00490934    0.00353653    0.00052389       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0279707     0.0345495     0.0416889     0.0331419    0.00410975       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00280719    0.00332268    0.00377269    0.00340956    0.00033066       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.19642       2.55304       3.74422       2.44826      0.748119        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.16574       2.47719       3.72403       2.39732      0.720689        10     
reco:emtrkmichelid:EmTrackMichelId                  90.3162       116.079       135.286       113.206       13.1024        10     
reco:crttag:SingleCRTMatchingProducer             0.00637057    0.00858346     0.0110419    0.00845414    0.00121981       10     
reco:crtreco:TwoCRTMatchingProducer               0.00393789    0.00547979    0.00660983    0.00543705    0.000743294      10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.48953      0.665411      0.888167      0.642689      0.118541        10     
reco:pandora2Track:LArPandoraTrackCreation          3.86689       6.59846       9.41656       6.34353       1.60526        10     
reco:pandora2calo:Calorimetry                       2.00055       3.59219       5.12634       3.40504      0.931642        10     
reco:pandora2calonosce:Calorimetry                  1.88745       3.42035       4.86725       3.25791      0.894024        10     
reco:pandora2pid:Chi2ParticleID                   0.00647137    0.00792702    0.00964432    0.00769177    0.000854557      10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0352802     0.0418168     0.0509785     0.0412112    0.00462355       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.006532     0.00784288    0.00963951    0.00766041    0.000859696      10     
reco:pandora2Shower:LArPandoraShowerCreation        1.48144       2.48135       3.46013       2.46723      0.563062        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.74122       3.15567       4.60579       3.05374      0.849521        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.58924       2.91681       4.21612       2.81897      0.782289        10     
[art]:TriggerResults:TriggerResultInserter        3.3231e-05    4.72739e-05   0.000103431   4.2376e-05    1.91265e-05      10     
end_path:out1:RootOutput                           9.02e-06     1.18532e-05    1.471e-05    1.24155e-05   1.68705e-06      10     
end_path:out1:RootOutput(write)                     1.52494       1.94955       2.32019       1.91256      0.246394        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4842.48 MB
  Peak resident set size usage (VmHWM): 4019.46 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.41 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9320712
-rw-r--r--. 1 duneprd np-comp 2211305465 Nov 23 15:31 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2099171665 Nov 23 15:29 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 1882546332 Nov 23 16:07 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1668921920 Nov 23 13:54 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1611061718 Nov 23 13:46 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   70785900 Nov 23 16:05 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242729 Nov 23 16:07 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Nov 23 16:07 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Nov 23 16:07 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Nov 23 16:07 time.db
-rw-r--r--. 1 duneprd np-comp        519 Nov 23 15:31 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Nov 23 13:54 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Nov 23 16:07 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Nov 23 13:39 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Nov 23 16:07 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Nov 23 15:34 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Nov 23 15:31 debugprod.log
justIN time: 2026-02-08 21:30:08 UTC       justIN version: 01.06.00