justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302563.0@dunegpschedd01.fnal.gov

Jobsub ID302563.0@dunegpschedd01.fnal.gov
Workflow ID12610
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-31 08:46:03
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce508
Last heartbeat2026-01-31 11:52:18
From worker nodeHostnameb9p11p9087.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-31 08:46:53
Input filesmonte-carlo-012610-000001
JobscriptExit code0
Real time3h (11046s)
CPU time2h (9784s = 88%)
Max RSS bytes3939389440 (3756 MiB)
Outputting started2026-01-31 11:51:01
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12610/1/001/reco_stage1.root
Finished2026-01-31 11:52:18
Saved logsjustin-logs:302563.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

l adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 148284 traces, 0 summary
Retagger: tagging trace set: gauss with 110754 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 110754 traces tagged "gauss"
FrameSaver: q=3.69455e+06 n=1950335 tag=gauss
wclsFrameSaver: saving 148284 traces tagged "wiener"
FrameSaver: q=3.93235e+06 n=1718431 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 47386 XUs and 31423 XVs -> 546 XUVs
C:0 T:1 145214 XUs and 95505 XVs -> 14241 XUVs
C:0 T:2 72836 XUs and 103655 XVs -> 4805 XUVs
C:0 T:3 37978 XUs and 40952 XVs -> 343 XUVs
C:0 T:4 15056 XUs and 12468 XVs -> 120 XUVs
C:0 T:5 107128 XUs and 53067 XVs -> 13050 XUVs
C:0 T:6 76786 XUs and 144498 XVs -> 6420 XUVs
C:0 T:7 28705 XUs and 51204 XVs -> 327 XUVs
C:0 T:8 10279 XUs and 10023 XVs -> 268 XUVs
C:0 T:9 156223 XUs and 115640 XVs -> 20652 XUVs
C:0 T:10 47488 XUs and 51177 XVs -> 5947 XUVs
C:0 T:11 190 XUs and 192 XVs -> 18 XUVs
66737 XUVs total
14678 collection wire objects
60563 potential space points
Neighbour search...
12227823 tests to find 5316596 neighbours
Iterating with no regularization...
Begin: 7.84297e+07
0 6.24784e+07
1 6.14332e+07
2 6.12745e+07
3 6.12245e+07
Now with regularization...
Begin: 5.19136e+07
0 5.18376e+07
1 5.17921e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412075061
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412075061
fLifetime: 30000 [us]
31-Jan-2026 12:50:59 CET  Closed output file "reco_stage1.root"
31-Jan-2026 12:50:59 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          198.379       260.24        317.085       262.324       39.9133        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00105034    0.00286015    0.00493113    0.00301101     0.0011511       10     
reco:ophit:OpHitFinder                              0.20386      0.252077      0.354385      0.238014      0.0408362       10     
reco:opflash:OpFlashFinder                         0.118024      0.141884      0.188002      0.140147      0.0201088       10     
reco:opslicer:OpSlicer                             0.389731      0.559731      0.875769      0.524313       0.12752        10     
reco:caldata:DataPrepModule                         3.90586       4.24716        5.479        4.11683      0.441109        10     
reco:wclsdatasp:WireCellToolkit                     25.3582       27.4762       29.3244       27.4037       1.12626        10     
reco:gaushit:GausHitFinder                         0.667529      0.967641       1.13046       0.9892       0.129265        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000264793   0.000413926   0.000724727   0.000392764   0.000118238      10     
reco:reco3d:SpacePointSolver                        6.72971       11.7774       14.564        12.1939       2.05526        10     
reco:hitpdune:DisambigFromSpacePoints              0.679454       1.29491       1.87506       1.31938      0.395832        10     
reco:pandora:StandardPandora                        12.3673       22.6981       40.8677       21.4191       8.07193        10     
reco:pandoraWriter:StandardPandora                 0.268109      0.318275      0.381792      0.311026      0.0317639       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.82689       2.4468        2.97319       2.37429      0.348268        10     
reco:pandoraShower:LArPandoraShowerCreation         1.64233       2.70966       3.91804       2.72582      0.672515        10     
reco:pandoracalo:Calorimetry                        1.0316        1.40146       1.78566       1.35893       0.22472        10     
reco:pandoracalonosce:Calorimetry                  0.883344       1.31416       1.73126       1.26942      0.257961        10     
reco:pandorapid:Chi2ParticleID                    0.00299075    0.00383689    0.00520262    0.00369222    0.000681543      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0322116     0.0411395     0.0520162     0.0400135    0.00651272       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00302948     0.0037639    0.00506131     0.0037435    0.000592477      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.56237       3.25319        4.772        3.52379      0.976559        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.68459       3.11009       4.38148       3.09556      0.929681        10     
reco:emtrkmichelid:EmTrackMichelId                  112.545       145.468       176.325       143.743       21.6307        10     
reco:crttag:SingleCRTMatchingProducer             0.00714323     0.0108108     0.0193146     0.010233     0.00325503       10     
reco:crtreco:TwoCRTMatchingProducer               0.00406739    0.00642819     0.0092266    0.00602781    0.00156093       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.55717      0.668528      0.814545      0.651546      0.0928228       10     
reco:pandora2Track:LArPandoraTrackCreation           3.86         8.19188       10.9704       8.84424       2.23981        10     
reco:pandora2calo:Calorimetry                       2.12388       4.57053       6.09776       4.74579       1.26166        10     
reco:pandora2calonosce:Calorimetry                  2.23095       4.26778       5.8518        4.59753       1.15202        10     
reco:pandora2pid:Chi2ParticleID                   0.00712167    0.00995843     0.0164478    0.00974745    0.00257176       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0403597     0.0545046     0.0840585     0.0477753     0.0144931       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00770063     0.0107096     0.016853     0.00952402    0.00296995       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.89007       3.16333       4.68259       3.04955      0.919396        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.04621       3.88654       5.3417        3.73316       1.09827        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.74043       3.47895       4.86806       3.41681       1.02764        10     
[art]:TriggerResults:TriggerResultInserter         3.187e-05    5.34054e-05   0.000137092   4.36755e-05   2.92923e-05      10     
end_path:out1:RootOutput                           8.68e-06     1.32073e-05   3.5651e-05    1.09705e-05   7.61941e-06      10     
end_path:out1:RootOutput(write)                     1.64863       2.35696       3.25308       2.26881      0.488183        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4779.84 MB
  Peak resident set size usage (VmHWM): 3939.39 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.68 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9423792
-rw-r--r--. 1 duneprd np-comp 2231518567 Jan 31 12:04 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2115239288 Jan 31 12:03 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 1913278536 Jan 31 12:50 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1691186091 Jan 31 10:13 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1624098032 Jan 31 09:53 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   73761840 Jan 31 12:47 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242965 Jan 31 12:51 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 31 12:50 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 31 12:50 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 31 12:50 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 12:04 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 10:13 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 12:50 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 31 09:46 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 31 12:51 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 31 12:10 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 31 12:04 debugprod.log
justIN time: 2026-02-04 09:20:53 UTC       justIN version: 01.06.00