justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265610.0@dunegpschedd02.fnal.gov

Jobsub ID265610.0@dunegpschedd02.fnal.gov
Workflow ID11482
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-03 04:01:16
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce02
Last heartbeat2026-01-03 07:52:09
From worker nodeHostnameccwcondor0036
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-03 04:29:59
Input filesmonte-carlo-011482-000001
JobscriptExit code0
Real time3h (12041s)
CPU time3h (11694s = 97%)
Max RSS bytes4228665344 (4032 MiB)
Outputting started2026-01-03 07:50:42
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11482/1/001/reco_stage1.root
Finished2026-01-03 07:52:09
Saved logsjustin-logs:265610.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

re:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 128605 traces, 0 summary
Retagger: tagging trace set: gauss with 95811 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95811 traces tagged "gauss"
FrameSaver: q=3.0861e+06 n=1638482 tag=gauss
wclsFrameSaver: saving 128605 traces tagged "wiener"
FrameSaver: q=3.27461e+06 n=1426132 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 18574 XUs and 20831 XVs -> 242 XUVs
C:0 T:1 49303 XUs and 55452 XVs -> 5447 XUVs
C:0 T:2 54189 XUs and 65003 XVs -> 3523 XUVs
C:0 T:3 25237 XUs and 24240 XVs -> 148 XUVs
C:0 T:4 14122 XUs and 14703 XVs -> 126 XUVs
C:0 T:5 49566 XUs and 44158 XVs -> 4025 XUVs
C:0 T:6 122116 XUs and 149744 XVs -> 11293 XUVs
C:0 T:7 45428 XUs and 46318 XVs -> 418 XUVs
C:0 T:8 8896 XUs and 8478 XVs -> 103 XUVs
C:0 T:9 48533 XUs and 61380 XVs -> 3144 XUVs
C:0 T:10 34953 XUs and 18834 XVs -> 4498 XUVs
C:0 T:11 575 XUs and 397 XVs -> 82 XUVs
33049 XUVs total
12507 collection wire objects
30657 potential space points
Neighbour search...
3062817 tests to find 1511392 neighbours
Iterating with no regularization...
Begin: 5.71442e+07
0 4.95175e+07
1 4.89276e+07
2 4.88495e+07
3 4.88265e+07
Now with regularization...
Begin: 3.96542e+07
0 3.96287e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411507825
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411507825
fLifetime: 30000 [us]
03-Jan-2026 08:50:40 CET  Closed output file "reco_stage1.root"
03-Jan-2026 08:50:40 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          304.511       374.11        473.957       353.014       58.2091        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00134942    0.00468945    0.00745041    0.00482754    0.00192296       10     
reco:ophit:OpHitFinder                             0.239275      0.307859      0.386139       0.30031      0.0450035       10     
reco:opflash:OpFlashFinder                         0.133764      0.175377      0.233926      0.162682      0.0351177       10     
reco:opslicer:OpSlicer                             0.552149      0.728621      0.858967      0.721217      0.0938992       10     
reco:caldata:DataPrepModule                         5.29233       5.93131       6.82425       5.95544      0.426923        10     
reco:wclsdatasp:WireCellToolkit                     32.8901       35.7793       39.8353       36.0203       1.99813        10     
reco:gaushit:GausHitFinder                         0.796708       1.26325       1.77461       1.21459      0.266399        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000368248   0.000636217   0.00107231    0.000491665   0.00025216       10     
reco:reco3d:SpacePointSolver                        7.49934       14.6762       29.9216       13.534        6.07309        10     
reco:hitpdune:DisambigFromSpacePoints              0.850602       1.48847       2.22165       1.38684      0.470049        10     
reco:pandora:StandardPandora                        20.5971       31.4192       55.3778       28.9669       10.8811        10     
reco:pandoraWriter:StandardPandora                 0.380158      0.494356      0.622158      0.483598      0.0749142       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.42465       5.55062       7.19296       5.25045       1.20411        10     
reco:pandoraShower:LArPandoraShowerCreation         3.71307       6.11784       9.26655       5.81653       1.6113         10     
reco:pandoracalo:Calorimetry                        2.0237        2.83042       4.39445       2.61749      0.749512        10     
reco:pandoracalonosce:Calorimetry                   2.03389       2.76016       3.54652       2.6785       0.485926        10     
reco:pandorapid:Chi2ParticleID                    0.00521157    0.00631742    0.00795303    0.00614915    0.000941519      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0501435     0.0632408     0.079641      0.061557     0.00782444       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00348727    0.00535979    0.00697821    0.00550224    0.00106751       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.02667       6.62141       10.3044       6.12822       2.13369        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.40025       6.00181       8.50748       5.47047       1.69941        10     
reco:emtrkmichelid:EmTrackMichelId                  167.811       197.745       240.991       190.94        24.6445        10     
reco:crttag:SingleCRTMatchingProducer             0.00937369     0.0194375     0.0325961     0.0181557    0.00660407       10     
reco:crtreco:TwoCRTMatchingProducer               0.00533965     0.0126477     0.0196831     0.0125106     0.0041456       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.74859       0.95068       1.29474      0.939489      0.157233        10     
reco:pandora2Track:LArPandoraTrackCreation          11.4073       16.6904       26.4798       15.4186       4.71364        10     
reco:pandora2calo:Calorimetry                       3.93191       7.66648       12.0682       7.30117       2.65908        10     
reco:pandora2calonosce:Calorimetry                   4.019        7.35982       11.3048       7.47755       2.34689        10     
reco:pandora2pid:Chi2ParticleID                   0.00743807     0.0126499     0.0177371     0.0129236    0.00333598       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0381587     0.069583      0.0978029     0.071392      0.021098        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0069893     0.0116615     0.0164502     0.0120293    0.00347418       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.39924       5.90719       8.82289       5.36944       1.87833        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.06541       6.86096       9.86097       6.76635       1.74035        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.61605       5.48886       8.07457       5.35712       1.3883         10     
[art]:TriggerResults:TriggerResultInserter        4.9465e-05    8.69187e-05   0.000188416   7.3927e-05    3.92709e-05      10     
end_path:out1:RootOutput                           9.454e-06    1.33512e-05   1.8969e-05    1.27895e-05   2.78756e-06      10     
end_path:out1:RootOutput(write)                     2.04147       2.87738       3.63259       2.73079      0.478595        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5064.15 MB
  Peak resident set size usage (VmHWM): 4228.67 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.31 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.62 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10099912
-rw-r--r-- 1 dune001 lbno 2379754867 Jan  3 07:46 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2258303762 Jan  3 07:44 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2059003641 Jan  3 08:50 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 1833468377 Jan  3 05:51 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1737867264 Jan  3 05:37 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   73298520 Jan  3 08:46 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242793 Jan  3 08:50 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan  3 08:50 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan  3 08:50 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan  3 08:50 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan  3 07:46 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan  3 05:51 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan  3 08:50 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan  3 05:30 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan  3 08:50 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan  3 07:53 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan  3 07:46 debugprod.log
justIN time: 2026-02-06 22:18:29 UTC       justIN version: 01.06.00