justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302521.1@dunegpschedd01.fnal.gov

Jobsub ID302521.1@dunegpschedd01.fnal.gov
Workflow ID12602
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-30 21:51:34
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce514
Last heartbeat2026-01-31 00:56:09
From worker nodeHostnameb9p12p2338.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-30 21:52:23
Input filesmonte-carlo-012602-000001
JobscriptExit code0
Real time3h (10944s)
CPU time2h (10394s = 94%)
Max RSS bytes4155793408 (3963 MiB)
Outputting started2026-01-31 00:54:48
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12602/1/001/reco_stage1.root
Finished2026-01-31 00:56:09
Saved logsjustin-logs:302521.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

eepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 131668 traces, 0 summary
Retagger: tagging trace set: gauss with 101270 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 101270 traces tagged "gauss"
FrameSaver: q=3.429e+06 n=1746672 tag=gauss
wclsFrameSaver: saving 131668 traces tagged "wiener"
FrameSaver: q=3.61829e+06 n=1535805 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32759 XUs and 32043 XVs -> 375 XUVs
C:0 T:1 65701 XUs and 54950 XVs -> 4859 XUVs
C:0 T:2 50875 XUs and 70087 XVs -> 2453 XUVs
C:0 T:3 30759 XUs and 35011 XVs -> 107 XUVs
C:0 T:4 19681 XUs and 16173 XVs -> 323 XUVs
C:0 T:5 93148 XUs and 56347 XVs -> 9139 XUVs
C:0 T:6 45252 XUs and 59977 XVs -> 2332 XUVs
C:0 T:8 10635 XUs and 9054 XVs -> 154 XUVs
C:0 T:9 71550 XUs and 81902 XVs -> 6080 XUVs
C:0 T:10 60309 XUs and 20157 XVs -> 6161 XUVs
C:0 T:11 170 XUs and 135 XVs -> 12 XUVs
31995 XUVs total
12179 collection wire objects
30256 potential space points
Neighbour search...
2943616 tests to find 1393322 neighbours
Iterating with no regularization...
Begin: 5.99363e+07
0 5.08403e+07
1 5.02572e+07
2 5.01671e+07
3 5.01391e+07
Now with regularization...
Begin: 4.00377e+07
0 4.00087e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412065904
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412065904
fLifetime: 30000 [us]
31-Jan-2026 01:54:45 CET  Closed output file "reco_stage1.root"
31-Jan-2026 01:54:45 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          179.744       281.397       358.746       271.587       53.1411        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000993515    0.002897     0.00530216    0.00288756    0.00129084       10     
reco:ophit:OpHitFinder                             0.190878      0.274126      0.360911      0.267015      0.0486596       10     
reco:opflash:OpFlashFinder                         0.116945      0.166556      0.222056      0.164857      0.0282246       10     
reco:opslicer:OpSlicer                             0.494254      0.658823      0.798186      0.649199      0.110923        10     
reco:caldata:DataPrepModule                         3.90304       4.97167       6.78609       4.85118      0.798559        10     
reco:wclsdatasp:WireCellToolkit                      24.77        30.0784       35.0718       30.8244       3.33775        10     
reco:gaushit:GausHitFinder                         0.681598      0.912622       1.18353      0.918834      0.146728        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000312635   0.000386899   0.000453163   0.00038563    4.08117e-05      10     
reco:reco3d:SpacePointSolver                        8.41548       12.0201       16.7217       11.1165       2.99255        10     
reco:hitpdune:DisambigFromSpacePoints              0.757185       1.29944       1.74304       1.41095      0.301908        10     
reco:pandora:StandardPandora                        15.0322       23.2925       35.1554       22.9904       5.48221        10     
reco:pandoraWriter:StandardPandora                 0.223358      0.365429      0.540118      0.326943      0.102427        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.40191       2.77999       5.05242       2.41276       1.02373        10     
reco:pandoraShower:LArPandoraShowerCreation         1.32676       2.96295       5.27864       2.83352        1.052         10     
reco:pandoracalo:Calorimetry                       0.851768        1.651        2.68852       1.56586      0.449495        10     
reco:pandoracalonosce:Calorimetry                  0.770813       1.55839       2.40434       1.46339      0.430157        10     
reco:pandorapid:Chi2ParticleID                    0.00273049    0.00519665    0.00719806    0.00562921    0.00155952       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0292362     0.0475141     0.0744444     0.0403152     0.0165987       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00263245     0.0045466    0.00708264    0.00380539    0.00166897       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.4618        3.33519       5.71177       3.09351       1.16475        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.44658       3.36715       5.78683       3.21279       1.19634        10     
reco:emtrkmichelid:EmTrackMichelId                  101.112       158.715       202.422       158.655       28.872         10     
reco:crttag:SingleCRTMatchingProducer             0.00742457     0.0130231     0.0188715     0.012778     0.00326409       10     
reco:crtreco:TwoCRTMatchingProducer               0.00506594     0.0077138     0.0110954    0.00783214    0.00177456       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.474775      0.805496       1.1183       0.803368      0.179466        10     
reco:pandora2Track:LArPandoraTrackCreation          4.00824       9.04751       15.1151       8.81302       2.96413        10     
reco:pandora2calo:Calorimetry                       2.13564       4.6822        8.13524       4.58177       1.6463         10     
reco:pandora2calonosce:Calorimetry                  1.99928       4.34404       5.83437       4.56699       1.28278        10     
reco:pandora2pid:Chi2ParticleID                   0.00656209     0.0113829     0.0165384     0.0114871    0.00343589       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0345432     0.0601591     0.0834297     0.0632333     0.0171054       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00635148     0.0124403     0.0172886     0.0136842    0.00417138       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.51216       3.29317       4.76445       3.35828       1.03071        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.90754       4.28105       6.3773        4.43022       1.36595        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.74887       3.90334       5.49724       4.2295        1.10956        10     
[art]:TriggerResults:TriggerResultInserter         3.04e-05     4.97586e-05   0.000112741   4.3786e-05    2.22297e-05      10     
end_path:out1:RootOutput                            8.4e-06     1.04712e-05    1.389e-05    1.0345e-05     1.732e-06       10     
end_path:out1:RootOutput(write)                     1.64856       2.39001       3.09706       2.38652      0.477734        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5011.55 MB
  Peak resident set size usage (VmHWM): 4155.79 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.14 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10178224
-rw-r--r--. 1 duneprd np-comp 2386589144 Jan 31 01:04 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2263974757 Jan 31 01:03 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2064833129 Jan 31 01:54 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1862348677 Jan 30 23:09 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1771248129 Jan 30 22:59 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   72765240 Jan 31 01:51 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242892 Jan 31 01:54 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 31 01:54 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 31 01:54 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 31 01:54 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 01:04 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 30 23:09 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 01:54 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 30 22:52 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 31 01:54 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 31 01:09 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 31 01:04 debugprod.log
justIN time: 2026-02-04 10:45:41 UTC       justIN version: 01.06.00