justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 275208.1@dunegpschedd01.fnal.gov

Jobsub ID275208.1@dunegpschedd01.fnal.gov
Workflow ID11733
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-11 22:53:21
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2026-01-12 01:26:30
From worker nodeHostnamewn2208241.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-11 22:54:49
Input filesmonte-carlo-011733-000001
JobscriptExit code0
Real time2h (9009s)
CPU time2h (8817s = 97%)
Max RSS bytes4601425920 (4388 MiB)
Outputting started2026-01-12 01:24:59
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11733/1/001/reco_stage1.root
Finished2026-01-12 01:26:30
Saved logsjustin-logs:275208.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

cKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 171308 traces, 0 summary
Retagger: tagging trace set: gauss with 135007 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 135007 traces tagged "gauss"
FrameSaver: q=4.67394e+06 n=2426682 tag=gauss
wclsFrameSaver: saving 171308 traces tagged "wiener"
FrameSaver: q=4.93383e+06 n=2165928 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 43550 XUs and 33320 XVs -> 497 XUVs
C:0 T:1 74066 XUs and 76585 XVs -> 9265 XUVs
C:0 T:2 132544 XUs and 109288 XVs -> 6369 XUVs
C:0 T:3 53320 XUs and 41238 XVs -> 420 XUVs
C:0 T:4 19091 XUs and 17976 XVs -> 145 XUVs
C:0 T:5 81248 XUs and 67425 XVs -> 6912 XUVs
C:0 T:6 107993 XUs and 116115 XVs -> 4298 XUVs
C:0 T:7 41906 XUs and 48057 XVs -> 460 XUVs
C:0 T:8 12407 XUs and 11496 XVs -> 176 XUVs
C:0 T:9 86709 XUs and 91734 XVs -> 4826 XUVs
C:0 T:10 79920 XUs and 59781 XVs -> 6074 XUVs
C:0 T:11 141 XUs and 99 XVs -> 22 XUVs
39464 XUVs total
15686 collection wire objects
36141 potential space points
Neighbour search...
2847553 tests to find 1296520 neighbours
Iterating with no regularization...
Begin: 1.23741e+08
0 1.00442e+08
1 9.89058e+07
2 9.87456e+07
3 9.86998e+07
Now with regularization...
Begin: 8.4355e+07
0 8.43116e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411684534
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411684534
fLifetime: 30000 [us]
12-Jan-2026 01:24:58 GMT  Closed output file "reco_stage1.root"
12-Jan-2026 01:24:58 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          181.938       260.259       336.492       248.441       48.1716        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000732155    0.0028308    0.00461726    0.00282992    0.00124758       10     
reco:ophit:OpHitFinder                             0.202517      0.242547      0.278951      0.248135      0.0235501       10     
reco:opflash:OpFlashFinder                         0.128703      0.178861      0.235463      0.185938      0.030952        10     
reco:opslicer:OpSlicer                             0.370756      0.514992      0.610807      0.519127      0.062511        10     
reco:caldata:DataPrepModule                         3.47374       3.59862       3.82606       3.56861      0.0943042       10     
reco:wclsdatasp:WireCellToolkit                     22.7446       24.2229       24.8702       24.2922      0.591032        10     
reco:gaushit:GausHitFinder                         0.703624      0.869152       1.04807       0.81715      0.133127        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000290747   0.000421754   0.000804742   0.000373432   0.000139882      10     
reco:reco3d:SpacePointSolver                        6.09001       13.2253       24.822        11.0456       6.17421        10     
reco:hitpdune:DisambigFromSpacePoints              0.910548       1.39889       1.96793       1.31264       0.33222        10     
reco:pandora:StandardPandora                        14.0395       30.6851       56.1701       28.7148       11.662         10     
reco:pandoraWriter:StandardPandora                 0.227308      0.312712      0.419294      0.302712      0.0534384       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.55614       2.91888       4.11818       2.86103      0.810795        10     
reco:pandoraShower:LArPandoraShowerCreation         1.80528       3.33813       5.11135       3.02192       1.03317        10     
reco:pandoracalo:Calorimetry                        1.01796       1.74467       2.32903       1.73508      0.419495        10     
reco:pandoracalonosce:Calorimetry                  0.929105       1.62288       2.19417       1.62154      0.401175        10     
reco:pandorapid:Chi2ParticleID                    0.00285159    0.00417411    0.00627528    0.00400275    0.000856427      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0297934     0.0359162     0.0429734     0.0340437    0.00448014       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00271118    0.00377593    0.00495113    0.00366279    0.000608042      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.08577       3.78211       5.68333       3.49367       1.12414        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.01605       3.70605       5.59025       3.44118       1.11071        10     
reco:emtrkmichelid:EmTrackMichelId                  102.065       132.839       162.859       127.91        19.0456        10     
reco:crttag:SingleCRTMatchingProducer             0.00720564     0.0109304     0.018366      0.0104601     0.0030142       10     
reco:crtreco:TwoCRTMatchingProducer               0.00415832    0.00577701    0.00697115    0.00568385    0.000813222      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.536663      0.714297      0.921101      0.730629      0.111228        10     
reco:pandora2Track:LArPandoraTrackCreation          5.12703       9.55549       14.1481       8.48621       2.83762        10     
reco:pandora2calo:Calorimetry                       2.87549       5.24176       7.64051       4.66556       1.45146        10     
reco:pandora2calonosce:Calorimetry                  2.76719       5.00576       7.25593       4.46842       1.37165        10     
reco:pandora2pid:Chi2ParticleID                   0.00670249    0.00991199     0.0130012    0.00943628    0.00185232       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0359754     0.0509292     0.0812995     0.045407      0.0139875       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00642494    0.00960951     0.0123096    0.00915744    0.00169769       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.04743       3.43302       4.98063       3.09439      0.925206        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.60936       4.54563       6.57607       4.12603       1.18432        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.51065       4.16586       5.96521       3.77093       1.05039        10     
[art]:TriggerResults:TriggerResultInserter        3.0628e-05    5.33961e-05   0.000188594   3.64285e-05   4.55555e-05      10     
end_path:out1:RootOutput                           1.03e-05     1.20768e-05   2.0839e-05    1.09655e-05   2.99225e-06      10     
end_path:out1:RootOutput(write)                     1.63779       2.20107       2.73694       2.08388      0.347717        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5412.86 MB
  Peak resident set size usage (VmHWM): 4601.43 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.27 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11522824
-rw-r--r-- 1 dune01 dune01 2660100865 Jan 12 00:40 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2524251533 Jan 12 00:38 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2394294008 Jan 12 01:24 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2103195815 Jan 11 23:08 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 2029745038 Jan 11 22:59 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   87155110 Jan 12 01:21 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242849 Jan 12 01:24 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Jan 12 01:24 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Jan 12 01:24 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Jan 12 01:24 time.db
-rw-r--r-- 1 dune01 dune01        519 Jan 12 00:40 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 11 23:08 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 12 01:24 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Jan 11 22:54 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Jan 12 01:24 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Jan 12 00:45 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Jan 12 00:40 debugprod.log
justIN time: 2026-02-06 10:21:19 UTC       justIN version: 01.06.00