justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 272561.0@dunegpschedd02.fnal.gov

Jobsub ID272561.0@dunegpschedd02.fnal.gov
Workflow ID11931
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-14 22:13:04
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce04
Last heartbeat2026-01-15 01:17:11
From worker nodeHostnameccwcondor0400
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-14 22:14:06
Input filesmonte-carlo-011931-000001
JobscriptExit code0
Real time3h (10898s)
CPU time2h (10611s = 97%)
Max RSS bytes4268728320 (4070 MiB)
Outputting started2026-01-15 01:15:46
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11931/1/001/reco_stage1.root
Finished2026-01-15 01:17:11
Saved logsjustin-logs:272561.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ng tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 163385 traces, 0 summary
Retagger: tagging trace set: gauss with 121448 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 121448 traces tagged "gauss"
FrameSaver: q=4.45546e+06 n=2123810 tag=gauss
wclsFrameSaver: saving 163385 traces tagged "wiener"
FrameSaver: q=4.74064e+06 n=1869077 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 26632 XUs and 19535 XVs -> 314 XUVs
C:0 T:1 40281 XUs and 33501 XVs -> 3180 XUVs
C:0 T:2 228667 XUs and 324229 XVs -> 17089 XUVs
C:0 T:3 96073 XUs and 110373 XVs -> 1514 XUVs
C:0 T:4 17861 XUs and 12545 XVs -> 118 XUVs
C:0 T:5 66692 XUs and 36622 XVs -> 3991 XUVs
C:0 T:6 108366 XUs and 116159 XVs -> 5025 XUVs
C:0 T:7 40154 XUs and 42004 XVs -> 380 XUVs
C:0 T:8 11403 XUs and 8924 XVs -> 133 XUVs
C:0 T:9 102679 XUs and 114823 XVs -> 14065 XUVs
C:0 T:10 33206 XUs and 17768 XVs -> 5160 XUVs
C:0 T:11 34 XUs and 19 XVs -> 1 XUVs
50970 XUVs total
14793 collection wire objects
46056 potential space points
Neighbour search...
4903468 tests to find 2354726 neighbours
Iterating with no regularization...
Begin: 1.14712e+08
0 9.16374e+07
1 9.00785e+07
2 8.98754e+07
3 8.98205e+07
Now with regularization...
Begin: 7.46619e+07
0 7.46066e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411744304
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411744304
fLifetime: 30000 [us]
15-Jan-2026 02:15:42 CET  Closed output file "reco_stage1.root"
15-Jan-2026 02:15:42 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          227.698       339.032       485.82        347.934       73.8007        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00106251    0.00398878    0.00661491    0.00401607    0.00161276       10     
reco:ophit:OpHitFinder                             0.230791      0.275194      0.316435      0.265755      0.0268612       10     
reco:opflash:OpFlashFinder                         0.156373      0.181815      0.237853      0.181386       0.02209        10     
reco:opslicer:OpSlicer                             0.431591      0.565219      0.705902      0.550367      0.105037        10     
reco:caldata:DataPrepModule                          3.653        4.57547       5.62781       4.5297       0.709096        10     
reco:wclsdatasp:WireCellToolkit                     27.3032       31.3891       39.235        31.6818       3.68475        10     
reco:gaushit:GausHitFinder                          0.65351      0.925923       1.25228      0.895644      0.222894        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000290134   0.000501662   0.000877463   0.000495472   0.000161143      10     
reco:reco3d:SpacePointSolver                        5.16767       14.9073       27.9527       14.1892       6.47438        10     
reco:hitpdune:DisambigFromSpacePoints              0.816157       1.47877       2.48424       1.36553       0.50238        10     
reco:pandora:StandardPandora                        13.8877        29.09        49.7351       26.2937       11.0693        10     
reco:pandoraWriter:StandardPandora                 0.301893      0.438166      0.666679      0.425655      0.0988038       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.54384       5.11191       9.5481        4.5464        2.21047        10     
reco:pandoraShower:LArPandoraShowerCreation         2.37572       5.04484       9.0911        4.95655       1.93238        10     
reco:pandoracalo:Calorimetry                        1.5099        2.4884        4.73706       2.34873      0.865834        10     
reco:pandoracalonosce:Calorimetry                   1.43799       2.43555       4.22773       2.25819      0.768416        10     
reco:pandorapid:Chi2ParticleID                    0.00341847     0.0045362    0.00585192    0.00441211    0.000875887      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0311502     0.0428114     0.0633078     0.0382612     0.0105916       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00356797    0.00453043    0.00590571     0.0045539    0.000762377      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.35641       5.47336       10.057        5.3833        1.94977        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.45118       5.32672       9.90235       4.60308       2.21364        10     
reco:emtrkmichelid:EmTrackMichelId                  131.067       175.273       235.865       180.288       32.3343        10     
reco:crttag:SingleCRTMatchingProducer             0.00950535     0.0209665     0.0379399     0.0176986    0.00958524       10     
reco:crtreco:TwoCRTMatchingProducer               0.00588093     0.0128999     0.0224438     0.0126609    0.00615344       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.518031      0.854407       1.16623      0.851351      0.183991        10     
reco:pandora2Track:LArPandoraTrackCreation          5.42965       16.4189       31.743        16.4269       6.98929        10     
reco:pandora2calo:Calorimetry                       2.32931       8.23208       12.6542       8.18074       2.92369        10     
reco:pandora2calonosce:Calorimetry                  2.7779        7.80321       11.888        8.38216       2.69849        10     
reco:pandora2pid:Chi2ParticleID                   0.00787977     0.0127893     0.0228649     0.0127287    0.00450059       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.043227      0.0704681     0.123909      0.0745652     0.0242292       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00801891     0.0121989     0.0212173     0.0120539    0.00391067       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.59869       6.07992       9.05238       6.3312        1.99265        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.01717       6.53823       9.90187       6.46765       2.04953        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.52205       5.27379       8.32115       4.57977       1.75324        10     
[art]:TriggerResults:TriggerResultInserter        4.4557e-05    7.94738e-05   0.000173309    6.029e-05    4.7272e-05       10     
end_path:out1:RootOutput                           8.573e-06    2.50286e-05   0.000139198   1.1743e-05    3.82835e-05      10     
end_path:out1:RootOutput(write)                     1.80501       2.46496       3.16089       2.44848      0.447903        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5289.73 MB
  Peak resident set size usage (VmHWM): 4268.73 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.76 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10966936
-rw-r--r-- 1 dune001 lbno 2534874809 Jan 15 01:17 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2405373108 Jan 15 01:15 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2230376297 Jan 15 02:15 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 2037614518 Jan 14 23:35 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1944660537 Jan 14 23:21 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   76631410 Jan 15 02:09 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242781 Jan 15 02:15 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan 15 02:15 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan 15 02:15 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan 15 02:15 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan 15 01:17 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 14 23:35 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 15 02:15 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan 14 23:14 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan 15 02:15 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan 15 01:23 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan 15 01:17 debugprod.log
justIN time: 2026-02-06 02:07:25 UTC       justIN version: 01.06.00