justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269486.0@dunegpschedd02.fnal.gov

Jobsub ID269486.0@dunegpschedd02.fnal.gov
Workflow ID11736
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-11 22:53:21
SiteUK_Oxford
EntryDUNE_UK_SGrid_Oxford_arc01
Last heartbeat2026-01-12 02:11:01
From worker nodeHostnamet2wn174.physics.ox.ac.uk
cpuinfoAMD EPYC 9655 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-11 22:55:06
Input filesmonte-carlo-011736-000001
JobscriptExit code0
Real time3h (11661s)
CPU time3h (11262s = 96%)
Max RSS bytes4663570432 (4447 MiB)
Outputting started2026-01-12 02:09:28
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11736/1/001/reco_stage1.root
Finished2026-01-12 02:11:01
Saved logsjustin-logs:269486.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

re:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 148139 traces, 0 summary
Retagger: tagging trace set: gauss with 113530 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 113530 traces tagged "gauss"
FrameSaver: q=3.96639e+06 n=1997924 tag=gauss
wclsFrameSaver: saving 148139 traces tagged "wiener"
FrameSaver: q=4.18781e+06 n=1773288 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 40573 XUs and 37182 XVs -> 596 XUVs
C:0 T:1 70023 XUs and 70907 XVs -> 4051 XUVs
C:0 T:2 84563 XUs and 79997 XVs -> 3251 XUVs
C:0 T:3 43129 XUs and 33154 XVs -> 246 XUVs
C:0 T:4 8601 XUs and 13671 XVs -> 65 XUVs
C:0 T:5 44845 XUs and 42801 XVs -> 4810 XUVs
C:0 T:6 108425 XUs and 83642 XVs -> 6917 XUVs
C:0 T:7 39852 XUs and 32188 XVs -> 1023 XUVs
C:0 T:8 9212 XUs and 11575 XVs -> 165 XUVs
C:0 T:9 60895 XUs and 90629 XVs -> 4346 XUVs
C:0 T:10 83697 XUs and 50355 XVs -> 9782 XUVs
C:0 T:11 202 XUs and 134 XVs -> 37 XUVs
35289 XUVs total
12346 collection wire objects
31682 potential space points
Neighbour search...
2263900 tests to find 1104890 neighbours
Iterating with no regularization...
Begin: 1.08397e+08
0 9.21428e+07
1 9.11709e+07
2 9.10511e+07
3 9.1017e+07
Now with regularization...
Begin: 7.88564e+07
0 7.88132e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411684498
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411684498
fLifetime: 30000 [us]
12-Jan-2026 02:09:26 GMT  Closed output file "reco_stage1.root"
12-Jan-2026 02:09:26 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          297.838       400.601       582.391       397.902       76.8212        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00123787    0.00403683    0.00683588    0.00407099    0.00174009       10     
reco:ophit:OpHitFinder                             0.308234      0.354858       0.41621      0.356083      0.0350608       10     
reco:opflash:OpFlashFinder                          0.17475      0.220891      0.285783      0.217154      0.0330879       10     
reco:opslicer:OpSlicer                             0.634441      0.819202       1.0272       0.834335      0.118926        10     
reco:caldata:DataPrepModule                         5.09954       5.38664       5.64119       5.39852      0.174898        10     
reco:wclsdatasp:WireCellToolkit                     30.4833       31.7632       33.9489       31.7873      0.974113        10     
reco:gaushit:GausHitFinder                         0.913835        1.514        2.05537       1.53302      0.285561        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000380175   0.000512438   0.000729924   0.000500848   9.20469e-05      10     
reco:reco3d:SpacePointSolver                        14.7785       22.9908       45.4317       19.8734       8.7435         10     
reco:hitpdune:DisambigFromSpacePoints               1.65831       2.35178       3.85957       2.19992      0.600937        10     
reco:pandora:StandardPandora                        21.3749       49.445        83.7952       47.1437       19.7243        10     
reco:pandoraWriter:StandardPandora                 0.502776      0.623488      0.830361      0.593319      0.100545        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.10203       5.92104       9.94678       5.40806       1.76315        10     
reco:pandoraShower:LArPandoraShowerCreation         3.64987       6.98774       14.2423       6.16791       2.84543        10     
reco:pandoracalo:Calorimetry                        2.15496       3.06001       4.58996       2.98791      0.726619        10     
reco:pandoracalonosce:Calorimetry                   2.00169       2.89432       3.82607       2.73371      0.638949        10     
reco:pandorapid:Chi2ParticleID                    0.00461548    0.00665047    0.00900182     0.0069818    0.00126176       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0396455     0.0729719     0.100926      0.0791142     0.017711        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00459712    0.00630208    0.00733102     0.0066002    0.000977086      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.81097       6.90828       12.7772       6.4822        2.38876        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.98669       6.59113       12.609        6.22979       2.3312         10     
reco:emtrkmichelid:EmTrackMichelId                  155.363       189.584       244.557       186.423       24.3053        10     
reco:crttag:SingleCRTMatchingProducer              0.0158394     0.0197347     0.0269978     0.0188179    0.00336803       10     
reco:crtreco:TwoCRTMatchingProducer               0.00952688     0.011282      0.0148521     0.0100961    0.00191716       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.671866      0.770326      0.946916      0.749213      0.0912361       10     
reco:pandora2Track:LArPandoraTrackCreation          11.6777       20.1088       35.9273       18.5085       6.35739        10     
reco:pandora2calo:Calorimetry                       5.77237       9.1683        14.2736       8.69768       2.23358        10     
reco:pandora2calonosce:Calorimetry                  5.46799       8.42746       13.8434       7.69227       2.31066        10     
reco:pandora2pid:Chi2ParticleID                    0.0118389     0.0170137     0.0241226     0.0168194    0.00339996       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0522591     0.0988535     0.139437      0.098661      0.0224092       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0112745     0.0167851     0.0238114     0.0166726    0.00339981       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.2815        6.60161       10.5891       6.12381       1.74338        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.80048       7.50293       11.5395       7.47966       1.76507        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.26329       6.71517       10.6405       6.49738       1.80287        10     
[art]:TriggerResults:TriggerResultInserter        4.8724e-05    8.7933e-05    0.00020529    7.10375e-05   4.84478e-05      10     
end_path:out1:RootOutput                           8.833e-06    1.05329e-05   1.6475e-05     9.935e-06    2.1491e-06       10     
end_path:out1:RootOutput(write)                     2.7468        3.46345       4.48036       3.52397      0.483838        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5579.45 MB
  Peak resident set size usage (VmHWM): 4663.57 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.42 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 12417072
-rw-r--r-- 1 dune020 dune 2843749962 Jan 12 01:00 detsim_stage2.root
-rw-r--r-- 1 dune020 dune 2700935870 Jan 12 00:57 detsim_stage1.root
-rw-r--r-- 1 dune020 dune 2595339732 Jan 12 02:09 reco_stage1.root
-rw-r--r-- 1 dune020 dune 2282033559 Jan 11 23:17 g4_stage2.root
-rw-r--r-- 1 dune020 dune 2201647483 Jan 11 23:02 g4_stage1.root
-rw-r--r-- 1 dune020 dune   90746830 Jan 12 02:05 Pandora_Events.pndr
-rw-r--r-- 1 dune020 dune     243029 Jan 12 02:09 jobscript.log
-rw-r--r-- 1 dune020 dune     217088 Jan 12 02:09 mem.db
-rw-r--r-- 1 dune020 dune      62376 Jan 12 02:09 Pandora_Geometry.xml
-rw-r--r-- 1 dune020 dune      36864 Jan 12 02:09 time.db
-rw-r--r-- 1 dune020 dune        519 Jan 12 01:00 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune020 dune        519 Jan 11 23:17 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune020 dune        519 Jan 12 02:09 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune020 dune         52 Jan 11 22:55 all-input-dids.txt
-rw-r--r-- 1 dune020 dune          7 Jan 12 02:09 justin-processed-pfns.txt
-rw-r--r-- 1 dune020 dune          0 Jan 12 01:07 ProtoDUNE.log
-rw-r--r-- 1 dune020 dune          0 Jan 12 01:00 debugprod.log
justIN time: 2026-02-06 10:21:15 UTC       justIN version: 01.06.00