justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267429.1@dunegpschedd01.fnal.gov

Jobsub ID267429.1@dunegpschedd01.fnal.gov
Workflow ID11191
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-12 18:07:41
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2025-12-12 22:51:34
From worker nodeHostnamewn-la-12.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-12 18:08:42
Input filesmonte-carlo-011191-000001
JobscriptExit code0
Real time4h (16877s)
CPU time4h (16724s = 99%)
Max RSS bytes4559228928 (4348 MiB)
Outputting started2025-12-12 22:50:01
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11191/1/001/reco_stage1.root
Finished2025-12-12 22:51:34
Saved logsjustin-logs:267429.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

gnalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 154286 traces, 0 summary
Retagger: tagging trace set: gauss with 120016 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 120016 traces tagged "gauss"
FrameSaver: q=4.10296e+06 n=2171415 tag=gauss
wclsFrameSaver: saving 154286 traces tagged "wiener"
FrameSaver: q=4.3285e+06 n=1934570 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 56232 XUs and 61010 XVs -> 1060 XUVs
C:0 T:1 132123 XUs and 134253 XVs -> 8694 XUVs
C:0 T:2 148126 XUs and 160442 XVs -> 7529 XUVs
C:0 T:3 67637 XUs and 61763 XVs -> 400 XUVs
C:0 T:4 11902 XUs and 14177 XVs -> 148 XUVs
C:0 T:5 56709 XUs and 72266 XVs -> 9645 XUVs
C:0 T:6 69987 XUs and 56090 XVs -> 2237 XUVs
C:0 T:7 26676 XUs and 22332 XVs -> 184 XUVs
C:0 T:8 7437 XUs and 8056 XVs -> 67 XUVs
C:0 T:9 39893 XUs and 66769 XVs -> 4493 XUVs
C:0 T:10 63172 XUs and 38141 XVs -> 8567 XUVs
C:0 T:11 158 XUs and 99 XVs -> 21 XUVs
43045 XUVs total
13999 collection wire objects
39382 potential space points
Neighbour search...
3306740 tests to find 1645670 neighbours
Iterating with no regularization...
Begin: 7.98396e+07
0 6.44122e+07
1 6.34373e+07
2 6.3294e+07
3 6.32543e+07
Now with regularization...
Begin: 5.19279e+07
0 5.18396e+07
1 5.18077e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411076750
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411076750
fLifetime: 30000 [us]
12-Dec-2025 23:49:58 CET  Closed output file "reco_stage1.root"
12-Dec-2025 23:49:58 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          284.529       474.273       619.034       485.443       102.05         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00131506    0.00611959    0.00943344    0.00690268    0.00292664       10     
reco:ophit:OpHitFinder                             0.296086      0.357624      0.485874      0.329612      0.061376        10     
reco:opflash:OpFlashFinder                         0.198046      0.257804       0.33915      0.242777      0.0523732       10     
reco:opslicer:OpSlicer                             0.543477      0.723927      0.965535      0.666938      0.133561        10     
reco:caldata:DataPrepModule                         4.80807       5.34553       6.98513       5.00851      0.779264        10     
reco:wclsdatasp:WireCellToolkit                     39.9678       45.9765       50.485        45.9739       3.16508        10     
reco:gaushit:GausHitFinder                          0.94093       1.33027       2.17283       1.33292      0.332025        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000621882   0.00101011    0.00165802    0.00102855     0.0002728       10     
reco:reco3d:SpacePointSolver                        8.38808       20.4502       37.7894       19.0399       9.21039        10     
reco:hitpdune:DisambigFromSpacePoints               0.8793        1.72305       2.86411       1.81061      0.578433        10     
reco:pandora:StandardPandora                        22.2799       40.692        71.0241       40.9682       14.0786        10     
reco:pandoraWriter:StandardPandora                  0.38178      0.538027      0.659755      0.561607      0.0843501       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.45479       10.9641       15.7378       11.9322       3.13648        10     
reco:pandoraShower:LArPandoraShowerCreation         4.20235       15.047        22.7497       15.2596       5.96415        10     
reco:pandoracalo:Calorimetry                        2.46426       5.72477       8.42598       5.97288       1.62064        10     
reco:pandoracalonosce:Calorimetry                   2.73831       5.27464       8.53927       5.51084       1.49854        10     
reco:pandorapid:Chi2ParticleID                    0.00464543    0.00591358    0.00792969    0.00581795    0.000898702      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.035417      0.0512898     0.0685695     0.0523458     0.0101778       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00456201    0.00572747    0.00689128    0.00567798    0.000699046      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.51888       13.6769       20.8591       13.8071        5.344         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.29162       12.3034       19.3408       12.2983       5.21425        10     
reco:emtrkmichelid:EmTrackMichelId                  127.692       180.414       217.775       183.898       27.2405        10     
reco:crttag:SingleCRTMatchingProducer              0.0134276     0.0273941     0.0393504     0.0284061    0.00784208       10     
reco:crtreco:TwoCRTMatchingProducer                0.0182941     0.026272      0.036525      0.0241133    0.00571691       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.645404      0.825209       1.11398      0.828386      0.137541        10     
reco:pandora2Track:LArPandoraTrackCreation          17.1552       37.0983       60.0714       37.4766       10.953         10     
reco:pandora2calo:Calorimetry                       7.38678       16.9007       29.1584       16.3319       5.99272        10     
reco:pandora2calonosce:Calorimetry                  5.08343       16.6367       23.8949       18.4754       5.44455        10     
reco:pandora2pid:Chi2ParticleID                   0.00922428     0.0129604     0.0155436     0.0133999    0.00176756       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0446151     0.063271      0.0764043     0.0666081    0.00912259       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00951678     0.0130851     0.0148244     0.0136094     0.0016222       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.33751       12.9006       22.6563       12.319        4.32218        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.4472        13.6334       23.3165       13.6435       4.73146        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.39167       12.015        21.3541       12.1311       5.10907        10     
[art]:TriggerResults:TriggerResultInserter        3.6234e-05    5.40819e-05   0.000125738   4.4431e-05    2.51541e-05      10     
end_path:out1:RootOutput                           7.792e-06    1.12809e-05   2.7582e-05     8.628e-06    5.82318e-06      10     
end_path:out1:RootOutput(write)                     2.0458        2.93633       3.38461       3.15244      0.468179        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5365.5 MB
  Peak resident set size usage (VmHWM): 4559.23 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.25 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11224352
-rw-r--r--. 1 dune009 dune 2607114709 Dec 12 22:28 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2475458963 Dec 12 22:26 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2319931560 Dec 12 23:49 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2047491701 Dec 12 19:33 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1962495460 Dec 12 19:17 g4_stage1.root
-rw-r--r--. 1 dune009 dune   80630240 Dec 12 23:43 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242926 Dec 12 23:50 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Dec 12 23:49 mem.db
-rw-r--r--. 1 dune009 dune      62376 Dec 12 23:49 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Dec 12 23:49 time.db
-rw-r--r--. 1 dune009 dune        519 Dec 12 22:28 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Dec 12 19:33 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Dec 12 23:49 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Dec 12 19:08 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Dec 12 23:50 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Dec 12 22:35 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Dec 12 22:28 debugprod.log
justIN time: 2025-12-18 18:07:35 UTC       justIN version: 01.05.03