justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 241703.1@dunegpschedd01.fnal.gov

Jobsub ID241703.1@dunegpschedd01.fnal.gov
Workflow ID9513
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-01 16:22:26
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-11-01 20:42:20
From worker nodeHostnamewn-da-01.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-01 16:23:16
Input filesmonte-carlo-009513-000001
JobscriptExit code0
Real time4h (15437s)
CPU time4h (15311s = 99%)
Max RSS bytes4341846016 (4140 MiB)
Outputting started2025-11-01 20:40:34
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09513/1/001/reco_stage1.root
Finished2025-11-01 20:42:20
Saved logsjustin-logs:241703.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

der
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 169888 traces, 0 summary
Retagger: tagging trace set: gauss with 125374 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 125374 traces tagged "gauss"
FrameSaver: q=4.65064e+06 n=2188208 tag=gauss
wclsFrameSaver: saving 169888 traces tagged "wiener"
FrameSaver: q=4.95631e+06 n=1922824 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33045 XUs and 33098 XVs -> 750 XUVs
C:0 T:1 70430 XUs and 73929 XVs -> 6415 XUVs
C:0 T:2 126609 XUs and 104163 XVs -> 10467 XUVs
C:0 T:3 39563 XUs and 38528 XVs -> 250 XUVs
C:0 T:4 11295 XUs and 12955 XVs -> 117 XUVs
C:0 T:5 34414 XUs and 34635 XVs -> 3528 XUVs
C:0 T:6 135364 XUs and 111719 XVs -> 5902 XUVs
C:0 T:7 53428 XUs and 43181 XVs -> 392 XUVs
C:0 T:8 13365 XUs and 14766 XVs -> 255 XUVs
C:0 T:9 93494 XUs and 128209 XVs -> 8570 XUVs
C:0 T:10 82771 XUs and 68575 XVs -> 7691 XUVs
C:0 T:11 762 XUs and 522 XVs -> 84 XUVs
44421 XUVs total
15529 collection wire objects
39367 potential space points
Neighbour search...
3229785 tests to find 1398184 neighbours
Iterating with no regularization...
Begin: 1.11749e+08
0 9.02536e+07
1 8.84765e+07
2 8.82005e+07
3 8.8119e+07
Now with regularization...
Begin: 7.07459e+07
0 7.06537e+07
1 7.05966e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410250822
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410250822
fLifetime: 30000 [us]
01-Nov-2025 21:40:32 CET  Closed output file "reco_stage1.root"
01-Nov-2025 21:40:32 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          411.47        493.529       614.788       480.355       60.0286        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00162221    0.00551719    0.00936625    0.00578499    0.00271615       10     
reco:ophit:OpHitFinder                             0.271706      0.334741      0.416097      0.319881      0.0406738       10     
reco:opflash:OpFlashFinder                         0.172637       0.24622      0.363647      0.235795      0.0534574       10     
reco:opslicer:OpSlicer                             0.651234      0.825027       1.0151       0.823287      0.111205        10     
reco:caldata:DataPrepModule                         5.0496         5.352        5.91285       5.28492      0.227283        10     
reco:wclsdatasp:WireCellToolkit                     40.5236       68.0935       78.5469       73.0643       13.5215        10     
reco:gaushit:GausHitFinder                          1.22711       1.46027       1.75888       1.43319       0.15113        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000756025   0.00101359    0.00144932    0.00100597    0.000219879      10     
reco:reco3d:SpacePointSolver                        8.98811       18.1343       24.6637       19.0515       4.32715        10     
reco:hitpdune:DisambigFromSpacePoints               1.2711        1.90608       2.42826       1.91786      0.343354        10     
reco:pandora:StandardPandora                        32.4215       41.9224       54.7705       41.2719       6.48901        10     
reco:pandoraWriter:StandardPandora                 0.455649       0.54576      0.626966      0.541861      0.0612071       10     
reco:pandoraTrack:LArPandoraTrackCreation           5.02699       7.59554       10.549        7.2043        1.94938        10     
reco:pandoraShower:LArPandoraShowerCreation         4.68202       8.5073        14.0458       8.16981       2.83905        10     
reco:pandoracalo:Calorimetry                        2.67151       3.95874       5.37995       3.84773      0.971206        10     
reco:pandoracalonosce:Calorimetry                   2.32955       3.73134       5.04851       3.60763      0.965044        10     
reco:pandorapid:Chi2ParticleID                    0.00564253    0.00690987    0.00913344     0.0068285    0.000933858      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0441576     0.0574658     0.0672508     0.0574561    0.00622663       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00549962    0.00654094    0.00754217    0.00648725    0.000619937      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.90818       8.33057       13.7036       8.14521       2.67839        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.69881       8.16507       13.0675       7.8786        2.33898        10     
reco:emtrkmichelid:EmTrackMichelId                  201.29        242.627       286.534       242.355       23.0173        10     
reco:crttag:SingleCRTMatchingProducer              0.0121852     0.0217809     0.0411345     0.0190442    0.00815946       10     
reco:crtreco:TwoCRTMatchingProducer                0.0113139     0.0159263     0.0260251     0.0139048    0.00485615       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.650867      0.982093       1.15345       1.00101      0.144781        10     
reco:pandora2Track:LArPandoraTrackCreation          17.071        22.616        35.3673       20.9646       5.98381        10     
reco:pandora2calo:Calorimetry                       7.01914       10.8558       17.3284       9.99703       3.37192        10     
reco:pandora2calonosce:Calorimetry                  6.66049       10.1842       16.0031       9.37626       3.09748        10     
reco:pandora2pid:Chi2ParticleID                    0.0141638     0.0164221      0.02102      0.0163844    0.00196728       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0620643     0.075236      0.0898681     0.0736734    0.00783591       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0139549     0.0163615     0.0207847      0.01634     0.00190965       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.1018        7.67863       12.3009       7.06802       2.11621        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.16634       8.42176       13.3232       7.59109       2.18921        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.11345       7.15979       11.7571       6.69397       1.9358         10     
[art]:TriggerResults:TriggerResultInserter        3.3594e-05     4.942e-05    0.000129396    4.197e-05    2.68237e-05      10     
end_path:out1:RootOutput                           6.823e-06    1.01321e-05   2.7582e-05     7.734e-06    6.01096e-06      10     
end_path:out1:RootOutput(write)                     2.81036       3.44398       3.94763       3.51098      0.378811        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5145.87 MB
  Peak resident set size usage (VmHWM): 4341.85 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.29 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11049560
-rw-r--r--. 1 dune009 dune 2556708439 Nov  1 20:15 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2421040574 Nov  1 20:13 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2277235278 Nov  1 21:40 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2028687586 Nov  1 17:46 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1947259967 Nov  1 17:30 g4_stage1.root
-rw-r--r--. 1 dune009 dune   83200390 Nov  1 21:32 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242875 Nov  1 21:40 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov  1 21:40 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov  1 21:40 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov  1 21:40 time.db
-rw-r--r--. 1 dune009 dune        519 Nov  1 20:15 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  1 17:46 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  1 21:40 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov  1 17:23 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov  1 21:40 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov  1 20:23 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov  1 20:16 debugprod.log
justIN time: 2025-11-03 18:30:31 UTC       justIN version: 01.05.01