justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 239366.0@dunegpschedd01.fnal.gov

Jobsub ID239366.0@dunegpschedd01.fnal.gov
Workflow ID9419
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-28 15:34:20
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2025-10-28 19:14:29
From worker nodeHostnamewn-la-25.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-28 15:36:02
Input filesmonte-carlo-009419-000001
JobscriptExit code0
Real time3h (13005s)
CPU time3h (12850s = 98%)
Max RSS bytes4195409920 (4001 MiB)
Outputting started2025-10-28 19:12:48
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09419/1/001/reco_stage1.root
Finished2025-10-28 19:14:29
Saved logsjustin-logs:239366.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

edRawDigitPrepService:prepare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 120875 traces, 0 summary
Retagger: tagging trace set: gauss with 95283 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95283 traces tagged "gauss"
FrameSaver: q=3.27788e+06 n=1696708 tag=gauss
wclsFrameSaver: saving 120875 traces tagged "wiener"
FrameSaver: q=3.45466e+06 n=1513399 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:1 45193 XUs and 44022 XVs -> 4481 XUVs
C:0 T:2 114652 XUs and 101787 XVs -> 6393 XUVs
C:0 T:3 58382 XUs and 39533 XVs -> 448 XUVs
C:0 T:4 10224 XUs and 10963 XVs -> 73 XUVs
C:0 T:5 33649 XUs and 30198 XVs -> 3158 XUVs
C:0 T:6 114689 XUs and 120518 XVs -> 5670 XUVs
C:0 T:7 49104 XUs and 49464 XVs -> 576 XUVs
C:0 T:8 8382 XUs and 6705 XVs -> 102 XUVs
C:0 T:9 45285 XUs and 47056 XVs -> 2814 XUVs
C:0 T:10 28377 XUs and 18427 XVs -> 2788 XUVs
C:0 T:11 301 XUs and 239 XVs -> 50 XUVs
26553 XUVs total
10612 collection wire objects
23868 potential space points
Neighbour search...
1114106 tests to find 608124 neighbours
Iterating with no regularization...
Begin: 6.21956e+07
0 5.34501e+07
1 5.27775e+07
2 5.26895e+07
3 5.26654e+07
Now with regularization...
Begin: 4.33772e+07
0 4.33576e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410169577
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410169577
fLifetime: 30000 [us]
28-Oct-2025 20:12:46 CET  Closed output file "reco_stage1.root"
28-Oct-2025 20:12:46 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          294.754       370.513       458.621       373.335       54.7851        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000946918    0.0050673    0.00824298    0.00509003    0.00237955       10     
reco:ophit:OpHitFinder                             0.278137      0.323974       0.35789      0.327867      0.0234519       10     
reco:opflash:OpFlashFinder                         0.173028      0.223161      0.263635      0.218358      0.0273216       10     
reco:opslicer:OpSlicer                             0.512837      0.674977      0.801304      0.683466      0.0763858       10     
reco:caldata:DataPrepModule                         4.82711       4.94972       5.38696       4.8644        0.16558        10     
reco:wclsdatasp:WireCellToolkit                     33.8922       42.8343       54.3413       43.5083       6.56179        10     
reco:gaushit:GausHitFinder                         0.942706       1.14675       1.5753        1.07994      0.205432        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00035445    0.00055166    0.000835351   0.000468612   0.000173528      10     
reco:reco3d:SpacePointSolver                        8.64382       16.7263       27.8902       15.9172       6.70999        10     
reco:hitpdune:DisambigFromSpacePoints               1.05272       1.46863       2.12097       1.42333      0.373681        10     
reco:pandora:StandardPandora                        20.6442       34.0132       53.1942       33.6864       9.19412        10     
reco:pandoraWriter:StandardPandora                 0.375491      0.440965      0.551109      0.433163      0.0566575       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.31105       6.64201       8.94786       6.67045       1.60078        10     
reco:pandoraShower:LArPandoraShowerCreation         5.59754       8.06884       11.8613       8.32836       1.95477        10     
reco:pandoracalo:Calorimetry                        2.00566       3.37817       4.70264       3.74854      0.834529        10     
reco:pandoracalonosce:Calorimetry                   2.04363       3.24814       4.65607       3.49878      0.779519        10     
reco:pandorapid:Chi2ParticleID                    0.00394447    0.00554538    0.00780788     0.0052666    0.00107464       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0370517     0.045878      0.0635339     0.0435093    0.00724049       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00390299    0.00489937    0.00718231    0.00478597    0.000844011      10     
reco:pandoraShowercalo:ShowerCalorimetry            5.63398       7.56718       10.1706       7.6385        1.70909        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.99268       7.37063       9.75938       7.88797       1.78985        10     
reco:emtrkmichelid:EmTrackMichelId                  140.572       166.604       203.069       167.088       22.2485        10     
reco:crttag:SingleCRTMatchingProducer              0.0120612     0.0186667     0.0284132     0.016379     0.00518192       10     
reco:crtreco:TwoCRTMatchingProducer               0.00893348     0.0135353     0.017538      0.0126235    0.00268331       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.642201      0.824703       1.01626      0.839888      0.118778        10     
reco:pandora2Track:LArPandoraTrackCreation          13.673        20.2658       25.989        19.7288       3.60711        10     
reco:pandora2calo:Calorimetry                       6.6684        9.83577       12.634        10.1844       1.77054        10     
reco:pandora2calonosce:Calorimetry                  6.60373       9.17143       11.0777       9.4367        1.5904         10     
reco:pandora2pid:Chi2ParticleID                    0.010046      0.0113901     0.0131905     0.011469     0.00103352       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.048905      0.0572064     0.0672433      0.05755     0.00588012       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0100827     0.0114244     0.0131923     0.0114679    0.000986966      10     
reco:pandora2Shower:LArPandoraShowerCreation        4.66035       7.00379       9.42088       6.76608       1.4967         10     
reco:pandora2Showercalo:ShowerCalorimetry           5.53405       7.88111       9.85955       8.03067       1.37202        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.00999       6.71029       8.54203       6.82602       1.0728         10     
[art]:TriggerResults:TriggerResultInserter        3.7135e-05    5.06029e-05   0.000105738   4.51985e-05   1.91827e-05      10     
end_path:out1:RootOutput                           6.329e-06    7.6614e-06    1.0976e-05    7.0755e-06    1.43135e-06      10     
end_path:out1:RootOutput(write)                     2.34592       2.74056       3.45061       2.74323      0.331532        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4999.62 MB
  Peak resident set size usage (VmHWM): 4195.41 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.06 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10511372
-rw-r--r--. 1 dune009 dune 2458841605 Oct 28 19:09 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2330918443 Oct 28 19:07 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2165551168 Oct 28 20:12 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1906016444 Oct 28 16:55 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1822576172 Oct 28 16:42 g4_stage1.root
-rw-r--r--. 1 dune009 dune   79123900 Oct 28 20:09 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242770 Oct 28 20:12 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Oct 28 20:12 mem.db
-rw-r--r--. 1 dune009 dune      62376 Oct 28 20:12 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Oct 28 20:12 time.db
-rw-r--r--. 1 dune009 dune        519 Oct 28 19:09 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 28 16:55 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 28 20:12 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Oct 28 16:36 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Oct 28 20:12 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Oct 28 19:14 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Oct 28 19:09 debugprod.log
justIN time: 2025-11-03 18:41:09 UTC       justIN version: 01.05.01