justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 236607.0@dunegpschedd02.fnal.gov

Jobsub ID236607.0@dunegpschedd02.fnal.gov
Workflow ID9544
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-03 10:56:44
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2025-11-03 16:21:17
From worker nodeHostnamewn-lb-02.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 10:58:24
Input filesmonte-carlo-009544-000001
JobscriptExit code0
Real time5h (19269s)
CPU time5h (19109s = 99%)
Max RSS bytes4605546496 (4392 MiB)
Outputting started2025-11-03 16:19:34
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09544/1/001/reco_stage1.root
Finished2025-11-03 16:21:17
Saved logsjustin-logs:236607.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

unning tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 144210 traces, 0 summary
Retagger: tagging trace set: gauss with 103091 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 103091 traces tagged "gauss"
FrameSaver: q=3.34949e+06 n=1747583 tag=gauss
wclsFrameSaver: saving 144210 traces tagged "wiener"
FrameSaver: q=3.58609e+06 n=1498473 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:1 49634 XUs and 59518 XVs -> 8136 XUVs
C:0 T:2 45245 XUs and 49885 XVs -> 1800 XUVs
C:0 T:3 25411 XUs and 21382 XVs -> 204 XUVs
C:0 T:4 13118 XUs and 14793 XVs -> 185 XUVs
C:0 T:5 61523 XUs and 66681 XVs -> 9625 XUVs
C:0 T:6 155100 XUs and 152795 XVs -> 11943 XUVs
C:0 T:7 42407 XUs and 51573 XVs -> 483 XUVs
C:0 T:9 105063 XUs and 145939 XVs -> 14682 XUVs
C:0 T:10 111229 XUs and 42535 XVs -> 11823 XUVs
C:0 T:11 818 XUs and 648 XVs -> 99 XUVs
58980 XUVs total
15238 collection wire objects
54212 potential space points
Neighbour search...
7192390 tests to find 3717242 neighbours
Iterating with no regularization...
Begin: 6.54437e+07
0 4.2987e+07
1 4.18925e+07
2 4.16955e+07
3 4.1622e+07
4 4.15869e+07
Now with regularization...
Begin: 3.22446e+07
0 3.21573e+07
1 3.20968e+07
2 3.20507e+07
3 3.20144e+07
4 3.19849e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410286437
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410286437
fLifetime: 30000 [us]
03-Nov-2025 17:19:31 CET  Closed output file "reco_stage1.root"
03-Nov-2025 17:19:31 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                           393.4        598.993       868.076       548.427       145.233        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00118964     0.0066417     0.0109728    0.00627056    0.00333772       10     
reco:ophit:OpHitFinder                             0.374934      0.460993       0.53834      0.452182      0.0424845       10     
reco:opflash:OpFlashFinder                         0.276426      0.355389      0.497759      0.339976      0.0543101       10     
reco:opslicer:OpSlicer                             0.600013       0.91596       1.23763       0.92568      0.161156        10     
reco:caldata:DataPrepModule                         6.19245       6.52906       7.10535       6.51334      0.226918        10     
reco:wclsdatasp:WireCellToolkit                     44.5053       61.6783       71.8344       65.4825       9.08449        10     
reco:gaushit:GausHitFinder                          1.20727       1.66248        2.313        1.52219      0.367288        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000587518   0.000825026   0.00122549    0.000806374   0.00016881       10     
reco:reco3d:SpacePointSolver                        15.8761       27.9808       47.4315       26.5365       8.7947         10     
reco:hitpdune:DisambigFromSpacePoints               1.43827       2.53096       4.3987        2.05353      0.968302        10     
reco:pandora:StandardPandora                        33.2441       62.3608       142.292       51.3574       32.8474        10     
reco:pandoraWriter:StandardPandora                 0.451019      0.624667       0.81519      0.577584      0.110749        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.12638       10.9587       15.8804       11.2769       3.08184        10     
reco:pandoraShower:LArPandoraShowerCreation         4.17756       14.1976       27.8269       13.3725       6.5683         10     
reco:pandoracalo:Calorimetry                        3.04518       5.97804       9.0271        6.35158       1.74612        10     
reco:pandoracalonosce:Calorimetry                   2.75917       5.68426       8.52772       5.91708       1.70898        10     
reco:pandorapid:Chi2ParticleID                    0.00648344    0.00762523     0.0088666    0.00757167    0.000805575      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0438772     0.0659946     0.0860314     0.0607005     0.0124564       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00630751    0.00741801    0.00840288    0.00760528    0.000744478      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.30801       13.4788       25.7304       12.621        6.27188        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.13752       13.0162       24.4762       12.2981       5.90323        10     
reco:emtrkmichelid:EmTrackMichelId                  188.08        262.091       345.595       240.264       49.0261        10     
reco:crttag:SingleCRTMatchingProducer              0.0135321     0.0254823     0.0365263     0.0257419    0.00675836       10     
reco:crtreco:TwoCRTMatchingProducer               0.00941679     0.020692      0.0314497     0.0209345    0.00658073       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.853707       1.19558       1.45819       1.18543      0.183272        10     
reco:pandora2Track:LArPandoraTrackCreation          12.9075       33.5028       49.7823       34.0767       12.3774        10     
reco:pandora2calo:Calorimetry                       5.52108       16.1847       24.786        16.8295       5.98992        10     
reco:pandora2calonosce:Calorimetry                  5.21309       15.6308       25.6226       16.314        6.05251        10     
reco:pandora2pid:Chi2ParticleID                    0.0126786     0.0171521     0.0217613     0.0159098    0.00300816       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0554559     0.0842412     0.111328      0.0760761     0.017405        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0128084     0.0171687      0.02195      0.0157536    0.00302905       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.97026       12.4166       21.6577       13.4867       4.86914        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.66612       13.479        23.3782       14.3042       5.31041        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.1465        11.3156       20.032        12.1753       4.38317        10     
[art]:TriggerResults:TriggerResultInserter        3.7155e-05    5.7099e-05    0.000133369   5.1512e-05    2.63296e-05      10     
end_path:out1:RootOutput                           7.41e-06     1.02171e-05   2.2474e-05     8.297e-06    4.49663e-06      10     
end_path:out1:RootOutput(write)                     2.90299       4.18787       5.44901       3.83154      0.771701        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5406.91 MB
  Peak resident set size usage (VmHWM): 4605.55 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.09 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11530388
-rw-r--r--. 1 dune009 dune 2656976685 Nov  3 15:37 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2519068329 Nov  3 15:34 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2407217514 Nov  3 17:19 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2115960542 Nov  3 12:25 g4_stage2.root
-rw-r--r--. 1 dune009 dune 2016345936 Nov  3 12:06 g4_stage1.root
-rw-r--r--. 1 dune009 dune   90875310 Nov  3 17:12 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243521 Nov  3 17:19 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov  3 17:19 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov  3 17:19 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov  3 17:19 time.db
-rw-r--r--. 1 dune009 dune        519 Nov  3 15:37 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  3 12:25 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  3 17:19 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov  3 11:58 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov  3 17:19 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov  3 15:47 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov  3 15:37 debugprod.log
justIN time: 2025-11-05 04:49:57 UTC       justIN version: 01.05.01