justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 241706.1@dunegpschedd01.fnal.gov

Jobsub ID241706.1@dunegpschedd01.fnal.gov
Workflow ID9516
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-01 16:22:27
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-11-01 21:31:33
From worker nodeHostnamewn-la-10.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-01 16:23:10
Input filesmonte-carlo-009516-000001
JobscriptExit code0
Real time5h (18398s)
CPU time5h (18243s = 99%)
Max RSS bytes4408446976 (4204 MiB)
Outputting started2025-11-01 21:29:50
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09516/1/001/reco_stage1.root
Finished2025-11-01 21:31:33
Saved logsjustin-logs:241706.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

unning tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 150379 traces, 0 summary
Retagger: tagging trace set: gauss with 115756 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 115756 traces tagged "gauss"
FrameSaver: q=4.08245e+06 n=2069236 tag=gauss
wclsFrameSaver: saving 150379 traces tagged "wiener"
FrameSaver: q=4.32207e+06 n=1840987 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 50912 XUs and 62187 XVs -> 1187 XUVs
C:0 T:1 110044 XUs and 126340 XVs -> 10596 XUVs
C:0 T:2 117250 XUs and 113023 XVs -> 6428 XUVs
C:0 T:3 50702 XUs and 46284 XVs -> 475 XUVs
C:0 T:4 12120 XUs and 13189 XVs -> 99 XUVs
C:0 T:5 86233 XUs and 82622 XVs -> 18601 XUVs
C:0 T:6 88165 XUs and 113289 XVs -> 4006 XUVs
C:0 T:7 38599 XUs and 50754 XVs -> 259 XUVs
C:0 T:8 13312 XUs and 13725 XVs -> 383 XUVs
C:0 T:9 164192 XUs and 160797 XVs -> 18661 XUVs
C:0 T:10 15303 XUs and 10080 XVs -> 1715 XUVs
C:0 T:11 60 XUs and 36 XVs -> 5 XUVs
62415 XUVs total
14055 collection wire objects
57215 potential space points
Neighbour search...
11882891 tests to find 4996354 neighbours
Iterating with no regularization...
Begin: 8.29987e+07
0 5.98992e+07
1 5.86261e+07
2 5.84285e+07
3 5.83585e+07
4 5.83199e+07
Now with regularization...
Begin: 4.68714e+07
0 4.67931e+07
1 4.67233e+07
2 4.66676e+07
3 4.66245e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410250790
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410250790
fLifetime: 30000 [us]
01-Nov-2025 22:29:47 CET  Closed output file "reco_stage1.root"
01-Nov-2025 22:29:47 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          384.155       554.111        646.6        563.101       66.3768        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00137137    0.00688524     0.0156276    0.00696753    0.00404451       10     
reco:ophit:OpHitFinder                             0.425282      0.450845      0.475056      0.449017      0.0160719       10     
reco:opflash:OpFlashFinder                          0.30544      0.338324      0.449766      0.326365      0.0412196       10     
reco:opslicer:OpSlicer                             0.780843      0.896534       1.04453      0.897853      0.0788046       10     
reco:caldata:DataPrepModule                         6.41779       6.53745       7.05446       6.49693      0.177468        10     
reco:wclsdatasp:WireCellToolkit                     44.8151       56.7134       64.4445       58.1665       5.87768        10     
reco:gaushit:GausHitFinder                          1.33614       1.55005       1.71162       1.58628      0.125044        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000637038   0.000917727   0.00135081    0.000860266   0.000215454      10     
reco:reco3d:SpacePointSolver                        12.3679       23.6442       35.5136       22.0494       6.80795        10     
reco:hitpdune:DisambigFromSpacePoints               1.4572        2.15422       2.66958       2.18422       0.34509        10     
reco:pandora:StandardPandora                        41.1745       53.1329       73.5851       50.6719       9.37796        10     
reco:pandoraWriter:StandardPandora                 0.488308      0.616927       0.69592      0.628531      0.0631841       10     
reco:pandoraTrack:LArPandoraTrackCreation           7.97391       11.5647       13.8277       11.3659       1.64248        10     
reco:pandoraShower:LArPandoraShowerCreation         7.50356       14.3123       19.9676       14.1786       3.66089        10     
reco:pandoracalo:Calorimetry                        4.89333       5.89899       7.08925       5.8983       0.807336        10     
reco:pandoracalonosce:Calorimetry                   4.60595       5.64631       6.76917       5.69381       0.5645         10     
reco:pandorapid:Chi2ParticleID                    0.00997855     0.0108189     0.0136638     0.0105176    0.00103111       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.055721      0.063409      0.0712661     0.0630257    0.00442628       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00651344    0.00722059    0.00822053    0.00729361    0.000552093      10     
reco:pandoraShowercalo:ShowerCalorimetry            7.68778       12.8387       16.554        13.2305       2.54992        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.88026       13.1647       16.4456       13.5473       2.55285        10     
reco:emtrkmichelid:EmTrackMichelId                  183.433       235.186       264.669       234.091       21.6666        10     
reco:crttag:SingleCRTMatchingProducer              0.013747      0.0264339     0.0515939     0.0214993     0.0125106       10     
reco:crtreco:TwoCRTMatchingProducer               0.00937793     0.0218648     0.0361939     0.0197413     0.0084665       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.906315       1.13277       1.37436       1.13567      0.134913        10     
reco:pandora2Track:LArPandoraTrackCreation          15.8274       34.6991       51.9181       34.6953       8.50016        10     
reco:pandora2calo:Calorimetry                       7.44655       17.1495       25.9397       16.2891       4.78227        10     
reco:pandora2calonosce:Calorimetry                  6.8613        16.5651       21.4386       17.1053       3.79193        10     
reco:pandora2pid:Chi2ParticleID                    0.0122438     0.015865      0.0183687     0.0158287    0.00164946       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0612535     0.0789381     0.0922003     0.0778901    0.00804223       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0123825     0.0159487     0.0183272     0.0160497    0.00163302       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.88458       11.9137       18.0062       12.4461       3.4661         10     
reco:pandora2Showercalo:ShowerCalorimetry           5.88184       13.1924       17.3974       13.5996       3.11512        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.13912       10.4858       14.6757       10.6194       2.33502        10     
[art]:TriggerResults:TriggerResultInserter         3.911e-05    6.19916e-05   0.000129604   5.6118e-05    2.40994e-05      10     
end_path:out1:RootOutput                           7.531e-06    9.0695e-06    1.4031e-05    8.4475e-06    1.88038e-06      10     
end_path:out1:RootOutput(write)                     2.94908       3.74615       4.21697        3.745       0.338195        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5207.22 MB
  Peak resident set size usage (VmHWM): 4408.45 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.06 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11321720
-rw-r--r--. 1 dune009 dune 2613640893 Nov  1 20:55 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2479195071 Nov  1 20:52 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2330779512 Nov  1 22:29 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2087986177 Nov  1 17:49 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1999004598 Nov  1 17:31 g4_stage1.root
-rw-r--r--. 1 dune009 dune   82217320 Nov  1 22:22 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242808 Nov  1 22:29 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov  1 22:29 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov  1 22:29 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov  1 22:29 time.db
-rw-r--r--. 1 dune009 dune        519 Nov  1 20:55 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  1 17:49 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  1 22:29 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov  1 17:23 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov  1 22:29 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov  1 21:03 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov  1 20:55 debugprod.log
justIN time: 2025-11-03 18:30:13 UTC       justIN version: 01.05.01