justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 254698.1@dunegpschedd01.fnal.gov

Jobsub ID254698.1@dunegpschedd01.fnal.gov
Workflow ID10441
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-21 00:17:44
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-11-21 03:38:04
From worker nodeHostnamewn-da-11.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-21 00:18:48
Input filesmonte-carlo-010441-000001
JobscriptExit code0
Real time3h (11863s)
CPU time3h (11731s = 98%)
Max RSS bytes4204277760 (4009 MiB)
Outputting started2025-11-21 03:36:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10441/1/001/reco_stage1.root
Finished2025-11-21 03:38:04
Saved logsjustin-logs:254698.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

RawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 188932 traces, 0 summary
Retagger: tagging trace set: gauss with 137635 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 137635 traces tagged "gauss"
FrameSaver: q=4.65288e+06 n=2384380 tag=gauss
wclsFrameSaver: saving 188932 traces tagged "wiener"
FrameSaver: q=4.96254e+06 n=2086665 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 41357 XUs and 40013 XVs -> 659 XUVs
C:0 T:1 97209 XUs and 89170 XVs -> 10856 XUVs
C:0 T:2 111429 XUs and 135420 XVs -> 6553 XUVs
C:0 T:3 54547 XUs and 58099 XVs -> 718 XUVs
C:0 T:4 20747 XUs and 26668 XVs -> 389 XUVs
C:0 T:5 114733 XUs and 128353 XVs -> 9738 XUVs
C:0 T:6 139760 XUs and 169649 XVs -> 11585 XUVs
C:0 T:7 51883 XUs and 58112 XVs -> 1233 XUVs
C:0 T:8 9468 XUs and 12022 XVs -> 134 XUVs
C:0 T:9 69104 XUs and 110393 XVs -> 8772 XUVs
C:0 T:10 46747 XUs and 29922 XVs -> 5511 XUVs
C:0 T:11 255 XUs and 197 XVs -> 37 XUVs
56185 XUVs total
18217 collection wire objects
49243 potential space points
Neighbour search...
3067051 tests to find 1657584 neighbours
Iterating with no regularization...
Begin: 8.46204e+07
0 6.73087e+07
1 6.58747e+07
2 6.56323e+07
3 6.55557e+07
4 6.5522e+07
Now with regularization...
Begin: 5.23487e+07
0 5.23166e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410639664
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410639664
fLifetime: 30000 [us]
21-Nov-2025 04:36:31 CET  Closed output file "reco_stage1.root"
21-Nov-2025 04:36:31 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          259.735       377.166       515.94        372.49        78.3294        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000784995    0.0038788    0.00790286     0.0042824    0.00202915       10     
reco:ophit:OpHitFinder                              0.2304       0.259345       0.29609      0.263326      0.0198636       10     
reco:opflash:OpFlashFinder                         0.122851      0.160914      0.183443      0.168519      0.0217239       10     
reco:opslicer:OpSlicer                             0.481184      0.665401      0.786188      0.692324      0.0944622       10     
reco:caldata:DataPrepModule                         4.38567       4.47207       4.69301       4.44928      0.0895055       10     
reco:wclsdatasp:WireCellToolkit                     32.6582       45.5572       56.9592       48.9955       9.15518        10     
reco:gaushit:GausHitFinder                         0.934362       1.20982       1.49553       1.21387      0.179653        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00046682    0.000720892   0.000878473   0.000743045   0.000115192      10     
reco:reco3d:SpacePointSolver                        5.71643       14.7975       24.2613       14.7613       5.89873        10     
reco:hitpdune:DisambigFromSpacePoints               1.04478       1.56202       2.12677       1.60568      0.383556        10     
reco:pandora:StandardPandora                        17.4591       36.0397       62.7035       27.807        16.5852        10     
reco:pandoraWriter:StandardPandora                 0.310187      0.406426      0.522579      0.408143      0.0664685       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.6413        4.15545       6.4616        4.14663       1.10451        10     
reco:pandoraShower:LArPandoraShowerCreation         2.59588       5.04971       8.73448       4.83373       1.78883        10     
reco:pandoracalo:Calorimetry                        1.58317       2.23579       3.29426       2.25212      0.490617        10     
reco:pandoracalonosce:Calorimetry                   1.41792       2.06287       3.08856       2.09262      0.468286        10     
reco:pandorapid:Chi2ParticleID                     0.0046512    0.00552568    0.00637633    0.00551556    0.000573054      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0374089     0.0476815     0.0622208     0.0475913    0.00699375       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00454884     0.0052004    0.00617207    0.00521421    0.000506834      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.56189       5.05391       8.55539       4.92542       1.63752        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.45413       4.91924       8.53188       4.81316       1.68721        10     
reco:emtrkmichelid:EmTrackMichelId                  155.62        201.391       258.364       206.089       33.5477        10     
reco:crttag:SingleCRTMatchingProducer             0.00763081     0.011783      0.0163988     0.0114771    0.00225478       10     
reco:crtreco:TwoCRTMatchingProducer               0.00655724    0.00921846     0.012371     0.00937211    0.00150929       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.507392      0.855947       1.17401      0.847402      0.180064        10     
reco:pandora2Track:LArPandoraTrackCreation          7.52002       14.1648       23.6062       13.9346       4.71794        10     
reco:pandora2calo:Calorimetry                       3.75266       6.6737        10.8829       6.77069       2.07988        10     
reco:pandora2calonosce:Calorimetry                  3.4531        6.38909       10.5171       6.37246       1.95981        10     
reco:pandora2pid:Chi2ParticleID                    0.0106632     0.0137039     0.0168636     0.0136284    0.00198111       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0476465     0.0612773     0.0807168     0.0620826    0.00981796       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0107964     0.0136895     0.016847      0.0136682    0.00197337       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.05593       5.17314       8.3045        5.09991       1.50098        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.18236       5.63936       9.27529       5.73051       1.7062         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.85857       5.08473       8.56569       5.11769       1.58305        10     
[art]:TriggerResults:TriggerResultInserter        2.7332e-05    3.69885e-05   9.6123e-05    3.00525e-05   1.98562e-05      10     
end_path:out1:RootOutput                           5.59e-06     6.6955e-06    1.0389e-05     6.392e-06    1.29782e-06      10     
end_path:out1:RootOutput(write)                     2.21405       2.87132       3.60556       2.92135      0.454296        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5007.05 MB
  Peak resident set size usage (VmHWM): 4204.28 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.78 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10215236
-rw-r--r--. 1 dune009 dune 2397457437 Nov 21 03:31 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2271795162 Nov 21 03:30 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2115545853 Nov 21 04:36 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1839773389 Nov 21 01:36 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1753158672 Nov 21 01:24 g4_stage1.root
-rw-r--r--. 1 dune009 dune   82052320 Nov 21 04:30 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243820 Nov 21 04:36 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov 21 04:36 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov 21 04:36 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov 21 04:36 time.db
-rw-r--r--. 1 dune009 dune        519 Nov 21 03:31 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 21 01:36 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 21 04:36 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov 21 01:18 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov 21 04:36 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov 21 03:36 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov 21 03:31 debugprod.log
justIN time: 2026-02-09 01:56:47 UTC       justIN version: 01.06.00