justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271145.0@dunegpschedd02.fnal.gov

Jobsub ID271145.0@dunegpschedd02.fnal.gov
Workflow ID11791
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-13 09:37:10
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2026-01-13 13:34:16
From worker nodeHostnamewn-da-16.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-13 09:38:39
Input filesmonte-carlo-011791-000001
JobscriptExit code0
Real time3h (14036s)
CPU time3h (13854s = 98%)
Max RSS bytes4361080832 (4159 MiB)
Outputting started2026-01-13 13:32:36
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11791/1/001/reco_stage1.root
Finished2026-01-13 13:34:16
Saved logsjustin-logs:271145.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Source: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 113116 traces, 0 summary
Retagger: tagging trace set: gauss with 85832 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 85832 traces tagged "gauss"
FrameSaver: q=3.29783e+06 n=1496825 tag=gauss
wclsFrameSaver: saving 113116 traces tagged "wiener"
FrameSaver: q=3.51118e+06 n=1328101 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 41594 XUs and 43227 XVs -> 490 XUVs
C:0 T:1 77167 XUs and 73374 XVs -> 5251 XUVs
C:0 T:2 71593 XUs and 121381 XVs -> 4419 XUVs
C:0 T:3 36886 XUs and 45562 XVs -> 404 XUVs
C:0 T:4 25732 XUs and 16355 XVs -> 273 XUVs
C:0 T:5 174291 XUs and 89124 XVs -> 15504 XUVs
C:0 T:6 53091 XUs and 49743 XVs -> 5592 XUVs
C:0 T:7 22795 XUs and 22956 XVs -> 208 XUVs
C:0 T:8 4407 XUs and 4698 XVs -> 109 XUVs
C:0 T:9 43426 XUs and 64188 XVs -> 13045 XUVs
C:0 T:10 4009 XUs and 6331 XVs -> 992 XUVs
C:0 T:11 37 XUs and 20 XVs -> 5 XUVs
46292 XUVs total
9141 collection wire objects
42635 potential space points
Neighbour search...
6471091 tests to find 2803550 neighbours
Iterating with no regularization...
Begin: 9.27254e+07
0 5.58138e+07
1 5.28891e+07
2 5.22977e+07
3 5.20704e+07
4 5.19525e+07
5 5.18796e+07
6 5.18341e+07
Now with regularization...
Begin: 3.83849e+07
0 3.81962e+07
1 3.80604e+07
2 3.7972e+07
3 3.79111e+07
4 3.78667e+07
5 3.78312e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411710775
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411710775
fLifetime: 30000 [us]
13-Jan-2026 14:32:34 CET  Closed output file "reco_stage1.root"
13-Jan-2026 14:32:34 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          325.039       460.005       589.204       482.736       85.0429        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00134537    0.00603224    0.00973123     0.0066979    0.00277133       10     
reco:ophit:OpHitFinder                             0.269288      0.308927      0.368252      0.310034      0.0272721       10     
reco:opflash:OpFlashFinder                         0.168586      0.215965      0.312753      0.216287      0.0392121       10     
reco:opslicer:OpSlicer                             0.625365      0.756211      0.887523      0.752761      0.0724545       10     
reco:caldata:DataPrepModule                         4.80318       5.07416       5.49897       5.07647      0.177919        10     
reco:wclsdatasp:WireCellToolkit                     38.6145       54.036        63.0009       58.8389       9.20349        10     
reco:gaushit:GausHitFinder                          1.00598       1.36536       1.60159       1.48807      0.231382        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000625512   0.000999211   0.00138664    0.00102288    0.000273593      10     
reco:reco3d:SpacePointSolver                        10.9805       16.1021       20.7567       16.357        2.81825        10     
reco:hitpdune:DisambigFromSpacePoints               1.07475       1.73247       2.38706       1.86482      0.402532        10     
reco:pandora:StandardPandora                        24.1213       42.8098       57.9147       45.7311       9.45824        10     
reco:pandoraWriter:StandardPandora                 0.384412      0.520633      0.614828      0.559352      0.0755936       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.39329       8.44368       12.9073       9.20165       2.84049        10     
reco:pandoraShower:LArPandoraShowerCreation         2.93166       9.21294       13.4022       10.367        3.34671        10     
reco:pandoracalo:Calorimetry                        1.69702       4.56274       7.82369       4.90114       1.88544        10     
reco:pandoracalonosce:Calorimetry                   1.6053        4.39125       7.49728       4.71886       1.85268        10     
reco:pandorapid:Chi2ParticleID                    0.00566272    0.00731062    0.00969109    0.00695809    0.00145472       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0397973     0.0542942     0.0643091     0.0586801     0.008866        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00513602    0.00664986    0.00792117    0.00702479    0.00101179       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.7255        8.90825       14.0117       9.63063       3.78994        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.58313       9.07149       13.5434       10.0358       3.86219        10     
reco:emtrkmichelid:EmTrackMichelId                  173.429       218.322       253.329       229.388       28.798         10     
reco:crttag:SingleCRTMatchingProducer             0.00820081     0.0242093     0.0584177     0.022145      0.0137803       10     
reco:crtreco:TwoCRTMatchingProducer               0.00696586     0.0173164     0.0288811     0.0166951     0.0075641       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.612807      0.866198       1.01615      0.929009      0.142228        10     
reco:pandora2Track:LArPandoraTrackCreation          8.66748       21.8787       36.2019       24.1746       8.43438        10     
reco:pandora2calo:Calorimetry                       3.47171       11.0825       20.3732       12.6423       5.03368        10     
reco:pandora2calonosce:Calorimetry                  3.26571       10.8604       18.8945       11.9251       5.13495        10     
reco:pandora2pid:Chi2ParticleID                    0.011876      0.0153333     0.0178781     0.016054     0.00198498       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0496656     0.0689587     0.0810326     0.0738703     0.0113904       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0119609     0.0158106     0.0185631     0.0166963    0.00234332       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.05718       8.22076       12.5029       8.70036       3.32346        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.01547       9.54986       16.4436       10.1083       4.67401        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.76907       8.18289       14.1646       8.62844       3.90873        10     
[art]:TriggerResults:TriggerResultInserter        3.4034e-05    5.35618e-05   0.000151081   4.03325e-05   3.35767e-05      10     
end_path:out1:RootOutput                           7.944e-06    9.3341e-06    1.3656e-05     8.872e-06    1.72988e-06      10     
end_path:out1:RootOutput(write)                     2.36815       3.08202       3.5526        3.24507      0.465373        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5162.68 MB
  Peak resident set size usage (VmHWM): 4361.08 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.10 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10583660
-rw-r--r--. 1 dune009 dune 2468259636 Jan 13 13:13 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2338237442 Jan 13 13:11 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2166724944 Jan 13 14:32 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1927048090 Jan 13 10:58 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1858745926 Jan 13 10:45 g4_stage1.root
-rw-r--r--. 1 dune009 dune   78037320 Jan 13 14:28 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243157 Jan 13 14:32 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Jan 13 14:32 mem.db
-rw-r--r--. 1 dune009 dune      62376 Jan 13 14:32 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Jan 13 14:32 time.db
-rw-r--r--. 1 dune009 dune        519 Jan 13 13:13 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 13 10:58 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 13 14:32 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Jan 13 10:38 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Jan 13 14:32 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Jan 13 13:20 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Jan 13 13:13 debugprod.log
justIN time: 2026-02-06 07:39:25 UTC       justIN version: 01.06.00