justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271723.0@dunegpschedd02.fnal.gov

Jobsub ID271723.0@dunegpschedd02.fnal.gov
Workflow ID11860
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-13 21:59:53
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2026-01-14 03:23:22
From worker nodeHostnamewn-la-04.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-13 22:01:21
Input filesmonte-carlo-011860-000001
JobscriptExit code0
Real time5h (19224s)
CPU time5h (19132s = 99%)
Max RSS bytes4503093248 (4294 MiB)
Outputting started2026-01-14 03:21:46
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11860/1/001/reco_stage1.root
Finished2026-01-14 03:23:22
Saved logsjustin-logs:271723.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

edRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 138325 traces, 0 summary
Retagger: tagging trace set: gauss with 105938 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 105938 traces tagged "gauss"
FrameSaver: q=3.71858e+06 n=1872482 tag=gauss
wclsFrameSaver: saving 138325 traces tagged "wiener"
FrameSaver: q=3.96306e+06 n=1667558 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 24707 XUs and 20931 XVs -> 184 XUVs
C:0 T:1 48438 XUs and 43199 XVs -> 3627 XUVs
C:0 T:2 105579 XUs and 128558 XVs -> 4874 XUVs
C:0 T:3 51657 XUs and 54518 XVs -> 415 XUVs
C:0 T:4 17841 XUs and 14774 XVs -> 141 XUVs
C:0 T:5 116961 XUs and 86275 XVs -> 15855 XUVs
C:0 T:6 90799 XUs and 84778 XVs -> 3156 XUVs
C:0 T:7 37251 XUs and 28891 XVs -> 270 XUVs
C:0 T:8 11859 XUs and 12078 XVs -> 289 XUVs
C:0 T:9 126164 XUs and 274316 XVs -> 23964 XUVs
C:0 T:10 42354 XUs and 20369 XVs -> 4016 XUVs
C:0 T:11 144 XUs and 91 XVs -> 14 XUVs
56805 XUVs total
12692 collection wire objects
52756 potential space points
Neighbour search...
9904268 tests to find 4238172 neighbours
Iterating with no regularization...
Begin: 9.22618e+07
0 6.96855e+07
1 6.86002e+07
2 6.84288e+07
3 6.83747e+07
Now with regularization...
Begin: 5.73351e+07
0 5.72743e+07
1 5.72308e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411723978
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411723978
fLifetime: 30000 [us]
14-Jan-2026 04:21:44 CET  Closed output file "reco_stage1.root"
14-Jan-2026 04:21:44 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          364.341       582.498       720.685       602.91        112.202        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00137864    0.00645194     0.0108774    0.00706809    0.00306927       10     
reco:ophit:OpHitFinder                             0.405192      0.461046      0.541702      0.451074       0.04659        10     
reco:opflash:OpFlashFinder                         0.267041      0.341308      0.448091      0.337659      0.061591        10     
reco:opslicer:OpSlicer                             0.749979      0.942196       1.15767      0.939009      0.126939        10     
reco:caldata:DataPrepModule                         6.34225       6.60185       7.17051       6.56662      0.204328        10     
reco:wclsdatasp:WireCellToolkit                     45.4557       60.2132       68.2714       64.1802       8.82367        10     
reco:gaushit:GausHitFinder                          1.23097       1.66869       1.96722       1.64024      0.238051        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000401068   0.000569772   0.000809898   0.00054283    0.000108246      10     
reco:reco3d:SpacePointSolver                        11.2946       30.1232       50.9079       28.4441       9.86054        10     
reco:hitpdune:DisambigFromSpacePoints               1.26722       2.60892       3.81712       2.53411      0.783513        10     
reco:pandora:StandardPandora                        27.9907       61.4284       86.2977       65.2679       16.4744        10     
reco:pandoraWriter:StandardPandora                 0.418693      0.602797      0.693731      0.648297      0.0913063       10     
reco:pandoraTrack:LArPandoraTrackCreation           6.68504       8.88185       11.329        8.75052       1.58446        10     
reco:pandoraShower:LArPandoraShowerCreation         6.24881       11.1102       16.1365       11.1654       2.6836         10     
reco:pandoracalo:Calorimetry                        2.95803       4.81607       6.31741       4.90646      0.992761        10     
reco:pandoracalonosce:Calorimetry                   2.4567        4.49287       5.9493        4.45784      0.994626        10     
reco:pandorapid:Chi2ParticleID                    0.00586106     0.007317     0.00876192    0.00724395    0.000877902      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0480122     0.0666368     0.0818622     0.0656747     0.0103741       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00594756    0.00695637    0.00798454    0.00683537    0.000725786      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.77772       10.9356       16.3412       11.0938       3.25673        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.61031       10.504        15.7088       10.4872       3.20316        10     
reco:emtrkmichelid:EmTrackMichelId                  185.373       266.016       320.022       265.573       42.881         10     
reco:crttag:SingleCRTMatchingProducer              0.0123656     0.0262801     0.0476544     0.0260361     0.0107409       10     
reco:crtreco:TwoCRTMatchingProducer                0.010107      0.0195088     0.0306089     0.0196718    0.00609402       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.959989       1.28038       1.60771       1.24104      0.205778        10     
reco:pandora2Track:LArPandoraTrackCreation          13.0446       32.2425       49.6018       30.4122       12.2289        10     
reco:pandora2calo:Calorimetry                       5.59892       16.0287       26.1406       15.3874       6.09735        10     
reco:pandora2calonosce:Calorimetry                  5.2803        15.1847       26.5526       14.3339       6.05309        10     
reco:pandora2pid:Chi2ParticleID                    0.0122851     0.0178612     0.0217876     0.0178745    0.00295788       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0577133     0.0877933      0.10611      0.0880936     0.0149025       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0125612     0.0178061     0.0216681     0.0177714    0.00285487       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.99331       10.1298       16.1881       10.2213       3.20437        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.77656       11.4689       16.9963       11.7368       3.44461        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.27387       9.70709       13.6665       10.1234       2.71309        10     
[art]:TriggerResults:TriggerResultInserter        3.9439e-05    5.72723e-05   0.000152417   4.39045e-05   3.32005e-05      10     
end_path:out1:RootOutput                           7.361e-06    8.4874e-06     1.387e-05    8.0165e-06    1.82064e-06      10     
end_path:out1:RootOutput(write)                     2.79636       4.20358       5.12086       4.26361      0.704837        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5303.11 MB
  Peak resident set size usage (VmHWM): 4503.09 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.14 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11553244
-rw-r--r--. 1 dune009 dune 2677429405 Jan 14 02:42 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2539589423 Jan 14 02:39 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2431888854 Jan 14 04:21 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2092076400 Jan 13 23:26 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1997686779 Jan 13 23:08 g4_stage1.root
-rw-r--r--. 1 dune009 dune   91241060 Jan 14 04:15 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243297 Jan 14 04:21 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Jan 14 04:21 mem.db
-rw-r--r--. 1 dune009 dune      62376 Jan 14 04:21 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Jan 14 04:21 time.db
-rw-r--r--. 1 dune009 dune        519 Jan 14 02:42 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 13 23:26 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 14 04:21 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Jan 13 23:01 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Jan 14 04:21 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Jan 14 02:51 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Jan 14 02:42 debugprod.log
justIN time: 2026-02-06 04:51:39 UTC       justIN version: 01.06.00