justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 246902.0@dunegpschedd01.fnal.gov

Jobsub ID246902.0@dunegpschedd01.fnal.gov
Workflow ID9918
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-11 01:45:42
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-11-11 04:59:21
From worker nodeHostnamewn-da-03.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-11 01:45:59
Input filesmonte-carlo-009918-000001
JobscriptExit code0
Real time3h (11502s)
CPU time3h (11410s = 99%)
Max RSS bytes4036079616 (3849 MiB)
Outputting started2025-11-11 04:57:42
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09918/1/001/reco_stage1.root
Finished2025-11-11 04:59:21
Saved logsjustin-logs:246902.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

repare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 112680 traces, 0 summary
Retagger: tagging trace set: gauss with 84442 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 84442 traces tagged "gauss"
FrameSaver: q=3.03292e+06 n=1464914 tag=gauss
wclsFrameSaver: saving 112680 traces tagged "wiener"
FrameSaver: q=3.23972e+06 n=1285638 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 40408 XUs and 37262 XVs -> 402 XUVs
C:0 T:1 103147 XUs and 90544 XVs -> 8292 XUVs
C:0 T:2 56667 XUs and 64417 XVs -> 4339 XUVs
C:0 T:3 26646 XUs and 27983 XVs -> 142 XUVs
C:0 T:4 12370 XUs and 11168 XVs -> 140 XUVs
C:0 T:5 46224 XUs and 31227 XVs -> 4765 XUVs
C:0 T:6 76449 XUs and 83934 XVs -> 5644 XUVs
C:0 T:7 27418 XUs and 34283 XVs -> 554 XUVs
C:0 T:8 8243 XUs and 7868 XVs -> 164 XUVs
C:0 T:9 71147 XUs and 75720 XVs -> 7081 XUVs
C:0 T:10 27449 XUs and 11041 XVs -> 5056 XUVs
C:0 T:11 264 XUs and 154 XVs -> 26 XUVs
36605 XUVs total
10847 collection wire objects
33352 potential space points
Neighbour search...
3450046 tests to find 1634446 neighbours
Iterating with no regularization...
Begin: 8.39156e+07
0 6.34031e+07
1 6.17187e+07
2 6.15102e+07
3 6.14469e+07
4 6.14183e+07
Now with regularization...
Begin: 4.94167e+07
0 4.93131e+07
1 4.92731e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
run: 1 ; subrun: 0 ; event: 10
evttime: 410444752
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410444752
fLifetime: 30000 [us]
11-Nov-2025 11:57:40 CET  Closed output file "reco_stage1.root"
11-Nov-2025 11:57:40 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          276.874       360.738       501.462       354.826       71.2615        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000873536   0.00413523    0.00912227    0.00411455     0.0023771       10     
reco:ophit:OpHitFinder                             0.245655      0.274887      0.314033      0.279245      0.0213971       10     
reco:opflash:OpFlashFinder                         0.118429      0.155349       0.18717      0.151718      0.0234361       10     
reco:opslicer:OpSlicer                             0.511541      0.643925      0.765659      0.638757      0.0701321       10     
reco:caldata:DataPrepModule                         4.78626       5.01556       5.28995       5.00746      0.121898        10     
reco:wclsdatasp:WireCellToolkit                     35.7904       57.0234       67.9422       60.3605       11.104         10     
reco:gaushit:GausHitFinder                         0.875135       1.12259       1.5318        1.07295      0.205166        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000393346   0.000578104   0.000783054   0.000559025   0.000117161      10     
reco:reco3d:SpacePointSolver                        6.08524       13.1418       21.2077       12.611        4.48585        10     
reco:hitpdune:DisambigFromSpacePoints              0.769369       1.30362       2.05472       1.25761      0.361805        10     
reco:pandora:StandardPandora                        13.5138       31.7168       55.2197       25.5946       14.9902        10     
reco:pandoraWriter:StandardPandora                 0.296524      0.375078      0.514733      0.366706      0.0666577       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.10808       3.32893       4.85088       3.39874       0.97638        10     
reco:pandoraShower:LArPandoraShowerCreation         1.91385       3.88798       7.87004       3.69898       1.7685         10     
reco:pandoracalo:Calorimetry                        1.11167       1.84425       2.67722       1.94618      0.533414        10     
reco:pandoracalonosce:Calorimetry                  0.989028       1.69513       2.46554       1.74629      0.529273        10     
reco:pandorapid:Chi2ParticleID                    0.00415413    0.00534817    0.00654001    0.00546387    0.000704969      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0332675     0.0457935     0.0599549     0.046347     0.00822748       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00427954    0.00507832    0.00590247    0.00511712    0.000526271      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.88018       3.99297       7.78162       3.87327       1.81191        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.77676       3.81314       7.60891       3.81555       1.76987        10     
reco:emtrkmichelid:EmTrackMichelId                  146.958       192.47        254.628       192.606       32.7234        10     
reco:crttag:SingleCRTMatchingProducer             0.00715883     0.0107254     0.0170494     0.0103453    0.00283955       10     
reco:crtreco:TwoCRTMatchingProducer               0.00637822    0.00831819     0.0114161     0.0082987    0.00163717       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.60793      0.798889       1.03231      0.779399      0.144765        10     
reco:pandora2Track:LArPandoraTrackCreation          5.53749       11.2373       19.8673       10.8697       4.48081        10     
reco:pandora2calo:Calorimetry                       2.66887       5.42797       9.9829        5.39758       2.24673        10     
reco:pandora2calonosce:Calorimetry                  2.46556       5.14068       9.22768       5.15552       2.07692        10     
reco:pandora2pid:Chi2ParticleID                    0.0100385     0.0132892     0.0169774     0.0130379    0.00229635       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0439406     0.0585278     0.0800877     0.0576867     0.0110885       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0100681     0.0131996     0.0168801     0.0129483    0.00219566       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.36557       4.2517        7.44419       4.22237       1.59754        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.45352       4.73721       8.70585       4.79181       1.96707        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.01866       4.32816       8.6228        4.3351        1.9009         10     
[art]:TriggerResults:TriggerResultInserter        3.0498e-05    4.30476e-05   0.000103556   3.41805e-05   2.09151e-05      10     
end_path:out1:RootOutput                           6.722e-06    1.07081e-05   4.1128e-05     7.354e-06    1.01485e-05      10     
end_path:out1:RootOutput(write)                     2.01927       2.72391       3.6577        2.72922      0.497659        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4840.57 MB
  Peak resident set size usage (VmHWM): 4036.08 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.12 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9516412
-rw-r--r--. 1 dune009 dune 2255341513 Nov 11 10:55 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2142826130 Nov 11 10:53 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 1913339398 Nov 11 11:57 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1711877495 Nov 11 09:02 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1652387830 Nov 11 08:51 g4_stage1.root
-rw-r--r--. 1 dune009 dune   68425630 Nov 11 11:53 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242944 Nov 11 11:57 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov 11 11:57 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov 11 11:57 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov 11 11:57 time.db
-rw-r--r--. 1 dune009 dune        519 Nov 11 10:55 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 11 09:02 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 11 11:57 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov 11 08:46 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov 11 11:57 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov 11 11:00 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov 11 10:55 debugprod.log
justIN time: 2026-02-10 07:56:15 UTC       justIN version: 01.06.00