justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302597.1@dunegpschedd01.fnal.gov

Jobsub ID302597.1@dunegpschedd01.fnal.gov
Workflow ID12608
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-31 09:38:05
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce504
Last heartbeat2026-01-31 11:50:18
From worker nodeHostnameb9p34p1002.cern.ch
cpuinfoAMD EPYC 9655 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-31 09:38:39
Input filesmonte-carlo-012608-000001
JobscriptExit code0
Real time2h (7815s)
CPU time1h (7016s = 89%)
Max RSS bytes4530573312 (4320 MiB)
Outputting started2026-01-31 11:48:55
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12608/1/001/reco_stage1.root
Finished2026-01-31 11:50:18
Saved logsjustin-logs:302597.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

eBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 173277 traces, 0 summary
Retagger: tagging trace set: gauss with 134091 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 134091 traces tagged "gauss"
FrameSaver: q=4.85211e+06 n=2489730 tag=gauss
wclsFrameSaver: saving 173277 traces tagged "wiener"
FrameSaver: q=5.13133e+06 n=2234035 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 35567 XUs and 30386 XVs -> 361 XUVs
C:0 T:1 73768 XUs and 69515 XVs -> 7011 XUVs
C:0 T:2 124109 XUs and 134915 XVs -> 5795 XUVs
C:0 T:3 63860 XUs and 57056 XVs -> 535 XUVs
C:0 T:4 24464 XUs and 28878 XVs -> 399 XUVs
C:0 T:5 134357 XUs and 129688 XVs -> 7664 XUVs
C:0 T:6 110949 XUs and 105812 XVs -> 6555 XUVs
C:0 T:7 44326 XUs and 41928 XVs -> 425 XUVs
C:0 T:8 11982 XUs and 12989 XVs -> 203 XUVs
C:0 T:9 92902 XUs and 120926 XVs -> 8026 XUVs
C:0 T:10 94400 XUs and 50911 XVs -> 12016 XUVs
C:0 T:11 180 XUs and 123 XVs -> 33 XUVs
49023 XUVs total
16225 collection wire objects
45403 potential space points
Neighbour search...
4580365 tests to find 2169216 neighbours
Iterating with no regularization...
Begin: 1.27074e+08
0 1.09318e+08
1 1.08128e+08
2 1.07992e+08
3 1.07954e+08
Now with regularization...
Begin: 9.39255e+07
0 9.3909e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
run: 1 ; subrun: 0 ; event: 10
evttime: 412075085
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412075085
fLifetime: 30000 [us]
31-Jan-2026 12:48:53 CET  Closed output file "reco_stage1.root"
31-Jan-2026 12:48:53 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          199.112       248.064       321.614       239.946       36.8146        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000990002   0.00307507    0.00489244    0.00311246    0.00123886       10     
reco:ophit:OpHitFinder                             0.183954      0.215268      0.265219      0.214892      0.0212242       10     
reco:opflash:OpFlashFinder                         0.095694      0.112723      0.143115      0.109638      0.0127734       10     
reco:opslicer:OpSlicer                             0.460317       0.58792      0.720568      0.588746      0.0742488       10     
reco:caldata:DataPrepModule                         2.56116       3.2391        4.29996        3.026       0.528223        10     
reco:wclsdatasp:WireCellToolkit                     17.1995       20.8369       24.7326       20.2538       2.22912        10     
reco:gaushit:GausHitFinder                         0.644735      0.946482       1.30025      0.964391      0.208562        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000230213   0.000344655   0.000620623   0.000320356   0.000107223      10     
reco:reco3d:SpacePointSolver                        6.04439       11.8757       17.6616       11.5358       3.47379        10     
reco:hitpdune:DisambigFromSpacePoints              0.926374       1.52906       2.24928       1.56159      0.400382        10     
reco:pandora:StandardPandora                        17.3763       34.4568       54.5454       31.3902       10.1727        10     
reco:pandoraWriter:StandardPandora                 0.260466      0.497292      0.673981      0.501551       0.12471        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.74149       3.08968       3.79781       3.11497      0.575541        10     
reco:pandoraShower:LArPandoraShowerCreation         2.12096       3.89049       5.28685       3.97819      0.923789        10     
reco:pandoracalo:Calorimetry                        1.37094       1.7289        2.29283       1.78219      0.256781        10     
reco:pandoracalonosce:Calorimetry                   1.00477       1.49994       2.0395        1.52251      0.279063        10     
reco:pandorapid:Chi2ParticleID                    0.00286877    0.00398095    0.00514449    0.00397338    0.000882751      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0282878     0.0436102     0.0573843     0.0442788    0.00877615       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00273905    0.00374605    0.00476668    0.00359945     0.0007173       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.54208       4.11059       5.95794       4.04728      0.961149        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.34099       3.8861        5.59933       3.67609      0.892033        10     
reco:emtrkmichelid:EmTrackMichelId                  95.5636       116.594       142.918       111.605       14.8187        10     
reco:crttag:SingleCRTMatchingProducer             0.00822359     0.0114385     0.0145007     0.0109887     0.0018276       10     
reco:crtreco:TwoCRTMatchingProducer               0.00517347    0.00629962    0.00860752    0.00579808    0.00104146       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.331386      0.458146      0.530116      0.458829      0.058272        10     
reco:pandora2Track:LArPandoraTrackCreation          8.78419       11.9083       18.5767       10.7689       2.82837        10     
reco:pandora2calo:Calorimetry                       4.34056       5.7321        7.80121       5.21329       1.09849        10     
reco:pandora2calonosce:Calorimetry                  4.26845       5.11598       6.55037       4.86847      0.738961        10     
reco:pandora2pid:Chi2ParticleID                   0.00802016     0.0102025     0.0132814    0.00997406    0.00181315       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0421763     0.0522242     0.0699668     0.0491003    0.00869461       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00806241     0.0102792     0.0129323     0.0106809    0.00142772       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.27122       3.8704        5.31008       3.66275      0.693941        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.77486       4.72477       5.80664       4.65959      0.638769        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.60162       4.15183       5.35093       3.87535      0.608276        10     
[art]:TriggerResults:TriggerResultInserter        2.7601e-05    4.74818e-05   0.000109523   4.0601e-05    2.26468e-05      10     
end_path:out1:RootOutput                           8.763e-06    1.08381e-05   1.5353e-05    1.0541e-05    1.81477e-06      10     
end_path:out1:RootOutput(write)                     2.32207       2.73527       3.84931       2.6143       0.417172        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5406.2 MB
  Peak resident set size usage (VmHWM): 4530.57 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.02 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.01 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.02 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11643680
-rw-r--r--. 1 duneprd np-comp 2687010966 Jan 31 12:05 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2551727242 Jan 31 12:03 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2439833583 Jan 31 12:48 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 2120221271 Jan 31 11:01 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 2032082959 Jan 31 10:43 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   91629140 Jan 31 12:45 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242991 Jan 31 12:48 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 31 12:48 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 31 12:48 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 31 12:48 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 12:05 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 11:01 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 12:48 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 31 10:38 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 31 12:48 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 31 12:10 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 31 12:05 debugprod.log
justIN time: 2026-02-04 09:20:21 UTC       justIN version: 01.06.00