justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 294537.1@dunegpschedd02.fnal.gov

Jobsub ID294537.1@dunegpschedd02.fnal.gov
Workflow ID12365
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-24 11:01:35
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce504
Last heartbeat2026-01-24 14:43:37
From worker nodeHostnameb9p06p0505.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-24 11:02:48
Input filesmonte-carlo-012365-000001
JobscriptExit code0
Real time3h (13116s)
CPU time3h (12387s = 94%)
Max RSS bytes4099272704 (3909 MiB)
Outputting started2026-01-24 14:41:25
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12365/1/001/reco_stage1.root
Finished2026-01-24 14:43:37
Saved logsjustin-logs:294537.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 134037 traces, 0 summary
Retagger: tagging trace set: gauss with 95988 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95988 traces tagged "gauss"
FrameSaver: q=3.32337e+06 n=1628551 tag=gauss
wclsFrameSaver: saving 134037 traces tagged "wiener"
FrameSaver: q=3.55254e+06 n=1406353 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28889 XUs and 29360 XVs -> 276 XUVs
C:0 T:1 67044 XUs and 56422 XVs -> 4612 XUVs
C:0 T:2 206334 XUs and 198149 XVs -> 16666 XUVs
C:0 T:3 68064 XUs and 62125 XVs -> 1144 XUVs
C:0 T:4 9639 XUs and 6775 XVs -> 102 XUVs
C:0 T:5 42957 XUs and 23227 XVs -> 3713 XUVs
C:0 T:6 109502 XUs and 141620 XVs -> 9586 XUVs
C:0 T:7 37291 XUs and 39950 XVs -> 400 XUVs
C:0 T:8 6998 XUs and 7013 XVs -> 59 XUVs
C:0 T:9 39151 XUs and 47745 XVs -> 3530 XUVs
C:0 T:10 35094 XUs and 32611 XVs -> 6630 XUVs
C:0 T:11 88 XUs and 57 XVs -> 10 XUVs
46728 XUVs total
12510 collection wire objects
41629 potential space points
Neighbour search...
4519645 tests to find 2264868 neighbours
Iterating with no regularization...
Begin: 7.16466e+07
0 5.67338e+07
1 5.56017e+07
2 5.54162e+07
3 5.53566e+07
4 5.53291e+07
Now with regularization...
Begin: 4.57578e+07
0 4.56935e+07
1 4.56584e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411936082
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411936082
fLifetime: 30000 [us]
24-Jan-2026 15:41:22 CET  Closed output file "reco_stage1.root"
24-Jan-2026 15:41:22 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          252.95        348.973       456.613       343.48        63.2991        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00102204    0.00372572    0.00660551    0.00379211    0.00172506       10     
reco:ophit:OpHitFinder                             0.262453      0.312638      0.389464      0.318321      0.0389912       10     
reco:opflash:OpFlashFinder                         0.138571      0.173733       0.22296       0.17258      0.0261838       10     
reco:opslicer:OpSlicer                             0.540468      0.682555      0.973647      0.657124      0.128366        10     
reco:caldata:DataPrepModule                         5.15066       5.8802        6.96177       5.77171      0.486699        10     
reco:wclsdatasp:WireCellToolkit                     34.7842       38.7516       41.8336        38.9         2.3421         10     
reco:gaushit:GausHitFinder                         0.743529       1.1834        1.7417        1.15426      0.247792        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000414164   0.000515222   0.000639915   0.000500499   7.41576e-05      10     
reco:reco3d:SpacePointSolver                        8.21703       14.9381       22.0246       14.2538       4.59049        10     
reco:hitpdune:DisambigFromSpacePoints               1.15492       1.65144       2.26518       1.66511       0.34475        10     
reco:pandora:StandardPandora                        17.7013       36.3336       63.649        33.5736       15.2929        10     
reco:pandoraWriter:StandardPandora                 0.316121      0.406046      0.511578      0.395758      0.0635533       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.5389        3.55501       5.17399       3.53025      0.729151        10     
reco:pandoraShower:LArPandoraShowerCreation         1.54787       3.56076       5.50029       3.76708       1.15394        10     
reco:pandoracalo:Calorimetry                        1.41253       2.11813       3.05569       2.17463       0.47308        10     
reco:pandoracalonosce:Calorimetry                   1.45807       2.02726       3.01896       2.03843      0.448791        10     
reco:pandorapid:Chi2ParticleID                    0.00569081    0.00668605    0.00858472    0.00632256    0.000925614      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0457509     0.056601      0.0757371     0.054653     0.00909139       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00488229    0.00627814    0.00867424     0.0060408    0.00104613       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.84852       4.0677        7.02048       3.96365       1.48375        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.57512       3.9315        6.62467       3.77266       1.44522        10     
reco:emtrkmichelid:EmTrackMichelId                  146.971       188.477       225.981       191.391       26.9925        10     
reco:crttag:SingleCRTMatchingProducer             0.00984915     0.0139842     0.0161583     0.014543     0.00187105       10     
reco:crtreco:TwoCRTMatchingProducer               0.00575491    0.00746598    0.00912001     0.0078425    0.00120641       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.656959      0.880825      0.984659      0.951077      0.112106        10     
reco:pandora2Track:LArPandoraTrackCreation          5.90976       11.6359       18.4159       12.0232       3.56996        10     
reco:pandora2calo:Calorimetry                       2.72156       6.01865       9.45397       6.23553       2.04964        10     
reco:pandora2calonosce:Calorimetry                  2.32024       5.7153        10.3407       5.92938       2.29807        10     
reco:pandora2pid:Chi2ParticleID                   0.00664155     0.0135117     0.023684      0.0127891    0.00457054       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0340191     0.0664888     0.115364      0.0655035     0.0220943       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00619811     0.0139771     0.0230143     0.0134363    0.00443156       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.16855       3.9913        6.7781        4.10327       1.3097         10     
reco:pandora2Showercalo:ShowerCalorimetry           2.44809       5.05424       8.34113       5.41872       1.71665        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.2101        4.61096       6.94297       4.89896       1.46132        10     
[art]:TriggerResults:TriggerResultInserter         4.184e-05    6.16176e-05   0.000119141   5.60555e-05   2.07107e-05      10     
end_path:out1:RootOutput                           6.78e-06     1.00681e-05     1.3e-05      1.01e-05     1.67916e-06      10     
end_path:out1:RootOutput(write)                     2.04452       2.73934       3.90445       2.66412      0.533539        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4961.61 MB
  Peak resident set size usage (VmHWM): 4099.27 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.62 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10221292
-rw-r--r--. 1 duneprd np-comp 2405647394 Jan 24 14:39 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2287082996 Jan 24 14:37 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2084817496 Jan 24 15:41 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1846309977 Jan 24 12:23 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1768233501 Jan 24 12:11 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   73898020 Jan 24 15:37 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242976 Jan 24 15:41 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 24 15:41 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 24 15:41 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 24 15:41 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 14:39 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 12:23 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 15:41 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 24 12:02 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 24 15:41 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 24 14:46 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 24 14:39 debugprod.log
justIN time: 2026-02-05 01:20:16 UTC       justIN version: 01.06.00