justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 296900.1@dunegpschedd02.fnal.gov

Jobsub ID296900.1@dunegpschedd02.fnal.gov
Workflow ID12600
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-30 21:51:34
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce504
Last heartbeat2026-01-31 00:10:32
From worker nodeHostnameb9p34p1002.cern.ch
cpuinfoAMD EPYC 9655 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-30 21:52:30
Input filesmonte-carlo-012600-000001
JobscriptExit code0
Real time2h (8205s)
CPU time1h (6438s = 78%)
Max RSS bytes3998040064 (3812 MiB)
Outputting started2026-01-31 00:09:17
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12600/1/001/reco_stage1.root
Finished2026-01-31 00:10:32
Saved logsjustin-logs:296900.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 161128 traces, 0 summary
Retagger: tagging trace set: gauss with 125714 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 125714 traces tagged "gauss"
FrameSaver: q=4.56634e+06 n=2282923 tag=gauss
wclsFrameSaver: saving 161128 traces tagged "wiener"
FrameSaver: q=4.8357e+06 n=2040994 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33639 XUs and 46704 XVs -> 676 XUVs
C:0 T:1 80763 XUs and 99776 XVs -> 7779 XUVs
C:0 T:2 175300 XUs and 185371 XVs -> 10851 XUVs
C:0 T:3 77517 XUs and 70053 XVs -> 779 XUVs
C:0 T:4 8837 XUs and 11273 XVs -> 63 XUVs
C:0 T:5 28685 XUs and 26843 XVs -> 2070 XUVs
C:0 T:6 183455 XUs and 160362 XVs -> 9438 XUVs
C:0 T:7 66324 XUs and 51484 XVs -> 830 XUVs
C:0 T:8 10222 XUs and 9270 XVs -> 250 XUVs
C:0 T:9 70551 XUs and 72391 XVs -> 9820 XUVs
C:0 T:10 47440 XUs and 24332 XVs -> 4587 XUVs
C:0 T:11 85 XUs and 48 XVs -> 8 XUVs
47151 XUVs total
14826 collection wire objects
42099 potential space points
Neighbour search...
3341281 tests to find 1713864 neighbours
Iterating with no regularization...
Begin: 9.28654e+07
0 7.76089e+07
1 7.62314e+07
2 7.60015e+07
3 7.59335e+07
Now with regularization...
Begin: 6.22328e+07
0 6.22103e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 412065955
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412065955
fLifetime: 30000 [us]
31-Jan-2026 01:09:14 CET  Closed output file "reco_stage1.root"
31-Jan-2026 01:09:14 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          158.954       218.815       282.42        217.311       32.4037        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00124026    0.00357865    0.00504946    0.00369114    0.00131889       10     
reco:ophit:OpHitFinder                             0.176389       0.21528      0.280952      0.213885      0.028342        10     
reco:opflash:OpFlashFinder                         0.0782288     0.107498      0.178539      0.097178      0.0288048       10     
reco:opslicer:OpSlicer                             0.425253      0.496701      0.607295       0.49138      0.0580334       10     
reco:caldata:DataPrepModule                         3.11156       3.41414       3.75613       3.3814       0.214174        10     
reco:wclsdatasp:WireCellToolkit                     20.3985       21.4516       22.549        21.2576      0.815831        10     
reco:gaushit:GausHitFinder                         0.622501       0.80554      0.994098      0.779949      0.130109        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000226687   0.000331301   0.000580032   0.000306545   0.00010716       10     
reco:reco3d:SpacePointSolver                        7.06887       11.662        21.556        9.71226       4.55769        10     
reco:hitpdune:DisambigFromSpacePoints              0.787516       1.16921       1.60576       1.11323      0.240882        10     
reco:pandora:StandardPandora                        12.2573       24.1645       45.0085       22.2094       8.84487        10     
reco:pandoraWriter:StandardPandora                 0.267518      0.429256      0.701069      0.405253      0.126152        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.98961       2.98336       4.52765       2.74375      0.741951        10     
reco:pandoraShower:LArPandoraShowerCreation         1.5952        3.01828       4.28652       3.04396      0.755492        10     
reco:pandoracalo:Calorimetry                        1.01038       1.73481       3.5452        1.43543       0.68268        10     
reco:pandoracalonosce:Calorimetry                  0.934354       1.40512       1.82872       1.29158       0.30777        10     
reco:pandorapid:Chi2ParticleID                    0.00306832    0.00362157    0.00483466    0.00345525    0.000552173      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0274656     0.0358923     0.0436318     0.0353278    0.00467556       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00312751    0.00364309     0.0045487    0.00339997    0.000513575      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.42938       3.2112        4.76644       3.22808      0.887891        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.33835       3.42376       6.4106        3.12122       1.30558        10     
reco:emtrkmichelid:EmTrackMichelId                  81.1199       106.027       126.251       104.553       12.9536        10     
reco:crttag:SingleCRTMatchingProducer             0.00936638     0.0132947      0.01952      0.011717     0.00324676       10     
reco:crtreco:TwoCRTMatchingProducer               0.00519906    0.00683132    0.00862359     0.0067495    0.00122397       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.351816      0.472302      0.691524      0.443716      0.113799        10     
reco:pandora2Track:LArPandoraTrackCreation          5.86102       9.89778       15.8331       9.26482       2.80594        10     
reco:pandora2calo:Calorimetry                       2.78639       4.65567       6.79177       4.43982       1.01556        10     
reco:pandora2calonosce:Calorimetry                  2.40962       4.32422       6.07943       4.18315      0.938576        10     
reco:pandora2pid:Chi2ParticleID                   0.00758999    0.00930299     0.0119855    0.00908615    0.00131978       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0408459     0.0535159     0.0784218     0.0504675     0.0119591       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00747834    0.00986103     0.0123202    0.00999826    0.00149312       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.22155       3.37589       4.5465        3.28516      0.694084        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.15961       3.83007       5.10018       3.84417      0.788364        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.38595       3.56872       4.5363        3.27596      0.708695        10     
[art]:TriggerResults:TriggerResultInserter        3.2919e-05    5.21537e-05   0.000142773   4.39455e-05   3.07438e-05      10     
end_path:out1:RootOutput                           7.521e-06     8.852e-06    1.4061e-05     8.262e-06    1.86684e-06      10     
end_path:out1:RootOutput(write)                     1.90141       2.7029        3.27364       2.81783      0.499452        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4951.15 MB
  Peak resident set size usage (VmHWM): 3998.04 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.02 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.10 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.02 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10453712
-rw-r--r--. 1 duneprd np-comp 2443426573 Jan 31 00:28 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2319058082 Jan 31 00:27 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2147638546 Jan 31 01:09 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1900201942 Jan 30 23:26 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1814734077 Jan 30 22:58 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   78925240 Jan 31 01:05 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242890 Jan 31 01:09 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 31 01:09 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 31 01:09 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 31 01:09 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 00:28 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 30 23:26 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 01:09 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 30 22:52 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 31 01:09 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 31 00:34 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 31 00:28 debugprod.log
justIN time: 2026-02-04 10:45:26 UTC       justIN version: 01.06.00