justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302562.0@dunegpschedd01.fnal.gov

Jobsub ID302562.0@dunegpschedd01.fnal.gov
Workflow ID12609
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-31 08:46:03
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce514
Last heartbeat2026-01-31 12:39:30
From worker nodeHostnameb9p10p7953.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-31 08:46:58
Input filesmonte-carlo-012609-000001
JobscriptExit code0
Real time3h (13858s)
CPU time3h (12630s = 91%)
Max RSS bytes4527140864 (4317 MiB)
Outputting started2026-01-31 12:37:57
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12609/1/001/reco_stage1.root
Finished2026-01-31 12:39:30
Saved logsjustin-logs:302562.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 141538 traces, 0 summary
Retagger: tagging trace set: gauss with 105721 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 105721 traces tagged "gauss"
FrameSaver: q=3.77117e+06 n=1830232 tag=gauss
wclsFrameSaver: saving 141538 traces tagged "wiener"
FrameSaver: q=4.01497e+06 n=1616165 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37575 XUs and 33198 XVs -> 506 XUVs
C:0 T:1 76530 XUs and 62209 XVs -> 10157 XUVs
C:0 T:2 85710 XUs and 96118 XVs -> 3053 XUVs
C:0 T:3 40782 XUs and 41867 XVs -> 406 XUVs
C:0 T:4 15155 XUs and 18285 XVs -> 157 XUVs
C:0 T:5 50501 XUs and 50415 XVs -> 6265 XUVs
C:0 T:6 96888 XUs and 96947 XVs -> 3878 XUVs
C:0 T:7 40735 XUs and 39525 XVs -> 301 XUVs
C:0 T:8 10419 XUs and 9923 XVs -> 198 XUVs
C:0 T:9 90796 XUs and 116303 XVs -> 12109 XUVs
C:0 T:10 23349 XUs and 14448 XVs -> 3015 XUVs
C:0 T:11 16 XUs and 12 XVs -> 0 XUVs
40045 XUVs total
12286 collection wire objects
36938 potential space points
Neighbour search...
4294352 tests to find 1936684 neighbours
Iterating with no regularization...
Begin: 8.96491e+07
0 7.38675e+07
1 7.27298e+07
2 7.25683e+07
3 7.25216e+07
Now with regularization...
Begin: 5.9896e+07
0 5.98335e+07
1 5.98025e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412075049
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412075049
fLifetime: 30000 [us]
31-Jan-2026 13:37:54 CET  Closed output file "reco_stage1.root"
31-Jan-2026 13:37:54 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          254.783       348.223       469.445       345.041       70.9704        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00117645    0.00339586    0.00583458    0.00342691    0.00149253       10     
reco:ophit:OpHitFinder                             0.217769      0.289587      0.358498      0.293751      0.044926        10     
reco:opflash:OpFlashFinder                          0.10995      0.170715      0.219994      0.171415      0.0324373       10     
reco:opslicer:OpSlicer                             0.442446      0.651091      0.847025      0.707624      0.142417        10     
reco:caldata:DataPrepModule                         4.22054       5.40122       6.43271       5.71985      0.708682        10     
reco:wclsdatasp:WireCellToolkit                     26.2004       31.7516        36.24        31.0766       3.16581        10     
reco:gaushit:GausHitFinder                         0.688137       1.14907       1.63749       1.1897       0.243384        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000339432   0.000523855   0.000846784   0.000497293   0.00015929       10     
reco:reco3d:SpacePointSolver                        9.68385       17.7273       35.3867       16.1584       7.10632        10     
reco:hitpdune:DisambigFromSpacePoints               1.11071       1.65247       2.57189       1.64704      0.414483        10     
reco:pandora:StandardPandora                        20.8496       49.2262       175.586       31.8824       43.9258        10     
reco:pandoraWriter:StandardPandora                 0.281689      0.421387      0.585461      0.410115      0.101568        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.32494       3.47165       4.95945       3.59187       0.85875        10     
reco:pandoraShower:LArPandoraShowerCreation         2.27351       3.74067       6.30368       3.53448       1.12027        10     
reco:pandoracalo:Calorimetry                        1.36401       2.0854        2.91299       2.04523      0.477038        10     
reco:pandoracalonosce:Calorimetry                   1.24401       1.82602       2.45849       1.87117       0.39247        10     
reco:pandorapid:Chi2ParticleID                    0.00331292     0.0050408    0.00805811    0.00463854    0.00140094       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0348117     0.0590546     0.0881549     0.0581989     0.0178746       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00329338    0.00532967    0.00768366    0.00510637    0.00141055       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.64921       4.20006       6.67129       3.9235        1.1418         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.65418       3.97572       6.46775       3.60712       1.19061        10     
reco:emtrkmichelid:EmTrackMichelId                  137.959       179.032       237.271       178.808       27.0214        10     
reco:crttag:SingleCRTMatchingProducer              0.0093272     0.0135807     0.0212812     0.0123963    0.00391117       10     
reco:crtreco:TwoCRTMatchingProducer               0.00466657    0.00807251     0.0163821    0.00732377    0.00303744       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.669203      0.898005       1.21527      0.885351       0.16385        10     
reco:pandora2Track:LArPandoraTrackCreation          6.79757       11.4458       20.2198       10.6654       3.52131        10     
reco:pandora2calo:Calorimetry                       3.54967       5.85949       9.2902        5.78071       1.64717        10     
reco:pandora2calonosce:Calorimetry                  3.54721       5.80371       7.74704       6.17034       1.47233        10     
reco:pandora2pid:Chi2ParticleID                   0.00804688      0.01335      0.0199073     0.0120618    0.00389682       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0440827     0.0694479     0.0989355     0.0672453     0.0195436       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00771459     0.0133327     0.0200586     0.0126733    0.00399111       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.72381       4.06284       6.29667       4.03714       1.05364        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.42816       5.14722       7.63734       4.9482        1.2236         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.12944       5.00112       7.13994       4.77327       1.30369        10     
[art]:TriggerResults:TriggerResultInserter         3.424e-05    6.34472e-05   0.000158891   5.9225e-05    3.39409e-05      10     
end_path:out1:RootOutput                           8.521e-06    1.44391e-05    4.478e-05     1.163e-05    1.02198e-05      10     
end_path:out1:RootOutput(write)                     2.07466       2.93951       4.00289       2.88353      0.576719        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5403.6 MB
  Peak resident set size usage (VmHWM): 4527.14 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.39 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10557744
-rw-r--r--. 1 duneprd np-comp 2478022584 Jan 31 12:36 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2358691921 Jan 31 12:33 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2179260691 Jan 31 13:37 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1900654775 Jan 31 10:16 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1814653392 Jan 31 09:55 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   79227190 Jan 31 13:33 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242883 Jan 31 13:37 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 31 13:37 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 31 13:37 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 31 13:37 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 12:36 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 10:16 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 13:37 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 31 09:46 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 31 13:37 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 31 12:43 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 31 12:36 debugprod.log
justIN time: 2026-02-04 09:21:35 UTC       justIN version: 01.06.00