justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265174.1@dunegpschedd02.fnal.gov

Jobsub ID265174.1@dunegpschedd02.fnal.gov
Workflow ID11428
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-30 21:37:06
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce503
Last heartbeat2025-12-31 01:40:13
From worker nodeHostnameb9p11p2849.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-30 21:38:05
Input filesmonte-carlo-011428-000001
JobscriptExit code0
Real time4h (14444s)
CPU time3h (13115s = 90%)
Max RSS bytes4109996032 (3919 MiB)
Outputting started2025-12-31 01:38:50
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11428/1/001/reco_stage1.root
Finished2025-12-31 01:40:13
Saved logsjustin-logs:265174.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 159379 traces, 0 summary
Retagger: tagging trace set: gauss with 113615 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 113615 traces tagged "gauss"
FrameSaver: q=4.36048e+06 n=1943121 tag=gauss
wclsFrameSaver: saving 159379 traces tagged "wiener"
FrameSaver: q=4.67852e+06 n=1697074 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 39385 XUs and 37778 XVs -> 922 XUVs
C:0 T:1 98579 XUs and 117827 XVs -> 15255 XUVs
C:0 T:2 71747 XUs and 83665 XVs -> 4050 XUVs
C:0 T:3 38027 XUs and 39819 XVs -> 462 XUVs
C:0 T:4 10707 XUs and 13301 XVs -> 95 XUVs
C:0 T:5 51075 XUs and 69239 XVs -> 9499 XUVs
C:0 T:6 154954 XUs and 143060 XVs -> 7788 XUVs
C:0 T:7 58239 XUs and 51583 XVs -> 815 XUVs
C:0 T:8 9340 XUs and 9105 XVs -> 183 XUVs
C:0 T:9 69242 XUs and 104432 XVs -> 10956 XUVs
C:0 T:10 44838 XUs and 37965 XVs -> 9342 XUVs
C:0 T:11 346 XUs and 289 XVs -> 45 XUVs
59412 XUVs total
12881 collection wire objects
53740 potential space points
Neighbour search...
7064430 tests to find 3358454 neighbours
Iterating with no regularization...
Begin: 9.88732e+07
0 7.9177e+07
1 7.72831e+07
2 7.69887e+07
3 7.69076e+07
4 7.68754e+07
Now with regularization...
Begin: 6.14296e+07
0 6.13653e+07
1 6.13237e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411441906
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411441906
fLifetime: 30000 [us]
31-Dec-2025 02:38:47 CET  Closed output file "reco_stage1.root"
31-Dec-2025 02:38:47 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          254.026       363.491       459.694       379.838       64.7658        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000992494   0.00359857    0.00513512    0.00388101    0.00134509       10     
reco:ophit:OpHitFinder                             0.230478      0.317609      0.382224      0.336053      0.0555906       10     
reco:opflash:OpFlashFinder                         0.128561      0.172867      0.213715       0.17434      0.0296006       10     
reco:opslicer:OpSlicer                             0.497412      0.755903      0.964432      0.798805       0.15803        10     
reco:caldata:DataPrepModule                         4.93216       5.87579       7.78227       5.86035      0.913345        10     
reco:wclsdatasp:WireCellToolkit                     33.4977       36.2308       41.0925       35.299        2.34636        10     
reco:gaushit:GausHitFinder                          0.76421       1.1907        1.8152        1.07461      0.344057        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000295984   0.000534339   0.00074792    0.000554748   0.000144744      10     
reco:reco3d:SpacePointSolver                        8.56023       18.7158       29.046        17.1383       6.42817        10     
reco:hitpdune:DisambigFromSpacePoints              0.974733       1.90963       2.54738       2.11306      0.571408        10     
reco:pandora:StandardPandora                        18.3578       39.8443       92.9842       38.6477       20.2277        10     
reco:pandoraWriter:StandardPandora                 0.247448      0.441151       0.64494      0.395121      0.133845        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.79033       3.62852       4.93726       4.1063       0.946979        10     
reco:pandoraShower:LArPandoraShowerCreation         1.78008       3.71768       5.64399       3.87636       1.0555         10     
reco:pandoracalo:Calorimetry                       0.976449       2.04842       2.78596       2.06027      0.577767        10     
reco:pandoracalonosce:Calorimetry                  0.999293       1.98725       2.5182        2.05852      0.504298        10     
reco:pandorapid:Chi2ParticleID                     0.0034366    0.00648088    0.00923759    0.00674661    0.00185607       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0415074     0.0648039     0.0841401     0.0670325     0.0136923       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00367658    0.00647308    0.00859821    0.00655479    0.00148509       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.16883       4.16089       5.46799       4.70151       1.23634        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.79611       3.92055       5.45373       4.29074       1.29252        10     
reco:emtrkmichelid:EmTrackMichelId                  146.464       194.481       229.527       202.946        28.71         10     
reco:crttag:SingleCRTMatchingProducer             0.00872659     0.0138896     0.0193156     0.0139758    0.00267715       10     
reco:crtreco:TwoCRTMatchingProducer               0.00582005    0.00879383     0.011071     0.00897022    0.00172023       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.666286       0.92634       1.2348       0.941963      0.194397        10     
reco:pandora2Track:LArPandoraTrackCreation          6.58915       12.4008       18.6391       12.7379       3.90244        10     
reco:pandora2calo:Calorimetry                       2.62245       6.22535       9.0245        6.4463        2.05349        10     
reco:pandora2calonosce:Calorimetry                  2.70545       5.98032       8.33647       6.03697       1.84459        10     
reco:pandora2pid:Chi2ParticleID                   0.00934377     0.0156209     0.0201806     0.0174513    0.00387698       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0471858     0.0798294      0.10794      0.0849576     0.0186125       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00915293     0.015408      0.0194149     0.0166963    0.00357645       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.84441       4.52149       6.31562       4.51536       1.1612         10     
reco:pandora2Showercalo:ShowerCalorimetry           2.73469       5.3732        7.32566       5.42197       1.47998        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.06648       5.18787       7.05011       4.86244       1.26372        10     
[art]:TriggerResults:TriggerResultInserter        5.0021e-05    8.58971e-05   0.000170202   7.4206e-05    3.28748e-05      10     
end_path:out1:RootOutput                           9.51e-06     1.27702e-05    2.031e-05    1.23105e-05   2.77093e-06      10     
end_path:out1:RootOutput(write)                     2.34221       3.14918       3.87207       3.32505      0.535267        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4956.65 MB
  Peak resident set size usage (VmHWM): 4110 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.26 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.67 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9711372
-rw-r--r--. 1 duneprd np-comp 2296933447 Dec 31 01:35 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2179439566 Dec 31 01:33 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 1980207666 Dec 31 02:38 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1737350939 Dec 30 23:10 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1675504552 Dec 30 22:46 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   74266520 Dec 31 02:33 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243215 Dec 31 02:38 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Dec 31 02:38 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Dec 31 02:38 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Dec 31 02:38 time.db
-rw-r--r--. 1 duneprd np-comp        519 Dec 31 01:35 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Dec 30 23:10 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Dec 31 02:38 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Dec 30 22:38 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Dec 31 02:38 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Dec 31 01:42 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Dec 31 01:35 debugprod.log
justIN time: 2026-02-07 00:49:52 UTC       justIN version: 01.06.00