justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270505.0@dunegpschedd01.fnal.gov

Jobsub ID270505.0@dunegpschedd01.fnal.gov
Workflow ID11382
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-28 15:22:07
SiteUS_FNAL-T1
EntryCMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat2025-12-28 17:54:00
From worker nodeHostnamedunegli-49579-0-cmswn6018.fnal.gov
cpuinfoAMD EPYC 9354 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-28 15:23:12
Input filesmonte-carlo-011382-000001
JobscriptExit code0
Real time2h (9037s)
CPU time2h (8992s = 99%)
Max RSS bytes4665913344 (4449 MiB)
Outputting started2025-12-28 17:53:49
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11382/1/001/reco_stage1.root
Finished2025-12-28 17:54:00
Saved logsjustin-logs:270505.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

asedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 191825 traces, 0 summary
Retagger: tagging trace set: gauss with 139828 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 139828 traces tagged "gauss"
FrameSaver: q=4.9813e+06 n=2427693 tag=gauss
wclsFrameSaver: saving 191825 traces tagged "wiener"
FrameSaver: q=5.30794e+06 n=2117241 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 42646 XUs and 37083 XVs -> 424 XUVs
C:0 T:1 88148 XUs and 81342 XVs -> 7333 XUVs
C:0 T:2 226633 XUs and 176039 XVs -> 36139 XUVs
C:0 T:3 84648 XUs and 50520 XVs -> 870 XUVs
C:0 T:4 24908 XUs and 22265 XVs -> 561 XUVs
C:0 T:5 117808 XUs and 95752 XVs -> 13180 XUVs
C:0 T:6 118168 XUs and 100705 XVs -> 4801 XUVs
C:0 T:7 48539 XUs and 38889 XVs -> 497 XUVs
C:0 T:8 8524 XUs and 8400 XVs -> 150 XUVs
C:0 T:9 56131 XUs and 113060 XVs -> 8476 XUVs
C:0 T:10 78047 XUs and 51025 XVs -> 11992 XUVs
C:0 T:11 156 XUs and 72 XVs -> 25 XUVs
84448 XUVs total
17372 collection wire objects
77728 potential space points
Neighbour search...
14480978 tests to find 6347282 neighbours
Iterating with no regularization...
Begin: 8.76505e+07
0 6.39716e+07
1 6.24547e+07
2 6.2208e+07
3 6.21309e+07
4 6.2097e+07
Now with regularization...
Begin: 4.74325e+07
0 4.73391e+07
1 4.72801e+07
2 4.72339e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411396587
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411396587
fLifetime: 30000 [us]
28-Dec-2025 17:53:48 UTC  Closed output file "reco_stage1.root"
28-Dec-2025 17:53:48 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          218.18        256.989       311.448       253.976       34.8973        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000540245   0.00286866     0.0044134    0.00318618    0.00126496       10     
reco:ophit:OpHitFinder                             0.222543      0.244157      0.269293      0.247416      0.0135844       10     
reco:opflash:OpFlashFinder                         0.161709       0.19207      0.223457      0.189611      0.0204501       10     
reco:opslicer:OpSlicer                             0.411513      0.496972      0.531763      0.501401       0.03376        10     
reco:caldata:DataPrepModule                         3.13298       3.21199       3.36741       3.20959      0.0630257       10     
reco:wclsdatasp:WireCellToolkit                     20.1802       21.3969       22.396        21.5396       0.67577        10     
reco:gaushit:GausHitFinder                         0.698762      0.779659      0.928905      0.770979      0.0713008       10     
reco:nhitsfilter:NumberOfHitsFilter               0.000227057   0.000275172   0.000331466   0.000273558   3.14651e-05      10     
reco:reco3d:SpacePointSolver                        9.20271       17.6979       30.0749       16.9165       5.43357        10     
reco:hitpdune:DisambigFromSpacePoints              0.933669       1.24459       1.59782       1.24232      0.235675        10     
reco:pandora:StandardPandora                        18.5921       25.1131       33.7585       24.5013       5.38853        10     
reco:pandoraWriter:StandardPandora                 0.243483      0.293203      0.353819      0.290306      0.038847        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.18899       2.77158       3.62898        2.682       0.479039        10     
reco:pandoraShower:LArPandoraShowerCreation         2.46791       3.39989       4.58422       3.32604      0.862146        10     
reco:pandoracalo:Calorimetry                        1.36745       1.63151        2.038        1.58894      0.225777        10     
reco:pandoracalonosce:Calorimetry                   1.27519       1.52758       1.89597       1.49605      0.209654        10     
reco:pandorapid:Chi2ParticleID                     0.0031845    0.00345688    0.00411757    0.00341655    0.000266507      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0293242     0.0346602     0.0412958     0.0348899    0.00375336       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00301093    0.00324805    0.00355818    0.00322599    0.000181377      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.78625       3.75926       5.01162       3.66645      0.857227        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.72426       3.67825       4.89252       3.60784      0.827163        10     
reco:emtrkmichelid:EmTrackMichelId                  115.792       134.858       162.45        133.748       15.4494        10     
reco:crttag:SingleCRTMatchingProducer             0.00623566    0.00837622     0.0104867    0.00813823    0.00118654       10     
reco:crtreco:TwoCRTMatchingProducer                0.0045539    0.00561816    0.00748623    0.00535388    0.000878498      10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.47285      0.612794       0.73758      0.617047      0.0793432       10     
reco:pandora2Track:LArPandoraTrackCreation          6.96857       9.70466       13.1062       9.6238        2.28541        10     
reco:pandora2calo:Calorimetry                       3.88984       5.11004       6.50072       5.07825      0.996098        10     
reco:pandora2calonosce:Calorimetry                  3.7322        4.89356       6.21417       4.87141      0.945932        10     
reco:pandora2pid:Chi2ParticleID                   0.00728568    0.00834701     0.0101175    0.00809286    0.000943135      10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0352082     0.0413457     0.0473627     0.0409549    0.00440765       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00707741     0.008166     0.00988905    0.00797879    0.000940864      10     
reco:pandora2Shower:LArPandoraShowerCreation        2.59697        3.484        4.54749       3.33568      0.746392        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.38752       4.4357        5.56464       4.34249      0.826228        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.12507       4.10961       5.31721       4.00357       0.79691        10     
[art]:TriggerResults:TriggerResultInserter        2.6119e-05    3.30784e-05   7.6156e-05    2.8008e-05    1.44805e-05      10     
end_path:out1:RootOutput                           5.809e-06    6.3286e-06     9.214e-06     5.989e-06    9.75382e-07      10     
end_path:out1:RootOutput(write)                     1.83252       2.14917       2.5999        2.11686      0.269269        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5483.68 MB
  Peak resident set size usage (VmHWM): 4665.91 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.02 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.01 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.02 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 12374920
-rw-r--r-- 1 dunegli fnalgrid 2804123553 Dec 28 17:09 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2651946297 Dec 28 17:08 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2576292871 Dec 28 17:53 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2315102684 Dec 28 15:36 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2229164772 Dec 28 15:26 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   94678230 Dec 28 17:50 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     243326 Dec 28 17:53 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Dec 28 17:53 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Dec 28 17:53 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Dec 28 17:53 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Dec 28 17:09 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Dec 28 15:36 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Dec 28 17:53 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Dec 28 15:23 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Dec 28 17:53 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Dec 28 17:13 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Dec 28 17:09 debugprod.log
justIN time: 2026-02-07 02:18:58 UTC       justIN version: 01.06.00