justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265426.1@dunegpschedd02.fnal.gov

Jobsub ID265426.1@dunegpschedd02.fnal.gov
Workflow ID11466
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-01 16:11:23
SiteUS_FNAL-T1
EntryCMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat2026-01-01 19:26:27
From worker nodeHostnamedunegli-49724-0-cmswn2357.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-01 16:27:12
Input filesmonte-carlo-011466-000001
JobscriptExit code0
Real time2h (10740s)
CPU time2h (10677s = 99%)
Max RSS bytes4002816000 (3817 MiB)
Outputting started2026-01-01 19:26:13
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11466/1/001/reco_stage1.root
Finished2026-01-01 19:26:27
Saved logsjustin-logs:265426.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 107518 traces, 0 summary
Retagger: tagging trace set: gauss with 79878 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 79878 traces tagged "gauss"
FrameSaver: q=2.93687e+06 n=1395970 tag=gauss
wclsFrameSaver: saving 107518 traces tagged "wiener"
FrameSaver: q=3.16433e+06 n=1243326 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 31824 XUs and 34614 XVs -> 307 XUVs
C:0 T:1 57272 XUs and 59176 XVs -> 3300 XUVs
C:0 T:2 83436 XUs and 84551 XVs -> 6263 XUVs
C:0 T:3 38622 XUs and 33802 XVs -> 419 XUVs
C:0 T:4 13572 XUs and 15666 XVs -> 197 XUVs
C:0 T:5 67926 XUs and 62575 XVs -> 6201 XUVs
C:0 T:6 67759 XUs and 61849 XVs -> 3652 XUVs
C:0 T:7 29523 XUs and 27889 XVs -> 258 XUVs
C:0 T:8 3737 XUs and 4372 XVs -> 180 XUVs
C:0 T:9 17670 XUs and 25792 XVs -> 2298 XUVs
C:0 T:10 18401 XUs and 12277 XVs -> 2295 XUVs
C:0 T:11 108 XUs and 92 XVs -> 17 XUVs
25387 XUVs total
8586 collection wire objects
22800 potential space points
Neighbour search...
1316920 tests to find 628364 neighbours
Iterating with no regularization...
Begin: 7.04745e+07
0 5.48603e+07
1 5.33883e+07
2 5.31506e+07
3 5.30847e+07
4 5.30581e+07
Now with regularization...
Begin: 4.38726e+07
0 4.38211e+07
1 4.38041e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411477744
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411477744
fLifetime: 30000 [us]
01-Jan-2026 19:26:11 UTC  Closed output file "reco_stage1.root"
01-Jan-2026 19:26:11 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          299.241       350.708       450.399       339.081       45.6932        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000857937   0.00310169    0.00593407    0.00254467    0.00158809       10     
reco:ophit:OpHitFinder                             0.212664      0.255166      0.282212      0.253595      0.0211498       10     
reco:opflash:OpFlashFinder                         0.126353      0.159933       0.20599      0.160174      0.0229105       10     
reco:opslicer:OpSlicer                             0.528699      0.645056      0.751911      0.659421      0.0783079       10     
reco:caldata:DataPrepModule                         5.46765       5.82373       5.95708       5.84006      0.140147        10     
reco:wclsdatasp:WireCellToolkit                     42.8921       53.0113       60.6014       53.9412       6.11592        10     
reco:gaushit:GausHitFinder                          1.18081       1.37923       1.69029       1.3517       0.168176        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000348873   0.00047784    0.000628256   0.000464158   8.68174e-05      10     
reco:reco3d:SpacePointSolver                        9.29973       12.6103       23.5012       10.3352       4.62278        10     
reco:hitpdune:DisambigFromSpacePoints               1.67934       2.10584       2.95829       1.98323      0.366356        10     
reco:pandora:StandardPandora                        26.4525       34.2492       50.4288       31.1049       8.30734        10     
reco:pandoraWriter:StandardPandora                 0.363914      0.422714      0.563671      0.393919      0.0625945       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.06013       3.99028       5.10339       3.79301      0.757163        10     
reco:pandoraShower:LArPandoraShowerCreation          2.924        3.9602        6.12451       3.8554       0.993646        10     
reco:pandoracalo:Calorimetry                        1.68804       2.11414       2.63743       2.00166      0.374803        10     
reco:pandoracalonosce:Calorimetry                   1.46266       1.91672       2.41357       1.92014       0.35781        10     
reco:pandorapid:Chi2ParticleID                    0.00445417     0.0055794    0.00638761     0.0056394    0.000640492      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0592176     0.0664935     0.0797329     0.0657004    0.00616341       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00455487    0.00526021    0.00601078    0.00530162    0.000515564      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.78349       4.34134       7.70092       4.0663        1.50911        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.70805       4.15961       7.17701       3.95757       1.41927        10     
reco:emtrkmichelid:EmTrackMichelId                  160.117       174.76        207.428       169.413       15.0209        10     
reco:crttag:SingleCRTMatchingProducer              0.0136917     0.0166419     0.0245475     0.0157929    0.00301313       10     
reco:crtreco:TwoCRTMatchingProducer               0.00790184    0.00974395     0.0128673    0.00895633    0.00163967       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.3669        1.67345       1.98895       1.66887       0.19461        10     
reco:pandora2Track:LArPandoraTrackCreation          9.27054       13.4293       20.5562       12.8184       3.45497        10     
reco:pandora2calo:Calorimetry                       4.23392       6.09506       8.75482       5.87642       1.62791        10     
reco:pandora2calonosce:Calorimetry                  3.94311       5.73397       8.28941       5.49327       1.56678        10     
reco:pandora2pid:Chi2ParticleID                    0.0114586     0.0135767     0.0166514     0.0133659     0.0017222       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0734505     0.0820667     0.0979286     0.0781113    0.00788577       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0110241     0.0132399     0.0166085     0.0131733     0.0017255       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.4217        4.60275       6.63226       4.28304       1.02374        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.71323       5.42078       8.0128        5.06811       1.44552        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.47346       5.01519       7.45231       4.65318       1.38128        10     
[art]:TriggerResults:TriggerResultInserter        4.6132e-05    6.48887e-05   0.000120965   5.87165e-05   2.02753e-05      10     
end_path:out1:RootOutput                           8.184e-06    1.02867e-05    1.441e-05    1.01815e-05   1.53594e-06      10     
end_path:out1:RootOutput(write)                     2.29257       2.53481       3.11995       2.40722      0.285023        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4817.1 MB
  Peak resident set size usage (VmHWM): 4002.82 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.73 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9427592
-rw-r--r-- 1 dunegli fnalgrid 2241952477 Jan  1 18:25 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2131540748 Jan  1 18:23 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1895445982 Jan  1 19:26 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1695708151 Jan  1 16:43 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1621531664 Jan  1 16:31 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   66996180 Jan  1 19:22 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     242901 Jan  1 19:26 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Jan  1 19:26 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Jan  1 19:26 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Jan  1 19:26 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Jan  1 18:25 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Jan  1 16:43 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Jan  1 19:26 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Jan  1 16:27 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Jan  1 19:26 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Jan  1 18:30 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Jan  1 18:25 debugprod.log
justIN time: 2026-02-06 22:22:29 UTC       justIN version: 01.06.00