justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 252108.0@dunegpschedd01.fnal.gov

Jobsub ID252108.0@dunegpschedd01.fnal.gov
Workflow ID10219
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-15 17:20:00
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-11-15 19:28:09
From worker nodeHostnamedunegli-7767970-0-fnpc23028.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-15 17:32:03
Input filesmonte-carlo-010219-000001
JobscriptExit code0
Real time1h (6818s)
CPU time1h (6746s = 98%)
Max RSS bytes4107390976 (3917 MiB)
Outputting started2025-11-15 19:25:41
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10219/1/001/reco_stage1.root
Finished2025-11-15 19:28:09
Saved logsjustin-logs:252108.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 109213 traces, 0 summary
Retagger: tagging trace set: gauss with 84750 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 84750 traces tagged "gauss"
FrameSaver: q=2.87811e+06 n=1507347 tag=gauss
wclsFrameSaver: saving 109213 traces tagged "wiener"
FrameSaver: q=3.06057e+06 n=1342743 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 22992 XUs and 20885 XVs -> 198 XUVs
C:0 T:1 39292 XUs and 33894 XVs -> 2786 XUVs
C:0 T:2 189583 XUs and 146245 XVs -> 10119 XUVs
C:0 T:3 58701 XUs and 55639 XVs -> 496 XUVs
C:0 T:4 9177 XUs and 5713 XVs -> 38 XUVs
C:0 T:5 38136 XUs and 15625 XVs -> 2729 XUVs
C:0 T:6 84436 XUs and 78288 XVs -> 4754 XUVs
C:0 T:7 33252 XUs and 30345 XVs -> 240 XUVs
C:0 T:8 9317 XUs and 6779 XVs -> 65 XUVs
C:0 T:9 60087 XUs and 56518 XVs -> 3748 XUVs
C:0 T:10 37588 XUs and 20441 XVs -> 3606 XUVs
C:0 T:11 1425 XUs and 867 XVs -> 146 XUVs
28925 XUVs total
9516 collection wire objects
26385 potential space points
Neighbour search...
2072133 tests to find 994326 neighbours
Iterating with no regularization...
Begin: 6.25028e+07
0 4.67421e+07
1 4.59382e+07
2 4.58001e+07
3 4.5752e+07
4 4.57292e+07
Now with regularization...
Begin: 3.83063e+07
0 3.82889e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410532752
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410532752
fLifetime: 30000 [us]
15-Nov-2025 19:25:40 UTC  Closed output file "reco_stage1.root"
15-Nov-2025 19:25:40 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          126.869        187.5        231.296       183.625       31.0762        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0006911    0.00280118    0.00450249    0.00301319    0.00125076       10     
reco:ophit:OpHitFinder                             0.169496      0.196845      0.219747      0.198224      0.0158758       10     
reco:opflash:OpFlashFinder                         0.0892707     0.117616       0.13735      0.121569       0.01701        10     
reco:opslicer:OpSlicer                             0.322467      0.449487      0.555305      0.446841      0.0600338       10     
reco:caldata:DataPrepModule                         3.34516       3.50855       3.73408       3.49989      0.101665        10     
reco:wclsdatasp:WireCellToolkit                     22.0243       23.0045       23.8677       23.1404      0.637059        10     
reco:gaushit:GausHitFinder                         0.491822      0.687918      0.837077      0.673288      0.115475        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000210993   0.000299453   0.000438017   0.000285696   6.05811e-05      10     
reco:reco3d:SpacePointSolver                        3.2933        7.24023       10.9249       7.39877       2.38707        10     
reco:hitpdune:DisambigFromSpacePoints              0.454689      0.882011       1.2374       0.860892      0.245592        10     
reco:pandora:StandardPandora                        7.90917       15.855        20.4169       16.4193       3.65734        10     
reco:pandoraWriter:StandardPandora                 0.157907       0.22679       0.29118      0.219734      0.0417929       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.1569        1.75296       2.25406       1.78724      0.358817        10     
reco:pandoraShower:LArPandoraShowerCreation         1.00638       2.00966       2.96053       1.94025       0.6175         10     
reco:pandoracalo:Calorimetry                       0.657395       1.08431       1.41251       1.14397      0.242266        10     
reco:pandoracalonosce:Calorimetry                  0.598589      0.996698       1.30563       1.05598      0.228411        10     
reco:pandorapid:Chi2ParticleID                    0.00257691    0.00318057    0.00422853    0.00312367    0.000477231      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0207633     0.0288985     0.0356577     0.0295251     0.0044488       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00239847    0.00281403    0.00334281    0.00284992    0.000268583      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.0423        2.29766       3.41447       2.1865       0.781144        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.01896       2.24011       3.34554       2.1205       0.765955        10     
reco:emtrkmichelid:EmTrackMichelId                  71.3224       102.611       123.56        100.526       15.8176        10     
reco:crttag:SingleCRTMatchingProducer             0.00447069    0.00835468     0.0122803    0.00895857     0.0020176       10     
reco:crtreco:TwoCRTMatchingProducer               0.00307289    0.00458598    0.00576277    0.00455012    0.000872149      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.329657      0.536245       0.70552      0.553766      0.121371        10     
reco:pandora2Track:LArPandoraTrackCreation          3.01852       5.82494       8.37648       5.77773       1.61916        10     
reco:pandora2calo:Calorimetry                       1.56927       3.2411        4.75886       3.21038      0.981496        10     
reco:pandora2calonosce:Calorimetry                  1.48078       3.08766       4.55449       3.05349      0.950163        10     
reco:pandora2pid:Chi2ParticleID                   0.00578421    0.00741556    0.00885605    0.00755112    0.000956923      10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.024541      0.0362347     0.0436132     0.0361323    0.00587917       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00562914    0.00719817    0.00869228    0.00730301    0.000957323      10     
reco:pandora2Shower:LArPandoraShowerCreation        1.18373       2.20886       3.16248       2.1331       0.592048        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.44225       2.89342       4.25919       2.79292      0.880376        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.34446       2.68367       3.96642       2.54622      0.819109        10     
[art]:TriggerResults:TriggerResultInserter        3.0889e-05    3.91889e-05   9.1385e-05    3.3023e-05    1.75884e-05      10     
end_path:out1:RootOutput                           4.378e-06    7.1074e-06    1.1241e-05     7.439e-06    1.95763e-06      10     
end_path:out1:RootOutput(write)                     1.17677       1.71004       2.05895       1.70869      0.277499        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4919.68 MB
  Peak resident set size usage (VmHWM): 4107.39 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.21 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9628040
-rw-r--r-- 1 dunegli fnalgrid 2276701871 Nov 15 18:53 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2164484887 Nov 15 18:52 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1936249890 Nov 15 19:25 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1740392686 Nov 15 17:41 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1672228848 Nov 15 17:35 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   68430030 Nov 15 19:23 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     242548 Nov 15 19:25 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Nov 15 19:25 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Nov 15 19:25 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Nov 15 19:25 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Nov 15 18:53 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Nov 15 17:41 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Nov 15 19:25 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Nov 15 17:32 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Nov 15 19:25 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Nov 15 18:56 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Nov 15 18:53 debugprod.log
justIN time: 2026-02-09 12:24:50 UTC       justIN version: 01.06.00