justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270914.1@dunegpschedd01.fnal.gov

Jobsub ID270914.1@dunegpschedd01.fnal.gov
Workflow ID11419
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-30 14:14:44
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2025-12-30 18:08:41
From worker nodeHostnameccwcondor0393
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-30 14:15:40
Input filesmonte-carlo-011419-000001
JobscriptExit code0
Real time3h (13863s)
CPU time3h (13463s = 97%)
Max RSS bytes4426858496 (4221 MiB)
Outputting started2025-12-30 18:06:44
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11419/1/001/reco_stage1.root
Finished2025-12-30 18:08:41
Saved logsjustin-logs:270914.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

prepare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 122204 traces, 0 summary
Retagger: tagging trace set: gauss with 93105 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 93105 traces tagged "gauss"
FrameSaver: q=3.15447e+06 n=1599452 tag=gauss
wclsFrameSaver: saving 122204 traces tagged "wiener"
FrameSaver: q=3.33553e+06 n=1404509 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38219 XUs and 40595 XVs -> 533 XUVs
C:0 T:1 64221 XUs and 66169 XVs -> 4446 XUVs
C:0 T:2 107672 XUs and 117831 XVs -> 9402 XUVs
C:0 T:3 45039 XUs and 45751 XVs -> 836 XUVs
C:0 T:4 15470 XUs and 10979 XVs -> 130 XUVs
C:0 T:5 57760 XUs and 34734 XVs -> 4184 XUVs
C:0 T:6 73538 XUs and 84073 XVs -> 3540 XUVs
C:0 T:7 29652 XUs and 34946 XVs -> 290 XUVs
C:0 T:8 6145 XUs and 5738 XVs -> 68 XUVs
C:0 T:9 35324 XUs and 42175 XVs -> 2522 XUVs
C:0 T:10 17612 XUs and 9426 XVs -> 3541 XUVs
C:0 T:11 81 XUs and 56 XVs -> 24 XUVs
29516 XUVs total
11089 collection wire objects
25791 potential space points
Neighbour search...
1572177 tests to find 810034 neighbours
Iterating with no regularization...
Begin: 5.38331e+07
0 4.63375e+07
1 4.57461e+07
2 4.56689e+07
3 4.56498e+07
Now with regularization...
Begin: 3.64912e+07
0 3.64804e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411435867
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411435867
fLifetime: 30000 [us]
30-Dec-2025 19:06:42 CET  Closed output file "reco_stage1.root"
30-Dec-2025 19:06:42 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          285.377       428.158       543.184       440.113       71.9839        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00107298    0.00372602    0.00607201    0.00389498    0.00142755       10     
reco:ophit:OpHitFinder                             0.314126      0.353064      0.433861      0.344627      0.0335441       10     
reco:opflash:OpFlashFinder                         0.162044      0.202177      0.264252      0.199805      0.0263428       10     
reco:opslicer:OpSlicer                              0.6999       0.821649      0.948052      0.820782      0.0821265       10     
reco:caldata:DataPrepModule                         6.2087        6.40333       6.60927       6.39677      0.107964        10     
reco:wclsdatasp:WireCellToolkit                     37.5762       40.1719       45.2466       39.0129       2.32573        10     
reco:gaushit:GausHitFinder                          1.02066       1.46263       1.86938        1.477       0.229351        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000456881   0.000597673   0.00091293    0.000579141   0.000122172      10     
reco:reco3d:SpacePointSolver                        12.4834       20.5399       35.3451       19.6546       6.67075        10     
reco:hitpdune:DisambigFromSpacePoints               1.01589       2.07796        3.068        2.20464      0.545392        10     
reco:pandora:StandardPandora                        20.6248       37.094        56.5026       37.0332       11.4469        10     
reco:pandoraWriter:StandardPandora                 0.396071      0.595664      0.739424      0.592228      0.0904427       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.49961       5.11829       6.15881       5.36921       1.04869        10     
reco:pandoraShower:LArPandoraShowerCreation         2.12852       6.30496       9.47793       6.25238       2.00844        10     
reco:pandoracalo:Calorimetry                        1.47335       3.02493       4.22069       3.08808      0.694366        10     
reco:pandoracalonosce:Calorimetry                   1.34755       2.6924        3.63629       2.70609      0.620476        10     
reco:pandorapid:Chi2ParticleID                    0.00523837    0.00637715    0.00791452    0.00622009    0.000838566      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0549339     0.0822634     0.133059      0.0816336     0.0198851       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00481979    0.00613924    0.00737628    0.00608334    0.000698072      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.30766       6.85956       9.94032       6.59561       2.36593        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.1596        6.60285       10.0122       6.35442       2.34398        10     
reco:emtrkmichelid:EmTrackMichelId                  164.962       227.93        276.707       231.17        29.4964        10     
reco:crttag:SingleCRTMatchingProducer              0.0123448     0.0205014     0.0455797     0.0165535    0.00943473       10     
reco:crtreco:TwoCRTMatchingProducer               0.00786299     0.0126597     0.025307      0.0122028    0.00456774       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.841485       1.02021       1.29801       1.00013      0.141266        10     
reco:pandora2Track:LArPandoraTrackCreation          7.91608       18.1562       25.368        17.7792       5.13333        10     
reco:pandora2calo:Calorimetry                       3.78565       8.81169       11.999        9.30733       2.55359        10     
reco:pandora2calonosce:Calorimetry                  3.42725       8.36012       12.9556       8.35366       2.69709        10     
reco:pandora2pid:Chi2ParticleID                    0.0108426     0.0160278     0.0190611     0.0172673    0.00290016       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0705647     0.0981799     0.124323      0.0966648     0.0171606       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0121383     0.0165573     0.0282781     0.0158212    0.00432391       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.72765       5.8677        9.11812       5.94438       1.7958         10     
reco:pandora2Showercalo:ShowerCalorimetry           2.96803       7.40773       12.1444       7.12321       2.61148        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.91355       6.45953       9.51127       6.32542       1.95535        10     
[art]:TriggerResults:TriggerResultInserter        5.9851e-05    9.15516e-05   0.000184519   7.2595e-05    4.39523e-05      10     
end_path:out1:RootOutput                           9.604e-06    1.13172e-05   1.7176e-05    1.05765e-05   2.09259e-06      10     
end_path:out1:RootOutput(write)                     2.46629       3.3735        4.14247       3.37498      0.470072        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5264.36 MB
  Peak resident set size usage (VmHWM): 4426.86 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.33 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.93 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10896576
-rw-r--r-- 1 dune001 lbno 2526811491 Dec 30 17:53 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2399976885 Dec 30 17:50 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2237782788 Dec 30 19:06 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 2006304543 Dec 30 15:36 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1906099667 Dec 30 15:22 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   80503960 Dec 30 19:02 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242766 Dec 30 19:06 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Dec 30 19:06 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Dec 30 19:06 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Dec 30 19:06 time.db
-rw-r--r-- 1 dune001 lbno        519 Dec 30 17:53 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Dec 30 15:36 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Dec 30 19:06 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Dec 30 15:15 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Dec 30 19:06 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Dec 30 17:59 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Dec 30 17:53 debugprod.log
justIN time: 2026-02-07 00:50:23 UTC       justIN version: 01.06.00