justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264904.0@dunegpschedd01.fnal.gov

Jobsub ID264904.0@dunegpschedd01.fnal.gov
Workflow ID11081
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-08 13:28:30
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-12-08 17:45:30
From worker nodeHostnamewn-s03-05-11-02.cr.cnaf.infn.it
cpuinfoAMD EPYC 7282 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-08 13:30:03
Input filesmonte-carlo-011081-000001
JobscriptExit code0
Real time4h (15260s)
CPU time4h (14665s = 96%)
Max RSS bytes4215197696 (4019 MiB)
Outputting started2025-12-08 17:44:24
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11081/1/001/reco_stage1.root
Finished2025-12-08 17:45:30
Saved logsjustin-logs:264904.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

epService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 105003 traces, 0 summary
Retagger: tagging trace set: gauss with 81948 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 81948 traces tagged "gauss"
FrameSaver: q=2.88529e+06 n=1476237 tag=gauss
wclsFrameSaver: saving 105003 traces tagged "wiener"
FrameSaver: q=3.06205e+06 n=1329218 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37053 XUs and 24608 XVs -> 415 XUVs
C:0 T:1 91008 XUs and 53212 XVs -> 6949 XUVs
C:0 T:2 75506 XUs and 110118 XVs -> 2921 XUVs
C:0 T:3 42655 XUs and 47711 XVs -> 412 XUVs
C:0 T:4 10348 XUs and 9366 XVs -> 178 XUVs
C:0 T:5 31149 XUs and 23191 XVs -> 2649 XUVs
C:0 T:6 46841 XUs and 91540 XVs -> 3671 XUVs
C:0 T:7 18354 XUs and 34826 XVs -> 212 XUVs
C:0 T:8 5635 XUs and 5650 XVs -> 67 XUVs
C:0 T:9 32392 XUs and 41732 XVs -> 2648 XUVs
C:0 T:10 30857 XUs and 47775 XVs -> 5845 XUVs
C:0 T:11 375 XUs and 280 XVs -> 62 XUVs
26029 XUVs total
8920 collection wire objects
23846 potential space points
Neighbour search...
1845936 tests to find 946394 neighbours
Iterating with no regularization...
Begin: 7.13887e+07
0 6.14439e+07
1 6.07922e+07
2 6.06943e+07
3 6.06646e+07
Now with regularization...
Begin: 5.26024e+07
0 5.25612e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410992604
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410992604
fLifetime: 30000 [us]
08-Dec-2025 18:44:21 CET  Closed output file "reco_stage1.root"
08-Dec-2025 18:44:21 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          343.481       473.074       616.297       462.419       76.6297        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00117125    0.00500452    0.00842316    0.00546545    0.00242823       10     
reco:ophit:OpHitFinder                             0.242375      0.367485      0.465809      0.367898      0.0645124       10     
reco:opflash:OpFlashFinder                         0.152314       0.2168       0.287968      0.202805      0.0404463       10     
reco:opslicer:OpSlicer                             0.518654      0.916486       1.21161      0.929239      0.214913        10     
reco:caldata:DataPrepModule                         4.79913       6.62577       7.69632       6.82539      0.964646        10     
reco:wclsdatasp:WireCellToolkit                     38.5477       44.7064       49.9403       45.1882       3.33288        10     
reco:gaushit:GausHitFinder                         0.999124       1.48987       1.90941       1.52893      0.296392        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000491125   0.000733218   0.00100818    0.000718102   0.000165408      10     
reco:reco3d:SpacePointSolver                        11.4932       18.3567       38.6335       14.3142       8.39295        10     
reco:hitpdune:DisambigFromSpacePoints               1.03465       1.83882       2.89878       1.67259      0.541287        10     
reco:pandora:StandardPandora                        24.1805       39.0513       58.6913       37.8406       9.66796        10     
reco:pandoraWriter:StandardPandora                 0.344281      0.573533      0.778454      0.571128      0.128202        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.42965       6.28473       8.35221       6.32779       1.77887        10     
reco:pandoraShower:LArPandoraShowerCreation         3.19578       6.70454       9.91353       6.30456       2.30596        10     
reco:pandoracalo:Calorimetry                        1.84991       3.32612       4.74686       3.2444       0.977054        10     
reco:pandoracalonosce:Calorimetry                   1.74204       2.95301       4.45462       2.8726        0.89577        10     
reco:pandorapid:Chi2ParticleID                     0.0049157    0.00745724     0.0102576    0.00703571    0.00185352       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0416071     0.0714475     0.102758      0.0661131     0.0195669       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00481643    0.00723265    0.00930702    0.00726082     0.0013555       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.00573       6.62253       10.069        6.37325       2.60438        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.82154       7.06232       10.4406       7.51786       2.68423        10     
reco:emtrkmichelid:EmTrackMichelId                  208.865       265.941       328.044       257.644       36.8614        10     
reco:crttag:SingleCRTMatchingProducer             0.00850217     0.0200676     0.0385421     0.0190566    0.00766173       10     
reco:crtreco:TwoCRTMatchingProducer               0.00713065     0.0137073     0.0204757     0.0141517    0.00349313       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.72498       1.03908       1.47999       1.0185        0.21074        10     
reco:pandora2Track:LArPandoraTrackCreation          7.66573       18.1889       25.2862       19.0552       5.08844        10     
reco:pandora2calo:Calorimetry                       5.07999       8.87161       13.3649       8.82034       2.50368        10     
reco:pandora2calonosce:Calorimetry                  4.74632       8.16648       11.5128       8.22055       2.14158        10     
reco:pandora2pid:Chi2ParticleID                    0.0150961     0.0192643     0.0242768     0.0190695    0.00337796       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0684551     0.0956904     0.122484      0.0921252     0.0177684       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0142615     0.0190261     0.0231906     0.0196959    0.00280531       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.00921       6.73393       9.09168       6.73606       1.57972        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.76006       7.03629       10.3228       6.77228       1.96932        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.21039       6.09282       9.22884       6.01331       2.02715        10     
[art]:TriggerResults:TriggerResultInserter         4.714e-05    8.00988e-05   0.000159041   7.1611e-05    3.33138e-05      10     
end_path:out1:RootOutput                           9.25e-06     1.1563e-05     1.521e-05    1.0615e-05    1.99705e-06      10     
end_path:out1:RootOutput(write)                     2.55308       3.46071       4.35896       3.3207       0.503903        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5021.88 MB
  Peak resident set size usage (VmHWM): 4215.2 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.03 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10342884
-rw-r--r-- 1 neutrino002 neutrino 2409160009 Dec  8 17:22 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2283828413 Dec  8 17:19 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2086671157 Dec  8 18:44 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 1905611090 Dec  8 14:56 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1832668189 Dec  8 14:38 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   72541060 Dec  8 18:38 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     242697 Dec  8 18:44 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Dec  8 18:44 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Dec  8 18:44 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Dec  8 18:44 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Dec  8 17:22 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec  8 14:56 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec  8 18:44 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Dec  8 14:30 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Dec  8 18:44 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Dec  8 17:32 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Dec  8 17:22 debugprod.log
justIN time: 2026-02-07 16:19:59 UTC       justIN version: 01.06.00