justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 244489.0@dunegpschedd01.fnal.gov

Jobsub ID244489.0@dunegpschedd01.fnal.gov
Workflow ID9849
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-06 17:50:55
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-11-06 20:43:37
From worker nodeHostnamecn-leo-198.cr.cnaf.infn.it
cpuinfoIntel(R) Xeon(R) Platinum 8480+
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-06 17:55:30
Input filesmonte-carlo-009849-000001
JobscriptExit code0
Real time2h (9873s)
CPU time2h (9524s = 96%)
Max RSS bytes4290011136 (4091 MiB)
Outputting started2025-11-06 20:40:04
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09849/1/001/reco_stage1.root
Finished2025-11-06 20:43:37
Saved logsjustin-logs:244489.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 153845 traces, 0 summary
Retagger: tagging trace set: gauss with 114306 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 114306 traces tagged "gauss"
FrameSaver: q=3.91835e+06 n=2007191 tag=gauss
wclsFrameSaver: saving 153845 traces tagged "wiener"
FrameSaver: q=4.18233e+06 n=1769578 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 62835 XUs and 50704 XVs -> 1154 XUVs
C:0 T:1 170068 XUs and 115781 XVs -> 18239 XUVs
C:0 T:2 84779 XUs and 155358 XVs -> 4756 XUVs
C:0 T:3 38295 XUs and 62787 XVs -> 375 XUVs
C:0 T:4 12078 XUs and 10706 XVs -> 128 XUVs
C:0 T:5 63179 XUs and 47402 XVs -> 5610 XUVs
C:0 T:6 96819 XUs and 146921 XVs -> 4840 XUVs
C:0 T:7 36776 XUs and 56238 XVs -> 307 XUVs
C:0 T:8 10031 XUs and 7441 XVs -> 176 XUVs
C:0 T:9 71178 XUs and 76446 XVs -> 11110 XUVs
C:0 T:10 40715 XUs and 32318 XVs -> 5828 XUVs
C:0 T:11 335 XUs and 262 XVs -> 20 XUVs
52543 XUVs total
14856 collection wire objects
47705 potential space points
Neighbour search...
6562031 tests to find 2992034 neighbours
Iterating with no regularization...
Begin: 7.48583e+07
0 6.09834e+07
1 6.00587e+07
2 5.99087e+07
3 5.98596e+07
Now with regularization...
Begin: 4.83981e+07
0 4.83302e+07
1 4.82901e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410352515
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410352515
fLifetime: 30000 [us]
06-Nov-2025 21:40:02 CET  Closed output file "reco_stage1.root"
06-Nov-2025 21:40:02 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          311.517       365.392       439.236       351.958       45.4044        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000884901   0.00613357     0.0108732    0.00585117    0.00301933       10     
reco:ophit:OpHitFinder                             0.266764      0.307981      0.338225      0.314263      0.0210032       10     
reco:opflash:OpFlashFinder                          0.12496      0.159126      0.186595       0.16267      0.019489        10     
reco:opslicer:OpSlicer                             0.417907      0.521996      0.620247      0.522275      0.0534714       10     
reco:caldata:DataPrepModule                         4.52905       4.66537       4.90976       4.63496      0.105277        10     
reco:wclsdatasp:WireCellToolkit                     61.0949       62.9967       66.5027       62.751        1.44647        10     
reco:gaushit:GausHitFinder                          0.88823       1.06571       1.30302       1.06911      0.135891        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000709165   0.000804363   0.000964397   0.000780267   9.10336e-05      10     
reco:reco3d:SpacePointSolver                        9.05855       14.4544       18.1964       15.1678       3.11118        10     
reco:hitpdune:DisambigFromSpacePoints               1.03591       1.29412       1.65428       1.2705        0.20622        10     
reco:pandora:StandardPandora                        27.2839       41.185        76.7845       37.9014       13.526         10     
reco:pandoraWriter:StandardPandora                 0.434689      0.500506      0.646806      0.477063      0.0614247       10     
reco:pandoraTrack:LArPandoraTrackCreation           5.08067       6.90538       10.2981       6.37012       1.59119        10     
reco:pandoraShower:LArPandoraShowerCreation         4.68193       6.84071       10.1381       6.48284       1.75234        10     
reco:pandoracalo:Calorimetry                        2.44818       3.2219        4.70182       3.08783      0.643885        10     
reco:pandoracalonosce:Calorimetry                   2.27397       3.02179       4.1863        2.91077      0.589542        10     
reco:pandorapid:Chi2ParticleID                    0.00454215    0.00523872    0.00639775    0.00517885    0.000519139      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP          0.03659      0.0600697     0.203518      0.0421242     0.0480968       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.004445     0.00496437    0.00571385    0.00491478    0.000353397      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.36902       6.23786       9.4707        6.14181       1.49161        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.20763       5.93516       9.03302       5.72214       1.48937        10     
reco:emtrkmichelid:EmTrackMichelId                  128.857       147.193       180.898       139.919       17.1371        10     
reco:crttag:SingleCRTMatchingProducer              0.0202645     0.0262997     0.0326848     0.0268394    0.00400992       10     
reco:crtreco:TwoCRTMatchingProducer                0.010952      0.0151047     0.0211454     0.0143255    0.00300752       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.774542      0.993678       1.14581      0.981006      0.104194        10     
reco:pandora2Track:LArPandoraTrackCreation          14.0321       19.3495       27.4114       18.0122       4.68662        10     
reco:pandora2calo:Calorimetry                       6.42237       8.51931       11.8994       8.18391       1.76876        10     
reco:pandora2calonosce:Calorimetry                  5.92403       7.93798       11.2724        7.697        1.68279        10     
reco:pandora2pid:Chi2ParticleID                    0.0101761     0.0123787     0.013907      0.0124166    0.000963887      10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0490435     0.0578588     0.0699574     0.056991     0.00601076       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0102249     0.0120208     0.013699      0.0121675    0.00105538       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.73294       6.28461       9.17119       5.87119       1.3544         10     
reco:pandora2Showercalo:ShowerCalorimetry           5.03707       6.84017       9.90092       6.43192       1.43171        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.30073       6.03842       8.95333       5.54109       1.35239        10     
[art]:TriggerResults:TriggerResultInserter        4.9179e-05    7.29788e-05   0.000139762   6.5089e-05    2.65797e-05      10     
end_path:out1:RootOutput                          1.1396e-05    1.43701e-05   2.3943e-05     1.283e-05    4.04024e-06      10     
end_path:out1:RootOutput(write)                     2.2115        2.5541        3.06819       2.45717      0.281962        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5107.09 MB
  Peak resident set size usage (VmHWM): 4290.01 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.71 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11143240
-rw-r--r-- 1 neutrino002 neutrino 2585758958 Nov  6 20:37 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2446217038 Nov  6 20:35 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2308503512 Nov  6 21:40 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2043867682 Nov  6 19:15 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1941818492 Nov  6 19:02 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   83900870 Nov  6 21:35 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     243041 Nov  6 21:40 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Nov  6 21:40 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Nov  6 21:40 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Nov  6 21:40 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Nov  6 20:37 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Nov  6 19:15 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Nov  6 21:40 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Nov  6 18:55 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Nov  6 21:40 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Nov  6 20:42 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Nov  6 20:37 debugprod.log
justIN time: 2026-02-10 13:27:15 UTC       justIN version: 01.06.00