justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264905.1@dunegpschedd01.fnal.gov

Jobsub ID264905.1@dunegpschedd01.fnal.gov
Workflow ID11082
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-08 13:28:30
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-12-08 18:20:01
From worker nodeHostnamewn-s03-05-11-02.cr.cnaf.infn.it
cpuinfoAMD EPYC 7282 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-08 13:30:18
Input filesmonte-carlo-011082-000001
JobscriptExit code0
Real time4h (17316s)
CPU time4h (16750s = 96%)
Max RSS bytes4370190336 (4167 MiB)
Outputting started2025-12-08 18:18:55
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11082/1/001/reco_stage1.root
Finished2025-12-08 18:20:01
Saved logsjustin-logs:264905.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 147178 traces, 0 summary
Retagger: tagging trace set: gauss with 109020 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 109020 traces tagged "gauss"
FrameSaver: q=3.77851e+06 n=1874148 tag=gauss
wclsFrameSaver: saving 147178 traces tagged "wiener"
FrameSaver: q=4.04007e+06 n=1643604 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34502 XUs and 28545 XVs -> 445 XUVs
C:0 T:1 57761 XUs and 50312 XVs -> 4677 XUVs
C:0 T:2 150485 XUs and 183242 XVs -> 16439 XUVs
C:0 T:3 51535 XUs and 60424 XVs -> 807 XUVs
C:0 T:4 5768 XUs and 5415 XVs -> 27 XUVs
C:0 T:5 13676 XUs and 11770 XVs -> 1128 XUVs
C:0 T:6 214063 XUs and 242839 XVs -> 18221 XUVs
C:0 T:7 67025 XUs and 67843 XVs -> 949 XUVs
C:0 T:8 5042 XUs and 5702 XVs -> 98 XUVs
C:0 T:9 26452 XUs and 35741 XVs -> 2008 XUVs
C:0 T:10 69524 XUs and 58216 XVs -> 9630 XUVs
C:0 T:11 497 XUs and 282 XVs -> 44 XUVs
54473 XUVs total
13532 collection wire objects
48901 potential space points
Neighbour search...
6358079 tests to find 3175690 neighbours
Iterating with no regularization...
Begin: 8.55455e+07
0 6.94385e+07
1 6.82193e+07
2 6.80262e+07
3 6.79686e+07
Now with regularization...
Begin: 5.64609e+07
0 5.63713e+07
1 5.63031e+07
2 5.6255e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410992591
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410992591
fLifetime: 30000 [us]
08-Dec-2025 19:18:53 CET  Closed output file "reco_stage1.root"
08-Dec-2025 19:18:53 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          435.809       606.165       788.962       594.426       109.191        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00164376    0.00471434    0.00950346    0.00443115    0.00234497       10     
reco:ophit:OpHitFinder                             0.308835      0.387127      0.439998      0.390202      0.0459554       10     
reco:opflash:OpFlashFinder                         0.188982      0.239865      0.297378      0.244917      0.0325178       10     
reco:opslicer:OpSlicer                             0.683335       1.02058       1.19865      0.989688      0.155429        10     
reco:caldata:DataPrepModule                         5.13864       6.62302       7.40948       6.96935      0.764014        10     
reco:wclsdatasp:WireCellToolkit                     36.9916       44.3767       56.4793       44.3033       5.70883        10     
reco:gaushit:GausHitFinder                          1.30922       1.74886       2.3242        1.70847       0.29924        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000673107   0.000963611   0.00165458    0.00094098    0.000258634      10     
reco:reco3d:SpacePointSolver                        12.1386       22.9651       33.5831       23.0858       6.06897        10     
reco:hitpdune:DisambigFromSpacePoints               1.72147       2.58552       3.91154       2.48853       0.73011        10     
reco:pandora:StandardPandora                        34.2493       73.8728       131.638       66.9022       28.5254        10     
reco:pandoraWriter:StandardPandora                 0.459529      0.682134      0.865656      0.702478      0.126078        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.68971       8.01422       11.1549       8.06905       2.01374        10     
reco:pandoraShower:LArPandoraShowerCreation         6.80327       10.4399       15.7874       9.50121       3.38202        10     
reco:pandoracalo:Calorimetry                        2.19087       4.40921       6.19535       4.53076       1.28881        10     
reco:pandoracalonosce:Calorimetry                   2.24636       4.27385       5.95636       4.38384       1.31396        10     
reco:pandorapid:Chi2ParticleID                    0.00659696     0.0090988     0.0113515    0.00971591    0.00177691       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0534622     0.0899391      0.11154      0.0943883     0.018353        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00526979    0.00811047     0.0106512    0.00856208    0.00165006       10     
reco:pandoraShowercalo:ShowerCalorimetry            6.46911       10.285        14.8372       9.61181       3.17356        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.57546       9.98456       14.6387       9.6457        3.18322        10     
reco:emtrkmichelid:EmTrackMichelId                  231.327       316.083       401.353       322.707       55.8487        10     
reco:crttag:SingleCRTMatchingProducer              0.0126197     0.0249006     0.037354      0.0253115    0.00698863       10     
reco:crtreco:TwoCRTMatchingProducer               0.00844276     0.0191976     0.0297791     0.0200515    0.00556318       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.852782       1.32061       1.69512       1.30399      0.254396        10     
reco:pandora2Track:LArPandoraTrackCreation          16.6881       27.9185       44.9288       25.842        9.90441        10     
reco:pandora2calo:Calorimetry                       8.16122       13.7706       20.7694       12.6842       4.69472        10     
reco:pandora2calonosce:Calorimetry                  7.59347       11.9132       19.2454       11.0633       4.37442        10     
reco:pandora2pid:Chi2ParticleID                     0.01229      0.0212449     0.029977      0.0199479    0.00613657       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0656311     0.107355      0.155266      0.104376      0.0310945       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0141035     0.0209652     0.0285543     0.0191948    0.00510581       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.44581       9.39475       14.9681       9.32497       2.99907        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.35548       10.4076       16.5785       10.8121       3.37556        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.25483       8.84287       14.6466       9.03771       3.16281        10     
[art]:TriggerResults:TriggerResultInserter        5.2621e-05    8.84808e-05   0.000251432   7.31805e-05   5.50239e-05      10     
end_path:out1:RootOutput                           1.012e-05    1.44593e-05    2.885e-05    1.2291e-05    5.72309e-06      10     
end_path:out1:RootOutput(write)                     2.69787       4.08788       5.44868       4.40292       0.98955        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5173.14 MB
  Peak resident set size usage (VmHWM): 4370.19 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.15 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11081476
-rw-r--r-- 1 neutrino002 neutrino 2559359984 Dec  8 17:35 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2425107422 Dec  8 17:32 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2293310126 Dec  8 19:18 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2031477609 Dec  8 14:58 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1951469098 Dec  8 14:39 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   86074910 Dec  8 19:12 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     243096 Dec  8 19:18 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Dec  8 19:18 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Dec  8 19:18 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Dec  8 19:18 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Dec  8 17:35 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec  8 14:58 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec  8 19:18 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Dec  8 14:30 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Dec  8 19:18 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Dec  8 17:45 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Dec  8 17:35 debugprod.log
justIN time: 2026-02-07 16:25:46 UTC       justIN version: 01.06.00