justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 252859.1@dunegpschedd01.fnal.gov

Jobsub ID252859.1@dunegpschedd01.fnal.gov
Workflow ID10262
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-16 16:33:05
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-11-16 18:54:23
From worker nodeHostnamewn-s04-05-09-03.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-16 16:34:12
Input filesmonte-carlo-010262-000001
JobscriptExit code0
Real time2h (8092s)
CPU time2h (7763s = 95%)
Max RSS bytes4336062464 (4135 MiB)
Outputting started2025-11-16 18:49:05
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10262/1/001/reco_stage1.root
Finished2025-11-16 18:54:23
Saved logsjustin-logs:252859.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

epare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 151476 traces, 0 summary
Retagger: tagging trace set: gauss with 111117 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 111117 traces tagged "gauss"
FrameSaver: q=3.89664e+06 n=1956254 tag=gauss
wclsFrameSaver: saving 151476 traces tagged "wiener"
FrameSaver: q=4.1222e+06 n=1702721 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 27906 XUs and 31259 XVs -> 449 XUVs
C:0 T:1 56426 XUs and 57123 XVs -> 5646 XUVs
C:0 T:2 151029 XUs and 120236 XVs -> 20574 XUVs
C:0 T:3 40309 XUs and 38344 XVs -> 236 XUVs
C:0 T:4 14145 XUs and 14691 XVs -> 146 XUVs
C:0 T:5 61993 XUs and 51488 XVs -> 4641 XUVs
C:0 T:6 142907 XUs and 95689 XVs -> 7345 XUVs
C:0 T:7 49952 XUs and 32656 XVs -> 416 XUVs
C:0 T:8 10938 XUs and 12936 XVs -> 129 XUVs
C:0 T:9 84649 XUs and 120113 XVs -> 6167 XUVs
C:0 T:10 50526 XUs and 32939 XVs -> 5507 XUVs
C:0 T:11 457 XUs and 393 XVs -> 33 XUVs
51289 XUVs total
14322 collection wire objects
46898 potential space points
Neighbour search...
7469462 tests to find 3229308 neighbours
Iterating with no regularization...
Begin: 8.23711e+07
0 6.98885e+07
1 6.89249e+07
2 6.87858e+07
3 6.87433e+07
Now with regularization...
Begin: 5.71289e+07
0 5.70817e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410552726
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410552726
fLifetime: 30000 [us]
16-Nov-2025 19:49:04 CET  Closed output file "reco_stage1.root"
16-Nov-2025 19:49:04 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          152.48        228.478       351.851       216.42        52.1596        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000564481   0.00269536    0.00443191    0.00272595     0.0012528       10     
reco:ophit:OpHitFinder                             0.162475      0.207968      0.266376      0.200388      0.0266302       10     
reco:opflash:OpFlashFinder                         0.0937947     0.134651      0.198807      0.127991      0.0286618       10     
reco:opslicer:OpSlicer                              0.31228       0.46585      0.575705      0.463367      0.0802233       10     
reco:caldata:DataPrepModule                         3.36425       3.48205       3.68057       3.47333      0.0839889       10     
reco:wclsdatasp:WireCellToolkit                     21.9389       23.2863       24.9193       22.9979       1.04798        10     
reco:gaushit:GausHitFinder                         0.535705      0.773177       1.09681       0.79068      0.145675        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000269186   0.000346504   0.000511692   0.00031913    8.07759e-05      10     
reco:reco3d:SpacePointSolver                        7.18021       12.0749       27.0301       9.5605        6.4525         10     
reco:hitpdune:DisambigFromSpacePoints              0.670612       1.1432        1.94173       1.04019      0.352722        10     
reco:pandora:StandardPandora                        10.5028       24.3982       51.1368       21.5626       10.6271        10     
reco:pandoraWriter:StandardPandora                 0.196191      0.282379      0.413603      0.266321      0.0596532       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.38588       2.08987       3.79062       2.05141      0.631105        10     
reco:pandoraShower:LArPandoraShowerCreation         1.32812       2.6902        5.40886       2.42501       1.09481        10     
reco:pandoracalo:Calorimetry                       0.804599       1.26377       2.09738       1.22575      0.335346        10     
reco:pandoracalonosce:Calorimetry                  0.729674       1.1631        1.95093       1.13654      0.314668        10     
reco:pandorapid:Chi2ParticleID                    0.00256366    0.00325094    0.00414651      0.00316     0.000486307      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.025834      0.0333891     0.0443264     0.0321268    0.00483998       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00244984    0.00301365    0.00398438    0.00294473    0.000397082      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.34752       3.05093       5.94759       2.76046       1.2252         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.31055       2.97134       5.87491       2.68301       1.21284        10     
reco:emtrkmichelid:EmTrackMichelId                  84.4982       120.758       164.972       117.523       20.7249        10     
reco:crttag:SingleCRTMatchingProducer             0.00627445    0.00955178     0.0143379    0.00927643    0.00232871       10     
reco:crtreco:TwoCRTMatchingProducer               0.00366644    0.00510833    0.00668255    0.00489519    0.000834933      10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.46176       0.64547      0.966635      0.585599      0.139991        10     
reco:pandora2Track:LArPandoraTrackCreation          3.99242       7.59463       14.9386       7.0081        2.88697        10     
reco:pandora2calo:Calorimetry                       2.02181       4.11886       7.65845       3.91779       1.46715        10     
reco:pandora2calonosce:Calorimetry                  1.90977       3.94373       7.30356       3.79936       1.39988        10     
reco:pandora2pid:Chi2ParticleID                   0.00624787    0.00831568     0.012546      0.0078582    0.00161938       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0300716     0.0419558     0.0586101     0.0400124    0.00735923       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00623093    0.00862501     0.0139332    0.00803749     0.0022921       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.53641       2.79536       4.96464       2.64631      0.911115        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.76429       3.59436       6.37401       3.40231       1.21023        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.65773       3.33856       5.84025       3.1351        1.11405        10     
[art]:TriggerResults:TriggerResultInserter        3.0187e-05    4.6246e-05    9.5399e-05    3.5281e-05    2.10652e-05      10     
end_path:out1:RootOutput                           6.462e-06     8.021e-06    1.2173e-05     7.559e-06    1.49628e-06      10     
end_path:out1:RootOutput(write)                     1.37838       2.03089       2.8732        1.91519      0.402258        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5136.71 MB
  Peak resident set size usage (VmHWM): 4336.06 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.20 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10685704
-rw-r--r-- 1 neutrino002 neutrino 2481416924 Nov 16 19:09 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2357262671 Nov 16 19:08 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2190412214 Nov 16 19:49 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 1954957941 Nov 16 17:49 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1876650075 Nov 16 17:40 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   80832310 Nov 16 19:46 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     242818 Nov 16 19:49 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Nov 16 19:49 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Nov 16 19:49 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Nov 16 19:49 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Nov 16 19:09 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Nov 16 17:49 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Nov 16 19:49 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Nov 16 17:34 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Nov 16 19:49 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Nov 16 19:14 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Nov 16 19:09 debugprod.log
justIN time: 2026-02-09 10:58:17 UTC       justIN version: 01.06.00