justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 247703.1@dunegpschedd02.fnal.gov

Jobsub ID247703.1@dunegpschedd02.fnal.gov
Workflow ID10287
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-17 15:18:44
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-11-17 17:55:23
From worker nodeHostnamewn-s04-05-03-03.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-17 15:19:27
Input filesmonte-carlo-010287-000001
JobscriptExit code0
Real time2h (9035s)
CPU time2h (8709s = 96%)
Max RSS bytes4385972224 (4182 MiB)
Outputting started2025-11-17 17:50:03
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10287/1/001/reco_stage1.root
Finished2025-11-17 17:55:23
Saved logsjustin-logs:247703.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

awDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 117621 traces, 0 summary
Retagger: tagging trace set: gauss with 86080 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 86080 traces tagged "gauss"
FrameSaver: q=2.94817e+06 n=1461549 tag=gauss
wclsFrameSaver: saving 117621 traces tagged "wiener"
FrameSaver: q=3.13359e+06 n=1259347 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 25341 XUs and 17819 XVs -> 409 XUVs
C:0 T:1 83092 XUs and 78522 XVs -> 10249 XUVs
C:0 T:2 85023 XUs and 106086 XVs -> 5470 XUVs
C:0 T:3 42468 XUs and 43627 XVs -> 268 XUVs
C:0 T:4 7879 XUs and 12568 XVs -> 148 XUVs
C:0 T:5 30862 XUs and 59207 XVs -> 5222 XUVs
C:0 T:6 93567 XUs and 110287 XVs -> 5992 XUVs
C:0 T:7 34824 XUs and 40052 XVs -> 409 XUVs
C:0 T:9 18845 XUs and 29659 XVs -> 1632 XUVs
C:0 T:10 59993 XUs and 57953 XVs -> 7401 XUVs
C:0 T:11 261 XUs and 142 XVs -> 22 XUVs
37222 XUVs total
11336 collection wire objects
33969 potential space points
Neighbour search...
3426999 tests to find 1673426 neighbours
Iterating with no regularization...
Begin: 7.12089e+07
0 4.55628e+07
1 4.40342e+07
2 4.37908e+07
3 4.37097e+07
4 4.36702e+07
Now with regularization...
Begin: 3.44396e+07
0 3.43846e+07
1 3.43531e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410567248
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410567248
fLifetime: 30000 [us]
17-Nov-2025 18:50:01 CET  Closed output file "reco_stage1.root"
17-Nov-2025 18:50:01 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          185.008        249.6        336.27        229.119       45.0769        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00088208    0.00342092    0.00694524    0.00359777    0.00163905       10     
reco:ophit:OpHitFinder                             0.175194       0.25212      0.312657      0.271161      0.0499576       10     
reco:opflash:OpFlashFinder                         0.0929369     0.131195      0.163848      0.131064      0.0224614       10     
reco:opslicer:OpSlicer                             0.323644      0.586799      0.765977      0.657343      0.161248        10     
reco:caldata:DataPrepModule                         3.71257       4.93105       5.89355       5.55734      0.962698        10     
reco:wclsdatasp:WireCellToolkit                     23.9316       30.5093       34.5474       31.0154       3.2367         10     
reco:gaushit:GausHitFinder                         0.653141      0.935568       1.23943      0.935219      0.203278        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000338066   0.000428847   0.000517555   0.000428691   6.08361e-05      10     
reco:reco3d:SpacePointSolver                        6.28828       13.0596       20.2142       13.7475       4.86086        10     
reco:hitpdune:DisambigFromSpacePoints              0.624247       1.06225       1.69807      0.984756      0.296472        10     
reco:pandora:StandardPandora                        10.323        21.6659       34.8322       21.0739       6.71877        10     
reco:pandoraWriter:StandardPandora                 0.242472       0.34178      0.487249      0.326416      0.0779704       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.25366       2.44631       3.53159       2.53409      0.616233        10     
reco:pandoraShower:LArPandoraShowerCreation         1.1331        2.40436       4.20365       2.47605      0.806182        10     
reco:pandoracalo:Calorimetry                       0.822298       1.51026       2.10406       1.45705      0.325802        10     
reco:pandoracalonosce:Calorimetry                  0.726978       1.41564       2.10893       1.38299      0.334154        10     
reco:pandorapid:Chi2ParticleID                    0.00331084    0.00488243    0.00776075    0.00465463    0.00138665       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0295322     0.0446235     0.059528      0.0455696     0.0118881       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00297932     0.0046636    0.00592511    0.00520288    0.00107502       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.18461       2.68398       4.57837       2.75581      0.855328        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.14973       2.58953       4.29843       2.65194      0.797424        10     
reco:emtrkmichelid:EmTrackMichelId                  106.718       133.804       185.495       116.846       27.9818        10     
reco:crttag:SingleCRTMatchingProducer             0.00929856     0.0115849     0.0153294     0.0111995    0.00172719       10     
reco:crtreco:TwoCRTMatchingProducer                0.004592     0.00723707     0.0123883    0.00718401    0.00193675       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.488478       0.73996       0.92515      0.748416      0.114083        10     
reco:pandora2Track:LArPandoraTrackCreation          3.7653        7.9681        12.0007       8.32539       1.99547        10     
reco:pandora2calo:Calorimetry                       2.02484       4.15032       5.96686       4.17204       0.92095        10     
reco:pandora2calonosce:Calorimetry                  1.77482       3.93742       5.9206        3.99607      0.955005        10     
reco:pandora2pid:Chi2ParticleID                   0.00749981     0.0115374     0.0154691     0.0120554     0.0026205       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.040686      0.0603892     0.0728712     0.0666995     0.0118386       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00715135     0.0121163     0.0183183     0.0117967    0.00359672       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.18232       2.96779       4.46437       3.03122      0.789098        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.31207       3.62811       5.50039       3.78275      0.987347        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.33694       3.34316       4.97005       3.48467      0.879232        10     
[art]:TriggerResults:TriggerResultInserter        3.4415e-05    5.68742e-05   0.000136447   4.74295e-05   2.81791e-05      10     
end_path:out1:RootOutput                           8.816e-06    1.0702e-05      1.6e-05     1.01435e-05   1.97225e-06      10     
end_path:out1:RootOutput(write)                     1.65111       2.28901       2.75602       2.39255      0.402115        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5191.91 MB
  Peak resident set size usage (VmHWM): 4385.97 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.03 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9767332
-rw-r--r-- 1 neutrino002 neutrino 2305394913 Nov 17 18:06 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2194587902 Nov 17 18:05 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 1961283419 Nov 17 18:50 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 1772181686 Nov 17 16:36 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1699952246 Nov 17 16:26 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   67717010 Nov 17 18:46 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     243165 Nov 17 18:50 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Nov 17 18:50 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Nov 17 18:50 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Nov 17 18:50 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Nov 17 18:06 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Nov 17 16:36 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Nov 17 18:50 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Nov 17 16:19 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Nov 17 18:50 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Nov 17 18:10 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Nov 17 18:06 debugprod.log
justIN time: 2026-02-09 09:17:42 UTC       justIN version: 01.06.00