justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302152.1@dunegpschedd01.fnal.gov

Jobsub ID302152.1@dunegpschedd01.fnal.gov
Workflow ID12528
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-29 16:40:09
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2026-01-29 19:58:38
From worker nodeHostnamehnode70.pic.es
cpuinfoAMD EPYC 7443P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-29 16:40:49
Input filesmonte-carlo-012528-000001
JobscriptExit code0
Real time3h (11798s)
CPU time2h (10382s = 87%)
Max RSS bytes4101074944 (3911 MiB)
Outputting started2026-01-29 19:57:28
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12528/1/001/reco_stage1.root
Finished2026-01-29 19:58:38
Saved logsjustin-logs:302152.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

epare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 131531 traces, 0 summary
Retagger: tagging trace set: gauss with 103920 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 103920 traces tagged "gauss"
FrameSaver: q=3.44963e+06 n=1862994 tag=gauss
wclsFrameSaver: saving 131531 traces tagged "wiener"
FrameSaver: q=3.62451e+06 n=1655289 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37767 XUs and 20277 XVs -> 308 XUVs
C:0 T:1 80611 XUs and 61069 XVs -> 8097 XUVs
C:0 T:2 110224 XUs and 98263 XVs -> 6706 XUVs
C:0 T:3 47355 XUs and 37037 XVs -> 354 XUVs
C:0 T:4 15248 XUs and 11238 XVs -> 129 XUVs
C:0 T:5 64427 XUs and 38739 XVs -> 5703 XUVs
C:0 T:6 124511 XUs and 124360 XVs -> 4647 XUVs
C:0 T:7 54256 XUs and 48462 XVs -> 531 XUVs
C:0 T:8 3946 XUs and 5863 XVs -> 48 XUVs
C:0 T:9 21607 XUs and 34201 XVs -> 1530 XUVs
C:0 T:10 52945 XUs and 33478 XVs -> 5563 XUVs
C:0 T:11 59 XUs and 44 XVs -> 7 XUVs
33623 XUVs total
12351 collection wire objects
30965 potential space points
Neighbour search...
3316013 tests to find 1515922 neighbours
Iterating with no regularization...
Begin: 7.73274e+07
0 6.67456e+07
1 6.62112e+07
2 6.61351e+07
3 6.61135e+07
Now with regularization...
Begin: 5.67479e+07
0 5.6729e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412041413
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412041413
fLifetime: 30000 [us]
29-Jan-2026 20:57:26 CET  Closed output file "reco_stage1.root"
29-Jan-2026 20:57:26 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                           213.1        344.75        510.237       348.806       85.9239        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000906833   0.00359803    0.00720601    0.00362125    0.00172544       10     
reco:ophit:OpHitFinder                             0.181492      0.300017      0.384921      0.305999      0.0537559       10     
reco:opflash:OpFlashFinder                         0.0985577     0.153511      0.229706      0.149219      0.034905        10     
reco:opslicer:OpSlicer                             0.435688       0.75398        1.047       0.748475      0.170591        10     
reco:caldata:DataPrepModule                         6.0001        6.46389       6.84736       6.48688      0.283552        10     
reco:wclsdatasp:WireCellToolkit                     26.5711       34.6219       40.4588       34.0755       3.91201        10     
reco:gaushit:GausHitFinder                         0.815699       1.24366       1.80537       1.1342       0.350542        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000350388   0.000537495   0.000699367   0.000551094   0.00011461       10     
reco:reco3d:SpacePointSolver                        5.01405       14.5282       26.8512       13.8817       6.29564        10     
reco:hitpdune:DisambigFromSpacePoints              0.895164       1.57564       2.35553       1.48926      0.523743        10     
reco:pandora:StandardPandora                        18.5411       33.2023       53.2295       33.7839       10.6126        10     
reco:pandoraWriter:StandardPandora                 0.309461      0.445114      0.586184      0.471638      0.101027        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.07051       4.13324       7.61095       3.63191       1.82886        10     
reco:pandoraShower:LArPandoraShowerCreation         1.37229       3.79476       7.4039        3.68341       1.77633        10     
reco:pandoracalo:Calorimetry                       0.916309       2.38472       4.87501       1.93246       1.2164         10     
reco:pandoracalonosce:Calorimetry                  0.862528       2.18739       4.6616        1.77273       1.16193        10     
reco:pandorapid:Chi2ParticleID                    0.00323241    0.00574459    0.00923656    0.00571938    0.00207727       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0235424     0.0502882     0.0790443     0.040262      0.0190616       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0030538     0.0054743    0.00864554    0.00487003    0.00209315       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.39529       4.43545       7.98872       4.45129       2.19814        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.35514       4.23467       7.73211       4.09007       2.23753        10     
reco:emtrkmichelid:EmTrackMichelId                  119.335       182.662       263.786       176.118       42.0012        10     
reco:crttag:SingleCRTMatchingProducer             0.00530516     0.0421475     0.299178      0.012326      0.0859284       10     
reco:crtreco:TwoCRTMatchingProducer               0.00440244    0.00922698     0.0179368    0.00890051    0.00375429       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.431438      0.844516       1.11458      0.924409      0.229619        10     
reco:pandora2Track:LArPandoraTrackCreation          3.32722       13.1937       24.257        13.3272       5.91284        10     
reco:pandora2calo:Calorimetry                       1.63782       7.26031       12.9037       7.68634       3.28871        10     
reco:pandora2calonosce:Calorimetry                  1.72013       6.6472        12.1947       7.22655       3.16145        10     
reco:pandora2pid:Chi2ParticleID                   0.00667219     0.0138151     0.0210931     0.0150684    0.00485626       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0309478     0.0702337     0.106186      0.0850777     0.0268844       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00698454     0.0140249     0.0214982     0.0150315    0.00506199       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.4093        4.64318       7.8497        4.52784       1.99339        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.40746       6.1348        9.41184       6.77054       2.59595        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.49455       5.67823       9.31847       6.08855       2.16652        10     
[art]:TriggerResults:TriggerResultInserter        3.2441e-05    5.69633e-05   0.000136523   5.0056e-05    2.83233e-05      10     
end_path:out1:RootOutput                           5.201e-06    4.82544e-05   0.000418841    6.68e-06     0.000123548      10     
end_path:out1:RootOutput(write)                     1.79597       2.88237       3.83411       2.86773      0.627599        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4922.69 MB
  Peak resident set size usage (VmHWM): 4101.07 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.26 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9558480
-rw-r--r-- 1 dune001 dune 2276116351 Jan 29 19:57 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2163172165 Jan 29 19:56 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 1948897925 Jan 29 20:57 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1698454211 Jan 29 18:09 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1628541163 Jan 29 17:50 g4_stage1.root
-rw-r--r-- 1 dune001 dune   72066960 Jan 29 20:51 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     242836 Jan 29 20:57 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Jan 29 20:57 mem.db
-rw-r--r-- 1 dune001 dune      62376 Jan 29 20:57 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Jan 29 20:57 time.db
-rw-r--r-- 1 dune001 dune        519 Jan 29 19:57 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 29 18:09 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 29 20:57 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Jan 29 17:40 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Jan 29 20:57 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Jan 29 20:03 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Jan 29 19:57 debugprod.log
justIN time: 2026-02-04 15:17:00 UTC       justIN version: 01.06.00