justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271809.0@dunegpschedd01.fnal.gov

Jobsub ID271809.0@dunegpschedd01.fnal.gov
Workflow ID11531
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-05 08:00:06
SiteUK_Bristol
EntryCMSHTPC_T2_UK_SGrid_Bristol_lcgce02
Last heartbeat2026-01-05 13:09:12
From worker nodeHostnamehd85.dice.priv
cpuinfoAMD EPYC 7551P 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit259200 (72 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-05 08:00:41
Input filesmonte-carlo-011531-000001
JobscriptExit code0
Real time5h (18421s)
CPU time4h (17923s = 97%)
Max RSS bytes4147195904 (3955 MiB)
Outputting started2026-01-05 13:07:43
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11531/1/001/reco_stage1.root
Finished2026-01-05 13:09:12
Saved logsjustin-logs:271809.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 112650 traces, 0 summary
Retagger: tagging trace set: gauss with 83765 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 83765 traces tagged "gauss"
FrameSaver: q=2.84665e+06 n=1498263 tag=gauss
wclsFrameSaver: saving 112650 traces tagged "wiener"
FrameSaver: q=3.03126e+06 n=1319520 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34648 XUs and 40909 XVs -> 1199 XUVs
C:0 T:1 84105 XUs and 93961 XVs -> 9657 XUVs
C:0 T:2 149077 XUs and 88794 XVs -> 22450 XUVs
C:0 T:3 31090 XUs and 26399 XVs -> 268 XUVs
C:0 T:4 13296 XUs and 16659 XVs -> 254 XUVs
C:0 T:5 89983 XUs and 83282 XVs -> 10301 XUVs
C:0 T:6 61389 XUs and 69597 XVs -> 2114 XUVs
C:0 T:8 4299 XUs and 7308 XVs -> 68 XUVs
C:0 T:9 53101 XUs and 210617 XVs -> 16171 XUVs
C:0 T:10 42567 XUs and 14358 XVs -> 3548 XUVs
C:0 T:11 61 XUs and 47 XVs -> 9 XUVs
66039 XUVs total
11059 collection wire objects
60275 potential space points
Neighbour search...
14218695 tests to find 5905530 neighbours
Iterating with no regularization...
Begin: 5.90673e+07
0 4.33964e+07
1 4.25121e+07
2 4.23693e+07
3 4.23194e+07
4 4.2295e+07
Now with regularization...
Begin: 3.51157e+07
0 3.50501e+07
1 3.50072e+07
2 3.49728e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411551339
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411551339
fLifetime: 30000 [us]
05-Jan-2026 13:07:40 UTC  Closed output file "reco_stage1.root"
05-Jan-2026 13:07:40 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          413.044       543.447       724.116       529.693       82.7539        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00120792    0.00354053    0.00517633    0.00380871    0.00123086       10     
reco:ophit:OpHitFinder                             0.304244      0.355567      0.397773      0.354421      0.0251395       10     
reco:opflash:OpFlashFinder                         0.173867      0.215308      0.269799      0.209093      0.0333766       10     
reco:opslicer:OpSlicer                             0.835292       1.07785       1.38239       1.06278      0.148442        10     
reco:caldata:DataPrepModule                         7.07665       7.38298       8.0896        7.28043      0.276526        10     
reco:wclsdatasp:WireCellToolkit                     43.7144       50.0322       59.0185       48.8409       4.60401        10     
reco:gaushit:GausHitFinder                          1.51069       1.92529       2.48085       1.91189      0.289247        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000524247   0.000651032   0.000814964   0.000636974   8.82977e-05      10     
reco:reco3d:SpacePointSolver                        12.9988       25.0456       38.3825       23.2056       8.05584        10     
reco:hitpdune:DisambigFromSpacePoints               1.73036       2.3551        3.70128       2.32767      0.553034        10     
reco:pandora:StandardPandora                        31.9322       51.0082       98.2902       45.9868       17.9109        10     
reco:pandoraWriter:StandardPandora                 0.491712      0.581443      0.754512      0.574188      0.0790256       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.65592       6.11697       9.38231       5.93687       1.59331        10     
reco:pandoraShower:LArPandoraShowerCreation         4.92569       7.52491       12.8063       6.61085       2.42648        10     
reco:pandoracalo:Calorimetry                        2.42119       3.54487       5.16236       3.4249       0.806748        10     
reco:pandoracalonosce:Calorimetry                   2.12403       3.20898       4.76543       3.00962      0.809105        10     
reco:pandorapid:Chi2ParticleID                    0.00680546    0.00809821     0.0103642    0.00781646    0.00113859       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0673777     0.0852548     0.101968      0.0875111    0.00997108       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00663901    0.00780932    0.00990069    0.00751942    0.000883763      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.32677       7.72879       13.2164       7.0564        2.61223        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.16699       7.57002       12.9675       6.68276       2.67981        10     
reco:emtrkmichelid:EmTrackMichelId                  235.737       293.113       350.895       290.471       33.7754        10     
reco:crttag:SingleCRTMatchingProducer              0.0143443     0.0216381     0.0336815     0.0211011     0.005001        10     
reco:crtreco:TwoCRTMatchingProducer                0.0110743     0.0168053     0.0276068     0.0154349    0.00466337       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.08287       1.34608       1.81396       1.34036      0.193165        10     
reco:pandora2Track:LArPandoraTrackCreation          13.1277       21.8628       36.1983       19.4888       7.19512        10     
reco:pandora2calo:Calorimetry                       6.09844       10.9493       20.093        9.91297       4.06648        10     
reco:pandora2calonosce:Calorimetry                  5.6182        10.296        17.7763       10.1023       3.59997        10     
reco:pandora2pid:Chi2ParticleID                    0.0151579     0.0217726     0.0381971     0.0201719     0.006133        10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0869592     0.111485      0.172163      0.106043      0.0227131       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0157045     0.021469      0.0376768     0.0198275    0.00587621       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.28018       8.13909       13.3465       7.77149       2.39872        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.07346       9.45699       15.0892       9.21092       3.00663        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.21335       8.09901       11.4464       8.21316       2.24772        10     
[art]:TriggerResults:TriggerResultInserter        6.4381e-05    8.58155e-05   0.000158789   7.78415e-05   2.57391e-05      10     
end_path:out1:RootOutput                          1.1221e-05    1.35303e-05   1.9156e-05     1.308e-05    2.13395e-06      10     
end_path:out1:RootOutput(write)                     3.17868       4.06695       5.5209        3.91245      0.701247        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4956.93 MB
  Peak resident set size usage (VmHWM): 4147.2 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.73 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9699032
-rw-r--r-- 1 dnpil001 users 2294955205 Jan  5 11:34 detsim_stage2.root
-rw-r--r-- 1 dnpil001 users 2179436835 Jan  5 11:31 detsim_stage1.root
-rw-r--r-- 1 dnpil001 users 1978625008 Jan  5 13:07 reco_stage1.root
-rw-r--r-- 1 dnpil001 users 1733042788 Jan  5 08:24 g4_stage2.root
-rw-r--r-- 1 dnpil001 users 1671102028 Jan  5 08:08 g4_stage1.root
-rw-r--r-- 1 dnpil001 users   74032990 Jan  5 13:01 Pandora_Events.pndr
-rw-r--r-- 1 dnpil001 users     242824 Jan  5 13:07 jobscript.log
-rw-r--r-- 1 dnpil001 users     217088 Jan  5 13:07 mem.db
-rw-r--r-- 1 dnpil001 users      62376 Jan  5 13:07 Pandora_Geometry.xml
-rw-r--r-- 1 dnpil001 users      36864 Jan  5 13:07 time.db
-rw-r--r-- 1 dnpil001 users        519 Jan  5 11:34 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Jan  5 08:24 g4_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Jan  5 13:07 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dnpil001 users         52 Jan  5 08:00 all-input-dids.txt
-rw-r--r-- 1 dnpil001 users          7 Jan  5 13:07 justin-processed-pfns.txt
-rw-r--r-- 1 dnpil001 users          0 Jan  5 11:42 ProtoDUNE.log
-rw-r--r-- 1 dnpil001 users          0 Jan  5 11:34 debugprod.log
justIN time: 2026-02-06 19:46:29 UTC       justIN version: 01.06.00