justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 268365.0@dunegpschedd02.fnal.gov

Jobsub ID268365.0@dunegpschedd02.fnal.gov
Workflow ID11708
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-11 05:48:26
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce02
Last heartbeat2026-01-11 10:10:33
From worker nodeHostnameccwcondor0732
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-11 05:49:35
Input filesmonte-carlo-011708-000001
JobscriptExit code0
Real time4h (15558s)
CPU time4h (15069s = 96%)
Max RSS bytes4677816320 (4461 MiB)
Outputting started2026-01-11 10:08:54
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11708/1/001/reco_stage1.root
Finished2026-01-11 10:10:33
Saved logsjustin-logs:268365.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

are:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 138206 traces, 0 summary
Retagger: tagging trace set: gauss with 101670 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 101670 traces tagged "gauss"
FrameSaver: q=3.41132e+06 n=1755392 tag=gauss
wclsFrameSaver: saving 138206 traces tagged "wiener"
FrameSaver: q=3.61617e+06 n=1526508 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 24594 XUs and 28364 XVs -> 280 XUVs
C:0 T:1 55243 XUs and 58451 XVs -> 4820 XUVs
C:0 T:2 119453 XUs and 156901 XVs -> 9426 XUVs
C:0 T:3 52062 XUs and 60055 XVs -> 410 XUVs
C:0 T:4 16467 XUs and 15487 XVs -> 149 XUVs
C:0 T:5 78886 XUs and 61787 XVs -> 7961 XUVs
C:0 T:6 83571 XUs and 104878 XVs -> 4218 XUVs
C:0 T:7 33048 XUs and 40356 XVs -> 210 XUVs
C:0 T:8 9789 XUs and 9246 XVs -> 93 XUVs
C:0 T:9 57092 XUs and 73691 XVs -> 4313 XUVs
C:0 T:10 19740 XUs and 8812 XVs -> 1857 XUVs
C:0 T:11 167 XUs and 94 XVs -> 17 XUVs
33754 XUVs total
12735 collection wire objects
31348 potential space points
Neighbour search...
1809700 tests to find 993836 neighbours
Iterating with no regularization...
Begin: 5.62443e+07
0 4.67083e+07
1 4.59635e+07
2 4.58578e+07
3 4.58276e+07
Now with regularization...
Begin: 3.56769e+07
0 3.56418e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411670262
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411670262
fLifetime: 30000 [us]
11-Jan-2026 11:08:51 CET  Closed output file "reco_stage1.root"
11-Jan-2026 11:08:51 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          358.773       507.44        659.852       478.172       93.278         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00106646    0.00517591     0.0102466     0.0050239     0.0026879       10     
reco:ophit:OpHitFinder                             0.284525      0.366198      0.427702      0.370906      0.0450486       10     
reco:opflash:OpFlashFinder                         0.174676      0.227902      0.266521      0.225787      0.0285654       10     
reco:opslicer:OpSlicer                             0.668851      0.978435       1.19453       0.99155      0.152637        10     
reco:caldata:DataPrepModule                         6.0661        6.62488       7.36884       6.63122      0.379669        10     
reco:wclsdatasp:WireCellToolkit                     41.9365       45.5709       52.3072       44.4335       3.34707        10     
reco:gaushit:GausHitFinder                          1.22113       1.50675       2.07648       1.46398      0.256314        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000686502   0.000843507   0.00101335    0.00084879    0.000124093      10     
reco:reco3d:SpacePointSolver                        8.74007       19.4148       28.9877       18.4428       5.40415        10     
reco:hitpdune:DisambigFromSpacePoints               1.30427       1.91286       2.54402       1.87828      0.364085        10     
reco:pandora:StandardPandora                        21.0428       55.5998       148.836       40.1669       35.3867        10     
reco:pandoraWriter:StandardPandora                 0.326732      0.576138      0.849562      0.580404      0.122018        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.64003       5.86714       8.80262       5.8517        1.76518        10     
reco:pandoraShower:LArPandoraShowerCreation         2.70848       5.88738       8.88509       5.27585       1.94379        10     
reco:pandoracalo:Calorimetry                        1.63586       3.00687       4.01056       2.79309      0.717724        10     
reco:pandoracalonosce:Calorimetry                   1.97273       2.87433       3.66605       2.83389      0.605873        10     
reco:pandorapid:Chi2ParticleID                    0.00628308    0.00805618     0.0109574    0.00770118    0.00121477       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0590118     0.0783582     0.099897      0.0780941     0.0103798       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00599344    0.00749501    0.00890981    0.00745215    0.000837454      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.54011       6.34747       9.51682       5.87596       1.84742        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.44466       6.35591       9.86936       5.69368       2.07876        10     
reco:emtrkmichelid:EmTrackMichelId                  228.614       282.405       341.336       278.416       36.4967        10     
reco:crttag:SingleCRTMatchingProducer              0.0123823     0.0336439     0.177487      0.0177088     0.0481078       10     
reco:crtreco:TwoCRTMatchingProducer                0.008547      0.0126289     0.0176553     0.0117092     0.0028744       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.710128       1.15619       1.38431       1.21101      0.188632        10     
reco:pandora2Track:LArPandoraTrackCreation          11.1214       18.4929       28.5778       16.4165       6.15791        10     
reco:pandora2calo:Calorimetry                       3.28872       8.55918       12.1896       8.31696       2.72462        10     
reco:pandora2calonosce:Calorimetry                  3.17012       8.37642       12.7606       7.89304       2.83153        10     
reco:pandora2pid:Chi2ParticleID                    0.010988      0.0187317     0.0256403     0.0188725    0.00459475       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0507098     0.101013      0.169298      0.104225      0.034463        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0109979     0.0191694     0.0270186     0.0188688    0.00536795       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.10563       6.8782        12.9843       6.17206       2.56802        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.02362       7.5117        13.367        7.17798       2.68178        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.59389       6.63654       10.0723       5.88799       1.95818        10     
[art]:TriggerResults:TriggerResultInserter        5.0215e-05    8.30328e-05   0.000225385   7.0909e-05    4.8159e-05       10     
end_path:out1:RootOutput                           8.105e-06    1.22973e-05   2.2372e-05    1.18025e-05   3.56025e-06      10     
end_path:out1:RootOutput(write)                     3.07951       3.87278       5.06418       3.71822      0.628294        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5489.46 MB
  Peak resident set size usage (VmHWM): 4677.82 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.05 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11426044
-rw-r--r-- 1 dune001 lbno 2645668153 Jan 11 09:41 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2512284117 Jan 11 09:38 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2354044592 Jan 11 11:08 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 2101202101 Jan 11 07:16 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 2006115291 Jan 11 06:58 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   80340170 Jan 11 11:02 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242955 Jan 11 11:08 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan 11 11:08 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan 11 11:08 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan 11 11:08 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan 11 09:41 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 11 07:16 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 11 11:08 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan 11 06:49 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan 11 11:08 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan 11 09:51 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan 11 09:41 debugprod.log
justIN time: 2026-02-06 11:42:46 UTC       justIN version: 01.06.00