justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270851.1@dunegpschedd02.fnal.gov

Jobsub ID270851.1@dunegpschedd02.fnal.gov
Workflow ID11770
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-12 21:48:34
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce507
Last heartbeat2026-01-13 00:51:12
From worker nodeHostnameb9p11p4299.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-12 21:49:33
Input filesmonte-carlo-011770-000001
JobscriptExit code0
Real time3h (10813s)
CPU time2h (10243s = 94%)
Max RSS bytes4746227712 (4526 MiB)
Outputting started2026-01-13 00:49:47
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11770/1/001/reco_stage1.root
Finished2026-01-13 00:51:12
Saved logsjustin-logs:270851.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 151455 traces, 0 summary
Retagger: tagging trace set: gauss with 119717 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 119717 traces tagged "gauss"
FrameSaver: q=4.13716e+06 n=2178637 tag=gauss
wclsFrameSaver: saving 151455 traces tagged "wiener"
FrameSaver: q=4.34748e+06 n=1946262 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 46395 XUs and 47361 XVs -> 854 XUVs
C:0 T:1 80114 XUs and 77037 XVs -> 4626 XUVs
C:0 T:2 116587 XUs and 122141 XVs -> 5207 XUVs
C:0 T:3 57070 XUs and 45294 XVs -> 304 XUVs
C:0 T:4 11353 XUs and 22067 XVs -> 123 XUVs
C:0 T:5 64766 XUs and 85391 XVs -> 9127 XUVs
C:0 T:6 146319 XUs and 138137 XVs -> 7592 XUVs
C:0 T:7 58052 XUs and 46268 XVs -> 529 XUVs
C:0 T:8 7932 XUs and 8840 XVs -> 101 XUVs
C:0 T:9 42421 XUs and 58001 XVs -> 2896 XUVs
C:0 T:10 44467 XUs and 25885 XVs -> 2874 XUVs
C:0 T:11 363 XUs and 386 XVs -> 91 XUVs
34324 XUVs total
13829 collection wire objects
31266 potential space points
Neighbour search...
2065388 tests to find 1032700 neighbours
Iterating with no regularization...
Begin: 7.5213e+07
0 6.4289e+07
1 6.35926e+07
2 6.35081e+07
3 6.34875e+07
Now with regularization...
Begin: 5.24624e+07
0 5.2446e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411703751
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411703751
fLifetime: 30000 [us]
13-Jan-2026 01:49:45 CET  Closed output file "reco_stage1.root"
13-Jan-2026 01:49:45 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          226.295       275.015       346.198       270.571       38.1932        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000871334   0.00347904    0.00567679     0.0036086    0.00150384       10     
reco:ophit:OpHitFinder                             0.230996      0.256482      0.317055      0.245464      0.0270593       10     
reco:opflash:OpFlashFinder                         0.131115      0.179461      0.272099       0.16181      0.0408085       10     
reco:opslicer:OpSlicer                             0.460728      0.549314      0.680731      0.540802      0.0703073       10     
reco:caldata:DataPrepModule                         4.04444       4.41641       6.17375       4.1143       0.654835        10     
reco:wclsdatasp:WireCellToolkit                     26.1683       27.4986       29.2893       27.4704      0.920933        10     
reco:gaushit:GausHitFinder                         0.760302      0.939777       1.2124       0.914563      0.132014        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000357012   0.000445625   0.000580193   0.000421752   6.83253e-05      10     
reco:reco3d:SpacePointSolver                        7.79828       11.4475       16.3458       10.4357       2.88448        10     
reco:hitpdune:DisambigFromSpacePoints              0.855037       1.31714       1.82012       1.2948       0.275083        10     
reco:pandora:StandardPandora                        15.2966       25.7687       43.8204       23.8794       8.71553        10     
reco:pandoraWriter:StandardPandora                  0.28729      0.350532      0.422649      0.339196      0.0361429       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.3422        3.32644       6.42574       2.98757        1.117         10     
reco:pandoraShower:LArPandoraShowerCreation         2.28584       3.44034       5.69593       3.2838        0.96711        10     
reco:pandoracalo:Calorimetry                        1.49269       1.89548       2.57328       1.81507      0.339037        10     
reco:pandoracalonosce:Calorimetry                   1.36543       1.78595       2.56412       1.70298      0.365744        10     
reco:pandorapid:Chi2ParticleID                    0.00362166    0.00486967    0.00833953     0.0042948     0.0012898       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0364064     0.0426108     0.073967      0.0384186     0.0107441       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00331055    0.00456493    0.00834659    0.00400327    0.00138769       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.55058       4.00159       5.80934       3.86059      0.978177        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.50666       3.71967       5.23006       3.66223      0.832169        10     
reco:emtrkmichelid:EmTrackMichelId                  128.29        145.904       175.844       145.714       15.0768        10     
reco:crttag:SingleCRTMatchingProducer              0.0079766     0.0121759     0.0173183     0.0117496    0.00247963       10     
reco:crtreco:TwoCRTMatchingProducer                0.0058902    0.00786678     0.0117824    0.00688332    0.00188039       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.604609      0.788421       1.15643      0.759356      0.162588        10     
reco:pandora2Track:LArPandoraTrackCreation          7.19634       11.3097       16.3403       11.0856       2.97681        10     
reco:pandora2calo:Calorimetry                       3.8441        5.56813       7.36831       5.69403       1.02591        10     
reco:pandora2calonosce:Calorimetry                  3.64476       5.27427       6.97051       5.52466      0.979367        10     
reco:pandora2pid:Chi2ParticleID                   0.00816975     0.010539      0.0142397     0.010382     0.00167404       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0458536     0.0527869     0.063682      0.051636     0.00561707       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0089218     0.0106521     0.0134314     0.0102759    0.00130459       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.5059        3.58632       4.73406       3.59041      0.638378        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.31468       4.7701        6.18746       4.86052      0.849839        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.04153       4.36621       5.62673       4.26157      0.872338        10     
[art]:TriggerResults:TriggerResultInserter         4.036e-05    6.54352e-05   0.00016612    5.56355e-05   3.4157e-05       10     
end_path:out1:RootOutput                           6.58e-06     1.23511e-05    1.947e-05     1.227e-05    3.01716e-06      10     
end_path:out1:RootOutput(write)                     1.98279       2.3035        2.82204       2.24101      0.258471        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5268.21 MB
  Peak resident set size usage (VmHWM): 4436.36 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.82 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10813836
-rw-r--r--. 1 duneprd np-comp 2521590098 Jan 13 01:01 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2393660150 Jan 13 00:59 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2226866399 Jan 13 01:49 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1971305800 Jan 12 23:11 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1879564727 Jan 12 22:57 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   79635730 Jan 13 01:45 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242717 Jan 13 01:49 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 13 01:49 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 13 01:49 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 13 01:49 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 13 01:01 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 12 23:11 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 13 01:49 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 12 22:49 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 13 01:49 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 13 01:06 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 13 01:01 debugprod.log
justIN time: 2026-02-06 09:02:13 UTC       justIN version: 01.06.00