justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270055.1@dunegpschedd02.fnal.gov

Jobsub ID270055.1@dunegpschedd02.fnal.gov
Workflow ID11740
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-12 08:41:54
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2026-01-12 14:03:53
From worker nodeHostnamewn-la-06.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-12 08:43:02
Input filesmonte-carlo-011740-000001
JobscriptExit code0
Real time5h (19155s)
CPU time5h (19007s = 99%)
Max RSS bytes4508442624 (4299 MiB)
Outputting started2026-01-12 14:02:19
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11740/1/001/reco_stage1.root
Finished2026-01-12 14:03:53
Saved logsjustin-logs:270055.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

l adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 142611 traces, 0 summary
Retagger: tagging trace set: gauss with 109903 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 109903 traces tagged "gauss"
FrameSaver: q=3.77892e+06 n=1962503 tag=gauss
wclsFrameSaver: saving 142611 traces tagged "wiener"
FrameSaver: q=3.98835e+06 n=1742662 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33881 XUs and 33996 XVs -> 379 XUVs
C:0 T:1 92995 XUs and 67901 XVs -> 8645 XUVs
C:0 T:2 80262 XUs and 114449 XVs -> 2772 XUVs
C:0 T:3 43019 XUs and 51304 XVs -> 404 XUVs
C:0 T:4 14834 XUs and 7478 XVs -> 102 XUVs
C:0 T:5 68931 XUs and 28352 XVs -> 6479 XUVs
C:0 T:6 90638 XUs and 84248 XVs -> 7146 XUVs
C:0 T:7 35670 XUs and 31144 XVs -> 617 XUVs
C:0 T:8 12433 XUs and 8509 XVs -> 85 XUVs
C:0 T:9 80986 XUs and 72320 XVs -> 4851 XUVs
C:0 T:10 82189 XUs and 116299 XVs -> 13011 XUVs
C:0 T:11 151 XUs and 63 XVs -> 15 XUVs
44506 XUVs total
12417 collection wire objects
41346 potential space points
Neighbour search...
5009838 tests to find 2407730 neighbours
Iterating with no regularization...
Begin: 7.25289e+07
0 6.04895e+07
1 5.96784e+07
2 5.95659e+07
3 5.95342e+07
Now with regularization...
Begin: 4.91752e+07
0 4.91397e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411692797
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411692797
fLifetime: 30000 [us]
12-Jan-2026 15:02:15 CET  Closed output file "reco_stage1.root"
12-Jan-2026 15:02:15 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          291.126       573.646       788.968       541.926       143.661        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00131819    0.00690291     0.0137546    0.00699834    0.00367489       10     
reco:ophit:OpHitFinder                             0.400721      0.477892      0.522889      0.474481      0.0334524       10     
reco:opflash:OpFlashFinder                         0.283877      0.374502      0.427248      0.364634      0.0435237       10     
reco:opslicer:OpSlicer                             0.815712       1.0115        1.18296       1.02096      0.118276        10     
reco:caldata:DataPrepModule                         6.15723       6.53402       6.99397       6.52828      0.209749        10     
reco:wclsdatasp:WireCellToolkit                     43.3636       51.301        58.6763       53.9464       5.49362        10     
reco:gaushit:GausHitFinder                         0.892036       1.58744       2.08499       1.62385       0.31777        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000335243   0.00057899    0.000761439   0.000573122   0.00012017       10     
reco:reco3d:SpacePointSolver                        11.6689       31.4158       46.3216       33.4575       12.5301        10     
reco:hitpdune:DisambigFromSpacePoints              0.880625       2.36777       3.55205       2.38937       0.71105        10     
reco:pandora:StandardPandora                        21.2942       60.0316       97.3553       55.6962       23.043         10     
reco:pandoraWriter:StandardPandora                 0.334788      0.578712      0.704181      0.576868      0.103044        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.9979        10.3406       16.0394       9.91382       3.23367        10     
reco:pandoraShower:LArPandoraShowerCreation         3.23816       13.972        25.8809       12.0605       6.47359        10     
reco:pandoracalo:Calorimetry                        2.02607       5.59757       9.3337        5.04766       2.14038        10     
reco:pandoracalonosce:Calorimetry                   1.87675       5.32306       8.91991       4.7664        2.01478        10     
reco:pandorapid:Chi2ParticleID                    0.00570178    0.00737666    0.00850588    0.00732679    0.000935682      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0397383     0.0644194     0.0856259     0.0635699      0.01108        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00552231    0.00723273    0.00853079    0.00682824    0.000958983      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.43915       12.8465       23.2689       10.7492       5.91285        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.39909       11.9676       22.0497       10.3404       5.36854        10     
reco:emtrkmichelid:EmTrackMichelId                  150.623       259.062       334.589       258.817       48.0565        10     
reco:crttag:SingleCRTMatchingProducer             0.00866409     0.020829      0.0310002     0.0202934    0.00608584       10     
reco:crtreco:TwoCRTMatchingProducer               0.00753952     0.0179974     0.0280476     0.0174793    0.00505757       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.683641       1.19631       1.45604       1.1968       0.207421        10     
reco:pandora2Track:LArPandoraTrackCreation          7.90196       30.6548       54.4724       28.0579       12.7096        10     
reco:pandora2calo:Calorimetry                       3.63593       14.9436       27.062        13.8346       6.57458        10     
reco:pandora2calonosce:Calorimetry                  3.42421       14.6627       26.5836       12.8449       6.99151        10     
reco:pandora2pid:Chi2ParticleID                    0.0108683     0.0170833     0.0223011     0.0163276    0.00346878       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0485818     0.0816859     0.109487      0.0808677     0.0156445       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0108102     0.017056      0.0220929     0.0163518    0.00343977       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.06745       10.6734       19.0323       9.41775       4.58124        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.12809       12.1605       19.7245       11.4402       5.05928        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.88049       10.0242       15.6517       9.76599       3.62073        10     
[art]:TriggerResults:TriggerResultInserter         3.98e-05     5.35221e-05   0.000127932   4.35155e-05   2.52091e-05      10     
end_path:out1:RootOutput                           6.76e-06     8.0994e-06    1.4192e-05     7.416e-06    2.08445e-06      10     
end_path:out1:RootOutput(write)                     2.30745       4.01981       5.20579       4.07595      0.737598        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5309.6 MB
  Peak resident set size usage (VmHWM): 4508.44 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.08 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11380568
-rw-r--r--. 1 dune009 dune 2625436670 Jan 12 13:24 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2481837746 Jan 12 13:21 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2365631251 Jan 12 15:02 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2093690631 Jan 12 10:09 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1999052963 Jan 12 09:51 g4_stage1.root
-rw-r--r--. 1 dune009 dune   87433740 Jan 12 14:55 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242970 Jan 12 15:02 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Jan 12 15:02 mem.db
-rw-r--r--. 1 dune009 dune      62376 Jan 12 15:02 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Jan 12 15:02 time.db
-rw-r--r--. 1 dune009 dune        519 Jan 12 13:24 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 12 10:09 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 12 15:02 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Jan 12 09:43 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Jan 12 15:02 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Jan 12 13:31 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Jan 12 13:24 debugprod.log
justIN time: 2026-02-06 10:20:39 UTC       justIN version: 01.06.00