justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 275210.1@dunegpschedd01.fnal.gov

Jobsub ID275210.1@dunegpschedd01.fnal.gov
Workflow ID11737
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-11 22:53:21
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2026-01-12 02:00:16
From worker nodeHostnamewn6219290.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-11 22:54:41
Input filesmonte-carlo-011737-000001
JobscriptExit code0
Real time3h (11040s)
CPU time2h (10795s = 97%)
Max RSS bytes4512133120 (4303 MiB)
Outputting started2026-01-12 01:58:43
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11737/1/001/reco_stage1.root
Finished2026-01-12 02:00:16
Saved logsjustin-logs:275210.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nning tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 118411 traces, 0 summary
Retagger: tagging trace set: gauss with 94888 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 94888 traces tagged "gauss"
FrameSaver: q=3.3158e+06 n=1714503 tag=gauss
wclsFrameSaver: saving 118411 traces tagged "wiener"
FrameSaver: q=3.47836e+06 n=1540553 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 25353 XUs and 16205 XVs -> 98 XUVs
C:0 T:1 34092 XUs and 27800 XVs -> 2012 XUVs
C:0 T:2 67307 XUs and 81070 XVs -> 2423 XUVs
C:0 T:3 38821 XUs and 38209 XVs -> 246 XUVs
C:0 T:4 14155 XUs and 11359 XVs -> 63 XUVs
C:0 T:5 45918 XUs and 37817 XVs -> 5038 XUVs
C:0 T:6 136834 XUs and 132021 XVs -> 4806 XUVs
C:0 T:7 59209 XUs and 53207 XVs -> 634 XUVs
C:0 T:8 8572 XUs and 8563 XVs -> 90 XUVs
C:0 T:9 50492 XUs and 69177 XVs -> 3923 XUVs
C:0 T:10 30126 XUs and 20189 XVs -> 3198 XUVs
C:0 T:11 62 XUs and 59 XVs -> 4 XUVs
22535 XUVs total
10326 collection wire objects
20921 potential space points
Neighbour search...
811871 tests to find 417040 neighbours
Iterating with no regularization...
Begin: 7.94753e+07
0 7.18052e+07
1 7.11965e+07
2 7.11227e+07
3 7.11079e+07
Now with regularization...
Begin: 6.06396e+07
0 6.06374e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411684486
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411684486
fLifetime: 30000 [us]
12-Jan-2026 01:58:41 GMT  Closed output file "reco_stage1.root"
12-Jan-2026 01:58:41 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          272.413       331.608       387.367       337.48        39.6619        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000673391   0.00300855    0.00434326     0.0034348    0.00123286       10     
reco:ophit:OpHitFinder                             0.245388      0.275579      0.326482      0.277228      0.0228126       10     
reco:opflash:OpFlashFinder                         0.160912      0.199569      0.228907      0.204273      0.0201133       10     
reco:opslicer:OpSlicer                             0.579266      0.678511      0.771207      0.688413      0.0604282       10     
reco:caldata:DataPrepModule                         4.23303       4.3418        4.6371        4.31398      0.111097        10     
reco:wclsdatasp:WireCellToolkit                     25.5792       29.5036       33.8742       29.2276       2.42902        10     
reco:gaushit:GausHitFinder                          0.86142       1.0613        1.30851       1.03605      0.138701        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000485105   0.000579419   0.000755075   0.000564405   8.11435e-05      10     
reco:reco3d:SpacePointSolver                        9.47952       12.392        17.9682       11.7711       2.69586        10     
reco:hitpdune:DisambigFromSpacePoints               1.20006       1.54057       1.93013       1.51588      0.234128        10     
reco:pandora:StandardPandora                        22.2713       32.6074       42.7498       32.5766       7.69662        10     
reco:pandoraWriter:StandardPandora                 0.340964      0.409239      0.494896      0.406314      0.0495982       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.47901       4.02076       5.64128       4.11621      0.895541        10     
reco:pandoraShower:LArPandoraShowerCreation         2.77275       4.22689       5.81561       4.22473       0.96602        10     
reco:pandoracalo:Calorimetry                        1.35038       1.98133       2.60347       2.0776       0.381712        10     
reco:pandoracalonosce:Calorimetry                   1.20819       1.82231       2.4217        1.90483      0.367094        10     
reco:pandorapid:Chi2ParticleID                    0.00385585    0.00551744    0.00665294    0.00554281    0.000815794      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0399369     0.0457568     0.0542957     0.0450065    0.00419901       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00382231    0.00525778     0.0063759    0.00533994    0.000686029      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.62478       4.02202       5.3057        4.03555      0.895843        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.57429       3.92563       5.21527       3.93988       0.87107        10     
reco:emtrkmichelid:EmTrackMichelId                  159.785       186.234       213.161       186.849       17.8661        10     
reco:crttag:SingleCRTMatchingProducer             0.00762496     0.0109049     0.0158623     0.0103746    0.00270404       10     
reco:crtreco:TwoCRTMatchingProducer               0.00635309    0.00763001    0.00893021    0.00753021    0.000886504      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.660614      0.809857       0.95871      0.791743      0.0984455       10     
reco:pandora2Track:LArPandoraTrackCreation          9.2584        13.1013       17.8776       13.1714       2.66443        10     
reco:pandora2calo:Calorimetry                       4.01397       5.67765       7.50824       5.75827       1.12275        10     
reco:pandora2calonosce:Calorimetry                  3.82594       5.39354       7.05723       5.46948       1.04718        10     
reco:pandora2pid:Chi2ParticleID                    0.0113442     0.0139529     0.0162737     0.0143044    0.00146325       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0515796     0.0596715     0.0686282     0.0599386    0.00565592       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0110875     0.0138314     0.0161766     0.0141548    0.00150623       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.53054       4.79687       6.16002       4.8261       0.856417        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.54996       4.90842       6.2397        4.97416      0.893892        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.3404        4.53205       5.65726       4.57402      0.781255        10     
[art]:TriggerResults:TriggerResultInserter         2.144e-05    3.11116e-05   0.000102925   2.3669e-05    2.39588e-05      10     
end_path:out1:RootOutput                           4.378e-06    5.1465e-06     9.518e-06     4.699e-06    1.47714e-06      10     
end_path:out1:RootOutput(write)                     2.4537        2.87718       3.22149       2.87504      0.280867        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5332.09 MB
  Peak resident set size usage (VmHWM): 4512.13 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.74 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11631528
-rw-r--r-- 1 dune01 dune01 2680375903 Jan 12 01:01 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2542700491 Jan 12 00:59 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2410334741 Jan 12 01:58 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2144391870 Jan 11 23:13 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 2046599424 Jan 11 23:00 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   85649980 Jan 12 01:55 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242909 Jan 12 01:58 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Jan 12 01:58 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Jan 12 01:58 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Jan 12 01:58 time.db
-rw-r--r-- 1 dune01 dune01        519 Jan 12 01:01 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 11 23:13 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 12 01:58 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Jan 11 22:54 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Jan 12 01:58 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Jan 12 01:07 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Jan 12 01:01 debugprod.log
justIN time: 2026-02-06 10:22:29 UTC       justIN version: 01.06.00