justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 246905.1@dunegpschedd01.fnal.gov

Jobsub ID246905.1@dunegpschedd01.fnal.gov
Workflow ID9922
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-11 01:45:42
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-11 07:22:10
From worker nodeHostnamewn1906250.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-11 01:46:08
Input filesmonte-carlo-009922-000001
JobscriptExit code0
Real time5h (20072s)
CPU time5h (19580s = 97%)
Max RSS bytes4139630592 (3947 MiB)
Outputting started2025-11-11 07:20:41
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09922/1/001/reco_stage1.root
Finished2025-11-11 07:22:10
Saved logsjustin-logs:246905.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 111988 traces, 0 summary
Retagger: tagging trace set: gauss with 84776 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 84776 traces tagged "gauss"
FrameSaver: q=2.97335e+06 n=1447677 tag=gauss
wclsFrameSaver: saving 111988 traces tagged "wiener"
FrameSaver: q=3.16123e+06 n=1278456 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 31851 XUs and 34980 XVs -> 696 XUVs
C:0 T:1 55954 XUs and 54664 XVs -> 4271 XUVs
C:0 T:2 34711 XUs and 41508 XVs -> 1294 XUVs
C:0 T:3 19866 XUs and 18878 XVs -> 147 XUVs
C:0 T:4 13531 XUs and 12671 XVs -> 194 XUVs
C:0 T:5 71176 XUs and 56531 XVs -> 15055 XUVs
C:0 T:6 77317 XUs and 97152 XVs -> 4127 XUVs
C:0 T:7 32927 XUs and 38544 XVs -> 463 XUVs
C:0 T:8 6818 XUs and 7810 XVs -> 64 XUVs
C:0 T:9 52482 XUs and 117223 XVs -> 11223 XUVs
C:0 T:10 23301 XUs and 10516 XVs -> 2821 XUVs
C:0 T:11 174 XUs and 100 XVs -> 33 XUVs
40388 XUVs total
9418 collection wire objects
37645 potential space points
Neighbour search...
7407519 tests to find 3161182 neighbours
Iterating with no regularization...
Begin: 6.83457e+07
0 5.92746e+07
1 5.85927e+07
2 5.84966e+07
3 5.8468e+07
Now with regularization...
Begin: 4.91349e+07
0 4.90926e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410444720
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410444720
fLifetime: 30000 [us]
11-Nov-2025 13:20:35 GMT  Closed output file "reco_stage1.root"
11-Nov-2025 13:20:35 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          490.769       717.773       975.113       761.293       129.077        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00166218    0.00452968    0.00665111    0.00451491     0.0014409       10     
reco:ophit:OpHitFinder                             0.310826      0.390762      0.472838      0.385609      0.0491104       10     
reco:opflash:OpFlashFinder                         0.168749      0.236573      0.309624      0.232458      0.0478891       10     
reco:opslicer:OpSlicer                             0.777082       1.01341       1.28417      0.994325       0.15666        10     
reco:caldata:DataPrepModule                         6.9489        9.67837       11.1895       10.4022       1.54623        10     
reco:wclsdatasp:WireCellToolkit                     61.9719       69.7642       78.1907       70.9857       5.17879        10     
reco:gaushit:GausHitFinder                          1.28519       2.28816       2.86693       2.32866      0.466732        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000543968   0.000911203   0.00121996    0.000924833   0.000173269      10     
reco:reco3d:SpacePointSolver                        16.1081       22.4611       33.525        22.6119       5.41399        10     
reco:hitpdune:DisambigFromSpacePoints               1.53331       3.44556       5.40576       3.42418       1.05215        10     
reco:pandora:StandardPandora                        33.8012       61.9237       96.843        62.5181       18.9998        10     
reco:pandoraWriter:StandardPandora                 0.636467       0.81745      0.997535      0.825445      0.131961        10     
reco:pandoraTrack:LArPandoraTrackCreation           8.81171       13.7364       19.7374       14.7052       3.30158        10     
reco:pandoraShower:LArPandoraShowerCreation         7.64281       14.7769       21.8401       15.7802       4.17211        10     
reco:pandoracalo:Calorimetry                        4.07825       6.66135       9.19996       7.4349        1.73522        10     
reco:pandoracalonosce:Calorimetry                   3.87775       6.39037       8.75751       7.22413       1.72259        10     
reco:pandorapid:Chi2ParticleID                    0.00914326     0.0114092     0.0131441     0.0121596    0.00141402       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0834611     0.105485      0.127383       0.10493      0.0144695       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00858996     0.0106254     0.0122529     0.0109274     0.0013761       10     
reco:pandoraShowercalo:ShowerCalorimetry            7.11148       14.9397       21.5481       16.4728       4.89366        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.76746       14.428        19.5742       16.7904       4.49378        10     
reco:emtrkmichelid:EmTrackMichelId                  245.069       335.279       446.623       341.471       50.8215        10     
reco:crttag:SingleCRTMatchingProducer              0.0291767     0.0425643     0.0696184     0.0428222     0.0115936       10     
reco:crtreco:TwoCRTMatchingProducer                0.0242101     0.0329784     0.0506138     0.0320887    0.00793858       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.27429       1.79119       2.81254       1.72461      0.412132        10     
reco:pandora2Track:LArPandoraTrackCreation          24.2065       44.8255       68.2273       47.0799       11.8649        10     
reco:pandora2calo:Calorimetry                       10.5795       20.3816       32.5569       22.478        6.63915        10     
reco:pandora2calonosce:Calorimetry                  10.2761       20.0951       30.8744       21.6746       6.13758        10     
reco:pandora2pid:Chi2ParticleID                    0.015919      0.0274139     0.0329303     0.0293277    0.00506695       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0857861     0.134815       0.1765       0.142567      0.0243222       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0133516     0.0262865     0.0321904     0.0279038    0.00555095       10     
reco:pandora2Shower:LArPandoraShowerCreation        8.48193       14.5371       21.7475       14.9402       3.89987        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.91723       17.755        26.3253       19.7132       5.64916        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.56264       15.4065       22.2132       17.6755       4.81236        10     
[art]:TriggerResults:TriggerResultInserter        8.0376e-05    0.000129019   0.000207239   0.000127317   3.15963e-05      10     
end_path:out1:RootOutput                          1.0251e-05    1.53908e-05   2.4303e-05     1.468e-05    4.08608e-06      10     
end_path:out1:RootOutput(write)                     2.84479       4.14841       5.50421       4.52237      0.799657        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4956.89 MB
  Peak resident set size usage (VmHWM): 4139.63 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.38 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.05 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.33 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9798280
-rw-r--r-- 1 dune01 dune01 2322263285 Nov 11 11:17 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2210450156 Nov 11 11:13 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1989051908 Nov 11 13:20 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1756561887 Nov 11 08:15 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1684049053 Nov 11 07:55 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   70430930 Nov 11 13:12 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242906 Nov 11 13:20 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 11 13:20 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 11 13:20 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 11 13:20 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 11 11:17 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 11 08:15 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 11 13:20 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 11 07:46 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 11 13:20 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 11 11:29 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 11 11:17 debugprod.log
justIN time: 2026-02-10 07:57:25 UTC       justIN version: 01.06.00