justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 247211.1@dunegpschedd01.fnal.gov

Jobsub ID247211.1@dunegpschedd01.fnal.gov
Workflow ID9945
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-11 18:26:26
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-11-11 21:30:33
From worker nodeHostnamewn2208251.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-11 18:27:46
Input filesmonte-carlo-009945-000001
JobscriptExit code0
Real time3h (10867s)
CPU time2h (10608s = 97%)
Max RSS bytes3999412224 (3814 MiB)
Outputting started2025-11-11 21:28:55
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09945/1/001/reco_stage1.root
Finished2025-11-11 21:30:33
Saved logsjustin-logs:247211.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

er
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 137935 traces, 0 summary
Retagger: tagging trace set: gauss with 107302 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 107302 traces tagged "gauss"
FrameSaver: q=3.83463e+06 n=1940143 tag=gauss
wclsFrameSaver: saving 137935 traces tagged "wiener"
FrameSaver: q=4.03039e+06 n=1725972 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33025 XUs and 41674 XVs -> 639 XUVs
C:0 T:1 101212 XUs and 108180 XVs -> 15349 XUVs
C:0 T:2 95248 XUs and 146356 XVs -> 9350 XUVs
C:0 T:3 40763 XUs and 46385 XVs -> 493 XUVs
C:0 T:4 11323 XUs and 15898 XVs -> 146 XUVs
C:0 T:5 52975 XUs and 82206 XVs -> 6721 XUVs
C:0 T:6 103383 XUs and 131389 XVs -> 13144 XUVs
C:0 T:7 35314 XUs and 38296 XVs -> 488 XUVs
C:0 T:8 12076 XUs and 10794 XVs -> 120 XUVs
C:0 T:9 72472 XUs and 87250 XVs -> 5422 XUVs
C:0 T:10 66187 XUs and 71632 XVs -> 19921 XUVs
C:0 T:11 159 XUs and 122 XVs -> 15 XUVs
71808 XUVs total
13255 collection wire objects
67221 potential space points
Neighbour search...
18837747 tests to find 7502494 neighbours
Iterating with no regularization...
Begin: 9.15603e+07
0 7.13804e+07
1 7.0458e+07
2 7.03044e+07
3 7.02504e+07
Now with regularization...
Begin: 5.8744e+07
0 5.86851e+07
1 5.86345e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410452181
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410452181
fLifetime: 30000 [us]
11-Nov-2025 21:28:53 GMT  Closed output file "reco_stage1.root"
11-Nov-2025 21:28:53 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          265.047       348.935       484.746       335.732       65.4265        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000978348   0.00363114    0.00654321    0.00389558    0.00154721       10     
reco:ophit:OpHitFinder                             0.227776      0.282907      0.345087      0.277542      0.0356753       10     
reco:opflash:OpFlashFinder                         0.117199      0.146352      0.178108      0.146518      0.0149957       10     
reco:opslicer:OpSlicer                             0.462306      0.657701       0.83098       0.69137      0.125718        10     
reco:caldata:DataPrepModule                         4.26949       5.64452       7.46885       5.47186       1.1887         10     
reco:wclsdatasp:WireCellToolkit                     32.5278       37.1171        45.64        37.3206       3.63151        10     
reco:gaushit:GausHitFinder                          0.86506       1.18355       1.58996       1.11931      0.209882        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000395493   0.000558769   0.000749549   0.000563233   0.000103354      10     
reco:reco3d:SpacePointSolver                        6.76845       16.1898       28.5417       15.7663       5.3154         10     
reco:hitpdune:DisambigFromSpacePoints               1.0366        1.65613       2.53361       1.4466       0.558014        10     
reco:pandora:StandardPandora                        14.0455       30.4457       51.6237       23.9866       13.5438        10     
reco:pandoraWriter:StandardPandora                 0.265772      0.450055      0.642291      0.444675      0.125605        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.63591       3.40099       5.79273       3.01753       1.3012         10     
reco:pandoraShower:LArPandoraShowerCreation         1.47216       3.85555       7.45936       2.87501       1.94699        10     
reco:pandoracalo:Calorimetry                       0.971665       2.03151       3.58472       1.61008       1.02525        10     
reco:pandoracalonosce:Calorimetry                  0.852137       1.83316       3.2507        1.50175      0.898742        10     
reco:pandorapid:Chi2ParticleID                    0.00324146    0.00528135    0.00889159    0.00472011    0.00179841       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0308571     0.0520134     0.0851981     0.0407868     0.0204894       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00320554    0.00513184    0.00818819    0.00428401     0.0018656       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.5927        4.29322       9.00681       3.19824       2.36662        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.57672       4.14022       8.35864       3.29075       2.20714        10     
reco:emtrkmichelid:EmTrackMichelId                  150.146       192.961       240.81        191.008       26.7852        10     
reco:crttag:SingleCRTMatchingProducer              0.0100938     0.0139269     0.0193537     0.0138091    0.00264697       10     
reco:crtreco:TwoCRTMatchingProducer               0.00678091    0.00891239     0.0114772     0.0089455    0.00140167       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.669508      0.859998       1.16678      0.818095      0.131261        10     
reco:pandora2Track:LArPandoraTrackCreation          8.47438       12.2128       19.1526       10.5176       3.67529        10     
reco:pandora2calo:Calorimetry                       3.99173       6.41836       10.5122       5.7536        2.31567        10     
reco:pandora2calonosce:Calorimetry                  3.49307       6.01358       11.2189       5.35587       2.31878        10     
reco:pandora2pid:Chi2ParticleID                   0.00923669     0.0149035     0.0215754     0.0156243    0.00348318       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0484023     0.0737362     0.113495      0.0749502     0.0177182       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00929205     0.0148146     0.0208483     0.0150152    0.00334234       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.4401        4.21768       8.47076       4.03801       1.62504        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.85831       5.30445       9.86899       5.03236       1.90561        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.8476        4.56872       8.24057       4.3251        1.7215         10     
[art]:TriggerResults:TriggerResultInserter        4.0677e-05    6.72283e-05   0.000169197   5.94165e-05   3.52912e-05      10     
end_path:out1:RootOutput                           9.498e-06    1.30835e-05    2.129e-05    1.2188e-05    3.23299e-06      10     
end_path:out1:RootOutput(write)                     2.18637       2.73849       3.3256        2.78961      0.375708        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4823.69 MB
  Peak resident set size usage (VmHWM): 3999.41 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.50 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9362568
-rw-r--r-- 1 dune01 dune01 2223877152 Nov 11 20:29 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2112916690 Nov 11 20:27 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1888231728 Nov 11 21:28 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1680045147 Nov 11 18:44 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1611829861 Nov 11 18:34 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   69737050 Nov 11 21:23 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242817 Nov 11 21:28 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 11 21:28 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 11 21:28 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 11 21:28 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 11 20:29 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 11 18:44 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 11 21:28 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 11 18:27 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 11 21:28 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 11 20:33 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 11 20:29 debugprod.log
justIN time: 2026-02-10 07:56:39 UTC       justIN version: 01.06.00