justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 295947.0@dunegpschedd02.fnal.gov

Jobsub ID295947.0@dunegpschedd02.fnal.gov
Workflow ID12447
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-27 14:23:30
SiteUK_Oxford
EntryDUNE_UK_SGrid_Oxford_arc01
Last heartbeat2026-01-27 17:00:44
From worker nodeHostnamet2wn173.physics.ox.ac.uk
cpuinfoAMD EPYC 9655 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-27 14:25:08
Input filesmonte-carlo-012447-000001
JobscriptExit code0
Real time2h (9193s)
CPU time2h (8795s = 95%)
Max RSS bytes4281761792 (4083 MiB)
Outputting started2026-01-27 16:58:22
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12447/1/001/reco_stage1.root
Finished2026-01-27 17:00:44
Saved logsjustin-logs:295947.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

g tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123039 traces, 0 summary
Retagger: tagging trace set: gauss with 94980 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 94980 traces tagged "gauss"
FrameSaver: q=3.3432e+06 n=1739646 tag=gauss
wclsFrameSaver: saving 123039 traces tagged "wiener"
FrameSaver: q=3.52075e+06 n=1545335 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 47818 XUs and 37188 XVs -> 702 XUVs
C:0 T:1 108663 XUs and 91363 XVs -> 8081 XUVs
C:0 T:2 59608 XUs and 158902 XVs -> 7371 XUVs
C:0 T:3 22633 XUs and 54200 XVs -> 166 XUVs
C:0 T:4 12068 XUs and 10695 XVs -> 182 XUVs
C:0 T:5 56219 XUs and 83850 XVs -> 14408 XUVs
C:0 T:6 84024 XUs and 115451 XVs -> 3854 XUVs
C:0 T:7 35653 XUs and 44781 XVs -> 234 XUVs
C:0 T:8 6562 XUs and 8187 XVs -> 126 XUVs
C:0 T:9 38795 XUs and 55703 XVs -> 3312 XUVs
C:0 T:10 23843 XUs and 20364 XVs -> 3609 XUVs
C:0 T:11 69 XUs and 44 XVs -> 7 XUVs
42052 XUVs total
12121 collection wire objects
39280 potential space points
Neighbour search...
6183096 tests to find 2706478 neighbours
Iterating with no regularization...
Begin: 7.03713e+07
0 5.56923e+07
1 5.47796e+07
2 5.46514e+07
3 5.46146e+07
Now with regularization...
Begin: 4.49247e+07
0 4.48586e+07
1 4.48216e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411999252
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411999252
fLifetime: 30000 [us]
27-Jan-2026 16:57:57 GMT  Closed output file "reco_stage1.root"
27-Jan-2026 16:57:57 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                           292.1        342.756       537.485       319.71        70.8332        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00155525    0.00392484    0.00569199    0.00416924    0.00138792       10     
reco:ophit:OpHitFinder                             0.243344      0.288247      0.309655      0.295994      0.0189221       10     
reco:opflash:OpFlashFinder                          0.10911      0.148052      0.169527      0.152004      0.018293        10     
reco:opslicer:OpSlicer                             0.533254      0.710597      0.800098      0.732102      0.0822544       10     
reco:caldata:DataPrepModule                         4.90284       5.12672       5.29998       5.16139       0.1104         10     
reco:wclsdatasp:WireCellToolkit                     27.5379       28.9804       30.6824       28.932        1.05914        10     
reco:gaushit:GausHitFinder                         0.919994       1.33374       1.71508       1.30932      0.208841        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000332915   0.000467454   0.000675066   0.00045751    9.75643e-05      10     
reco:reco3d:SpacePointSolver                         13.39        17.4297       26.4374       15.8191       3.91049        10     
reco:hitpdune:DisambigFromSpacePoints               1.42244       1.92357       3.25347       1.79371      0.512441        10     
reco:pandora:StandardPandora                        26.6784       38.0022       80.4817       32.5165       15.1218        10     
reco:pandoraWriter:StandardPandora                 0.396743       0.55076       0.65313      0.557249      0.0754395       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.38082       5.51015       10.8587       5.15607       1.89514        10     
reco:pandoraShower:LArPandoraShowerCreation         3.58195        6.377        14.095        5.72028       2.84347        10     
reco:pandoracalo:Calorimetry                        1.60175       2.75926       4.11455       2.54211      0.828426        10     
reco:pandoracalonosce:Calorimetry                   1.70389       2.49128       3.90631       2.13987      0.832232        10     
reco:pandorapid:Chi2ParticleID                    0.00357009    0.00544113    0.00721838    0.00542834    0.00111715       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0334181     0.066093      0.092646      0.0655424     0.016141        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0036787    0.00523984    0.00743984    0.00525884    0.000975029      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.8342        6.03965       12.4372       5.17704       2.34646        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.31883       5.76739       10.2795       5.29352       1.89562        10     
reco:emtrkmichelid:EmTrackMichelId                  141.88        162.548       228.342       155.095       24.8897        10     
reco:crttag:SingleCRTMatchingProducer              0.0152516     0.0193474     0.024883      0.0188756    0.00349566       10     
reco:crtreco:TwoCRTMatchingProducer               0.00764574     0.0126237     0.0223116     0.0110519    0.00438371       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.521139      0.702408       0.93698      0.703001      0.111406        10     
reco:pandora2Track:LArPandoraTrackCreation          13.0939       18.4949       36.9251       15.4979       6.88285        10     
reco:pandora2calo:Calorimetry                       5.62366       8.13792       13.8694       7.31739       2.28733        10     
reco:pandora2calonosce:Calorimetry                  5.82633       7.57749       14.2253       6.59261       2.38811        10     
reco:pandora2pid:Chi2ParticleID                    0.0110819     0.0140882     0.0210357     0.0138262    0.00248984       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0632403     0.0868027     0.121665      0.0821958     0.0160594       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0112469     0.014115      0.0204142     0.0136298    0.00233364       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.17773       5.55376       10.8071       4.90851       1.84197        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.13915       6.76488       12.534        6.00897       2.14253        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.32498       5.88093       10.1339       4.80077       1.94623        10     
[art]:TriggerResults:TriggerResultInserter         5.226e-05    7.6958e-05    0.000187434   6.33015e-05   3.80666e-05      10     
end_path:out1:RootOutput                           9.825e-06    1.19409e-05    2.012e-05    1.12115e-05   2.80967e-06      10     
end_path:out1:RootOutput(write)                     2.89424       3.26196       4.35218       3.09554      0.415695        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5227.11 MB
  Peak resident set size usage (VmHWM): 4281.76 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.25 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10581216
-rw-r--r-- 1 dune020 dune 2472898962 Jan 27 15:58 detsim_stage2.root
-rw-r--r-- 1 dune020 dune 2349125768 Jan 27 15:55 detsim_stage1.root
-rw-r--r-- 1 dune020 dune 2198708012 Jan 27 16:57 reco_stage1.root
-rw-r--r-- 1 dune020 dune 1900470154 Jan 27 14:41 g4_stage2.root
-rw-r--r-- 1 dune020 dune 1829106872 Jan 27 14:31 g4_stage1.root
-rw-r--r-- 1 dune020 dune   84236810 Jan 27 16:54 Pandora_Events.pndr
-rw-r--r-- 1 dune020 dune     242698 Jan 27 16:58 jobscript.log
-rw-r--r-- 1 dune020 dune     217088 Jan 27 16:57 mem.db
-rw-r--r-- 1 dune020 dune      62376 Jan 27 16:57 Pandora_Geometry.xml
-rw-r--r-- 1 dune020 dune      36864 Jan 27 16:57 time.db
-rw-r--r-- 1 dune020 dune        519 Jan 27 15:58 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune020 dune        519 Jan 27 14:41 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune020 dune        519 Jan 27 16:57 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune020 dune         52 Jan 27 14:25 all-input-dids.txt
-rw-r--r-- 1 dune020 dune          7 Jan 27 16:57 justin-processed-pfns.txt
-rw-r--r-- 1 dune020 dune          0 Jan 27 16:03 ProtoDUNE.log
-rw-r--r-- 1 dune020 dune          0 Jan 27 15:58 debugprod.log
justIN time: 2026-02-04 18:08:57 UTC       justIN version: 01.06.00