justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266360.1@dunegpschedd01.fnal.gov

Jobsub ID266360.1@dunegpschedd01.fnal.gov
Workflow ID11140
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-10 22:29:35
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-12-11 02:35:06
From worker nodeHostnamecn072.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-10 22:31:32
Input filesmonte-carlo-011140-000001
JobscriptExit code0
Real time4h (14407s)
CPU time3h (14166s = 98%)
Max RSS bytes4433702912 (4228 MiB)
Outputting started2025-12-11 02:31:39
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11140/1/001/reco_stage1.root
Finished2025-12-11 02:35:06
Saved logsjustin-logs:266360.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nning tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 131747 traces, 0 summary
Retagger: tagging trace set: gauss with 100622 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 100622 traces tagged "gauss"
FrameSaver: q=3.53443e+06 n=1750003 tag=gauss
wclsFrameSaver: saving 131747 traces tagged "wiener"
FrameSaver: q=3.7384e+06 n=1550601 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 30851 XUs and 29057 XVs -> 469 XUVs
C:0 T:1 46167 XUs and 43060 XVs -> 3401 XUVs
C:0 T:2 42815 XUs and 76003 XVs -> 3066 XUVs
C:0 T:3 24075 XUs and 31820 XVs -> 158 XUVs
C:0 T:4 16975 XUs and 16275 XVs -> 150 XUVs
C:0 T:5 68733 XUs and 50408 XVs -> 4311 XUVs
C:0 T:6 117263 XUs and 114923 XVs -> 6208 XUVs
C:0 T:7 47156 XUs and 45761 XVs -> 596 XUVs
C:0 T:8 8760 XUs and 8093 XVs -> 106 XUVs
C:0 T:9 62740 XUs and 70138 XVs -> 5178 XUVs
C:0 T:10 26881 XUs and 37257 XVs -> 5192 XUVs
C:0 T:11 721 XUs and 473 XVs -> 162 XUVs
28997 XUVs total
10996 collection wire objects
26876 potential space points
Neighbour search...
1688562 tests to find 890612 neighbours
Iterating with no regularization...
Begin: 7.95437e+07
0 6.82665e+07
1 6.75349e+07
2 6.74384e+07
3 6.7411e+07
Now with regularization...
Begin: 5.71473e+07
0 5.71269e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411032540
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411032540
fLifetime: 30000 [us]
11-Dec-2025 02:31:37 GMT  Closed output file "reco_stage1.root"
11-Dec-2025 02:31:37 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          440.198       625.26        937.488       588.266       129.414        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00177399    0.00433994    0.00791688    0.00427454    0.00190906       10     
reco:ophit:OpHitFinder                             0.243537      0.394168      0.519021      0.410683      0.0926123       10     
reco:opflash:OpFlashFinder                         0.157824       0.26374      0.333927      0.290332      0.057022        10     
reco:opslicer:OpSlicer                             0.562298      0.953167       1.2674        1.00326      0.234155        10     
reco:caldata:DataPrepModule                         5.00162       8.00059       9.45829       8.71099       1.61301        10     
reco:wclsdatasp:WireCellToolkit                     46.5486       59.5822       73.6406       58.5237       8.84065        10     
reco:gaushit:GausHitFinder                          1.13151       1.62767       2.54383       1.34752      0.510832        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000695355   0.000958592    0.0014763    0.00085205    0.00025822       10     
reco:reco3d:SpacePointSolver                        10.1409       18.3548       31.9815       15.1181       8.22016        10     
reco:hitpdune:DisambigFromSpacePoints               1.61097       2.66076       5.93771       2.15174       1.30391        10     
reco:pandora:StandardPandora                        30.5225       51.6056       103.061       45.8331       20.5184        10     
reco:pandoraWriter:StandardPandora                 0.447885      0.647645      0.922977      0.618739      0.168333        10     
reco:pandoraTrack:LArPandoraTrackCreation           7.96176       13.7838       20.868        13.3571       3.50472        10     
reco:pandoraShower:LArPandoraShowerCreation         8.53448       15.2099       28.1372       14.0309       5.01229        10     
reco:pandoracalo:Calorimetry                        3.51277       6.42937       10.0242       6.01709       1.73678        10     
reco:pandoracalonosce:Calorimetry                   3.34299       6.17386       9.56767       5.82875       1.64638        10     
reco:pandorapid:Chi2ParticleID                    0.00537238    0.00764079     0.0126491    0.00674563    0.00239413       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0509175     0.0733137     0.120295      0.0640969     0.0221429       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00514595    0.00713907     0.0119524    0.00636792    0.00204569       10     
reco:pandoraShowercalo:ShowerCalorimetry             7.85         14.9662       29.1196       13.4158       5.55746        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.44165       14.6904       29.4489       12.9622       5.79451        10     
reco:emtrkmichelid:EmTrackMichelId                  203.398       271.406       366.002       260.271       56.3691        10     
reco:crttag:SingleCRTMatchingProducer              0.0241365     0.0406979     0.0585538     0.0414647    0.00870504       10     
reco:crtreco:TwoCRTMatchingProducer                0.0191725     0.0331631     0.0435293     0.0325921    0.00677233       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.737111       1.51085       2.24888       1.5302       0.469268        10     
reco:pandora2Track:LArPandoraTrackCreation          19.2518       46.6006       79.2584       46.0903       14.4894        10     
reco:pandora2calo:Calorimetry                       7.77624       21.137        35.8127       19.7367       6.89594        10     
reco:pandora2calonosce:Calorimetry                  7.50294       19.3987       34.6795       17.8141       6.97359        10     
reco:pandora2pid:Chi2ParticleID                   0.00926045     0.0179136     0.0317038     0.013946     0.00752791       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0550104     0.0996676     0.169697      0.077243      0.0398052       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00923063     0.0177506     0.0308209     0.0133901    0.00741085       10     
reco:pandora2Shower:LArPandoraShowerCreation        7.19691       13.5873       21.3922       13.3836       3.60814        10     
reco:pandora2Showercalo:ShowerCalorimetry           9.72945       17.0266       28.6655       15.5861       5.30212        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      9.19732       15.3333       25.1312       13.1829       4.79281        10     
[art]:TriggerResults:TriggerResultInserter        4.4485e-05    8.07085e-05   0.000166071   6.1217e-05    4.10465e-05      10     
end_path:out1:RootOutput                           8.422e-06    1.15001e-05   1.6916e-05    1.0023e-05    3.32315e-06      10     
end_path:out1:RootOutput(write)                     2.24711       3.40475       4.93849       3.2944        1.06661        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5239.19 MB
  Peak resident set size usage (VmHWM): 4433.7 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.31 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.02 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10580248
-rw-r--r-- 1 pildune32 pildune 2470925781 Dec 11 00:44 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2344189744 Dec 11 00:42 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2159943226 Dec 11 02:31 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1939546871 Dec 10 22:59 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1843756047 Dec 10 22:39 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   75202510 Dec 11 02:23 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242804 Dec 11 02:31 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Dec 11 02:31 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Dec 11 02:31 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Dec 11 02:31 time.db
-rw-r--r-- 1 pildune32 pildune        519 Dec 11 00:44 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Dec 10 22:59 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Dec 11 02:31 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Dec 10 22:31 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Dec 11 02:31 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Dec 11 00:57 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Dec 11 00:44 debugprod.log
justIN time: 2026-02-07 13:29:33 UTC       justIN version: 01.06.00