justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242659.0@dunegpschedd01.fnal.gov

Jobsub ID242659.0@dunegpschedd01.fnal.gov
Workflow ID9575
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-03 21:49:19
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2025-11-04 02:35:16
From worker nodeHostnamecn080.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 21:50:36
Input filesmonte-carlo-009575-000001
JobscriptExit code0
Real time4h (16833s)
CPU time4h (16435s = 97%)
Max RSS bytes4497670144 (4289 MiB)
Outputting started2025-11-04 02:31:10
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09575/1/001/reco_stage1.root
Finished2025-11-04 02:35:16
Saved logsjustin-logs:242659.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

eToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 166952 traces, 0 summary
Retagger: tagging trace set: gauss with 127567 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 127567 traces tagged "gauss"
FrameSaver: q=4.3535e+06 n=2280635 tag=gauss
wclsFrameSaver: saving 166952 traces tagged "wiener"
FrameSaver: q=4.61949e+06 n=2028791 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 22468 XUs and 24533 XVs -> 162 XUVs
C:0 T:1 39667 XUs and 38285 XVs -> 2578 XUVs
C:0 T:2 75131 XUs and 140268 XVs -> 5406 XUVs
C:0 T:3 36447 XUs and 60667 XVs -> 348 XUVs
C:0 T:4 22643 XUs and 24063 XVs -> 367 XUVs
C:0 T:5 93595 XUs and 80491 XVs -> 5836 XUVs
C:0 T:6 257158 XUs and 221906 XVs -> 23851 XUVs
C:0 T:7 57874 XUs and 71511 XVs -> 947 XUVs
C:0 T:8 7940 XUs and 8487 XVs -> 68 XUVs
C:0 T:9 60095 XUs and 72733 XVs -> 7709 XUVs
C:0 T:10 110359 XUs and 66120 XVs -> 18082 XUVs
C:0 T:11 447 XUs and 385 XVs -> 47 XUVs
65401 XUVs total
15625 collection wire objects
60511 potential space points
Neighbour search...
10816477 tests to find 4654070 neighbours
Iterating with no regularization...
Begin: 7.67606e+07
0 6.19395e+07
1 6.08414e+07
2 6.06781e+07
3 6.06318e+07
Now with regularization...
Begin: 4.88924e+07
0 4.88278e+07
1 4.87792e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410295578
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410295578
fLifetime: 30000 [us]
04-Nov-2025 02:31:07 GMT  Closed output file "reco_stage1.root"
04-Nov-2025 02:31:07 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          526.699       740.174       932.415       704.561       128.758        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00134248    0.00807064     0.0350387    0.00532173    0.00922107       10     
reco:ophit:OpHitFinder                             0.239487      0.387882      0.662996      0.354628      0.124281        10     
reco:opflash:OpFlashFinder                         0.169715      0.273931      0.350824      0.298727      0.0540581       10     
reco:opslicer:OpSlicer                             0.664395      0.917763        1.272       0.900941      0.198469        10     
reco:caldata:DataPrepModule                         6.49212       7.89954       9.55685       7.99741       1.00873        10     
reco:wclsdatasp:WireCellToolkit                     48.7003       56.354        64.1988       54.6349       4.90183        10     
reco:gaushit:GausHitFinder                          1.24471       1.96318       2.81066       1.91827      0.422854        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00084634    0.00106383    0.00122354    0.00108442    0.000140862      10     
reco:reco3d:SpacePointSolver                        12.4969       23.1295       37.2603       21.6466       7.47393        10     
reco:hitpdune:DisambigFromSpacePoints               1.88898       3.69593       7.21546       3.27728       1.67537        10     
reco:pandora:StandardPandora                        35.1249       69.3479       142.284       55.7973       31.5987        10     
reco:pandoraWriter:StandardPandora                 0.536131      0.756753       1.14503      0.752075      0.157613        10     
reco:pandoraTrack:LArPandoraTrackCreation           10.7375       15.2872       19.4215       15.2323       2.57753        10     
reco:pandoraShower:LArPandoraShowerCreation         11.825        19.4187       27.5251       17.7016       5.27589        10     
reco:pandoracalo:Calorimetry                        4.93639       7.53309       10.7957       7.54327       1.52584        10     
reco:pandoracalonosce:Calorimetry                   4.67791       7.2721        10.2747       7.36003       1.52365        10     
reco:pandorapid:Chi2ParticleID                    0.00546234    0.00880707     0.0151974    0.00862596    0.00246863       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.064005      0.0883634     0.134147      0.0874349     0.0189515       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0060562    0.00866283     0.014737      0.0085749    0.00224482       10     
reco:pandoraShowercalo:ShowerCalorimetry            10.5132       19.2833       27.0659       18.0225       5.49681        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       10.3933       19.0542       26.9863       17.2164       5.6583         10     
reco:emtrkmichelid:EmTrackMichelId                  251.995       320.781       372.244       334.632       41.6245        10     
reco:crttag:SingleCRTMatchingProducer              0.0361041     0.0422748     0.0530597     0.0423626    0.00501258       10     
reco:crtreco:TwoCRTMatchingProducer                 0.0303       0.0354016     0.0460335     0.034706     0.00460417       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.36891       2.03525       3.2882        1.87184      0.603945        10     
reco:pandora2Track:LArPandoraTrackCreation          39.0393       57.7998       84.1806       53.3628       12.716         10     
reco:pandora2calo:Calorimetry                       13.355        25.2472       35.0595       24.4983       5.8092         10     
reco:pandora2calonosce:Calorimetry                  13.0426       23.7289       31.2257       23.5133        5.473         10     
reco:pandora2pid:Chi2ParticleID                    0.0126602     0.0212461     0.0269849     0.0209289    0.00434446       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0919367     0.124523      0.195126      0.120665      0.029872        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0157792     0.0224654     0.0367491     0.0220179    0.00587003       10     
reco:pandora2Shower:LArPandoraShowerCreation        9.84925       16.3156       20.8182       17.3502       3.47462        10     
reco:pandora2Showercalo:ShowerCalorimetry           11.1807       19.8754       27.2563       20.0566       4.68029        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      10.5236       17.3534       23.3613       17.4543       3.67862        10     
[art]:TriggerResults:TriggerResultInserter        5.3648e-05    8.18471e-05   0.000168562   7.3338e-05    3.20551e-05      10     
end_path:out1:RootOutput                           7.175e-06    9.7827e-06    1.3061e-05    9.5525e-06    1.80643e-06      10     
end_path:out1:RootOutput(write)                     2.91501       3.89638       5.51632       3.66689      0.712251        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5299.26 MB
  Peak resident set size usage (VmHWM): 4497.67 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.67 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11361976
-rw-r--r-- 1 pildune22 pildune 2624948699 Nov  4 00:23 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2488916375 Nov  4 00:19 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2356920479 Nov  4 02:31 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 2079325436 Nov  3 22:22 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1997852404 Nov  3 21:59 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   86083270 Nov  4 02:19 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     242868 Nov  4 02:31 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Nov  4 02:31 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Nov  4 02:31 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Nov  4 02:31 time.db
-rw-r--r-- 1 pildune22 pildune        519 Nov  4 00:23 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov  3 22:22 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov  4 02:31 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Nov  3 21:50 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Nov  4 02:31 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Nov  4 00:33 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Nov  4 00:23 debugprod.log
justIN time: 2026-02-11 12:01:34 UTC       justIN version: 01.06.00