justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 248816.1@dunegpschedd02.fnal.gov

Jobsub ID248816.1@dunegpschedd02.fnal.gov
Workflow ID10396
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-20 10:00:58
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2025-11-20 13:36:16
From worker nodeHostnamecn033.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-20 10:02:22
Input filesmonte-carlo-010396-000001
JobscriptExit code0
Real time3h (12617s)
CPU time3h (12357s = 97%)
Max RSS bytes4230987776 (4034 MiB)
Outputting started2025-11-20 13:32:40
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10396/1/001/reco_stage1.root
Finished2025-11-20 13:36:16
Saved logsjustin-logs:248816.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

leKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 143191 traces, 0 summary
Retagger: tagging trace set: gauss with 111472 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 111472 traces tagged "gauss"
FrameSaver: q=3.86929e+06 n=2004828 tag=gauss
wclsFrameSaver: saving 143191 traces tagged "wiener"
FrameSaver: q=4.07179e+06 n=1784424 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32454 XUs and 33100 XVs -> 378 XUVs
C:0 T:1 64038 XUs and 59092 XVs -> 3290 XUVs
C:0 T:2 161585 XUs and 177453 XVs -> 14807 XUVs
C:0 T:3 63707 XUs and 54722 XVs -> 546 XUVs
C:0 T:4 5983 XUs and 7257 XVs -> 88 XUVs
C:0 T:5 22136 XUs and 23043 XVs -> 2589 XUVs
C:0 T:6 168287 XUs and 137741 XVs -> 7525 XUVs
C:0 T:7 57546 XUs and 44044 XVs -> 634 XUVs
C:0 T:8 6887 XUs and 7744 XVs -> 70 XUVs
C:0 T:9 31166 XUs and 44537 XVs -> 2067 XUVs
C:0 T:10 128588 XUs and 43577 XVs -> 6884 XUVs
C:0 T:11 435 XUs and 473 XVs -> 58 XUVs
38936 XUVs total
13215 collection wire objects
35328 potential space points
Neighbour search...
4520894 tests to find 1917072 neighbours
Iterating with no regularization...
Begin: 8.72339e+07
0 6.61481e+07
1 6.50881e+07
2 6.49954e+07
3 6.49746e+07
Now with regularization...
Begin: 5.48799e+07
0 5.48228e+07
1 5.48054e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410627687
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410627687
fLifetime: 30000 [us]
20-Nov-2025 13:32:36 GMT  Closed output file "reco_stage1.root"
20-Nov-2025 13:32:36 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          402.372       515.965       683.951       518.316       73.4539        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000855105   0.00348901    0.00557552    0.00370332    0.00147139       10     
reco:ophit:OpHitFinder                             0.225176      0.298807      0.403318      0.270412      0.0647309       10     
reco:opflash:OpFlashFinder                         0.141048      0.204302      0.283746      0.199223      0.0418302       10     
reco:opslicer:OpSlicer                              0.56016      0.709166      0.964459      0.659182      0.128774        10     
reco:caldata:DataPrepModule                         6.63978       7.89318       10.4704       7.37234       1.29562        10     
reco:wclsdatasp:WireCellToolkit                     45.1702       51.808        66.5757       49.0597       6.7267         10     
reco:gaushit:GausHitFinder                          1.31769       1.6055        2.28803       1.53689       0.26823        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000839568   0.000973633   0.00121175    0.000911081   0.000139287      10     
reco:reco3d:SpacePointSolver                        7.98506       17.5163       23.9193       16.8509       4.62014        10     
reco:hitpdune:DisambigFromSpacePoints               1.97673       2.54839       3.33773       2.48659      0.390415        10     
reco:pandora:StandardPandora                        31.5114       50.7967       70.5498       47.8511       11.5029        10     
reco:pandoraWriter:StandardPandora                 0.417455      0.590718      0.790314      0.565585      0.111145        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.54425       11.7431       15.0344       12.2404       2.43838        10     
reco:pandoraShower:LArPandoraShowerCreation         8.0318        14.7218       20.457        14.9088       3.49178        10     
reco:pandoracalo:Calorimetry                        2.69827       5.38017       7.07765       5.39452       1.15264        10     
reco:pandoracalonosce:Calorimetry                   2.39628       5.18614       6.66806       5.32117       1.10668        10     
reco:pandorapid:Chi2ParticleID                    0.00463208    0.00717002     0.0107024    0.00651997    0.00174397       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0463387     0.0740449     0.0974494     0.0731793     0.0164706       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00454997    0.00681887    0.00880155    0.00650329    0.00137507       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.24195       13.3663       18.166        14.2755       3.67496        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.34302       12.7472       17.5552       12.8384       3.64468        10     
reco:emtrkmichelid:EmTrackMichelId                  162.422       206.079       301.515       194.617       36.0081        10     
reco:crttag:SingleCRTMatchingProducer              0.0211733     0.033171      0.0393866     0.0354122    0.00519237       10     
reco:crtreco:TwoCRTMatchingProducer                0.0170958     0.0271086     0.0322275     0.0278235    0.00427541       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.839103       1.24384       1.65396       1.25385      0.227333        10     
reco:pandora2Track:LArPandoraTrackCreation          25.6252       38.6934       52.463        38.626        7.54011        10     
reco:pandora2calo:Calorimetry                       11.7542        16.9         22.8387       17.1439       3.1204         10     
reco:pandora2calonosce:Calorimetry                  10.4654       15.8504       21.1473       16.2833       3.20039        10     
reco:pandora2pid:Chi2ParticleID                    0.0118684     0.015762      0.0204859     0.0150335    0.00261336       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0709871     0.0892435     0.108423      0.0898187     0.012602        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0118168     0.0155218     0.0206575     0.0146312    0.00259068       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.16126       11.7981       16.0747       12.1167       2.90832        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.35366       13.2911       18.5763       13.7184       3.11368        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.09799       11.5293       16.0989       11.7321       2.86525        10     
[art]:TriggerResults:TriggerResultInserter        4.0841e-05    5.80858e-05   0.000104997   5.00155e-05   1.87923e-05      10     
end_path:out1:RootOutput                           6.256e-06    8.1949e-06    1.1855e-05    7.7515e-06    1.44106e-06      10     
end_path:out1:RootOutput(write)                     2.60323       3.01859       3.50101       3.02187       0.32028        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5033.39 MB
  Peak resident set size usage (VmHWM): 4230.99 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.61 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10389952
-rw-r--r-- 1 pildune22 pildune 2440000896 Nov 20 12:04 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2320635078 Nov 20 12:00 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2145617854 Nov 20 13:32 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 1860502853 Nov 20 10:24 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1791882058 Nov 20 10:10 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   80053070 Nov 20 13:25 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     242977 Nov 20 13:32 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Nov 20 13:32 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Nov 20 13:32 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Nov 20 13:32 time.db
-rw-r--r-- 1 pildune22 pildune        519 Nov 20 12:04 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov 20 10:24 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov 20 13:32 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Nov 20 10:02 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Nov 20 13:32 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Nov 20 12:11 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Nov 20 12:04 debugprod.log
justIN time: 2026-02-09 03:28:27 UTC       justIN version: 01.06.00