justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 233496.1@dunegpschedd02.fnal.gov

Jobsub ID233496.1@dunegpschedd02.fnal.gov
Workflow ID9402
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-28 06:21:51
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-10-28 10:58:12
From worker nodeHostnamecn009.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-28 06:22:23
Input filesmonte-carlo-009402-000001
JobscriptExit code0
Real time4h (16339s)
CPU time4h (15946s = 97%)
Max RSS bytes4200390656 (4005 MiB)
Outputting started2025-10-28 10:54:43
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09402/1/001/reco_stage1.root
Finished2025-10-28 10:58:12
Saved logsjustin-logs:233496.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

wDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 155012 traces, 0 summary
Retagger: tagging trace set: gauss with 119235 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 119235 traces tagged "gauss"
FrameSaver: q=4.26857e+06 n=2118842 tag=gauss
wclsFrameSaver: saving 155012 traces tagged "wiener"
FrameSaver: q=4.50752e+06 n=1878794 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37530 XUs and 40013 XVs -> 923 XUVs
C:0 T:1 101250 XUs and 94875 XVs -> 10285 XUVs
C:0 T:2 155083 XUs and 106070 XVs -> 7812 XUVs
C:0 T:3 57671 XUs and 40290 XVs -> 309 XUVs
C:0 T:4 18026 XUs and 15018 XVs -> 113 XUVs
C:0 T:5 82298 XUs and 52231 XVs -> 6014 XUVs
C:0 T:6 163219 XUs and 122438 XVs -> 14492 XUVs
C:0 T:7 64973 XUs and 49314 XVs -> 1048 XUVs
C:0 T:8 10637 XUs and 8727 XVs -> 109 XUVs
C:0 T:9 69224 XUs and 70845 XVs -> 5412 XUVs
C:0 T:10 28274 XUs and 20514 XVs -> 2981 XUVs
49498 XUVs total
13598 collection wire objects
43522 potential space points
Neighbour search...
4959866 tests to find 2349052 neighbours
Iterating with no regularization...
Begin: 1.27358e+08
0 1.01034e+08
1 9.90812e+07
2 9.89008e+07
3 9.88439e+07
Now with regularization...
Begin: 8.51727e+07
0 8.51161e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410161903
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410161903
fLifetime: 30000 [us]
28-Oct-2025 10:54:38 GMT  Closed output file "reco_stage1.root"
28-Oct-2025 10:54:38 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          418.997       704.427       927.285       690.921       139.404        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00199518    0.00444625    0.00686267    0.00443571    0.00149502       10     
reco:ophit:OpHitFinder                             0.245728       0.37976      0.503588      0.361689      0.0784242       10     
reco:opflash:OpFlashFinder                         0.149384      0.248401      0.324412      0.249055      0.0510171       10     
reco:opslicer:OpSlicer                              0.57234       0.91074       1.25508       0.85008      0.201656        10     
reco:caldata:DataPrepModule                         4.98773       8.3457        9.14577       8.76374       1.1767         10     
reco:wclsdatasp:WireCellToolkit                     44.711        64.677        70.8876       66.8934       7.33173        10     
reco:gaushit:GausHitFinder                          1.27656       2.03555       2.74573       1.8804       0.480459        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000815069   0.00106967    0.00155234    0.000975128   0.00024525       10     
reco:reco3d:SpacePointSolver                        8.09439       25.8146       75.6759       19.7071       18.2362        10     
reco:hitpdune:DisambigFromSpacePoints               1.72283       3.45248       5.55787       3.11057       1.22041        10     
reco:pandora:StandardPandora                        32.145        58.2271       89.4913       55.7187       17.9532        10     
reco:pandoraWriter:StandardPandora                  0.46995      0.713691      0.941744      0.698132      0.158599        10     
reco:pandoraTrack:LArPandoraTrackCreation           7.4683        15.1651       23.428        14.6896       3.89439        10     
reco:pandoraShower:LArPandoraShowerCreation         6.73855       17.2269       24.9086       16.3306       5.02801        10     
reco:pandoracalo:Calorimetry                        3.24479       7.03771       9.92988       6.88581       1.81483        10     
reco:pandoracalonosce:Calorimetry                   2.83604       6.74309       9.9781        6.66694       1.92375        10     
reco:pandorapid:Chi2ParticleID                     0.0061337      0.01066      0.0147303     0.0107674    0.00217223       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0599851     0.0964557     0.142379      0.0987441     0.0212151       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00596084    0.00935049     0.0119558    0.00928469    0.00207013       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.52459       17.5215       26.6326       17.3654       6.16156        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.32312       17.3991       26.2367       17.2439       6.06654        10     
reco:emtrkmichelid:EmTrackMichelId                  205.086       302.61        372.34        305.873       46.8463        10     
reco:crttag:SingleCRTMatchingProducer              0.0236834     0.0419224     0.054461      0.0414072    0.00847938       10     
reco:crtreco:TwoCRTMatchingProducer                0.0192487     0.035524      0.0439329     0.0369303    0.00666669       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.992248       1.69606       2.25946       1.71537      0.363798        10     
reco:pandora2Track:LArPandoraTrackCreation          21.2701       51.7013       77.0848       51.6664       14.4678        10     
reco:pandora2calo:Calorimetry                       8.64679       22.8253       34.1663       22.4874       7.2179         10     
reco:pandora2calonosce:Calorimetry                  8.11563       22.0233       32.2808       22.5239       6.88532        10     
reco:pandora2pid:Chi2ParticleID                    0.0134144     0.024331      0.0383604     0.0238838    0.00650835       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0803927     0.123028      0.168501      0.122722      0.0300781       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0127558     0.0216439     0.0284494     0.0204534    0.00452339       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.82919       16.2823       23.4934       16.3903       4.36326        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.76166       19.3369       27.5994       20.2189       6.0006         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.16209       17.5521       24.772        18.2453       5.37572        10     
[art]:TriggerResults:TriggerResultInserter        5.9673e-05    0.000105252   0.000203829   0.000100362   3.85332e-05      10     
end_path:out1:RootOutput                           8.537e-06    1.21808e-05   1.8851e-05    1.09355e-05   2.89121e-06      10     
end_path:out1:RootOutput(write)                     2.7319        3.92172       5.3978        3.96526      0.780497        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5002.57 MB
  Peak resident set size usage (VmHWM): 4200.39 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.33 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.18 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9997404
-rw-r--r-- 1 pildune32 pildune 2356143893 Oct 28 08:53 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2235530651 Oct 28 08:49 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2045269517 Oct 28 10:54 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1796373764 Oct 28 06:51 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1727910965 Oct 28 06:30 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   75493900 Oct 28 10:42 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242899 Oct 28 10:54 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Oct 28 10:54 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Oct 28 10:54 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Oct 28 10:54 time.db
-rw-r--r-- 1 pildune32 pildune        519 Oct 28 08:53 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Oct 28 06:51 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Oct 28 10:54 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Oct 28 06:22 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Oct 28 10:54 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Oct 28 09:05 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Oct 28 08:53 debugprod.log
justIN time: 2025-11-04 01:54:01 UTC       justIN version: 01.05.01