justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 238690.0@dunegpschedd01.fnal.gov

Jobsub ID238690.0@dunegpschedd01.fnal.gov
Workflow ID9361
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-26 21:35:09
SiteUK_Sheffield
EntryDUNE_UK_Sheffield_lcgce2
Last heartbeat2025-10-27 03:21:06
From worker nodeHostnamewn085.hep
cpuinfoIntel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-26 21:47:26
Input filesmonte-carlo-009361-000001
JobscriptExit code0
Real time5h (19929s)
CPU time5h (19472s = 97%)
Max RSS bytes4386779136 (4183 MiB)
Outputting started2025-10-27 03:19:36
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09361/1/001/reco_stage1.root
Finished2025-10-27 03:21:06
Saved logsjustin-logs:238690.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

KeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 136640 traces, 0 summary
Retagger: tagging trace set: gauss with 107459 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 107459 traces tagged "gauss"
FrameSaver: q=3.68863e+06 n=1924664 tag=gauss
wclsFrameSaver: saving 136640 traces tagged "wiener"
FrameSaver: q=3.89604e+06 n=1722551 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37274 XUs and 49932 XVs -> 458 XUVs
C:0 T:1 83110 XUs and 92103 XVs -> 8014 XUVs
C:0 T:2 78818 XUs and 112708 XVs -> 4326 XUVs
C:0 T:3 39114 XUs and 46266 XVs -> 269 XUVs
C:0 T:4 6965 XUs and 8495 XVs -> 57 XUVs
C:0 T:5 33700 XUs and 44081 XVs -> 6511 XUVs
C:0 T:6 107981 XUs and 147754 XVs -> 6939 XUVs
C:0 T:7 42602 XUs and 51892 XVs -> 374 XUVs
C:0 T:8 10105 XUs and 11100 XVs -> 155 XUVs
C:0 T:9 77818 XUs and 116729 XVs -> 10235 XUVs
C:0 T:10 20765 XUs and 15619 XVs -> 2667 XUVs
C:0 T:11 484 XUs and 312 XVs -> 116 XUVs
40121 XUVs total
12721 collection wire objects
37179 potential space points
Neighbour search...
4183197 tests to find 2017000 neighbours
Iterating with no regularization...
Begin: 8.05409e+07
0 6.86521e+07
1 6.77785e+07
2 6.76421e+07
3 6.76021e+07
Now with regularization...
Begin: 5.66711e+07
0 5.66317e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410134502
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410134502
fLifetime: 30000 [us]
27-Oct-2025 03:19:33 GMT  Closed output file "reco_stage1.root"
27-Oct-2025 03:19:33 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          422.129       721.597       1254.38       724.064       216.767        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00141103    0.00482792    0.00792954    0.00475574    0.00184982       10     
reco:ophit:OpHitFinder                             0.269168      0.410983      0.518618      0.414468      0.0696591       10     
reco:opflash:OpFlashFinder                         0.176278       0.23212      0.306194      0.229323      0.0414748       10     
reco:opslicer:OpSlicer                             0.600668       1.0666        1.46059       1.12298      0.275385        10     
reco:caldata:DataPrepModule                         6.3502        8.82241       10.007        9.26537       1.14592        10     
reco:wclsdatasp:WireCellToolkit                     45.8203       69.6986       81.9509       71.8412       11.994         10     
reco:gaushit:GausHitFinder                          1.04871       2.51888       3.70783       2.73103      0.735856        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000471354   0.000851081   0.00114973    0.000912122   0.00020031       10     
reco:reco3d:SpacePointSolver                        7.3695        19.9643       33.7834       20.7662       7.7465         10     
reco:hitpdune:DisambigFromSpacePoints               1.46991       4.52361       9.32874       4.20687       2.09905        10     
reco:pandora:StandardPandora                        28.892        94.9257       324.114       75.9705       80.9161        10     
reco:pandoraWriter:StandardPandora                 0.522354      0.825879       1.2802       0.780135      0.221774        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.21244       8.76106       16.0854       7.76213       3.17592        10     
reco:pandoraShower:LArPandoraShowerCreation         3.84121       9.20279       20.9927       8.22924       4.62888        10     
reco:pandoracalo:Calorimetry                        2.8139        4.37053       7.61761       3.93931       1.47735        10     
reco:pandoracalonosce:Calorimetry                   2.34422       3.8376        6.98569       3.41639       1.38288        10     
reco:pandorapid:Chi2ParticleID                    0.00777045     0.0125227     0.0184661     0.0133147    0.00389909       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0791863     0.111051      0.159103      0.102465      0.0257009       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00749368     0.0129617     0.0181069     0.013973     0.00381596       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.60158       9.06204       20.6979       8.35316       4.63344        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.39148       9.11361       20.4696       8.20139       4.57898        10     
reco:emtrkmichelid:EmTrackMichelId                  253.044       370.915       505.971       371.391       75.6414        10     
reco:crttag:SingleCRTMatchingProducer              0.0208361     0.032104       0.04277      0.0329653    0.00706928       10     
reco:crtreco:TwoCRTMatchingProducer                0.0119571     0.0214365     0.0298996     0.0221604    0.00507466       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.6919        2.67396       3.4934        2.8204       0.520865        10     
reco:pandora2Track:LArPandoraTrackCreation          13.3503       34.3659       64.966        34.9866       13.7614        10     
reco:pandora2calo:Calorimetry                       5.82479       13.9209       26.3847       14.6443       5.54878        10     
reco:pandora2calonosce:Calorimetry                  5.37662       13.0658       24.8114       13.878        5.40016        10     
reco:pandora2pid:Chi2ParticleID                    0.0241369      0.04132      0.0606233     0.0434319    0.00993777       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.109301      0.162964      0.229798      0.169303      0.0365197       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0233369     0.0402398     0.0586196     0.042531     0.00960787       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.40528       10.1926       18.9632       10.5152       4.03472        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.00669       12.4083       22.758        12.9512       5.34298        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.54755       11.3722       20.762        12.0902       4.95816        10     
[art]:TriggerResults:TriggerResultInserter        6.0177e-05    9.77831e-05   0.000215167   8.8084e-05    4.13313e-05      10     
end_path:out1:RootOutput                           9.637e-06    1.38972e-05   2.2253e-05    1.3661e-05    3.23597e-06      10     
end_path:out1:RootOutput(write)                     2.68822       4.77471       6.99663       4.96735       1.35212        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5193.92 MB
  Peak resident set size usage (VmHWM): 4386.78 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.40 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.67 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.26 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10642064
-rw-r--r--. 1 dune004 dune 2477338404 Oct 27 01:15 detsim_stage2.root
-rw-r--r--. 1 dune004 dune 2348721271 Oct 27 01:10 detsim_stage1.root
-rw-r--r--. 1 dune004 dune 2191095022 Oct 27 03:19 reco_stage1.root
-rw-r--r--. 1 dune004 dune 1937330448 Oct 26 22:23 g4_stage2.root
-rw-r--r--. 1 dune004 dune 1861201907 Oct 26 22:02 g4_stage1.root
-rw-r--r--. 1 dune004 dune   81173090 Oct 27 03:11 Pandora_Events.pndr
-rw-r--r--. 1 dune004 dune     242838 Oct 27 03:19 jobscript.log
-rw-r--r--. 1 dune004 dune     217088 Oct 27 03:19 mem.db
-rw-r--r--. 1 dune004 dune      62376 Oct 27 03:19 Pandora_Geometry.xml
-rw-r--r--. 1 dune004 dune      36864 Oct 27 03:19 time.db
-rw-r--r--. 1 dune004 dune        519 Oct 27 01:15 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune        519 Oct 26 22:23 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune        519 Oct 27 03:19 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune004 dune         52 Oct 26 21:47 all-input-dids.txt
-rw-r--r--. 1 dune004 dune          7 Oct 27 03:19 justin-processed-pfns.txt
-rw-r--r--. 1 dune004 dune          0 Oct 27 01:27 ProtoDUNE.log
-rw-r--r--. 1 dune004 dune          0 Oct 27 01:15 debugprod.log
justIN time: 2025-12-19 05:16:40 UTC       justIN version: 01.05.03