justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 236981.0@dunegpschedd02.fnal.gov

Jobsub ID236981.0@dunegpschedd02.fnal.gov
Workflow ID9574
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-03 21:49:18
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2025-11-04 04:28:13
From worker nodeHostnamecn096.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 21:50:05
Input filesmonte-carlo-009574-000001
JobscriptExit code0
Real time6h (23660s)
CPU time6h (23304s = 98%)
Max RSS bytes4827959296 (4604 MiB)
Outputting started2025-11-04 04:24:26
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09574/1/001/reco_stage1.root
Finished2025-11-04 04:28:13
Saved logsjustin-logs:236981.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

eToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 156495 traces, 0 summary
Retagger: tagging trace set: gauss with 115050 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 115050 traces tagged "gauss"
FrameSaver: q=4.11131e+06 n=2021734 tag=gauss
wclsFrameSaver: saving 156495 traces tagged "wiener"
FrameSaver: q=4.36575e+06 n=1773864 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32029 XUs and 25985 XVs -> 318 XUVs
C:0 T:1 60655 XUs and 57127 XVs -> 6859 XUVs
C:0 T:2 74725 XUs and 122750 XVs -> 4923 XUVs
C:0 T:3 32709 XUs and 48530 XVs -> 357 XUVs
C:0 T:4 17264 XUs and 13078 XVs -> 129 XUVs
C:0 T:5 67073 XUs and 38612 XVs -> 5369 XUVs
C:0 T:6 157407 XUs and 155792 XVs -> 11171 XUVs
C:0 T:7 59052 XUs and 57460 XVs -> 788 XUVs
C:0 T:8 13915 XUs and 9417 XVs -> 231 XUVs
C:0 T:9 97611 XUs and 83028 XVs -> 8639 XUVs
C:0 T:10 47936 XUs and 47493 XVs -> 6720 XUVs
C:0 T:11 496 XUs and 305 XVs -> 52 XUVs
45556 XUVs total
14741 collection wire objects
41052 potential space points
Neighbour search...
3379800 tests to find 1760302 neighbours
Iterating with no regularization...
Begin: 7.30646e+07
0 5.56222e+07
1 5.43832e+07
2 5.42065e+07
3 5.41541e+07
Now with regularization...
Begin: 4.21411e+07
0 4.20822e+07
1 4.20512e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410295589
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410295589
fLifetime: 30000 [us]
04-Nov-2025 04:24:23 GMT  Closed output file "reco_stage1.root"
04-Nov-2025 04:24:23 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          934.53        1101.5        1716.37       1049.38        217.1         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00178258    0.00566386    0.00979437    0.00529831    0.00256716       10     
reco:ophit:OpHitFinder                             0.483943      0.626307      0.735229       0.6229       0.0750562       10     
reco:opflash:OpFlashFinder                         0.255574      0.415402      0.527443      0.409868      0.0871052       10     
reco:opslicer:OpSlicer                              1.15386       1.44208       1.91092       1.37154      0.254943        10     
reco:caldata:DataPrepModule                         8.57809       10.5968       11.2907       10.9505      0.867901        10     
reco:wclsdatasp:WireCellToolkit                     74.2003       77.8944       80.3995       78.5345       1.8075         10     
reco:gaushit:GausHitFinder                          2.61871       3.16333       4.14992       3.03554      0.453827        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00108202    0.00131816    0.00161674    0.00134251    0.000172558      10     
reco:reco3d:SpacePointSolver                        21.3127       33.362        53.4782       32.6811       8.58311        10     
reco:hitpdune:DisambigFromSpacePoints               4.54461       6.0293        10.2703       5.63431       1.62097        10     
reco:pandora:StandardPandora                        80.4896       118.848       279.822       98.0999       55.7837        10     
reco:pandoraWriter:StandardPandora                  0.87603       1.00652       1.37824      0.967246      0.140126        10     
reco:pandoraTrack:LArPandoraTrackCreation           17.6048       23.8799       36.3521       23.3262       4.65215        10     
reco:pandoraShower:LArPandoraShowerCreation         20.6913       26.8905       52.9164       24.0163       9.00969        10     
reco:pandoracalo:Calorimetry                        8.72298       11.7493       17.4754       11.4719       2.18225        10     
reco:pandoracalonosce:Calorimetry                   8.27116       11.3813       17.1767       11.0109       2.20138        10     
reco:pandorapid:Chi2ParticleID                     0.0104909     0.0144935     0.0174352     0.0149989    0.00201736       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.108682      0.134629       0.17272      0.134526      0.017183        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0102362     0.013574      0.0176449     0.0137055     0.0017783       10     
reco:pandoraShowercalo:ShowerCalorimetry            21.1815       28.7672       58.6277       25.8693       10.468         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       20.5631       28.1451       56.9914       25.7274       10.1221        10     
reco:emtrkmichelid:EmTrackMichelId                  417.685       469.987       600.242       456.759       51.4474        10     
reco:crttag:SingleCRTMatchingProducer              0.0453297     0.0537694     0.0646477     0.0514178    0.00614556       10     
reco:crtreco:TwoCRTMatchingProducer                0.0373455     0.0428755     0.0496226     0.0431621    0.00305193       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.67956       2.57932       3.13209       2.49611      0.430782        10     
reco:pandora2Track:LArPandoraTrackCreation          62.4265       79.2345       143.754       72.271        22.7924        10     
reco:pandora2calo:Calorimetry                       30.2264       38.5642       71.3722       34.7601       11.4762        10     
reco:pandora2calonosce:Calorimetry                  28.4391       36.4966       66.4068       33.1715       10.5488        10     
reco:pandora2pid:Chi2ParticleID                    0.0247151     0.033681      0.0527438     0.0330482    0.00735035       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP         0.14837      0.175575      0.245944       0.17445      0.0266757       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0234493     0.0329741     0.0505972     0.0315853    0.00676367       10     
reco:pandora2Shower:LArPandoraShowerCreation        17.5549       22.9902       36.7228       23.1773       5.31491        10     
reco:pandora2Showercalo:ShowerCalorimetry           25.7887       32.0341       55.794        30.4257       8.4403         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      22.7832       29.1007       50.2548       27.4787       7.79296        10     
[art]:TriggerResults:TriggerResultInserter        8.9912e-05    0.000108919   0.000198086   9.7172e-05    3.09219e-05      10     
end_path:out1:RootOutput                          1.1381e-05    1.36526e-05   1.6479e-05    1.3325e-05    1.60188e-06      10     
end_path:out1:RootOutput(write)                     4.83019       5.54434       7.30067       5.36148      0.691632        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5631.23 MB
  Peak resident set size usage (VmHWM): 4827.96 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.40 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.58 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 12098884
-rw-r--r-- 1 pildune22 pildune 2763473313 Nov  4 01:16 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2616699946 Nov  4 01:12 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2522344231 Nov  4 04:24 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 2240582831 Nov  3 22:31 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 2153317710 Nov  3 22:00 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   92222480 Nov  4 04:10 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     242801 Nov  4 04:24 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Nov  4 04:24 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Nov  4 04:24 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Nov  4 04:24 time.db
-rw-r--r-- 1 pildune22 pildune        519 Nov  4 01:17 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov  3 22:31 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov  4 04:24 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Nov  3 21:50 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Nov  4 04:24 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Nov  4 01:30 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Nov  4 01:17 debugprod.log
justIN time: 2026-02-11 12:01:33 UTC       justIN version: 01.06.00