justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 238781.1@dunegpschedd01.fnal.gov

Jobsub ID238781.1@dunegpschedd01.fnal.gov
Workflow ID9370
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-27 06:10:32
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-10-27 11:06:06
From worker nodeHostnamewn2204240.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-27 06:12:59
Input filesmonte-carlo-009370-000001
JobscriptExit code0
Real time4h (17470s)
CPU time4h (16804s = 96%)
Max RSS bytes4443938816 (4238 MiB)
Outputting started2025-10-27 11:04:09
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09370/1/001/reco_stage1.root
Finished2025-10-27 11:06:06
Saved logsjustin-logs:238781.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

gnalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 142145 traces, 0 summary
Retagger: tagging trace set: gauss with 104607 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 104607 traces tagged "gauss"
FrameSaver: q=3.49145e+06 n=1790273 tag=gauss
wclsFrameSaver: saving 142145 traces tagged "wiener"
FrameSaver: q=3.70533e+06 n=1555722 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 24868 XUs and 26271 XVs -> 703 XUVs
C:0 T:1 63805 XUs and 63131 XVs -> 10460 XUVs
C:0 T:2 171410 XUs and 168181 XVs -> 7798 XUVs
C:0 T:3 75963 XUs and 65060 XVs -> 546 XUVs
C:0 T:4 10978 XUs and 7887 XVs -> 86 XUVs
C:0 T:5 51953 XUs and 32021 XVs -> 6263 XUVs
C:0 T:6 88611 XUs and 74132 XVs -> 4653 XUVs
C:0 T:7 35154 XUs and 27260 XVs -> 190 XUVs
C:0 T:8 9588 XUs and 10151 XVs -> 148 XUVs
C:0 T:9 74448 XUs and 101359 XVs -> 8068 XUVs
C:0 T:10 28747 XUs and 11640 XVs -> 4101 XUVs
C:0 T:11 148 XUs and 80 XVs -> 21 XUVs
43037 XUVs total
13322 collection wire objects
39370 potential space points
Neighbour search...
4161092 tests to find 2152208 neighbours
Iterating with no regularization...
Begin: 6.88362e+07
0 5.398e+07
1 5.31118e+07
2 5.29807e+07
3 5.29407e+07
Now with regularization...
Begin: 4.31383e+07
0 4.30772e+07
1 4.30453e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410141637
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410141637
fLifetime: 30000 [us]
27-Oct-2025 11:04:07 GMT  Closed output file "reco_stage1.root"
27-Oct-2025 11:04:07 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          411.288       529.686       707.703       525.353       85.8675        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00140561    0.00505726    0.00820739    0.00526923    0.00231185       10     
reco:ophit:OpHitFinder                             0.274899      0.440825      0.518398      0.461237      0.0636233       10     
reco:opflash:OpFlashFinder                         0.212947       0.26019      0.339453      0.255917      0.0332929       10     
reco:opslicer:OpSlicer                             0.791234       1.07277       1.40544       1.10152      0.179783        10     
reco:caldata:DataPrepModule                         7.50223       7.9034        8.62225       7.79021      0.298659        10     
reco:wclsdatasp:WireCellToolkit                     39.7998       42.8649       45.4251       42.8644       1.60722        10     
reco:gaushit:GausHitFinder                          1.24266       1.94439       2.45932       2.01207      0.341254        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000647086   0.000774597   0.000907737   0.00078419    8.55392e-05      10     
reco:reco3d:SpacePointSolver                        16.9627       28.9226       48.2519       25.9837       9.58558        10     
reco:hitpdune:DisambigFromSpacePoints               2.17067       2.9975        4.38335       2.95022      0.650396        10     
reco:pandora:StandardPandora                        29.3073       60.8395       104.778       61.3113       21.2288        10     
reco:pandoraWriter:StandardPandora                 0.646567      0.742464      0.914447      0.716636      0.0836992       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.39324       6.76353       10.6135       6.64849       1.81806        10     
reco:pandoraShower:LArPandoraShowerCreation         4.8923        7.65668       12.9896       7.52731       2.28089        10     
reco:pandoracalo:Calorimetry                        2.58136       3.95135       6.2682        3.8403       0.981914        10     
reco:pandoracalonosce:Calorimetry                   2.35546       3.63787       5.58531       3.5202       0.864833        10     
reco:pandorapid:Chi2ParticleID                    0.00719164    0.00965086     0.011707     0.00986912    0.00139408       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0732009     0.0928454     0.121019      0.0888346     0.0124088       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00692599    0.00929308     0.0112998     0.0093555    0.00134672       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.66301       8.31041       13.2316       8.30346       2.06947        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.4655        7.88327       12.3595       7.87422       1.90103        10     
reco:emtrkmichelid:EmTrackMichelId                  233.94        269.529       335.529       265.378        29.14         10     
reco:crttag:SingleCRTMatchingProducer              0.0163903     0.0267731     0.0568316     0.0246502     0.0116319       10     
reco:crtreco:TwoCRTMatchingProducer               0.00914963     0.0136253     0.0191272     0.0129728    0.00281637       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.980519       1.28487       1.76097       1.28213      0.214914        10     
reco:pandora2Track:LArPandoraTrackCreation          14.6693       21.4201       35.4598       21.2776       5.96937        10     
reco:pandora2calo:Calorimetry                       6.4326        10.9378       17.5603       10.646        2.90544        10     
reco:pandora2calonosce:Calorimetry                  4.90411       10.5287       15.5884       10.5727       2.81894        10     
reco:pandora2pid:Chi2ParticleID                    0.009627       0.02381      0.0318479     0.0244346     0.0056986       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0528497     0.117837      0.147501      0.121825      0.0249815       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00934653     0.0229491     0.0294299     0.0236594    0.00518028       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.7727        7.39751       11.3425       7.64546       1.9408         10     
reco:pandora2Showercalo:ShowerCalorimetry           4.68356       9.1229        13.4965       9.41488       2.17405        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.16665       8.38948       12.5126       8.31511       1.67861        10     
[art]:TriggerResults:TriggerResultInserter        7.5472e-05    0.000101863   0.000217168   8.8506e-05    3.92633e-05      10     
end_path:out1:RootOutput                          1.4227e-05    2.05086e-05   4.5526e-05    1.57445e-05   1.03271e-05      10     
end_path:out1:RootOutput(write)                     2.79714       4.40252       5.81212       4.45648       0.73437        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5277.49 MB
  Peak resident set size usage (VmHWM): 4443.94 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.00 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11293820
-rw-r--r-- 1 dune01 dune01 2605486006 Oct 27 09:33 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2467731709 Oct 27 09:29 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2340711733 Oct 27 11:04 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2070920015 Oct 27 06:42 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1992623030 Oct 27 06:21 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   86770550 Oct 27 10:59 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242626 Oct 27 11:04 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Oct 27 11:04 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Oct 27 11:04 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Oct 27 11:04 time.db
-rw-r--r-- 1 dune01 dune01        519 Oct 27 09:33 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 27 06:42 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 27 11:04 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Oct 27 06:13 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Oct 27 11:04 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Oct 27 09:42 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Oct 27 09:33 debugprod.log
justIN time: 2025-11-03 18:41:01 UTC       justIN version: 01.05.01