justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 232087.1@dunegpschedd01.fnal.gov

Jobsub ID232087.1@dunegpschedd01.fnal.gov
Workflow ID8783
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-09 13:42:49
SiteUK_Lancaster
EntryUBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat2025-10-09 17:37:06
From worker nodeHostnamecomp14-03
cpuinfoIntel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-09 13:43:44
Input filesmonte-carlo-008783-000001
JobscriptExit code0
Real time3h (13904s)
CPU time3h (13726s = 98%)
Max RSS bytes4685783040 (4468 MiB)
Outputting started2025-10-09 17:35:29
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/08783/1/001/reco_stage1.root
Finished2025-10-09 17:37:06
Saved logsjustin-logs:232087.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

igitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 179045 traces, 0 summary
Retagger: tagging trace set: gauss with 133611 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 133611 traces tagged "gauss"
FrameSaver: q=4.64116e+06 n=2300609 tag=gauss
wclsFrameSaver: saving 179045 traces tagged "wiener"
FrameSaver: q=4.93001e+06 n=2016552 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 53340 XUs and 51947 XVs -> 768 XUVs
C:0 T:1 111776 XUs and 106755 XVs -> 7329 XUVs
C:0 T:2 93536 XUs and 150580 XVs -> 5091 XUVs
C:0 T:3 46614 XUs and 62082 XVs -> 372 XUVs
C:0 T:4 23167 XUs and 24093 XVs -> 345 XUVs
C:0 T:5 132355 XUs and 115150 XVs -> 10588 XUVs
C:0 T:6 82063 XUs and 117050 XVs -> 3917 XUVs
C:0 T:7 33130 XUs and 46928 XVs -> 328 XUVs
C:0 T:8 8880 XUs and 9119 XVs -> 117 XUVs
C:0 T:9 70370 XUs and 103293 XVs -> 9014 XUVs
C:0 T:10 21030 XUs and 24803 XVs -> 3580 XUVs
C:0 T:11 60 XUs and 33 XVs -> 7 XUVs
41456 XUVs total
17690 collection wire objects
37830 potential space points
Neighbour search...
1912152 tests to find 1020544 neighbours
Iterating with no regularization...
Begin: 8.83872e+07
0 7.44528e+07
1 7.33238e+07
2 7.31676e+07
3 7.31259e+07
Now with regularization...
Begin: 5.78802e+07
0 5.78208e+07
1 5.77946e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 409785891
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 409785891
fLifetime: 30000 [us]
09-Oct-2025 18:35:24 BST  Closed output file "reco_stage1.root"
09-Oct-2025 18:35:24 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          379.186       500.195        642.4        486.407       87.0071        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000785116   0.00419728    0.00648296    0.00467971    0.00180907       10     
reco:ophit:OpHitFinder                             0.321298       0.40377      0.534528      0.399425      0.0707181       10     
reco:opflash:OpFlashFinder                         0.179069      0.223592      0.288322      0.215916      0.0363431       10     
reco:opslicer:OpSlicer                             0.624788      0.753112      0.863411      0.772832      0.0778388       10     
reco:caldata:DataPrepModule                         6.37546       6.56386       6.94516       6.55838      0.173942        10     
reco:wclsdatasp:WireCellToolkit                     58.9451       61.8381       66.9877       61.4941       2.18142        10     
reco:gaushit:GausHitFinder                          1.34422       1.7165        2.07019       1.75113      0.214335        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000526862    0.0007254    0.000864634   0.000734547   9.74402e-05      10     
reco:reco3d:SpacePointSolver                        13.7302       20.4359       31.0149       18.3785       6.49306        10     
reco:hitpdune:DisambigFromSpacePoints               2.38731       3.50001       4.86384       3.42183      0.825636        10     
reco:pandora:StandardPandora                        33.2578       62.6112       94.9314       59.0587       21.9904        10     
reco:pandoraWriter:StandardPandora                 0.473969      0.604513       1.03966      0.555097      0.161029        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.21267       6.95447       10.2835       7.33588       1.86848        10     
reco:pandoraShower:LArPandoraShowerCreation         4.08883       6.5305        8.83899       6.75382       1.73517        10     
reco:pandoracalo:Calorimetry                        2.56117       3.53005       4.54578       3.70637       0.62107        10     
reco:pandoracalonosce:Calorimetry                   2.31972       3.21707       4.28471       3.27329       0.57902        10     
reco:pandorapid:Chi2ParticleID                    0.00724729    0.00876695     0.0100703    0.00885959    0.000876287      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0738365     0.0885084     0.108907      0.0849568     0.0121401       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00730051    0.00843101    0.00972493     0.0085113    0.000832069      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.88351       7.05268       9.7371        6.59929       1.69604        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.50021       6.99334       9.53937       6.97937       1.75908        10     
reco:emtrkmichelid:EmTrackMichelId                  190.065       233.194       284.622       228.293       31.8705        10     
reco:crttag:SingleCRTMatchingProducer              0.017536      0.0274756     0.0580437     0.0234513     0.0114695       10     
reco:crtreco:TwoCRTMatchingProducer                0.0100549     0.0136296     0.0195831     0.0135408    0.00263999       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.5121        2.18179       2.92159       2.11443      0.411769        10     
reco:pandora2Track:LArPandoraTrackCreation          13.8312       23.1572       35.6305       22.5829       6.98631        10     
reco:pandora2calo:Calorimetry                       6.78527       10.3514       14.898        10.6305       2.62353        10     
reco:pandora2calonosce:Calorimetry                  6.47438       9.94743       13.9465       9.85282       2.65221        10     
reco:pandora2pid:Chi2ParticleID                    0.0177036     0.0223929     0.0270294     0.0221378    0.00333082       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0892434      0.15122       0.23642      0.146923      0.0429232       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID           0.01722      0.0219047     0.0266245     0.0215677    0.00329966       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.83199        7.331        10.5248       7.35405       1.99115        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.14649       9.08371       12.2384       8.97358       2.17519        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.80928       8.30205       10.9851       7.99929       1.90219        10     
[art]:TriggerResults:TriggerResultInserter        4.8694e-05    6.02139e-05   0.000110785   5.57925e-05   1.71431e-05      10     
end_path:out1:RootOutput                           7.739e-06    1.08498e-05   2.3159e-05     9.188e-06    4.36281e-06      10     
end_path:out1:RootOutput(write)                     2.52731       3.25877       4.05679       3.23531      0.491739        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5500.44 MB
  Peak resident set size usage (VmHWM): 4685.78 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.30 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.11 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11270412
-rw-r--r-- 1 pltdune004 pltdune 2608388068 Oct  9 17:09 detsim_stage2.root
-rw-r--r-- 1 pltdune004 pltdune 2473637953 Oct  9 17:07 detsim_stage1.root
-rw-r--r-- 1 pltdune004 pltdune 2349226457 Oct  9 18:35 reco_stage1.root
-rw-r--r-- 1 pltdune004 pltdune 2043532103 Oct  9 15:06 g4_stage2.root
-rw-r--r-- 1 pltdune004 pltdune 1978224733 Oct  9 14:52 g4_stage1.root
-rw-r--r-- 1 pltdune004 pltdune   87327590 Oct  9 18:28 Pandora_Events.pndr
-rw-r--r-- 1 pltdune004 pltdune     242903 Oct  9 18:35 jobscript.log
-rw-r--r-- 1 pltdune004 pltdune     217088 Oct  9 18:35 mem.db
-rw-r--r-- 1 pltdune004 pltdune      62376 Oct  9 18:35 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune004 pltdune      36864 Oct  9 18:35 time.db
-rw-r--r-- 1 pltdune004 pltdune        519 Oct  9 17:09 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune004 pltdune        519 Oct  9 15:06 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune004 pltdune        519 Oct  9 18:35 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune004 pltdune         52 Oct  9 14:43 all-input-dids.txt
-rw-r--r-- 1 pltdune004 pltdune          7 Oct  9 18:35 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune004 pltdune          0 Oct  9 17:16 ProtoDUNE.log
-rw-r--r-- 1 pltdune004 pltdune          0 Oct  9 17:09 debugprod.log
justIN time: 2025-11-05 07:29:14 UTC       justIN version: 01.05.01