justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265176.0@dunegpschedd02.fnal.gov

Jobsub ID265176.0@dunegpschedd02.fnal.gov
Workflow ID11432
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-30 21:37:07
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce508
Last heartbeat2025-12-31 01:01:40
From worker nodeHostnameb9p12p9953.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-30 21:38:15
Input filesmonte-carlo-011432-000001
JobscriptExit code0
Real time3h (12120s)
CPU time3h (11362s = 93%)
Max RSS bytes4363382784 (4161 MiB)
Outputting started2025-12-31 01:00:16
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11432/1/001/reco_stage1.root
Finished2025-12-31 01:01:40
Saved logsjustin-logs:265176.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

els
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 164730 traces, 0 summary
Retagger: tagging trace set: gauss with 118346 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 118346 traces tagged "gauss"
FrameSaver: q=4.27323e+06 n=2043676 tag=gauss
wclsFrameSaver: saving 164730 traces tagged "wiener"
FrameSaver: q=4.56367e+06 n=1769775 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 46505 XUs and 37723 XVs -> 897 XUVs
C:0 T:1 157397 XUs and 118351 XVs -> 13854 XUVs
C:0 T:2 97850 XUs and 114729 XVs -> 9446 XUVs
C:0 T:4 14745 XUs and 10346 XVs -> 100 XUVs
C:0 T:5 80895 XUs and 43069 XVs -> 7520 XUVs
C:0 T:6 263431 XUs and 212067 XVs -> 32068 XUVs
C:0 T:7 49626 XUs and 55945 XVs -> 1242 XUVs
C:0 T:8 10685 XUs and 8661 XVs -> 136 XUVs
C:0 T:9 89559 XUs and 84839 XVs -> 11625 XUVs
C:0 T:10 57585 XUs and 31219 XVs -> 6258 XUVs
C:0 T:11 346 XUs and 221 XVs -> 28 XUVs
83174 XUVs total
15918 collection wire objects
75292 potential space points
Neighbour search...
13997252 tests to find 6291140 neighbours
Iterating with no regularization...
Begin: 8.59071e+07
0 6.26286e+07
1 6.08119e+07
2 6.05337e+07
3 6.04444e+07
4 6.04087e+07
Now with regularization...
Begin: 4.77289e+07
0 4.75805e+07
1 4.74893e+07
2 4.74213e+07
3 4.7367e+07
4 4.73227e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411441858
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411441858
fLifetime: 30000 [us]
31-Dec-2025 02:00:15 CET  Closed output file "reco_stage1.root"
31-Dec-2025 02:00:15 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          237.871       319.211       464.551       306.921       65.9607        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00102144    0.00339213    0.00506134    0.00360575    0.00140539       10     
reco:ophit:OpHitFinder                             0.225919      0.272359      0.336059      0.267234      0.0372685       10     
reco:opflash:OpFlashFinder                         0.135487      0.186073      0.270907      0.168084      0.0395776       10     
reco:opslicer:OpSlicer                             0.414272      0.593763      0.842856      0.578419      0.110522        10     
reco:caldata:DataPrepModule                         3.98971       4.51041       5.63988        4.452       0.480584        10     
reco:wclsdatasp:WireCellToolkit                     27.071        29.2306       33.9093       28.6399       1.8932         10     
reco:gaushit:GausHitFinder                         0.736521       1.02666       1.34892      0.990436      0.159068        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000337072   0.000463223   0.000579293   0.000443593   6.64122e-05      10     
reco:reco3d:SpacePointSolver                        11.3659       19.6715       41.5218       15.2974       9.8066         10     
reco:hitpdune:DisambigFromSpacePoints               1.20209       1.60238       2.19238       1.5106       0.320394        10     
reco:pandora:StandardPandora                        16.199        31.6688       59.3118       29.5091       12.3294        10     
reco:pandoraWriter:StandardPandora                  0.26827       0.37886      0.513124      0.370282      0.0653675       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.2745        3.49397       5.3844        3.16239      0.929007        10     
reco:pandoraShower:LArPandoraShowerCreation         2.17696       3.79039       6.23351       3.56201       1.17011        10     
reco:pandoracalo:Calorimetry                        1.45262       2.00829       2.61203       1.9611       0.407119        10     
reco:pandoracalonosce:Calorimetry                   1.30937       1.88916       2.70866       1.69894      0.435452        10     
reco:pandorapid:Chi2ParticleID                    0.00371166    0.00575788    0.00920188    0.00519026    0.00198075       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0343254     0.0543679     0.0802822     0.0492954     0.0165526       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00353691    0.00499906    0.00801652    0.00402752     0.0015575       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.07625       4.32302       6.48864       4.66065       1.22899        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.06675       4.23023       6.70993       3.91176       1.41467        10     
reco:emtrkmichelid:EmTrackMichelId                  131.637       169.187       232.438       164.742       29.4244        10     
reco:crttag:SingleCRTMatchingProducer             0.00777191     0.0110667     0.0178335     0.0103761    0.00286608       10     
reco:crtreco:TwoCRTMatchingProducer               0.00489217    0.00702737    0.00958726    0.00671252    0.00151714       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.62656      0.766843      0.990037      0.741393      0.0987707       10     
reco:pandora2Track:LArPandoraTrackCreation          5.97632       11.4704       22.259        9.78901       4.59569        10     
reco:pandora2calo:Calorimetry                       3.35485       6.05005       9.55517       5.85564       1.80025        10     
reco:pandora2calonosce:Calorimetry                  3.45549       5.64782       10.4463       5.51143       1.81825        10     
reco:pandora2pid:Chi2ParticleID                   0.00816847     0.0126849     0.023586      0.0103009     0.0047641       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0414901     0.0659997     0.116241      0.0529288     0.025075        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00807213     0.0126295     0.0230399     0.010407     0.00459482       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.61317       4.36366       8.73335       3.8104        1.66258        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.18344       5.20677       7.62932       5.25971       1.29707        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.05563       4.75611       6.78261       4.94159       1.00201        10     
[art]:TriggerResults:TriggerResultInserter         3.81e-05     5.60894e-05   0.000141721   4.6915e-05    2.97023e-05      10     
end_path:out1:RootOutput                           1.131e-05    1.29891e-05    1.544e-05     1.254e-05    1.55618e-06      10     
end_path:out1:RootOutput(write)                     2.01432       2.60979       3.3063        2.44651      0.419764        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5192.49 MB
  Peak resident set size usage (VmHWM): 4363.38 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.86 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10749064
-rw-r--r--. 1 duneprd np-comp 2499134683 Dec 31 01:05 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2369592301 Dec 31 01:03 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2220059553 Dec 31 02:00 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1960021414 Dec 30 23:02 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1874584901 Dec 30 22:46 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   83034400 Dec 31 01:56 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243400 Dec 31 02:00 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Dec 31 02:00 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Dec 31 02:00 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Dec 31 02:00 time.db
-rw-r--r--. 1 duneprd np-comp        519 Dec 31 01:05 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Dec 30 23:02 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Dec 31 02:00 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Dec 30 22:38 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Dec 31 02:00 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Dec 31 01:10 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Dec 31 01:05 debugprod.log
justIN time: 2026-02-07 00:49:52 UTC       justIN version: 01.06.00