justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269486.1@dunegpschedd01.fnal.gov

Jobsub ID269486.1@dunegpschedd01.fnal.gov
Workflow ID11294
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-21 09:33:10
SiteUK_Bristol
EntryCMSHTPC_T2_UK_SGrid_Bristol_lcgce02
Last heartbeat2025-12-21 15:23:03
From worker nodeHostnamehd77.dice.priv
cpuinfoAMD EPYC 7551P 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit259200 (72 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-21 09:34:33
Input filesmonte-carlo-011294-000001
JobscriptExit code0
Real time5h (20816s)
CPU time5h (20324s = 97%)
Max RSS bytes4270665728 (4072 MiB)
Outputting started2025-12-21 15:21:30
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11294/1/001/reco_stage1.root
Finished2025-12-21 15:23:03
Saved logsjustin-logs:269486.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 161033 traces, 0 summary
Retagger: tagging trace set: gauss with 121705 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 121705 traces tagged "gauss"
FrameSaver: q=4.16486e+06 n=2165716 tag=gauss
wclsFrameSaver: saving 161033 traces tagged "wiener"
FrameSaver: q=4.41942e+06 n=1916008 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37393 XUs and 33998 XVs -> 555 XUVs
C:0 T:1 82952 XUs and 70636 XVs -> 5214 XUVs
C:0 T:2 122256 XUs and 143937 XVs -> 6750 XUVs
C:0 T:3 56951 XUs and 58370 XVs -> 610 XUVs
C:0 T:4 18772 XUs and 14600 XVs -> 128 XUVs
C:0 T:5 59806 XUs and 40677 XVs -> 4819 XUVs
C:0 T:6 212664 XUs and 195684 XVs -> 12140 XUVs
C:0 T:7 63644 XUs and 64792 XVs -> 918 XUVs
C:0 T:8 10900 XUs and 6867 XVs -> 79 XUVs
C:0 T:9 74789 XUs and 66649 XVs -> 4461 XUVs
C:0 T:10 223522 XUs and 39513 XVs -> 9241 XUVs
C:0 T:11 19 XUs and 17 XVs -> 2 XUVs
44917 XUVs total
15580 collection wire objects
40948 potential space points
Neighbour search...
2822500 tests to find 1506598 neighbours
Iterating with no regularization...
Begin: 8.3219e+07
0 6.25153e+07
1 6.13359e+07
2 6.11972e+07
3 6.11558e+07
Now with regularization...
Begin: 4.9944e+07
0 4.98989e+07
Failure in algorithm Alg0061, LArCosmicRayTrackRecovery, STATUS_CODE_FAILURE
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411250943
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411250943
fLifetime: 30000 [us]
21-Dec-2025 15:21:27 UTC  Closed output file "reco_stage1.root"
21-Dec-2025 15:21:27 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          396.588       579.697       763.355       582.36        112.932        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00105774    0.00387965    0.00634476    0.00389735    0.00162657       10     
reco:ophit:OpHitFinder                             0.332304      0.391757      0.455616      0.391938      0.0323179       10     
reco:opflash:OpFlashFinder                         0.194561      0.281365      0.368446      0.271575      0.0521988       10     
reco:opslicer:OpSlicer                             0.763411       1.17898       1.38871       1.19529      0.160383        10     
reco:caldata:DataPrepModule                          6.952        7.1958        7.48943       7.18434      0.185016        10     
reco:wclsdatasp:WireCellToolkit                     42.8986       46.0999       48.3822       46.3275       1.72147        10     
reco:gaushit:GausHitFinder                          1.5582        2.13562       2.5527        2.26713       0.31587        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000503757   0.000694149   0.000914599   0.000678365   0.000124074      10     
reco:reco3d:SpacePointSolver                        14.4492       20.8395       32.2021       19.4182       5.33855        10     
reco:hitpdune:DisambigFromSpacePoints               1.75213       2.85957       3.63023       3.00625      0.658496        10     
reco:pandora:StandardPandora                        31.9297       72.823        159.471       68.3762       36.1003        10     
reco:pandoraWriter:StandardPandora                  0.46894      0.645752      0.773119      0.657685      0.0975299       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.3944        7.07648       9.7686        7.04091       1.71756        10     
reco:pandoraShower:LArPandoraShowerCreation         4.36955       8.63481       12.5369       8.33823       2.63488        10     
reco:pandoracalo:Calorimetry                        2.1473        3.7938        5.16867       3.78606      0.954185        10     
reco:pandoracalonosce:Calorimetry                   1.92437       3.5119        4.9621        3.47277      0.939591        10     
reco:pandorapid:Chi2ParticleID                    0.00740432    0.00906044     0.011007     0.00922166     0.0011677       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0697536     0.0990099     0.157745      0.0981166     0.0242183       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00716104    0.00859015     0.0104036    0.00880173    0.00102607       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.77045       8.71708       12.6462       8.66496       2.79722        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.58185       8.34402       12.1409       8.14498       2.66929        10     
reco:emtrkmichelid:EmTrackMichelId                  226.982       304.817       357.29        310.547       43.2365        10     
reco:crttag:SingleCRTMatchingProducer              0.0178513     0.024025       0.04047      0.0233231    0.00611918       10     
reco:crtreco:TwoCRTMatchingProducer               0.00979323     0.0151952     0.0191707     0.0158981    0.00283124       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.05489       1.41065       1.70862       1.49893      0.225142        10     
reco:pandora2Track:LArPandoraTrackCreation          13.2587       24.6434       33.6444       24.034        6.49031        10     
reco:pandora2calo:Calorimetry                       5.61552       11.5859       16.049        11.4324       3.24487        10     
reco:pandora2calonosce:Calorimetry                  5.40642       10.9877       15.1006       11.019        3.01886        10     
reco:pandora2pid:Chi2ParticleID                    0.0188169     0.0240462     0.0284062     0.0235633    0.00323872       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0879326     0.119712      0.147089      0.124058      0.0188845       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0177213     0.0232626     0.0285838     0.0231571    0.00323236       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.15571       8.92414       11.7896       8.96914       2.17368        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.79256       9.63485       12.5509       9.71086       2.49712        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.27046       8.4723        11.1772       8.54334       2.19197        10     
[art]:TriggerResults:TriggerResultInserter        5.6987e-05    7.3357e-05    0.000149561   6.56085e-05   2.58743e-05      10     
end_path:out1:RootOutput                          1.1021e-05    1.33782e-05   1.9096e-05    1.30545e-05   2.22949e-06      10     
end_path:out1:RootOutput(write)                     3.15928       4.21477       4.99505       4.26314      0.627332        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5082.18 MB
  Peak resident set size usage (VmHWM): 4270.67 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.68 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10968040
-rw-r--r-- 1 dnpil001 users 2553735849 Dec 21 13:42 detsim_stage2.root
-rw-r--r-- 1 dnpil001 users 2420036649 Dec 21 13:38 detsim_stage1.root
-rw-r--r-- 1 dnpil001 users 2274642261 Dec 21 15:21 reco_stage1.root
-rw-r--r-- 1 dnpil001 users 1991627795 Dec 21 10:03 g4_stage2.root
-rw-r--r-- 1 dnpil001 users 1907487891 Dec 21 09:42 g4_stage1.root
-rw-r--r-- 1 dnpil001 users   83126580 Dec 21 15:12 Pandora_Events.pndr
-rw-r--r-- 1 dnpil001 users     243444 Dec 21 15:21 jobscript.log
-rw-r--r-- 1 dnpil001 users     217088 Dec 21 15:21 mem.db
-rw-r--r-- 1 dnpil001 users      62376 Dec 21 15:21 Pandora_Geometry.xml
-rw-r--r-- 1 dnpil001 users      36864 Dec 21 15:21 time.db
-rw-r--r-- 1 dnpil001 users        519 Dec 21 13:42 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Dec 21 10:03 g4_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Dec 21 15:21 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dnpil001 users         52 Dec 21 09:34 all-input-dids.txt
-rw-r--r-- 1 dnpil001 users          7 Dec 21 15:21 justin-processed-pfns.txt
-rw-r--r-- 1 dnpil001 users          0 Dec 21 13:52 ProtoDUNE.log
-rw-r--r-- 1 dnpil001 users          0 Dec 21 13:42 debugprod.log
justIN time: 2026-02-07 06:33:59 UTC       justIN version: 01.06.00