justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271549.1@dunegpschedd01.fnal.gov

Jobsub ID271549.1@dunegpschedd01.fnal.gov
Workflow ID11510
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-04 09:48:54
SiteUK_Bristol
EntryCMSHTPC_T2_UK_SGrid_Bristol_lcgce02
Last heartbeat2026-01-04 16:16:50
From worker nodeHostnamehd85.dice.priv
cpuinfoAMD EPYC 7551P 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit259200 (72 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-04 10:17:55
Input filesmonte-carlo-011510-000001
JobscriptExit code0
Real time5h (18884s)
CPU time5h (18362s = 97%)
Max RSS bytes4081643520 (3892 MiB)
Outputting started2026-01-04 15:32:40
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11510/1/001/reco_stage1.root
Finished2026-01-04 16:16:50
Saved logsjustin-logs:271549.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Service:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 140288 traces, 0 summary
Retagger: tagging trace set: gauss with 104307 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 104307 traces tagged "gauss"
FrameSaver: q=3.72965e+06 n=1832968 tag=gauss
wclsFrameSaver: saving 140288 traces tagged "wiener"
FrameSaver: q=3.96197e+06 n=1616412 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 41836 XUs and 25591 XVs -> 486 XUVs
C:0 T:1 63369 XUs and 45237 XVs -> 5451 XUVs
C:0 T:2 53805 XUs and 60365 XVs -> 2092 XUVs
C:0 T:3 30062 XUs and 27815 XVs -> 167 XUVs
C:0 T:4 15120 XUs and 14721 XVs -> 132 XUVs
C:0 T:5 50890 XUs and 41552 XVs -> 4124 XUVs
C:0 T:6 109027 XUs and 152878 XVs -> 7921 XUVs
C:0 T:7 42598 XUs and 56979 XVs -> 625 XUVs
C:0 T:8 4942 XUs and 6986 XVs -> 68 XUVs
C:0 T:9 38036 XUs and 66417 XVs -> 4927 XUVs
C:0 T:10 68973 XUs and 65820 XVs -> 9875 XUVs
C:0 T:11 179 XUs and 136 XVs -> 16 XUVs
35884 XUVs total
12427 collection wire objects
33475 potential space points
Neighbour search...
2195143 tests to find 1160970 neighbours
Iterating with no regularization...
Begin: 7.65871e+07
0 6.28632e+07
1 6.18263e+07
2 6.16872e+07
3 6.16511e+07
Now with regularization...
Begin: 4.99761e+07
0 4.99475e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
run: 1 ; subrun: 0 ; event: 10
evttime: 411532774
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411532774
fLifetime: 30000 [us]
04-Jan-2026 15:32:38 UTC  Closed output file "reco_stage1.root"
04-Jan-2026 15:32:38 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          361.016       514.779       646.631       517.064       76.5362        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00150308    0.00334898    0.00630732    0.00330349    0.00125435       10     
reco:ophit:OpHitFinder                             0.303025      0.341527      0.405663      0.338789      0.0262798       10     
reco:opflash:OpFlashFinder                         0.168967      0.202432      0.262728      0.196975      0.0260703       10     
reco:opslicer:OpSlicer                             0.887406       1.04704       1.26348       1.01533      0.108566        10     
reco:caldata:DataPrepModule                         6.96818       7.14605       7.6638        7.07179      0.188468        10     
reco:wclsdatasp:WireCellToolkit                     41.2815       46.0746       48.9006       46.793        2.48934        10     
reco:gaushit:GausHitFinder                          1.49342       1.87724       2.15645       1.90227      0.194606        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000505151   0.000653002   0.000981267   0.000613094   0.000129268      10     
reco:reco3d:SpacePointSolver                        10.3938       18.2784       24.9777       17.7224       4.20567        10     
reco:hitpdune:DisambigFromSpacePoints               1.38789       2.33784       3.22467       2.31028      0.500273        10     
reco:pandora:StandardPandora                        26.3148       49.737        73.9574       46.834        13.844         10     
reco:pandoraWriter:StandardPandora                 0.435196      0.585629      0.713566       0.60147      0.0714876       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.03411       6.26037       8.70473       6.13738       1.47319        10     
reco:pandoraShower:LArPandoraShowerCreation         3.71819       6.95386       9.33759       7.41658       1.6451         10     
reco:pandoracalo:Calorimetry                        1.73841       3.53678       4.68026       3.67666      0.815698        10     
reco:pandoracalonosce:Calorimetry                   1.51747       3.24157       4.41431       3.29839      0.789139        10     
reco:pandorapid:Chi2ParticleID                    0.00611382    0.00822766     0.0102522    0.00813171    0.00118863       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0650723     0.087099      0.110979      0.0881159     0.0119007       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00588299    0.00774359    0.00963706    0.00766107    0.000965793      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.69148       7.18628       9.12584       7.84905       1.70847        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.5576        7.20971       9.39958       7.9896        1.74776        10     
reco:emtrkmichelid:EmTrackMichelId                  216.85        282.637        350.1        284.113       35.8418        10     
reco:crttag:SingleCRTMatchingProducer              0.0173521     0.0235192     0.0294857     0.0229513    0.00420044       10     
reco:crtreco:TwoCRTMatchingProducer               0.00964041     0.0151767     0.0207466     0.0158107    0.00274332       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.929103       1.3593        1.81505       1.36775      0.223779        10     
reco:pandora2Track:LArPandoraTrackCreation          10.9864       20.4238       25.8761       20.7804       4.52298        10     
reco:pandora2calo:Calorimetry                       4.89348       10.2263       13.3384       10.9014       2.58746        10     
reco:pandora2calonosce:Calorimetry                  4.59817       9.63419       12.6235       10.1605       2.33887        10     
reco:pandora2pid:Chi2ParticleID                    0.0154614     0.0204855     0.0285007     0.0200994    0.00350343       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0812159     0.109968      0.136827      0.107935      0.0167699       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.015303      0.0203077     0.0282515     0.0198794    0.00350915       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.22757       7.76355       10.1824       7.93042       1.66545        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.27832       8.63206       11.2257       8.9605        1.98182        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.90189       7.7253        10.3531       8.17469       1.70634        10     
[art]:TriggerResults:TriggerResultInserter        5.2288e-05    8.36275e-05   0.000199536   7.24715e-05   4.02297e-05      10     
end_path:out1:RootOutput                          1.1181e-05    1.30016e-05   2.0549e-05    1.2248e-05    2.57802e-06      10     
end_path:out1:RootOutput(write)                     2.91041       3.93313       5.02624       3.96785      0.552847        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4895.56 MB
  Peak resident set size usage (VmHWM): 4081.64 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.67 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9639836
-rw-r--r-- 1 dnpil001 users 2282800692 Jan  4 14:04 detsim_stage2.root
-rw-r--r-- 1 dnpil001 users 2164923434 Jan  4 14:01 detsim_stage1.root
-rw-r--r-- 1 dnpil001 users 1974464133 Jan  4 15:32 reco_stage1.root
-rw-r--r-- 1 dnpil001 users 1726014913 Jan  4 10:44 g4_stage2.root
-rw-r--r-- 1 dnpil001 users 1646490923 Jan  4 10:26 g4_stage1.root
-rw-r--r-- 1 dnpil001 users   75885940 Jan  4 15:25 Pandora_Events.pndr
-rw-r--r-- 1 dnpil001 users     243070 Jan  4 15:32 jobscript.log
-rw-r--r-- 1 dnpil001 users     217088 Jan  4 15:32 mem.db
-rw-r--r-- 1 dnpil001 users      62376 Jan  4 15:32 Pandora_Geometry.xml
-rw-r--r-- 1 dnpil001 users      36864 Jan  4 15:32 time.db
-rw-r--r-- 1 dnpil001 users        519 Jan  4 14:04 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Jan  4 10:44 g4_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Jan  4 15:32 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dnpil001 users         52 Jan  4 10:17 all-input-dids.txt
-rw-r--r-- 1 dnpil001 users          7 Jan  4 15:32 justin-processed-pfns.txt
-rw-r--r-- 1 dnpil001 users          0 Jan  4 14:12 ProtoDUNE.log
-rw-r--r-- 1 dnpil001 users          0 Jan  4 14:04 debugprod.log
justIN time: 2026-02-06 21:03:35 UTC       justIN version: 01.06.00