justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265271.1@dunegpschedd02.fnal.gov

Jobsub ID265271.1@dunegpschedd02.fnal.gov
Workflow ID11443
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-31 16:14:03
SiteUK_Bristol
EntryCMSHTPC_T2_UK_SGrid_Bristol_lcgce02
Last heartbeat2025-12-31 21:32:17
From worker nodeHostnamehd68.dice.priv
cpuinfoAMD EPYC 7551P 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit259200 (72 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-31 16:32:07
Input filesmonte-carlo-011443-000001
JobscriptExit code0
Real time4h (17926s)
CPU time4h (17474s = 97%)
Max RSS bytes4095729664 (3905 MiB)
Outputting started2025-12-31 21:30:55
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11443/1/001/reco_stage1.root
Finished2025-12-31 21:32:17
Saved logsjustin-logs:265271.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

PrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 117528 traces, 0 summary
Retagger: tagging trace set: gauss with 88200 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 88200 traces tagged "gauss"
FrameSaver: q=2.96566e+06 n=1556771 tag=gauss
wclsFrameSaver: saving 117528 traces tagged "wiener"
FrameSaver: q=3.13919e+06 n=1370698 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37441 XUs and 32198 XVs -> 482 XUVs
C:0 T:1 122667 XUs and 93652 XVs -> 21299 XUVs
C:0 T:2 48240 XUs and 81295 XVs -> 5471 XUVs
C:0 T:3 22949 XUs and 33482 XVs -> 114 XUVs
C:0 T:4 11288 XUs and 10235 XVs -> 110 XUVs
C:0 T:5 63343 XUs and 46539 XVs -> 10417 XUVs
C:0 T:6 52567 XUs and 81574 XVs -> 2388 XUVs
C:0 T:7 23354 XUs and 33810 XVs -> 115 XUVs
C:0 T:8 8796 XUs and 9306 XVs -> 208 XUVs
C:0 T:9 78443 XUs and 88235 XVs -> 11722 XUVs
C:0 T:10 26586 XUs and 26141 XVs -> 6976 XUVs
C:0 T:11 97 XUs and 68 XVs -> 11 XUVs
59313 XUVs total
10215 collection wire objects
54533 potential space points
Neighbour search...
14198875 tests to find 5853080 neighbours
Iterating with no regularization...
Begin: 5.39954e+07
0 4.515e+07
1 4.45592e+07
2 4.44914e+07
3 4.44744e+07
Now with regularization...
Begin: 3.72467e+07
0 3.72041e+07
1 3.7161e+07
2 3.71253e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411457702
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411457702
fLifetime: 30000 [us]
31-Dec-2025 21:30:52 UTC  Closed output file "reco_stage1.root"
31-Dec-2025 21:30:52 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          331.351       479.93        695.313       468.626       95.6855        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00126008    0.00362386    0.00587106    0.00378564    0.00141406       10     
reco:ophit:OpHitFinder                             0.303862      0.348922      0.429567      0.339697      0.0369315       10     
reco:opflash:OpFlashFinder                         0.161593      0.220326      0.284345      0.214941      0.0421876       10     
reco:opslicer:OpSlicer                             0.775589       1.02681       1.36696       1.0037       0.170966        10     
reco:caldata:DataPrepModule                         6.88325       7.06673       7.44145       7.0255       0.168526        10     
reco:wclsdatasp:WireCellToolkit                     42.7337       45.8235       49.6434       45.5737       1.86801        10     
reco:gaushit:GausHitFinder                          1.20875       1.69503       2.36849       1.68598      0.320692        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000443155   0.00060985    0.000966201   0.000542017   0.000151452      10     
reco:reco3d:SpacePointSolver                        12.651        19.5481       41.558        16.7516       8.6396         10     
reco:hitpdune:DisambigFromSpacePoints               1.3021        2.18947        3.745        2.11663      0.652896        10     
reco:pandora:StandardPandora                        19.9702       41.7354       75.9519       39.7065       14.4325        10     
reco:pandoraWriter:StandardPandora                 0.399886      0.551939      0.750248      0.538017      0.0904786       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.29633       5.70909       8.81451       5.51302       1.52167        10     
reco:pandoraShower:LArPandoraShowerCreation         2.7765        6.52272       12.5286       6.23822       2.55121        10     
reco:pandoracalo:Calorimetry                        1.72144       3.05647       4.78489       2.99739      0.851085        10     
reco:pandoracalonosce:Calorimetry                   1.53755       2.81388       4.40645       2.72939      0.824161        10     
reco:pandorapid:Chi2ParticleID                    0.00681518    0.00834826    0.00983387    0.00846436    0.00101202       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0590265     0.0817122      0.11379      0.0806382     0.0144509       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00656866    0.00782053    0.00918774    0.00775505    0.000826389      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.56423       6.40382       12.4591       6.2495         2.639         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.49419       6.18641       12.1419       6.00943       2.5893         10     
reco:emtrkmichelid:EmTrackMichelId                  195.713       265.687       361.708       261.019       43.1958        10     
reco:crttag:SingleCRTMatchingProducer              0.0132831     0.0188095     0.0254167     0.0176885    0.00464445       10     
reco:crtreco:TwoCRTMatchingProducer               0.00868072     0.012933      0.0186799     0.0124233    0.00314199       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.01536       1.27164       1.66521       1.1942       0.207805        10     
reco:pandora2Track:LArPandoraTrackCreation          9.43376       18.9406       34.3719       18.1002       6.56254        10     
reco:pandora2calo:Calorimetry                       4.0191        8.9725        16.4927       8.71567       3.29173        10     
reco:pandora2calonosce:Calorimetry                  3.69214       8.43791       15.3726       8.07239       3.19772        10     
reco:pandora2pid:Chi2ParticleID                    0.0148673     0.0204492     0.0265725     0.0207889    0.00312095       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.074292      0.101787      0.140436      0.100169      0.0170354       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0148142     0.0202141     0.0262332     0.0204167     0.0029891       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.39917       7.03001       12.6023       6.77775       2.39973        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.43891       7.67908       13.9147       7.40106       2.78361        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.05592       6.87482       12.5171       6.67486       2.49007        10     
[art]:TriggerResults:TriggerResultInserter        5.6466e-05    7.80791e-05   0.000178396   6.8069e-05    3.37517e-05      10     
end_path:out1:RootOutput                          1.2243e-05    1.43019e-05   2.2232e-05    1.34305e-05   2.80602e-06      10     
end_path:out1:RootOutput(write)                     2.70406       3.74504       5.23033       3.63207      0.673061        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4910.62 MB
  Peak resident set size usage (VmHWM): 4095.73 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.26 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.63 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9735780
-rw-r--r-- 1 dnpil001 users 2306247111 Dec 31 20:08 detsim_stage2.root
-rw-r--r-- 1 dnpil001 users 2186600649 Dec 31 20:05 detsim_stage1.root
-rw-r--r-- 1 dnpil001 users 1982994498 Dec 31 21:30 reco_stage1.root
-rw-r--r-- 1 dnpil001 users 1750157215 Dec 31 16:57 g4_stage2.root
-rw-r--r-- 1 dnpil001 users 1670232910 Dec 31 16:39 g4_stage1.root
-rw-r--r-- 1 dnpil001 users   72526100 Dec 31 21:25 Pandora_Events.pndr
-rw-r--r-- 1 dnpil001 users     243115 Dec 31 21:30 jobscript.log
-rw-r--r-- 1 dnpil001 users     217088 Dec 31 21:30 mem.db
-rw-r--r-- 1 dnpil001 users      62376 Dec 31 21:30 Pandora_Geometry.xml
-rw-r--r-- 1 dnpil001 users      36864 Dec 31 21:30 time.db
-rw-r--r-- 1 dnpil001 users        519 Dec 31 20:08 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Dec 31 16:57 g4_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Dec 31 21:30 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dnpil001 users         52 Dec 31 16:32 all-input-dids.txt
-rw-r--r-- 1 dnpil001 users          7 Dec 31 21:30 justin-processed-pfns.txt
-rw-r--r-- 1 dnpil001 users          0 Dec 31 20:16 ProtoDUNE.log
-rw-r--r-- 1 dnpil001 users          0 Dec 31 20:08 debugprod.log
justIN time: 2026-02-06 23:31:45 UTC       justIN version: 01.06.00