justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 276086.0@dunegpschedd02.fnal.gov

Jobsub ID276086.0@dunegpschedd02.fnal.gov
Workflow ID12092
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-19 08:26:46
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce514
Last heartbeat2026-01-19 11:56:44
From worker nodeHostnameb9p05p8888.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-19 08:28:01
Input filesmonte-carlo-012092-000001
JobscriptExit code0
Real time3h (12430s)
CPU time3h (11725s = 94%)
Max RSS bytes4407877632 (4203 MiB)
Outputting started2026-01-19 11:55:12
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12092/1/001/reco_stage1.root
Finished2026-01-19 11:56:44
Saved logsjustin-logs:276086.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 120641 traces, 0 summary
Retagger: tagging trace set: gauss with 90372 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 90372 traces tagged "gauss"
FrameSaver: q=3.03137e+06 n=1571121 tag=gauss
wclsFrameSaver: saving 120641 traces tagged "wiener"
FrameSaver: q=3.22016e+06 n=1377996 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 31674 XUs and 25860 XVs -> 356 XUVs
C:0 T:1 78851 XUs and 56076 XVs -> 5582 XUVs
C:0 T:2 73449 XUs and 119056 XVs -> 5231 XUVs
C:0 T:3 32913 XUs and 44665 XVs -> 325 XUVs
C:0 T:4 6512 XUs and 5453 XVs -> 39 XUVs
C:0 T:5 16360 XUs and 10809 XVs -> 1333 XUVs
C:0 T:6 123971 XUs and 150108 XVs -> 9690 XUVs
C:0 T:7 44654 XUs and 51871 XVs -> 528 XUVs
C:0 T:8 5022 XUs and 6234 XVs -> 71 XUVs
C:0 T:9 25901 XUs and 43000 XVs -> 3270 XUVs
C:0 T:10 38592 XUs and 48000 XVs -> 5607 XUVs
C:0 T:11 265 XUs and 170 XVs -> 25 XUVs
32057 XUVs total
11019 collection wire objects
29117 potential space points
Neighbour search...
2071511 tests to find 1125908 neighbours
Iterating with no regularization...
Begin: 5.05007e+07
0 4.03654e+07
1 3.96845e+07
2 3.9572e+07
3 3.95334e+07
Now with regularization...
Begin: 3.12554e+07
0 3.12061e+07
1 3.11814e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411833338
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411833338
fLifetime: 30000 [us]
19-Jan-2026 12:55:10 CET  Closed output file "reco_stage1.root"
19-Jan-2026 12:55:10 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          255.963       331.403       450.309       305.569       66.9522        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00140047    0.00368628    0.00582942    0.00381543    0.00144421       10     
reco:ophit:OpHitFinder                             0.246683      0.300882       0.41754      0.280684      0.050459        10     
reco:opflash:OpFlashFinder                         0.142706      0.194374      0.259338      0.193828      0.0318827       10     
reco:opslicer:OpSlicer                             0.527979      0.652475      0.827467      0.631182       0.10646        10     
reco:caldata:DataPrepModule                         4.21212       4.96641       6.47999       4.71154      0.756282        10     
reco:wclsdatasp:WireCellToolkit                     29.9581       32.0128       34.3299       31.8958       1.38339        10     
reco:gaushit:GausHitFinder                         0.734458       1.1375        1.68254       1.11629      0.290071        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000387028   0.000547238   0.00089937    0.000541866   0.000144776      10     
reco:reco3d:SpacePointSolver                        8.10433       15.0366       26.439        13.5937       5.60645        10     
reco:hitpdune:DisambigFromSpacePoints              0.961884       1.61112       2.96394       1.45893      0.593667        10     
reco:pandora:StandardPandora                        15.5313       32.0977       74.4622       25.8163       16.7113        10     
reco:pandoraWriter:StandardPandora                  0.27103      0.407496      0.655292      0.383034      0.116973        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.9572        3.25655       6.13228       3.20363       1.16439        10     
reco:pandoraShower:LArPandoraShowerCreation         1.93138       3.51044       7.55137       2.83338       1.57698        10     
reco:pandoracalo:Calorimetry                        1.12611       2.01882        2.879        2.07631      0.481776        10     
reco:pandoracalonosce:Calorimetry                   1.10157       1.89372       2.56325       1.91391      0.456321        10     
reco:pandorapid:Chi2ParticleID                    0.00336731    0.00483508    0.00728082    0.00464445    0.00116097       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0344548     0.047163      0.0744845     0.0432202     0.0124212       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0033469    0.00463611    0.00703282    0.00432379    0.00106519       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.04095       3.96685       6.85932       3.88312       1.31127        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.05523       4.04028       7.31183       3.61269       1.55904        10     
reco:emtrkmichelid:EmTrackMichelId                  148.259       183.453       234.954       171.532       29.0272        10     
reco:crttag:SingleCRTMatchingProducer             0.00956641     0.0154924     0.024417      0.0140441    0.00418111       10     
reco:crtreco:TwoCRTMatchingProducer               0.00633094    0.00921488     0.0148744    0.00850384    0.00246675       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.758923      0.923175       1.22245      0.886386       0.15942        10     
reco:pandora2Track:LArPandoraTrackCreation          7.98272       11.4382       17.8248       10.777        3.05638        10     
reco:pandora2calo:Calorimetry                       4.17634       5.85374       9.35201       5.19593       1.57112        10     
reco:pandora2calonosce:Calorimetry                  4.03974       5.92988       10.0476       5.27193       1.85881        10     
reco:pandora2pid:Chi2ParticleID                   0.00734917     0.0116212     0.0182668     0.0116672    0.00337031       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0411302     0.0617844      0.10078      0.0569213     0.0195276       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00730628     0.0111764     0.0186933    0.00970497    0.00361813       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.10298       4.14607       8.2325        3.79379       1.70228        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.64557       4.97433       9.61731       4.8314        1.87015        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.55087       4.54738       8.21344       3.97276       1.66467        10     
[art]:TriggerResults:TriggerResultInserter         3.276e-05    6.15513e-05   0.000179544   5.0211e-05    4.04206e-05      10     
end_path:out1:RootOutput                           1.058e-05    1.33953e-05   2.0031e-05    1.30455e-05   2.66495e-06      10     
end_path:out1:RootOutput(write)                     2.26246       2.75429       3.54414       2.62884       0.40884        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5248.48 MB
  Peak resident set size usage (VmHWM): 4407.88 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.10 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10876208
-rw-r--r--. 1 duneprd np-comp 2531827325 Jan 19 11:56 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2403693443 Jan 19 11:54 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2233015474 Jan 19 12:55 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1976212126 Jan 19 09:48 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1913663132 Jan 19 09:37 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   78216400 Jan 19 12:51 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242707 Jan 19 12:55 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 19 12:55 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 19 12:55 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 19 12:55 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 19 11:56 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 19 09:48 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 19 12:55 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 19 09:28 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 19 12:55 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 19 12:03 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 19 11:56 debugprod.log
justIN time: 2026-02-05 19:25:52 UTC       justIN version: 01.06.00