justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270850.1@dunegpschedd02.fnal.gov

Jobsub ID270850.1@dunegpschedd02.fnal.gov
Workflow ID11769
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-12 21:48:33
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce504
Last heartbeat2026-01-13 01:43:22
From worker nodeHostnameb9p07p0229.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-12 21:49:36
Input filesmonte-carlo-011769-000001
JobscriptExit code0
Real time3h (13933s)
CPU time3h (13053s = 93%)
Max RSS bytes4526596096 (4316 MiB)
Outputting started2026-01-13 01:41:50
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11769/1/001/reco_stage1.root
Finished2026-01-13 01:43:22
Saved logsjustin-logs:270850.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ng tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 158505 traces, 0 summary
Retagger: tagging trace set: gauss with 123310 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 123310 traces tagged "gauss"
FrameSaver: q=4.36727e+06 n=2248066 tag=gauss
wclsFrameSaver: saving 158505 traces tagged "wiener"
FrameSaver: q=4.62392e+06 n=2012293 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 56708 XUs and 51613 XVs -> 984 XUVs
C:0 T:1 118589 XUs and 105687 XVs -> 10487 XUVs
C:0 T:2 94471 XUs and 151140 XVs -> 4208 XUVs
C:0 T:3 49175 XUs and 68120 XVs -> 480 XUVs
C:0 T:4 12114 XUs and 11514 XVs -> 125 XUVs
C:0 T:5 52379 XUs and 36774 XVs -> 4955 XUVs
C:0 T:6 176058 XUs and 196178 XVs -> 12323 XUVs
C:0 T:7 54812 XUs and 62422 XVs -> 601 XUVs
C:0 T:8 9975 XUs and 11204 XVs -> 116 XUVs
C:0 T:9 75898 XUs and 103045 XVs -> 10290 XUVs
C:0 T:10 31792 XUs and 25389 XVs -> 3652 XUVs
C:0 T:11 42 XUs and 55 XVs -> 1 XUVs
48222 XUVs total
15464 collection wire objects
43923 potential space points
Neighbour search...
4706161 tests to find 2121742 neighbours
Iterating with no regularization...
Begin: 9.40255e+07
0 7.60321e+07
1 7.45197e+07
2 7.42687e+07
3 7.41991e+07
Now with regularization...
Begin: 5.99246e+07
0 5.9826e+07
1 5.97796e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411703764
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411703764
fLifetime: 30000 [us]
13-Jan-2026 02:41:48 CET  Closed output file "reco_stage1.root"
13-Jan-2026 02:41:48 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          303.226       375.961       469.75        359.928       57.4921        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000867753   0.00390339    0.00620688    0.00388813    0.00155185       10     
reco:ophit:OpHitFinder                              0.2237       0.318491      0.373518      0.326065      0.0378242       10     
reco:opflash:OpFlashFinder                         0.149901      0.222569      0.282095      0.217747      0.0355924       10     
reco:opslicer:OpSlicer                             0.456676      0.699563       1.02901      0.677938      0.153067        10     
reco:caldata:DataPrepModule                         4.60514       4.91306       5.25775       4.91362       0.20759        10     
reco:wclsdatasp:WireCellToolkit                     31.5217       33.8253       36.2308       33.8172       1.68762        10     
reco:gaushit:GausHitFinder                          1.1277        1.34272       1.6121        1.28365      0.191937        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000400866   0.000540583   0.000876243   0.000491983   0.000127046      10     
reco:reco3d:SpacePointSolver                        14.9907       19.4056       27.6452       19.4817       3.67303        10     
reco:hitpdune:DisambigFromSpacePoints               1.28116       1.97291       2.86499       1.83248      0.513757        10     
reco:pandora:StandardPandora                        22.6736       41.844        69.1932       36.9058       14.6059        10     
reco:pandoraWriter:StandardPandora                  0.31631      0.395681      0.500426      0.389283      0.0542038       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.26629       3.81215       4.92223       3.73557      0.496875        10     
reco:pandoraShower:LArPandoraShowerCreation         3.17346       4.93792       7.48598       4.11099       1.52241        10     
reco:pandoracalo:Calorimetry                        1.72818       2.25206       2.93702       2.19246      0.311656        10     
reco:pandoracalonosce:Calorimetry                   1.50385       2.1254        2.94209       2.02389       0.36724        10     
reco:pandorapid:Chi2ParticleID                    0.00417175    0.00561478    0.00907661    0.00489963    0.00152982       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0379403     0.0544835     0.0947771     0.050126      0.0161982       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0038499    0.00534966    0.00900683    0.00440675    0.00175183       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.45192       5.56754       8.04312       5.00909       1.60819        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.34143       5.4081        8.05223       4.86156       1.60083        10     
reco:emtrkmichelid:EmTrackMichelId                  164.304       196.101       232.831       192.871       23.5451        10     
reco:crttag:SingleCRTMatchingProducer              0.0100818     0.0153545     0.0219005     0.0147216    0.00390636       10     
reco:crtreco:TwoCRTMatchingProducer                0.0058303    0.00878893     0.0120962    0.00811196     0.0022657       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.712734      0.989213       1.28275      0.977443      0.162885        10     
reco:pandora2Track:LArPandoraTrackCreation          9.14345       14.2225       22.7381       12.6789       4.08874        10     
reco:pandora2calo:Calorimetry                       5.68565       7.62069       11.0179       7.29838       1.70531        10     
reco:pandora2calonosce:Calorimetry                  4.6152        7.18081       9.64942       7.18052       1.77008        10     
reco:pandora2pid:Chi2ParticleID                    0.0103349     0.0156455     0.023228      0.0152318    0.00415977       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0545345     0.0771422     0.113642      0.0708342     0.0199977       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.009059      0.0158592     0.023317      0.0159413    0.00532782       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.18555       5.11335       7.23731       4.85596       1.38772        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.07607       6.49307       8.86979       6.28658       1.58716        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.29962       5.88467       7.70251       5.59096       1.26776        10     
[art]:TriggerResults:TriggerResultInserter         4.515e-05    6.6375e-05    0.000150942   5.7496e-05    2.94738e-05      10     
end_path:out1:RootOutput                            8.6e-06     1.30332e-05   3.7231e-05    1.0445e-05    8.12336e-06      10     
end_path:out1:RootOutput(write)                     2.33027       3.00348       3.80039       2.84724      0.482552        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5382.91 MB
  Peak resident set size usage (VmHWM): 4526.6 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.06 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11942412
-rw-r--r--. 1 duneprd np-comp 2732898702 Jan 13 01:36 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2593782260 Jan 13 01:33 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2480829690 Jan 13 02:41 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 2206702886 Jan 12 23:15 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 2124510363 Jan 12 22:58 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   89691380 Jan 13 02:36 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242967 Jan 13 02:41 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 13 02:41 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 13 02:41 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 13 02:41 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 13 01:36 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 12 23:15 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 13 02:41 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 12 22:49 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 13 02:41 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 13 01:42 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 13 01:36 debugprod.log
justIN time: 2026-02-06 09:01:33 UTC       justIN version: 01.06.00