justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 288559.0@dunegpschedd02.fnal.gov

Jobsub ID288559.0@dunegpschedd02.fnal.gov
Workflow ID12112
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-20 14:48:55
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce503
Last heartbeat2026-01-20 17:53:02
From worker nodeHostnameb9p09p5455.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-20 14:51:32
Input filesmonte-carlo-012112-000001
JobscriptExit code0
Real time2h (10788s)
CPU time2h (9911s = 91%)
Max RSS bytes4256161792 (4058 MiB)
Outputting started2026-01-20 17:51:21
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12112/1/001/reco_stage1.root
Finished2026-01-20 17:53:02
Saved logsjustin-logs:288559.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ning tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 162820 traces, 0 summary
Retagger: tagging trace set: gauss with 124238 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 124238 traces tagged "gauss"
FrameSaver: q=4.31276e+06 n=2187864 tag=gauss
wclsFrameSaver: saving 162820 traces tagged "wiener"
FrameSaver: q=4.54967e+06 n=1930549 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 18771 XUs and 19503 XVs -> 240 XUVs
C:0 T:1 23179 XUs and 22167 XVs -> 2100 XUVs
C:0 T:2 166720 XUs and 195926 XVs -> 14025 XUVs
C:0 T:3 69025 XUs and 69079 XVs -> 812 XUVs
C:0 T:4 13234 XUs and 15010 XVs -> 103 XUVs
C:0 T:5 47791 XUs and 40460 XVs -> 4020 XUVs
C:0 T:6 140940 XUs and 128473 XVs -> 8052 XUVs
C:0 T:7 54385 XUs and 41769 XVs -> 491 XUVs
C:0 T:8 12803 XUs and 11390 XVs -> 124 XUVs
C:0 T:9 86170 XUs and 112015 XVs -> 7726 XUVs
C:0 T:10 58129 XUs and 33142 XVs -> 3957 XUVs
C:0 T:11 250 XUs and 229 XVs -> 22 XUVs
41672 XUVs total
14600 collection wire objects
37604 potential space points
Neighbour search...
3029916 tests to find 1530274 neighbours
Iterating with no regularization...
Begin: 7.91633e+07
0 6.70586e+07
1 6.62339e+07
2 6.61084e+07
3 6.60718e+07
Now with regularization...
Begin: 5.36643e+07
0 5.3637e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411853228
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411853228
fLifetime: 30000 [us]
20-Jan-2026 18:51:19 CET  Closed output file "reco_stage1.root"
20-Jan-2026 18:51:19 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          200.02        258.04        390.667       242.589       52.175         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0010719    0.00377874    0.00550951    0.00433401    0.00162207       10     
reco:ophit:OpHitFinder                              0.22297      0.254986       0.28629      0.252562      0.0201495       10     
reco:opflash:OpFlashFinder                         0.135985      0.184725      0.239729      0.190019      0.0303507       10     
reco:opslicer:OpSlicer                             0.414168      0.531734      0.653745      0.521603      0.0739547       10     
reco:caldata:DataPrepModule                         3.70039       3.89475       4.20057       3.85238      0.136175        10     
reco:wclsdatasp:WireCellToolkit                     24.3658       25.8411       27.4326       25.9993      0.890025        10     
reco:gaushit:GausHitFinder                         0.708466      0.878988       1.16255      0.857763      0.127711        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000305267   0.000439819   0.000663964   0.000435109   9.95122e-05      10     
reco:reco3d:SpacePointSolver                        7.89958       13.165        25.5697       12.834        4.66447        10     
reco:hitpdune:DisambigFromSpacePoints              0.920615       1.33911       2.06415       1.24756      0.352901        10     
reco:pandora:StandardPandora                        17.3016       28.7865       78.7938       22.8659       17.1831        10     
reco:pandoraWriter:StandardPandora                 0.250254      0.309421      0.414468      0.292222       0.04662        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.91489       2.84311       4.01603       2.70868      0.638707        10     
reco:pandoraShower:LArPandoraShowerCreation         1.82735       3.10039       5.34601       2.70013       1.03085        10     
reco:pandoracalo:Calorimetry                        1.27444       1.71479       2.27459       1.61316       0.31903        10     
reco:pandoracalonosce:Calorimetry                   1.16027       1.59279       2.1302        1.52619      0.323391        10     
reco:pandorapid:Chi2ParticleID                    0.00344357    0.00484759    0.00701828    0.00446554    0.00127829       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0317961     0.0416199     0.0609657     0.037857     0.00928225       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00327479    0.00424568    0.00705122    0.00390077    0.00101149       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.15638       3.48087       5.93098       3.02825       1.11775        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.98892       3.42397       5.94929       2.96644       1.14923        10     
reco:emtrkmichelid:EmTrackMichelId                  109.51        133.279       172.519       130.018       18.1399        10     
reco:crttag:SingleCRTMatchingProducer             0.00755725     0.0107205     0.0196019     0.0087376    0.00393159       10     
reco:crtreco:TwoCRTMatchingProducer               0.00477744     0.0059435    0.00805354    0.00561767    0.000921683      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.522953      0.724939      0.859235      0.753584      0.117226        10     
reco:pandora2Track:LArPandoraTrackCreation          5.64313       9.1756        14.5158       8.37214       2.52456        10     
reco:pandora2calo:Calorimetry                       3.09013       4.96144       7.96871       4.40224       1.40367        10     
reco:pandora2calonosce:Calorimetry                  2.98031       4.6581        7.19816       4.18053       1.24386        10     
reco:pandora2pid:Chi2ParticleID                   0.00736113     0.0102639     0.0134408    0.00984389    0.00179144       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0381393     0.0487553     0.0633081     0.0448207    0.00788326       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00819517     0.0100401     0.0122876    0.00976382    0.00130698       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.17428       3.30842       5.45104       2.9697       0.898506        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.7981        4.2595        6.66618       3.84327       1.06989        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.5737        3.85935       6.18826       3.49088       1.0019         10     
[art]:TriggerResults:TriggerResultInserter         3.663e-05    5.95122e-05   0.000128923   4.69265e-05   2.75615e-05      10     
end_path:out1:RootOutput                           7.04e-06     1.25472e-05   2.5821e-05     1.074e-05    4.87573e-06      10     
end_path:out1:RootOutput(write)                     1.78039       2.25261       2.98745       2.0997       0.388945        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5106.52 MB
  Peak resident set size usage (VmHWM): 4256.16 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.43 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11035184
-rw-r--r--. 1 duneprd np-comp 2557207238 Jan 20 18:05 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2423868481 Jan 20 18:03 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2267821604 Jan 20 18:51 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 2025767489 Jan 20 16:13 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1943552473 Jan 20 15:59 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   81188050 Jan 20 18:47 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242983 Jan 20 18:51 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 20 18:51 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 20 18:51 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 20 18:51 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 20 18:05 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 20 16:13 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 20 18:51 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 20 15:51 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 20 18:51 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 20 18:10 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 20 18:05 debugprod.log
justIN time: 2026-02-04 07:20:29 UTC       justIN version: 01.06.00