justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 299749.1@dunegpschedd01.fnal.gov

Jobsub ID299749.1@dunegpschedd01.fnal.gov
Workflow ID12294
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-23 06:43:56
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce506
Last heartbeat2026-01-23 10:25:53
From worker nodeHostnameb9p05p8732.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-23 06:45:24
Input filesmonte-carlo-012294-000001
JobscriptExit code0
Real time3h (13140s)
CPU time3h (12240s = 93%)
Max RSS bytes4214534144 (4019 MiB)
Outputting started2026-01-23 10:24:25
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12294/1/001/reco_stage1.root
Finished2026-01-23 10:25:53
Saved logsjustin-logs:299749.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

unning tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 146244 traces, 0 summary
Retagger: tagging trace set: gauss with 108952 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 108952 traces tagged "gauss"
FrameSaver: q=3.86054e+06 n=1903014 tag=gauss
wclsFrameSaver: saving 146244 traces tagged "wiener"
FrameSaver: q=4.11541e+06 n=1689817 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38500 XUs and 34029 XVs -> 449 XUVs
C:0 T:1 65362 XUs and 61290 XVs -> 7816 XUVs
C:0 T:2 64653 XUs and 85377 XVs -> 3729 XUVs
C:0 T:3 30456 XUs and 34042 XVs -> 323 XUVs
C:0 T:4 12188 XUs and 13539 XVs -> 147 XUVs
C:0 T:5 46052 XUs and 51493 XVs -> 6972 XUVs
C:0 T:6 105052 XUs and 111646 XVs -> 5337 XUVs
C:0 T:7 45772 XUs and 43754 XVs -> 386 XUVs
C:0 T:8 8415 XUs and 7815 XVs -> 68 XUVs
C:0 T:9 50927 XUs and 55532 XVs -> 4289 XUVs
C:0 T:10 98178 XUs and 69073 XVs -> 10267 XUVs
C:0 T:11 502 XUs and 425 XVs -> 38 XUVs
39821 XUVs total
11850 collection wire objects
36956 potential space points
Neighbour search...
3468794 tests to find 1734102 neighbours
Iterating with no regularization...
Begin: 8.11271e+07
0 6.31245e+07
1 6.19614e+07
2 6.17905e+07
3 6.17407e+07
Now with regularization...
Begin: 5.02579e+07
0 5.01953e+07
1 5.0163e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411912392
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411912392
fLifetime: 30000 [us]
23-Jan-2026 11:24:23 CET  Closed output file "reco_stage1.root"
23-Jan-2026 11:24:23 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          249.501       322.509       482.838       321.717       64.3576        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000992248   0.00326844    0.00561787    0.00331799    0.00137689       10     
reco:ophit:OpHitFinder                             0.220997      0.304595      0.417342      0.306447      0.050964        10     
reco:opflash:OpFlashFinder                         0.149463      0.191431      0.236012       0.19059      0.0259146       10     
reco:opslicer:OpSlicer                             0.545585      0.710576       0.86379       0.70293      0.086855        10     
reco:caldata:DataPrepModule                         4.68546       5.36409       6.35407       5.37194      0.535793        10     
reco:wclsdatasp:WireCellToolkit                     30.6291       32.0151       33.9198       31.5676       1.08497        10     
reco:gaushit:GausHitFinder                         0.830549       1.16456       1.68357       1.08529      0.251018        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000300258   0.000507805   0.000816463   0.000467558   0.000161998      10     
reco:reco3d:SpacePointSolver                        9.85727       13.835        28.4601       11.8661       5.16759        10     
reco:hitpdune:DisambigFromSpacePoints               1.15046       1.72028       3.32052       1.67468      0.594695        10     
reco:pandora:StandardPandora                        16.8673       30.9686       64.9582       29.6017       13.3282        10     
reco:pandoraWriter:StandardPandora                 0.260145      0.410288       0.58619      0.399441      0.0893189       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.37288       3.35627       5.29049       3.36265      0.801233        10     
reco:pandoraShower:LArPandoraShowerCreation         2.18778       3.68832       7.1402        3.59022       1.4083         10     
reco:pandoracalo:Calorimetry                        1.32482       1.99219       3.11006       1.94902      0.493849        10     
reco:pandoracalonosce:Calorimetry                    1.113        1.76596       2.81585       1.72851      0.450272        10     
reco:pandorapid:Chi2ParticleID                    0.00346652    0.00533729    0.00780256    0.00515265    0.00138091       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0311423     0.0504101     0.077649      0.0499871     0.0131245       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00333323    0.00508489    0.00690504     0.0048941    0.00106289       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.55302       4.0823        7.42898       3.98627       1.39247        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.14553       4.03497       7.29009       3.99608       1.42785        10     
reco:emtrkmichelid:EmTrackMichelId                  142.509       176.479       239.953       176.059       27.3615        10     
reco:crttag:SingleCRTMatchingProducer             0.00993136     0.0152171     0.0212694     0.0150563    0.00304209       10     
reco:crtreco:TwoCRTMatchingProducer               0.00670232    0.00851128     0.0102285    0.00847294    0.00107737       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.708444       0.95252       1.19849      0.959432      0.143296        10     
reco:pandora2Track:LArPandoraTrackCreation          6.45003       11.4671       21.0265       10.9425       3.71252        10     
reco:pandora2calo:Calorimetry                       3.50489       5.78293       9.85244       5.38277       1.64524        10     
reco:pandora2calonosce:Calorimetry                  3.0993        5.50608       9.61414       5.23788       1.67754        10     
reco:pandora2pid:Chi2ParticleID                    0.0095876     0.0128647     0.0182178     0.0129406    0.00237682       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0476124     0.0653963     0.0942019     0.0665381     0.0136995       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00933821     0.0134977     0.0193818     0.0141635    0.00275692       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.4466        3.95581       7.09382       3.69596       1.20735        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.77996       5.05734       8.96499       4.84156       1.57758        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.63164       4.70605       8.28518       4.71113       1.43899        10     
[art]:TriggerResults:TriggerResultInserter        3.8951e-05    6.21597e-05   0.000150634   5.31915e-05   3.16499e-05      10     
end_path:out1:RootOutput                          1.0211e-05    1.26145e-05   1.6971e-05     1.198e-05    2.05544e-06      10     
end_path:out1:RootOutput(write)                     2.07779       2.7283        3.81182       2.64591      0.466673        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5054.78 MB
  Peak resident set size usage (VmHWM): 4214.53 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.38 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10685996
-rw-r--r--. 1 duneprd np-comp 2476914350 Jan 23 10:27 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2347215366 Jan 23 10:25 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2175519775 Jan 23 11:24 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1976662588 Jan 23 08:11 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1887678697 Jan 23 07:54 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   77854170 Jan 23 11:20 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243017 Jan 23 11:24 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 23 11:24 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 23 11:24 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 23 11:24 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 23 10:27 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 23 08:11 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 23 11:24 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 23 07:45 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 23 11:24 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 23 10:34 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 23 10:27 debugprod.log
justIN time: 2026-02-05 04:09:50 UTC       justIN version: 01.06.00