justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271060.1@dunegpschedd01.fnal.gov

Jobsub ID271060.1@dunegpschedd01.fnal.gov
Workflow ID11436
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-31 08:47:40
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-31 10:41:23
From worker nodeHostnamedunegli-8201300-0-fnpc23028.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-31 08:48:59
Input filesmonte-carlo-011436-000001
JobscriptExit code0
Real time1h (6731s)
CPU time1h (6649s = 98%)
Max RSS bytes4218785792 (4023 MiB)
Outputting started2025-12-31 10:41:11
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11436/1/001/reco_stage1.root
Finished2025-12-31 10:41:23
Saved logsjustin-logs:271060.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

g tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 104847 traces, 0 summary
Retagger: tagging trace set: gauss with 78562 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 78562 traces tagged "gauss"
FrameSaver: q=2.65526e+06 n=1357699 tag=gauss
wclsFrameSaver: saving 104847 traces tagged "wiener"
FrameSaver: q=2.81336e+06 n=1194303 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34545 XUs and 31943 XVs -> 324 XUVs
C:0 T:1 82877 XUs and 101260 XVs -> 16298 XUVs
C:0 T:2 95043 XUs and 139643 XVs -> 5329 XUVs
C:0 T:3 48714 XUs and 57286 XVs -> 411 XUVs
C:0 T:4 6223 XUs and 5299 XVs -> 87 XUVs
C:0 T:5 27116 XUs and 30997 XVs -> 5931 XUVs
C:0 T:6 49890 XUs and 75055 XVs -> 7677 XUVs
C:0 T:7 19139 XUs and 23303 XVs -> 297 XUVs
C:0 T:8 5513 XUs and 5794 XVs -> 82 XUVs
C:0 T:9 32867 XUs and 50964 XVs -> 4191 XUVs
C:0 T:10 16947 XUs and 13654 XVs -> 4643 XUVs
C:0 T:11 52 XUs and 32 XVs -> 5 XUVs
45275 XUVs total
9072 collection wire objects
41677 potential space points
Neighbour search...
8158723 tests to find 3576406 neighbours
Iterating with no regularization...
Begin: 5.38286e+07
0 4.78589e+07
1 4.73742e+07
2 4.72917e+07
3 4.72655e+07
Now with regularization...
Begin: 3.89737e+07
0 3.89327e+07
1 3.88926e+07
2 3.88613e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411451460
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411451460
fLifetime: 30000 [us]
31-Dec-2025 10:41:10 UTC  Closed output file "reco_stage1.root"
31-Dec-2025 10:41:10 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          116.513       190.486       287.079       178.864        53.92         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000903587   0.00284741    0.00434617    0.00306986    0.00113522       10     
reco:ophit:OpHitFinder                             0.159558      0.189755      0.247515      0.182434      0.0243012       10     
reco:opflash:OpFlashFinder                         0.0789472     0.116443      0.174418      0.109225      0.0267475       10     
reco:opslicer:OpSlicer                             0.272535      0.392727      0.524903      0.388782      0.0775241       10     
reco:caldata:DataPrepModule                         3.37122       3.52274       3.81741       3.49975      0.122543        10     
reco:wclsdatasp:WireCellToolkit                     22.4184       23.9118       24.6516       24.2443      0.743554        10     
reco:gaushit:GausHitFinder                          0.46812      0.668462      0.947262      0.626098      0.149987        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000218557   0.000374392   0.000663468   0.000343436   0.00012257       10     
reco:reco3d:SpacePointSolver                        3.64115       9.25891       26.5363       7.58681       6.43188        10     
reco:hitpdune:DisambigFromSpacePoints               0.36679      0.857506       1.4688       0.748394      0.348396        10     
reco:pandora:StandardPandora                        6.19499       16.9968       34.5805       13.9121       8.66262        10     
reco:pandoraWriter:StandardPandora                 0.144953      0.257047      0.366513      0.255264      0.0710673       10     
reco:pandoraTrack:LArPandoraTrackCreation          0.879328       1.99477       3.98168       1.83634      0.921836        10     
reco:pandoraShower:LArPandoraShowerCreation        0.780965       2.32404       6.30022       1.97689       1.54937        10     
reco:pandoracalo:Calorimetry                       0.546328       1.16051       2.2536        1.04374       0.50749        10     
reco:pandoracalonosce:Calorimetry                  0.481172       1.05572       1.99544       1.02003      0.466328        10     
reco:pandorapid:Chi2ParticleID                    0.00230809    0.00333105    0.00616126    0.00300952      0.00109        10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0197089     0.0297134     0.0406672     0.0298044     0.0063894       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00215594    0.00323137    0.00439153     0.0030698    0.00062434       10     
reco:pandoraShowercalo:ShowerCalorimetry           0.818214       2.4816        6.66637       2.0325        1.70099        10     
reco:pandoraShowercalonosce:ShowerCalorimetry      0.789528       2.35157       6.22332        1.899        1.58336        10     
reco:emtrkmichelid:EmTrackMichelId                  64.9585       100.028       133.409       97.8206       22.3967        10     
reco:crttag:SingleCRTMatchingProducer             0.00513787    0.00860924     0.0149479    0.00753058    0.00287892       10     
reco:crtreco:TwoCRTMatchingProducer               0.00313342    0.00482925    0.00751178    0.00442271    0.00132325       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.311604      0.519226      0.758861      0.526178      0.115345        10     
reco:pandora2Track:LArPandoraTrackCreation          2.3394        6.35345       15.9692       5.42714       3.91659        10     
reco:pandora2calo:Calorimetry                       1.27673       3.35986       7.0149        3.10968       1.7853         10     
reco:pandora2calonosce:Calorimetry                  1.1856        3.1265        6.13173       3.01705       1.60736        10     
reco:pandora2pid:Chi2ParticleID                   0.00482376    0.00733334     0.0106767    0.00679173    0.00206381       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0235571     0.036495      0.0501098     0.0358637     0.0076363       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00458422    0.00736197     0.0107555    0.00711707     0.0020621       10     
reco:pandora2Shower:LArPandoraShowerCreation       0.909755       2.23747       4.08014       2.20099       1.00305        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.11028       2.80172       5.09108       2.77184       1.3421         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.04017       2.58597       4.78692       2.6717        1.20769        10     
[art]:TriggerResults:TriggerResultInserter        3.2752e-05    5.88564e-05   0.000178661   4.7656e-05    4.0347e-05       10     
end_path:out1:RootOutput                           7.063e-06    1.17372e-05   1.9417e-05    1.1607e-05    3.48084e-06      10     
end_path:out1:RootOutput(write)                     1.07052       1.73723       2.45615       1.71229      0.430311        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5030.23 MB
  Peak resident set size usage (VmHWM): 4218.79 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.20 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9729756
-rw-r--r-- 1 dunegli fnalgrid 2308259051 Dec 31 10:08 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2199518270 Dec 31 10:07 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1960751696 Dec 31 10:41 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1758938107 Dec 31 08:59 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1668394923 Dec 31 08:52 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   66792790 Dec 31 10:39 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     242734 Dec 31 10:41 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Dec 31 10:41 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Dec 31 10:41 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Dec 31 10:41 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Dec 31 10:08 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Dec 31 08:59 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Dec 31 10:41 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Dec 31 08:49 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Dec 31 10:41 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Dec 31 10:12 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Dec 31 10:08 debugprod.log
justIN time: 2026-02-06 23:32:35 UTC       justIN version: 01.06.00