justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 299748.0@dunegpschedd01.fnal.gov

Jobsub ID299748.0@dunegpschedd01.fnal.gov
Workflow ID12293
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-23 06:43:56
SiteCZ_FZU
EntryNova_CZ_FZU_osgce1
Last heartbeat2026-01-23 10:58:51
From worker nodeHostnamemikan50.farm.particle.cz
cpuinfoIntel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-23 06:45:00
Input filesmonte-carlo-012293-000001
JobscriptExit code0
Real time4h (15180s)
CPU time4h (14864s = 97%)
Max RSS bytes4446154752 (4240 MiB)
Outputting started2026-01-23 10:58:01
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12293/1/001/reco_stage1.root
Finished2026-01-23 10:58:51
Saved logsjustin-logs:299748.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 171624 traces, 0 summary
Retagger: tagging trace set: gauss with 126271 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 126271 traces tagged "gauss"
FrameSaver: q=4.26893e+06 n=2172636 tag=gauss
wclsFrameSaver: saving 171624 traces tagged "wiener"
FrameSaver: q=4.56567e+06 n=1906259 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 39582 XUs and 40134 XVs -> 630 XUVs
C:0 T:1 109314 XUs and 99259 XVs -> 14104 XUVs
C:0 T:2 193875 XUs and 147714 XVs -> 14350 XUVs
C:0 T:3 75423 XUs and 51402 XVs -> 501 XUVs
C:0 T:4 14618 XUs and 14480 XVs -> 168 XUVs
C:0 T:5 69551 XUs and 65306 XVs -> 9904 XUVs
C:0 T:6 164372 XUs and 163144 XVs -> 12633 XUVs
C:0 T:7 60979 XUs and 54380 XVs -> 1010 XUVs
C:0 T:8 9335 XUs and 8660 XVs -> 93 XUVs
C:0 T:9 63718 XUs and 73688 XVs -> 6743 XUVs
C:0 T:10 42154 XUs and 51371 XVs -> 6471 XUVs
C:0 T:11 197 XUs and 136 XVs -> 17 XUVs
66624 XUVs total
15336 collection wire objects
61051 potential space points
Neighbour search...
8628129 tests to find 4130392 neighbours
Iterating with no regularization...
Begin: 7.97431e+07
0 6.66342e+07
1 6.5598e+07
2 6.54259e+07
3 6.53695e+07
Now with regularization...
Begin: 5.32524e+07
0 5.31912e+07
1 5.31374e+07
2 5.30964e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411912415
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411912415
fLifetime: 30000 [us]
23-Jan-2026 11:57:59 CET  Closed output file "reco_stage1.root"
23-Jan-2026 11:57:59 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          436.806       559.153       823.044       555.907       111.578        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00118068    0.00584898     0.0108372    0.00479634    0.00340212       10     
reco:ophit:OpHitFinder                             0.305356      0.365458      0.435358      0.358649      0.040248        10     
reco:opflash:OpFlashFinder                         0.192398      0.246339      0.356588      0.240598      0.0446294       10     
reco:opslicer:OpSlicer                             0.685274      0.853045       1.00968      0.851144      0.0925121       10     
reco:caldata:DataPrepModule                         6.35978       6.75942       7.06995       6.74171      0.196805        10     
reco:wclsdatasp:WireCellToolkit                     46.8767       51.7077       59.5747       51.8168       3.60921        10     
reco:gaushit:GausHitFinder                          1.31625       1.72612       2.04725       1.73244      0.220762        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000645668   0.000761375   0.000864796   0.00074965    8.35726e-05      10     
reco:reco3d:SpacePointSolver                        13.1369       26.8674       35.9804       27.9872       6.98949        10     
reco:hitpdune:DisambigFromSpacePoints               2.30296       3.50585       6.67754       3.14966       1.24462        10     
reco:pandora:StandardPandora                        34.6984       57.8399       99.1254       56.6127       17.9022        10     
reco:pandoraWriter:StandardPandora                 0.510864      0.699727       1.19769      0.635252      0.198715        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.32016       10.6568       20.8624       9.59158       4.23803        10     
reco:pandoraShower:LArPandoraShowerCreation         6.02774       11.0862       20.5515       9.82516       4.60001        10     
reco:pandoracalo:Calorimetry                        3.11453       4.86046       7.19835       4.87293       1.23703        10     
reco:pandoracalonosce:Calorimetry                   2.6394        4.35048       6.8174        3.94087       1.3943         10     
reco:pandorapid:Chi2ParticleID                    0.00710547    0.00894954     0.0141685    0.00848752    0.00189717       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0685453     0.0909802     0.112275      0.0949266     0.0138704       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00600271    0.00753188    0.00839673    0.00765909    0.000687229      10     
reco:pandoraShowercalo:ShowerCalorimetry            5.62414       9.88013       19.0905       8.60966       4.05635        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.39573       10.3269       19.8293       10.1939       4.41858        10     
reco:emtrkmichelid:EmTrackMichelId                  213.906       255.753       351.051       247.643       37.6456        10     
reco:crttag:SingleCRTMatchingProducer              0.0203986     0.0290556     0.0507316     0.0246943    0.00891663       10     
reco:crtreco:TwoCRTMatchingProducer                0.0145961     0.0226588     0.0447013     0.0179881     0.009463        10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.80433       2.22869       2.70139        2.198       0.314947        10     
reco:pandora2Track:LArPandoraTrackCreation          22.5031       33.2611       53.7749       32.0602       10.6866        10     
reco:pandora2calo:Calorimetry                       8.93392       14.3455       22.5456       14.8344       4.44686        10     
reco:pandora2calonosce:Calorimetry                  7.53281       13.7101       21.4248       13.6825       4.8537         10     
reco:pandora2pid:Chi2ParticleID                    0.0172955     0.0239453     0.0375121     0.0206015    0.00698303       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0864683     0.123885      0.178842      0.118982      0.0274474       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0164752     0.0220368     0.0332751     0.0202449    0.00557479       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.29079       10.9811       18.3065       11.9564       3.89578        10     
reco:pandora2Showercalo:ShowerCalorimetry           7.07215        11.67        20.4954       11.9633       4.05021        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.32841       11.094        20.1908       11.203        3.83348        10     
[art]:TriggerResults:TriggerResultInserter        5.8534e-05    8.90117e-05   0.000161317   8.18905e-05   2.87101e-05      10     
end_path:out1:RootOutput                          1.0224e-05    1.20336e-05   1.6226e-05    1.13585e-05   1.92907e-06      10     
end_path:out1:RootOutput(write)                     3.04441       3.88717       5.29525       3.76756      0.574383        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5260.39 MB
  Peak resident set size usage (VmHWM): 4446.15 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.26 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.18 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10957172
-rw-r--r--. 1 fermilab users 2539419119 Jan 23 10:22 detsim_stage2.root
-rw-r--r--. 1 fermilab users 2407981216 Jan 23 10:18 detsim_stage1.root
-rw-r--r--. 1 fermilab users 2260074228 Jan 23 11:57 reco_stage1.root
-rw-r--r--. 1 fermilab users 2011786958 Jan 23 08:13 g4_stage2.root
-rw-r--r--. 1 fermilab users 1917551619 Jan 23 07:52 g4_stage1.root
-rw-r--r--. 1 fermilab users   82700550 Jan 23 11:49 Pandora_Events.pndr
-rw-r--r--. 1 fermilab users     243138 Jan 23 11:58 jobscript.log
-rw-r--r--. 1 fermilab users     217088 Jan 23 11:57 mem.db
-rw-r--r--. 1 fermilab users      62376 Jan 23 11:57 Pandora_Geometry.xml
-rw-r--r--. 1 fermilab users      36864 Jan 23 11:57 time.db
-rw-r--r--. 1 fermilab users        519 Jan 23 10:22 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 fermilab users        519 Jan 23 08:13 g4_protoDUNE_hist.root
-rw-r--r--. 1 fermilab users        519 Jan 23 11:57 protoDUNE_reco1_hist.root
-rw-r--r--. 1 fermilab users         52 Jan 23 07:45 all-input-dids.txt
-rw-r--r--. 1 fermilab users          7 Jan 23 11:58 justin-processed-pfns.txt
-rw-r--r--. 1 fermilab users          0 Jan 23 10:29 ProtoDUNE.log
-rw-r--r--. 1 fermilab users          0 Jan 23 10:22 debugprod.log
justIN time: 2026-02-04 11:47:27 UTC       justIN version: 01.06.00